2024-04-20T10:00:52.093 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T10:00:52.094 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/kernel_untar_build}} --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445 --verbose --name rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi -- /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445/orig.config.yaml 2024-04-20T10:00:52.150 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T10:00:52.265 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445 branch: wip-rishabh-testing-20240416.193735 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/kernel_untar_build}} email: null first_in_suite: false job_id: '7664445' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: a8c84f5997950fd7d008204839253291d52e6eda subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7954 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCLxedFLQQooUioetMNOWMSW3V3DZBT3SZGtYGjkEavLNXRfqwvYiBnhj37FmDrLfhFL546HlP9/18VyZRadts= smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNvWiyW+vBj4WJa04epY2fu4M+1srgsmyOzK2JFbH4XVy32KcTmP8AxkpXBroArg2JrtL/lomIreJnIn9bWJW9I= smithi162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEGMrSYtx1ReTq6hR4AO9tNxgdLp5UhF+fXUZTlcHIimdAt8vJ5tY0gsMEKMLNemlxgMlXB8HEOfgG+5vfck9E8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - kernel_untar_build.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_09:50:15 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-20T10:00:52.265 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-20T10:00:52.266 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-20T10:00:52.266 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T10:00:52.268 INFO:teuthology.task.internal:Checking packages... 2024-04-20T10:00:52.291 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-20T10:00:52.292 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T10:00:52.292 INFO:teuthology.packaging:ref: None 2024-04-20T10:00:52.292 INFO:teuthology.packaging:tag: None 2024-04-20T10:00:52.292 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-20T10:00:52.292 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:00:52.292 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-20T10:00:52.522 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-20T10:00:52.523 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T10:00:52.530 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T10:00:52.530 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T10:00:52.584 INFO:teuthology.task.internal:Saving configuration 2024-04-20T10:00:52.605 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T10:00:52.642 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T10:00:52.666 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 09:51:24.035993', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCLxedFLQQooUioetMNOWMSW3V3DZBT3SZGtYGjkEavLNXRfqwvYiBnhj37FmDrLfhFL546HlP9/18VyZRadts='} 2024-04-20T10:00:52.685 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 09:51:24.032904', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNvWiyW+vBj4WJa04epY2fu4M+1srgsmyOzK2JFbH4XVy32KcTmP8AxkpXBroArg2JrtL/lomIreJnIn9bWJW9I='} 2024-04-20T10:00:52.709 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 09:51:24.034627', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEGMrSYtx1ReTq6hR4AO9tNxgdLp5UhF+fXUZTlcHIimdAt8vJ5tY0gsMEKMLNemlxgMlXB8HEOfgG+5vfck9E8='} 2024-04-20T10:00:52.709 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T10:00:52.715 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-20T10:00:52.715 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-20T10:00:52.715 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-20T10:00:52.716 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T10:00:52.847 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2fa518d160>, signals=[15]) 2024-04-20T10:00:52.848 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T10:00:52.854 INFO:teuthology.task.internal:Opening connections... 2024-04-20T10:00:52.854 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-04-20T10:00:52.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:00:52.932 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-04-20T10:00:52.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:00:53.006 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2024-04-20T10:00:53.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:00:53.082 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T10:00:53.089 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-04-20T10:00:53.107 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-04-20T10:00:53.108 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-04-20T10:00:53.165 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-04-20T10:00:53.166 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-04-20T10:00:53.166 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-04-20T10:00:53.166 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:00:53.166 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-04-20T10:00:53.166 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:00:53.167 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:00:53.167 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-04-20T10:00:53.167 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:00:53.167 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:00:53.167 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:00:53.167 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:00:53.167 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:00:53.167 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:00:53.168 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-04-20T10:00:53.196 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-04-20T10:00:53.214 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-04-20T10:00:53.215 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-04-20T10:00:53.273 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-04-20T10:00:53.273 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-04-20T10:00:53.273 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-04-20T10:00:53.274 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:00:53.274 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-04-20T10:00:53.274 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:00:53.274 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:00:53.274 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-04-20T10:00:53.274 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:00:53.274 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:00:53.274 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:00:53.274 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:00:53.274 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:00:53.274 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:00:53.275 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-04-20T10:00:53.295 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2024-04-20T10:00:53.313 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2024-04-20T10:00:53.313 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2024-04-20T10:00:53.370 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2024-04-20T10:00:53.371 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="9" 2024-04-20T10:00:53.371 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2024-04-20T10:00:53.371 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:00:53.371 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="9" 2024-04-20T10:00:53.371 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:00:53.371 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:00:53.371 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2024-04-20T10:00:53.372 INFO:teuthology.orchestra.run.smithi162.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:00:53.372 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:00:53.372 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:00:53.372 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:00:53.372 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:00:53.372 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:00:53.372 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2024-04-20T10:00:53.395 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T10:00:53.406 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T10:00:53.468 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T10:00:53.468 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:00:53.471 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:00:53.473 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:00:53.489 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T10:00:53.509 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T10:00:53.510 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:00:53.528 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:00:53.530 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:00:53.557 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T10:00:53.637 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T10:00:53.653 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-20T10:00:53.653 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-20T10:00:53.653 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-20T10:00:53.654 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T10:00:53.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:00:53.671 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-20T10:00:53.838 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:00:53.839 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T10:00:53.839 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-04-20T10:00:53.857 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-437.el9.x86_64 2024-04-20T10:00:53.857 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:00:53.857 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T10:00:53.858 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum upgrade -y linux-firmware 2024-04-20T10:00:57.347 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-04-20T10:00:59.164 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-20T10:01:05.077 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.2 MB 00:02 2024-04-20T10:01:09.184 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 5.6 kB/s | 16 kB 00:02 2024-04-20T10:01:10.437 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-20T10:01:15.651 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-20T10:01:17.747 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-20T10:01:17.748 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-04-20T10:01:17.749 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-20T10:01:17.845 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-20T10:01:17.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:01:18.108 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T10:01:18.109 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T10:01:18.111 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi042.front.sepia.ceph.com to complete... 2024-04-20T10:09:36.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20 10:09:36 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T10:09:36.428 INFO:teuthology.orchestra.run.smithi042.stderr:FINISHED --2024-04-20 10:09:36-- 2024-04-20T10:09:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:Total wall clock time: 8m 18s 2024-04-20T10:09:36.429 INFO:teuthology.orchestra.run.smithi042.stderr:Downloaded: 1 files, 341M in 8m 18s (701 KB/s) 2024-04-20T10:09:36.432 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-20T10:09:36.433 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T10:09:38.634 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying... ######################################## 2024-04-20T10:09:38.758 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing... ######################################## 2024-04-20T10:09:38.863 INFO:teuthology.orchestra.run.smithi042.stdout:Updating / installing... 2024-04-20T10:09:47.134 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T10:10:25.138 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi042.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T10:10:25.139 DEBUG:teuthology.orchestra.run.smithi042:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T10:10:25.264 INFO:teuthology.orchestra.run.smithi042.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:25.264 INFO:teuthology.orchestra.run.smithi042.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:25.264 INFO:teuthology.orchestra.run.smithi042.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:25.264 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T10:10:25.264 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T10:10:25.264 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T10:10:25.264 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T10:10:25.264 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T10:10:25.264 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T10:10:25.264 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T10:10:25.264 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T10:10:25.264 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T10:10:25.265 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T10:10:25.266 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T10:10:25.267 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T10:10:25.268 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T10:10:25.269 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T10:10:25.270 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T10:10:25.271 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T10:10:25.272 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T10:10:25.273 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T10:10:25.274 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T10:10:25.275 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T10:10:25.276 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T10:10:25.277 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T10:10:25.278 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T10:10:25.279 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T10:10:25.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T10:10:25.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T10:10:25.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T10:10:25.283 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T10:10:25.284 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T10:10:25.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T10:10:25.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T10:10:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T10:10:25.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T10:10:25.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T10:10:25.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T10:10:25.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T10:10:25.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T10:10:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T10:10:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T10:10:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T10:10:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T10:10:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T10:10:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T10:10:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T10:10:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T10:10:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T10:10:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T10:10:25.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T10:10:25.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T10:10:25.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T10:10:25.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T10:10:25.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T10:10:25.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T10:10:25.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T10:10:25.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T10:10:25.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T10:10:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T10:10:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T10:10:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T10:10:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T10:10:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T10:10:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T10:10:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T10:10:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T10:10:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T10:10:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T10:10:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T10:10:25.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T10:10:25.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T10:10:25.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T10:10:25.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T10:10:25.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T10:10:25.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T10:10:25.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T10:10:25.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T10:10:25.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T10:10:25.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T10:10:25.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T10:10:25.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T10:10:25.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T10:10:25.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T10:10:25.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T10:10:25.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T10:10:25.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T10:10:25.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T10:10:25.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T10:10:25.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T10:10:25.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T10:10:25.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T10:10:25.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T10:10:25.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T10:10:25.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T10:10:25.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T10:10:25.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T10:10:25.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T10:10:25.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T10:10:25.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T10:10:25.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T10:10:25.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T10:10:25.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T10:10:25.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T10:10:25.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T10:10:25.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T10:10:25.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T10:10:25.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T10:10:25.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T10:10:25.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T10:10:25.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T10:10:25.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T10:10:25.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T10:10:25.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T10:10:25.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T10:10:25.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T10:10:25.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T10:10:25.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T10:10:25.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T10:10:25.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T10:10:25.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T10:10:25.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T10:10:25.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T10:10:25.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T10:10:25.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T10:10:25.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T10:10:25.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T10:10:25.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T10:10:25.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T10:10:25.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T10:10:25.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T10:10:25.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T10:10:25.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T10:10:25.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T10:10:25.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T10:10:25.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T10:10:25.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T10:10:25.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T10:10:25.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T10:10:25.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T10:10:25.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T10:10:25.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T10:10:25.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T10:10:25.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T10:10:25.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T10:10:25.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T10:10:25.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T10:10:25.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T10:10:25.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T10:10:25.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T10:10:25.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T10:10:25.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T10:10:25.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T10:10:25.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T10:10:25.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T10:10:25.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T10:10:25.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T10:10:25.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T10:10:25.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T10:10:25.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T10:10:25.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T10:10:25.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T10:10:25.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T10:10:25.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T10:10:25.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T10:10:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T10:10:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T10:10:25.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T10:10:25.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T10:10:25.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T10:10:25.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T10:10:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T10:10:25.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T10:10:25.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T10:10:25.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T10:10:25.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T10:10:25.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T10:10:25.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T10:10:25.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T10:10:25.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T10:10:25.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T10:10:25.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T10:10:25.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T10:10:25.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T10:10:25.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T10:10:25.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T10:10:25.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T10:10:25.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T10:10:25.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T10:10:25.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T10:10:25.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T10:10:25.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T10:10:25.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T10:10:25.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T10:10:25.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T10:10:25.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T10:10:25.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T10:10:25.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T10:10:25.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T10:10:25.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T10:10:25.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T10:10:25.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T10:10:25.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T10:10:25.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T10:10:25.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T10:10:25.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T10:10:25.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T10:10:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T10:10:25.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T10:10:25.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T10:10:25.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T10:10:25.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T10:10:25.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T10:10:25.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T10:10:25.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T10:10:25.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T10:10:25.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T10:10:25.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T10:10:25.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T10:10:25.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T10:10:25.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T10:10:25.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T10:10:25.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T10:10:25.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T10:10:25.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T10:10:25.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T10:10:25.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T10:10:25.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T10:10:25.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T10:10:25.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T10:10:25.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T10:10:25.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T10:10:25.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T10:10:25.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T10:10:25.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T10:10:25.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T10:10:25.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T10:10:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T10:10:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T10:10:25.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T10:10:25.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T10:10:25.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T10:10:25.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T10:10:25.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:10:25.407 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:25.407 DEBUG:teuthology.orchestra.run.smithi042:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout:for file in vmlinuz System.map config; do 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: fi 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout:done 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:10:25.431 INFO:teuthology.orchestra.run.smithi042.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:10:25.432 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-04-20T10:10:25.432 INFO:teuthology.orchestra.run.smithi042.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T10:10:25.432 INFO:teuthology.orchestra.run.smithi042.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:25.432 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-04-20T10:10:25.432 INFO:teuthology.orchestra.run.smithi042.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T10:10:25.432 INFO:teuthology.orchestra.run.smithi042.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:10:25.432 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:25.432 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-04-20T10:10:25.433 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -qi grub2-tools 2024-04-20T10:10:25.514 INFO:teuthology.orchestra.run.smithi042.stdout:Name : grub2-tools 2024-04-20T10:10:25.514 INFO:teuthology.orchestra.run.smithi042.stdout:Epoch : 1 2024-04-20T10:10:25.515 INFO:teuthology.orchestra.run.smithi042.stdout:Version : 2.06 2024-04-20T10:10:25.515 INFO:teuthology.orchestra.run.smithi042.stdout:Release : 68.el9 2024-04-20T10:10:25.515 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: x86_64 2024-04-20T10:10:25.515 INFO:teuthology.orchestra.run.smithi042.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T10:10:25.515 INFO:teuthology.orchestra.run.smithi042.stdout:Group : Unspecified 2024-04-20T10:10:25.515 INFO:teuthology.orchestra.run.smithi042.stdout:Size : 8264254 2024-04-20T10:10:25.515 INFO:teuthology.orchestra.run.smithi042.stdout:License : GPLv3+ 2024-04-20T10:10:25.515 INFO:teuthology.orchestra.run.smithi042.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T10:10:25.516 INFO:teuthology.orchestra.run.smithi042.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T10:10:25.516 INFO:teuthology.orchestra.run.smithi042.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T10:10:25.516 INFO:teuthology.orchestra.run.smithi042.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T10:10:25.516 INFO:teuthology.orchestra.run.smithi042.stdout:Packager : builder@centos.org 2024-04-20T10:10:25.516 INFO:teuthology.orchestra.run.smithi042.stdout:Vendor : CentOS 2024-04-20T10:10:25.516 INFO:teuthology.orchestra.run.smithi042.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T10:10:25.516 INFO:teuthology.orchestra.run.smithi042.stdout:Summary : Support tools for GRUB. 2024-04-20T10:10:25.516 INFO:teuthology.orchestra.run.smithi042.stdout:Description : 2024-04-20T10:10:25.516 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:10:25.516 INFO:teuthology.orchestra.run.smithi042.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T10:10:25.516 INFO:teuthology.orchestra.run.smithi042.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T10:10:25.516 INFO:teuthology.orchestra.run.smithi042.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T10:10:25.517 INFO:teuthology.orchestra.run.smithi042.stdout:hardware devices. 2024-04-20T10:10:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:10:25.517 INFO:teuthology.orchestra.run.smithi042.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T10:10:25.518 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T10:10:25.518 INFO:teuthology.task.kernel:Updating grub on smithi042 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:25.518 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T10:10:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-04-20T10:10:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T10:10:27.420 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-04-20T10:10:27.422 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-04-20T10:10:27.441 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.S2HvtqBzCE 2024-04-20T10:10:27.441 DEBUG:teuthology.orchestra.run.smithi042:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.S2HvtqBzCE 2024-04-20T10:10:27.511 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.S2HvtqBzCE 2024-04-20T10:10:27.660 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.S2HvtqBzCE is 6KB 2024-04-20T10:10:27.709 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.S2HvtqBzCE 2024-04-20T10:10:27.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T10:10:27.788 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T10:10:27.789 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T10:10:27.789 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T10:10:27.789 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T10:10:27.790 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:27.930 DEBUG:teuthology.orchestra.run.smithi042:> sudo shutdown -r now 2024-04-20T10:10:57.933 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-20T10:10:57.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-20T10:10:57.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:11:57.937 DEBUG:teuthology.orchestra.remote:timed out 2024-04-20T10:12:06.939 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-20T10:12:06.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:12:07.301 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-04-20T10:12:07.931 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-20T10:12:07.931 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T10:12:07.931 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-04-20T10:12:07.977 INFO:teuthology.orchestra.run.smithi042.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T10:12:07.977 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T10:12:07.977 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T10:12:07.977 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T10:12:08.980 DEBUG:teuthology.parallel:result is None 2024-04-20T10:12:08.980 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T10:12:08.988 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T10:12:08.988 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T10:12:08.995 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T10:12:08.998 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T10:12:09.037 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T10:12:09.044 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T10:12:09.080 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T10:12:09.080 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T10:12:09.085 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T10:12:09.088 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T10:12:09.147 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T10:12:09.154 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T10:12:09.155 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T10:12:09.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:12:09.174 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T10:12:09.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:12:09.191 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T10:12:09.198 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T10:12:09.217 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T10:12:09.250 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:12:09.267 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:12:09.270 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:12:09.284 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:12:09.315 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:12:09.366 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:12:09.371 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T10:12:09.380 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T10:12:09.380 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T10:12:09.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T10:12:09.387 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T10:12:09.450 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T10:12:09.459 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T10:12:09.460 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T10:12:09.464 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T10:12:09.467 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T10:12:09.501 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:12:09.621 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:12:09.692 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:12:09.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:12:09.847 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:12:09.847 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T10:12:09.926 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:12:09.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:12:10.031 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:12:10.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:12:10.158 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-20T10:12:10.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T10:12:10.225 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:12:10.270 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:12:10.602 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:12:10.647 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:12:10.689 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-20T10:12:10.689 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T10:12:10.755 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-04-20T10:12:10.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-04-20T10:12:10.764 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2024-04-20T10:12:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:12:10.834 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:12:10.871 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:12:11.209 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T10:12:11.217 INFO:teuthology.task.internal:Starting timer... 2024-04-20T10:12:11.217 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T10:12:11.241 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T10:12:11.290 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-20T10:12:11.290 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2024-04-20T10:12:11.416 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2024-04-20T10:12:11.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-04-20T10:12:11.489 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-04-20T10:12:11.491 DEBUG:teuthology.orchestra.run.smithi162:> sudo service auditd rotate 2024-04-20T10:12:11.540 INFO:teuthology.orchestra.run.smithi162.stdout:Rotating logs: 2024-04-20T10:12:11.542 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T10:12:11.542 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2024-04-20T10:12:11.599 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2024-04-20T10:12:11.599 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-04-20T10:12:11.619 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-04-20T10:12:11.619 DEBUG:teuthology.orchestra.run.smithi162:> /usr/sbin/getenforce 2024-04-20T10:12:11.641 INFO:teuthology.orchestra.run.smithi162.stdout:Permissive 2024-04-20T10:12:11.641 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi042.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi162.front.sepia.ceph.com': 'permissive'} 2024-04-20T10:12:11.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:12:11.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:12:11.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:12:11.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:12:11.744 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:12:11.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:12:11.772 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T10:12:11.773 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2024-04-20T10:12:11.959 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-04-20T10:12:11.989 DEBUG:teuthology.orchestra.run.smithi162:> sudo /usr/sbin/setenforce permissive 2024-04-20T10:12:12.014 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T10:12:12.024 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T10:12:12.048 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T10:12:12.049 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi046.front.sepia.ceph.com,smithi162.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T10:14:48.106 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi162.front.sepia.ceph.com')] 2024-04-20T10:14:48.107 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-20T10:14:48.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:14:48.234 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-04-20T10:14:48.392 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-20T10:14:48.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-20T10:14:48.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:14:48.468 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-04-20T10:14:48.556 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-20T10:14:48.556 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-04-20T10:14:48.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:14:48.633 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-04-20T10:14:48.711 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-04-20T10:14:48.711 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T10:14:48.723 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T10:14:48.723 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T10:14:48.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:14:48.731 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T10:14:48.732 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:14:48.734 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T10:14:48.734 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:14:48.765 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T10:14:48.781 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T10:14:48.781 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T10:14:48.796 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T10:14:48.807 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-04-20T10:14:48.818 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-04-20T10:14:48.820 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: ntpd: command not found 2024-04-20T10:14:48.829 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T10:14:48.831 INFO:teuthology.orchestra.run.smithi162.stdout:506 Cannot talk to daemon 2024-04-20T10:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T10:14:48.844 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T10:14:48.847 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T10:14:48.858 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T10:14:48.875 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T10:14:48.903 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:14:48.907 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:14:48.907 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-04-20T10:14:48.907 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:48.907 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:48.907 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:48.907 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:48.914 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:14:48.917 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:14:48.918 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-04-20T10:14:48.918 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:48.918 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:48.918 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:48.918 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:48.952 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2024-04-20T10:14:48.986 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2024-04-20T10:14:49.026 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T10:14:49.068 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T10:14:49.216 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:14:49.228 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:14:49.228 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-04-20T10:14:49.228 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:49.228 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:49.229 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:49.229 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:49.229 INFO:teuthology.run_tasks:Running task install... 2024-04-20T10:14:49.242 DEBUG:teuthology.task.install:project ceph 2024-04-20T10:14:49.243 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-20T10:14:49.243 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-20T10:14:49.243 INFO:teuthology.task.install:Using flavor: default 2024-04-20T10:14:49.257 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T10:14:49.257 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-20T10:14:49.258 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T10:14:49.259 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:14:49.263 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T10:14:49.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:14:49.264 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T10:14:49.264 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:14:49.409 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T10:14:49.409 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T10:14:49.421 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T10:14:49.421 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T10:14:49.424 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T10:14:49.424 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T10:14:49.687 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T10:14:49.687 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:14:49.688 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T10:14:49.696 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T10:14:49.696 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-20T10:14:49.697 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T10:14:49.726 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T10:14:49.726 DEBUG:teuthology.orchestra.run.smithi162:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T10:14:49.758 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T10:14:49.758 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-20T10:14:49.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T10:14:49.780 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T10:14:49.780 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T10:14:49.789 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T10:14:49.789 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T10:14:49.793 DEBUG:teuthology.orchestra.run.smithi162:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T10:14:49.859 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T10:14:49.870 DEBUG:teuthology.orchestra.run.smithi162:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T10:14:49.915 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T10:14:49.934 INFO:teuthology.orchestra.run.smithi162.stdout:check_obsoletes = 1 2024-04-20T10:14:49.936 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum clean all 2024-04-20T10:14:49.940 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T10:14:50.007 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-04-20T10:14:50.009 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-04-20T10:14:50.038 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T10:14:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2024-04-20T10:14:50.127 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2024-04-20T10:14:50.164 INFO:teuthology.orchestra.run.smithi162.stdout:48 files removed 2024-04-20T10:14:50.191 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T10:14:50.250 INFO:teuthology.orchestra.run.smithi046.stdout:48 files removed 2024-04-20T10:14:50.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T10:14:50.451 INFO:teuthology.orchestra.run.smithi042.stdout:48 files removed 2024-04-20T10:14:50.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T10:14:51.206 INFO:teuthology.orchestra.run.smithi162.stdout:ceph packages for x86_64 111 kB/s | 84 kB 00:00 2024-04-20T10:14:51.401 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 98 kB/s | 84 kB 00:00 2024-04-20T10:14:51.837 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 93 kB/s | 84 kB 00:00 2024-04-20T10:14:51.939 INFO:teuthology.orchestra.run.smithi162.stdout:ceph noarch packages 17 kB/s | 12 kB 00:00 2024-04-20T10:14:52.177 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-04-20T10:14:52.607 INFO:teuthology.orchestra.run.smithi162.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-04-20T10:14:52.754 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 14 kB/s | 12 kB 00:00 2024-04-20T10:14:52.853 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-04-20T10:14:53.601 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 8.0 MB 00:00 2024-04-20T10:14:53.613 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 2.5 kB/s | 1.9 kB 00:00 2024-04-20T10:14:54.487 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-20T10:14:55.541 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-20T10:14:56.377 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.0 MB 00:03 2024-04-20T10:14:56.963 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-20T10:14:59.649 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-20T10:15:01.755 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 6.2 MB 00:00 2024-04-20T10:15:03.742 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-20T10:15:05.041 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-20T10:15:09.368 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 1.6 MB/s | 19 MB 00:11 2024-04-20T10:15:10.848 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 1.6 kB/s | 16 kB 00:10 2024-04-20T10:15:11.177 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-20T10:15:12.774 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-20T10:15:13.746 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:15:13.748 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:15:13.770 INFO:teuthology.orchestra.run.smithi042.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T10:15:13.771 INFO:teuthology.orchestra.run.smithi042.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T10:15:13.772 INFO:teuthology.orchestra.run.smithi042.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T10:15:13.774 INFO:teuthology.orchestra.run.smithi042.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T10:15:13.776 INFO:teuthology.orchestra.run.smithi042.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T10:15:13.780 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T10:15:13.781 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T10:15:13.859 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-20T10:15:13.875 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-04-20T10:15:13.875 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-04-20T10:15:13.875 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-04-20T10:15:13.875 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-20T10:15:13.876 INFO:teuthology.orchestra.run.smithi042.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T10:15:13.876 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T10:15:13.876 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:15:13.876 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T10:15:13.876 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T10:15:13.876 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T10:15:13.876 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T10:15:13.876 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T10:15:13.876 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T10:15:13.876 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T10:15:13.876 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T10:15:13.876 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T10:15:13.877 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T10:15:13.878 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T10:15:13.878 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T10:15:13.878 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T10:15:13.878 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T10:15:13.878 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T10:15:13.878 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T10:15:13.878 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T10:15:13.878 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:15:13.878 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T10:15:13.878 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T10:15:13.878 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T10:15:13.879 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T10:15:13.880 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T10:15:13.881 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T10:15:13.882 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T10:15:13.883 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T10:15:13.884 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T10:15:13.885 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T10:15:13.886 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T10:15:13.887 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T10:15:13.887 INFO:teuthology.orchestra.run.smithi042.stdout:Installing weak dependencies: 2024-04-20T10:15:13.887 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T10:15:13.887 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:15:13.887 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-20T10:15:13.887 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-04-20T10:15:13.887 INFO:teuthology.orchestra.run.smithi042.stdout:Install 148 Packages 2024-04-20T10:15:13.887 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 2 Packages 2024-04-20T10:15:13.887 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:15:13.888 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 218 M 2024-04-20T10:15:13.888 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-20T10:15:13.894 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.2 MB 00:00 2024-04-20T10:15:15.770 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-20T10:15:16.897 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-04-20T10:15:17.849 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-20T10:15:20.008 INFO:teuthology.orchestra.run.smithi162.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:15:20.009 INFO:teuthology.orchestra.run.smithi162.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:15:20.019 INFO:teuthology.orchestra.run.smithi162.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T10:15:20.020 INFO:teuthology.orchestra.run.smithi162.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T10:15:20.020 INFO:teuthology.orchestra.run.smithi162.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T10:15:20.021 INFO:teuthology.orchestra.run.smithi162.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T10:15:20.023 INFO:teuthology.orchestra.run.smithi162.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T10:15:20.025 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T10:15:20.026 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T10:15:20.085 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-04-20T10:15:20.098 INFO:teuthology.orchestra.run.smithi162.stdout:======================================================================================= 2024-04-20T10:15:20.098 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout:======================================================================================= 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T10:15:20.099 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T10:15:20.100 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrading: 2024-04-20T10:15:20.101 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T10:15:20.102 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T10:15:20.103 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T10:15:20.104 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T10:15:20.105 INFO:teuthology.orchestra.run.smithi162.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T10:15:20.106 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T10:15:20.107 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T10:15:20.108 INFO:teuthology.orchestra.run.smithi162.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T10:15:20.108 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T10:15:20.108 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T10:15:20.108 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T10:15:20.108 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T10:15:20.108 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T10:15:20.109 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T10:15:20.109 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T10:15:20.109 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T10:15:20.109 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T10:15:20.109 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T10:15:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout:Installing weak dependencies: 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout:======================================================================================= 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout:Install 148 Packages 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrade 2 Packages 2024-04-20T10:15:20.111 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:15:20.112 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 218 M 2024-04-20T10:15:20.112 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-04-20T10:15:22.163 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-04-20T10:15:24.415 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:15:24.415 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:15:24.426 INFO:teuthology.orchestra.run.smithi046.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T10:15:24.427 INFO:teuthology.orchestra.run.smithi046.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T10:15:24.428 INFO:teuthology.orchestra.run.smithi046.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T10:15:24.429 INFO:teuthology.orchestra.run.smithi046.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T10:15:24.430 INFO:teuthology.orchestra.run.smithi046.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T10:15:24.432 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T10:15:24.434 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T10:15:24.497 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-04-20T10:15:24.516 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-04-20T10:15:24.516 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-04-20T10:15:24.516 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-04-20T10:15:24.516 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-04-20T10:15:24.516 INFO:teuthology.orchestra.run.smithi046.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T10:15:24.516 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T10:15:24.516 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T10:15:24.517 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T10:15:24.518 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T10:15:24.518 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T10:15:24.518 INFO:teuthology.orchestra.run.smithi046.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T10:15:24.518 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T10:15:24.518 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T10:15:24.518 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T10:15:24.518 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T10:15:24.518 INFO:teuthology.orchestra.run.smithi046.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T10:15:24.518 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T10:15:24.518 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T10:15:24.518 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T10:15:24.518 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:15:24.519 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-04-20T10:15:24.520 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T10:15:24.520 INFO:teuthology.orchestra.run.smithi046.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T10:15:24.520 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T10:15:24.520 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T10:15:24.520 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T10:15:24.520 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T10:15:24.520 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T10:15:24.520 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T10:15:24.520 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T10:15:24.520 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T10:15:24.520 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T10:15:24.520 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T10:15:24.521 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T10:15:24.522 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T10:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T10:15:24.524 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T10:15:24.525 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T10:15:24.526 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T10:15:24.527 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-04-20T10:15:24.528 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T10:15:24.528 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:15:24.528 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-04-20T10:15:24.528 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-04-20T10:15:24.528 INFO:teuthology.orchestra.run.smithi046.stdout:Install 148 Packages 2024-04-20T10:15:24.528 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 2 Packages 2024-04-20T10:15:24.528 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:15:24.528 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 218 M 2024-04-20T10:15:24.528 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-04-20T10:15:25.020 INFO:teuthology.orchestra.run.smithi042.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-20T10:15:25.633 INFO:teuthology.orchestra.run.smithi162.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-20T10:15:25.806 INFO:teuthology.orchestra.run.smithi046.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-20T10:15:26.337 INFO:teuthology.orchestra.run.smithi042.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 678 kB/s | 893 kB 00:01 2024-04-20T10:15:26.646 INFO:teuthology.orchestra.run.smithi042.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 471 kB/s | 145 kB 00:00 2024-04-20T10:15:27.056 INFO:teuthology.orchestra.run.smithi046.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 714 kB/s | 893 kB 00:01 2024-04-20T10:15:27.541 INFO:teuthology.orchestra.run.smithi162.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 468 kB/s | 893 kB 00:01 2024-04-20T10:15:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 181 kB/s | 145 kB 00:00 2024-04-20T10:15:27.874 INFO:teuthology.orchestra.run.smithi162.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 435 kB/s | 145 kB 00:00 2024-04-20T10:15:28.554 INFO:teuthology.orchestra.run.smithi042.stdout:(4/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 1.3 MB/s | 5.4 MB 00:04 2024-04-20T10:15:30.024 INFO:teuthology.orchestra.run.smithi046.stdout:(4/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 1.1 MB/s | 5.4 MB 00:04 2024-04-20T10:15:30.165 INFO:teuthology.orchestra.run.smithi046.stdout:(5/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 1.0 MB/s | 2.4 MB 00:02 2024-04-20T10:15:31.088 INFO:teuthology.orchestra.run.smithi042.stdout:(5/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 544 kB/s | 2.4 MB 00:04 2024-04-20T10:15:31.146 INFO:teuthology.orchestra.run.smithi042.stdout:(6/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 592 kB/s | 1.5 MB 00:02 2024-04-20T10:15:31.524 INFO:teuthology.orchestra.run.smithi046.stdout:(6/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 1.0 MB/s | 1.5 MB 00:01 2024-04-20T10:15:33.882 INFO:teuthology.orchestra.run.smithi162.stdout:(4/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 638 kB/s | 5.4 MB 00:08 2024-04-20T10:15:35.632 INFO:teuthology.orchestra.run.smithi162.stdout:(5/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 312 kB/s | 2.4 MB 00:07 2024-04-20T10:15:37.190 INFO:teuthology.orchestra.run.smithi162.stdout:(6/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 464 kB/s | 1.5 MB 00:03 2024-04-20T10:15:43.054 INFO:teuthology.orchestra.run.smithi042.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 402 kB/s | 4.7 MB 00:11 2024-04-20T10:15:46.248 INFO:teuthology.orchestra.run.smithi046.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 299 kB/s | 4.7 MB 00:16 2024-04-20T10:15:53.289 INFO:teuthology.orchestra.run.smithi162.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 272 kB/s | 4.7 MB 00:17 2024-04-20T10:16:05.922 INFO:teuthology.orchestra.run.smithi046.stdout:(8/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x86 501 kB/s | 17 MB 00:34 2024-04-20T10:16:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 253 kB/s | 25 kB 00:00 2024-04-20T10:16:15.245 INFO:teuthology.orchestra.run.smithi042.stdout:(8/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 426 kB/s | 21 MB 00:50 2024-04-20T10:16:15.410 INFO:teuthology.orchestra.run.smithi042.stdout:(9/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el9 335 kB/s | 11 MB 00:32 2024-04-20T10:16:15.468 INFO:teuthology.orchestra.run.smithi042.stdout:(10/150): ceph-selinux-19.0.0-3133.ga8c84f59.el 112 kB/s | 25 kB 00:00 2024-04-20T10:16:16.794 INFO:teuthology.orchestra.run.smithi042.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 170 kB/s | 225 kB 00:01 2024-04-20T10:16:16.994 INFO:teuthology.orchestra.run.smithi042.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 135 kB/s | 27 kB 00:00 2024-04-20T10:16:17.253 INFO:teuthology.orchestra.run.smithi042.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 123 kB/s | 32 kB 00:00 2024-04-20T10:16:19.221 INFO:teuthology.orchestra.run.smithi046.stdout:(10/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 329 kB/s | 11 MB 00:32 2024-04-20T10:16:19.628 INFO:teuthology.orchestra.run.smithi042.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 315 kB/s | 748 kB 00:02 2024-04-20T10:16:19.658 INFO:teuthology.orchestra.run.smithi162.stdout:(8/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el9 411 kB/s | 11 MB 00:26 2024-04-20T10:16:19.758 INFO:teuthology.orchestra.run.smithi162.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 252 kB/s | 25 kB 00:00 2024-04-20T10:16:20.046 INFO:teuthology.orchestra.run.smithi046.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 272 kB/s | 225 kB 00:00 2024-04-20T10:16:20.103 INFO:teuthology.orchestra.run.smithi042.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 344 kB/s | 163 kB 00:00 2024-04-20T10:16:20.511 INFO:teuthology.orchestra.run.smithi042.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 310 kB/s | 126 kB 00:00 2024-04-20T10:16:20.579 INFO:teuthology.orchestra.run.smithi046.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 51 kB/s | 27 kB 00:00 2024-04-20T10:16:21.263 INFO:teuthology.orchestra.run.smithi046.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 46 kB/s | 32 kB 00:00 2024-04-20T10:16:21.720 INFO:teuthology.orchestra.run.smithi042.stdout:(17/150): libradosstriper1-19.0.0-3133.ga8c84f5 415 kB/s | 501 kB 00:01 2024-04-20T10:16:22.500 INFO:teuthology.orchestra.run.smithi162.stdout:(10/150): ceph-common-19.0.0-3133.ga8c84f59.el9 378 kB/s | 21 MB 00:57 2024-04-20T10:16:23.208 INFO:teuthology.orchestra.run.smithi162.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 317 kB/s | 225 kB 00:00 2024-04-20T10:16:23.392 INFO:teuthology.orchestra.run.smithi162.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 148 kB/s | 27 kB 00:00 2024-04-20T10:16:23.592 INFO:teuthology.orchestra.run.smithi162.stdout:(13/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 371 kB/s | 17 MB 00:46 2024-04-20T10:16:23.617 INFO:teuthology.orchestra.run.smithi162.stdout:(14/150): libcephfs-devel-19.0.0-3133.ga8c84f59 141 kB/s | 32 kB 00:00 2024-04-20T10:16:24.279 INFO:teuthology.orchestra.run.smithi046.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 248 kB/s | 748 kB 00:03 2024-04-20T10:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 190 kB/s | 163 kB 00:00 2024-04-20T10:16:25.201 INFO:teuthology.orchestra.run.smithi162.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 175 kB/s | 126 kB 00:00 2024-04-20T10:16:26.068 INFO:teuthology.orchestra.run.smithi162.stdout:(17/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 302 kB/s | 748 kB 00:02 2024-04-20T10:16:26.113 INFO:teuthology.orchestra.run.smithi046.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 89 kB/s | 163 kB 00:01 2024-04-20T10:16:26.920 INFO:teuthology.orchestra.run.smithi042.stdout:(18/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 309 kB/s | 17 MB 00:55 2024-04-20T10:16:26.988 INFO:teuthology.orchestra.run.smithi046.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 145 kB/s | 126 kB 00:00 2024-04-20T10:16:27.359 INFO:teuthology.orchestra.run.smithi162.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 232 kB/s | 501 kB 00:02 2024-04-20T10:16:27.511 INFO:teuthology.orchestra.run.smithi042.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 76 kB/s | 45 kB 00:00 2024-04-20T10:16:27.951 INFO:teuthology.orchestra.run.smithi162.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 76 kB/s | 45 kB 00:00 2024-04-20T10:16:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 148 kB/s | 135 kB 00:00 2024-04-20T10:16:28.526 INFO:teuthology.orchestra.run.smithi162.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 234 kB/s | 135 kB 00:00 2024-04-20T10:16:29.404 INFO:teuthology.orchestra.run.smithi042.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 165 kB/s | 162 kB 00:00 2024-04-20T10:16:29.909 INFO:teuthology.orchestra.run.smithi162.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 117 kB/s | 162 kB 00:01 2024-04-20T10:16:29.913 INFO:teuthology.orchestra.run.smithi046.stdout:(17/150): libradosstriper1-19.0.0-3133.ga8c84f5 171 kB/s | 501 kB 00:02 2024-04-20T10:16:31.287 INFO:teuthology.orchestra.run.smithi042.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 170 kB/s | 321 kB 00:01 2024-04-20T10:16:32.001 INFO:teuthology.orchestra.run.smithi162.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 153 kB/s | 321 kB 00:02 2024-04-20T10:16:32.362 INFO:teuthology.orchestra.run.smithi042.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 278 kB/s | 299 kB 00:01 2024-04-20T10:16:32.729 INFO:teuthology.orchestra.run.smithi042.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 272 kB/s | 99 kB 00:00 2024-04-20T10:16:33.038 INFO:teuthology.orchestra.run.smithi042.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 277 kB/s | 85 kB 00:00 2024-04-20T10:16:33.651 INFO:teuthology.orchestra.run.smithi162.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 181 kB/s | 299 kB 00:01 2024-04-20T10:16:34.309 INFO:teuthology.orchestra.run.smithi162.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 151 kB/s | 99 kB 00:00 2024-04-20T10:16:34.734 INFO:teuthology.orchestra.run.smithi162.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 200 kB/s | 85 kB 00:00 2024-04-20T10:16:36.071 INFO:teuthology.orchestra.run.smithi042.stdout:(26/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 383 kB/s | 5.4 MB 00:14 2024-04-20T10:16:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 297 kB/s | 170 kB 00:00 2024-04-20T10:16:36.822 INFO:teuthology.orchestra.run.smithi042.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 196 kB/s | 34 kB 00:00 2024-04-20T10:16:37.029 INFO:teuthology.orchestra.run.smithi046.stdout:(18/150): ceph-common-19.0.0-3133.ga8c84f59.el9 302 kB/s | 21 MB 01:11 2024-04-20T10:16:37.338 INFO:teuthology.orchestra.run.smithi046.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 146 kB/s | 45 kB 00:00 2024-04-20T10:16:37.489 INFO:teuthology.orchestra.run.smithi042.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 213 kB/s | 142 kB 00:00 2024-04-20T10:16:38.121 INFO:teuthology.orchestra.run.smithi046.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 172 kB/s | 135 kB 00:00 2024-04-20T10:16:39.246 INFO:teuthology.orchestra.run.smithi046.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 144 kB/s | 162 kB 00:01 2024-04-20T10:16:40.705 INFO:teuthology.orchestra.run.smithi046.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 220 kB/s | 321 kB 00:01 2024-04-20T10:16:42.263 INFO:teuthology.orchestra.run.smithi046.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 192 kB/s | 299 kB 00:01 2024-04-20T10:16:42.472 INFO:teuthology.orchestra.run.smithi042.stdout:(30/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 335 kB/s | 3.1 MB 00:09 2024-04-20T10:16:43.041 INFO:teuthology.orchestra.run.smithi046.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 128 kB/s | 99 kB 00:00 2024-04-20T10:16:43.539 INFO:teuthology.orchestra.run.smithi046.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 170 kB/s | 85 kB 00:00 2024-04-20T10:16:45.483 INFO:teuthology.orchestra.run.smithi162.stdout:(26/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 283 kB/s | 5.4 MB 00:19 2024-04-20T10:16:46.399 INFO:teuthology.orchestra.run.smithi162.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 186 kB/s | 170 kB 00:00 2024-04-20T10:16:46.608 INFO:teuthology.orchestra.run.smithi162.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 164 kB/s | 34 kB 00:00 2024-04-20T10:16:47.308 INFO:teuthology.orchestra.run.smithi162.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 203 kB/s | 142 kB 00:00 2024-04-20T10:16:48.983 INFO:teuthology.orchestra.run.smithi162.stdout:(30/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 222 kB/s | 3.1 MB 00:14 2024-04-20T10:16:49.996 INFO:teuthology.orchestra.run.smithi042.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 290 kB/s | 3.5 MB 00:12 2024-04-20T10:16:50.813 INFO:teuthology.orchestra.run.smithi042.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 305 kB/s | 249 kB 00:00 2024-04-20T10:16:51.005 INFO:teuthology.orchestra.run.smithi042.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 258 kB/s | 49 kB 00:00 2024-04-20T10:16:51.105 INFO:teuthology.orchestra.run.smithi042.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 166 kB/s | 16 kB 00:00 2024-04-20T10:16:52.047 INFO:teuthology.orchestra.run.smithi042.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 290 kB/s | 273 kB 00:00 2024-04-20T10:16:54.205 INFO:teuthology.orchestra.run.smithi046.stdout:(26/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 297 kB/s | 3.1 MB 00:10 2024-04-20T10:16:54.796 INFO:teuthology.orchestra.run.smithi046.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 288 kB/s | 170 kB 00:00 2024-04-20T10:16:55.005 INFO:teuthology.orchestra.run.smithi046.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 164 kB/s | 34 kB 00:00 2024-04-20T10:16:55.089 INFO:teuthology.orchestra.run.smithi042.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 251 kB/s | 763 kB 00:03 2024-04-20T10:16:55.189 INFO:teuthology.orchestra.run.smithi042.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 185 kB/s | 18 kB 00:00 2024-04-20T10:16:55.331 INFO:teuthology.orchestra.run.smithi042.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 569 kB/s | 80 kB 00:00 2024-04-20T10:16:55.505 INFO:teuthology.orchestra.run.smithi046.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 284 kB/s | 142 kB 00:00 2024-04-20T10:16:55.523 INFO:teuthology.orchestra.run.smithi042.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-04-20T10:16:55.632 INFO:teuthology.orchestra.run.smithi042.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-20T10:16:55.719 INFO:teuthology.orchestra.run.smithi042.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-20T10:16:55.957 INFO:teuthology.orchestra.run.smithi042.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-04-20T10:16:56.016 INFO:teuthology.orchestra.run.smithi042.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-20T10:16:56.075 INFO:teuthology.orchestra.run.smithi042.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-20T10:16:56.133 INFO:teuthology.orchestra.run.smithi042.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-20T10:16:56.192 INFO:teuthology.orchestra.run.smithi042.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-20T10:16:56.525 INFO:teuthology.orchestra.run.smithi042.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-04-20T10:16:56.642 INFO:teuthology.orchestra.run.smithi042.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-04-20T10:16:56.701 INFO:teuthology.orchestra.run.smithi042.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-20T10:16:56.768 INFO:teuthology.orchestra.run.smithi042.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 326 kB/s | 21 kB 00:00 2024-04-20T10:16:56.843 INFO:teuthology.orchestra.run.smithi042.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-20T10:16:56.910 INFO:teuthology.orchestra.run.smithi042.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 494 kB/s | 33 kB 00:00 2024-04-20T10:16:57.077 INFO:teuthology.orchestra.run.smithi042.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-04-20T10:16:57.144 INFO:teuthology.orchestra.run.smithi042.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 277 kB/s | 18 kB 00:00 2024-04-20T10:16:57.212 INFO:teuthology.orchestra.run.smithi042.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-20T10:16:57.278 INFO:teuthology.orchestra.run.smithi042.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-20T10:16:57.362 INFO:teuthology.orchestra.run.smithi042.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 550 kB/s | 45 kB 00:00 2024-04-20T10:16:57.454 INFO:teuthology.orchestra.run.smithi042.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-20T10:16:57.571 INFO:teuthology.orchestra.run.smithi042.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-04-20T10:16:57.638 INFO:teuthology.orchestra.run.smithi042.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 385 kB/s | 25 kB 00:00 2024-04-20T10:16:57.705 INFO:teuthology.orchestra.run.smithi042.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 712 kB/s | 47 kB 00:00 2024-04-20T10:16:57.780 INFO:teuthology.orchestra.run.smithi042.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-20T10:16:57.847 INFO:teuthology.orchestra.run.smithi042.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-20T10:16:57.914 INFO:teuthology.orchestra.run.smithi042.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 531 kB/s | 35 kB 00:00 2024-04-20T10:16:58.131 INFO:teuthology.orchestra.run.smithi042.stdout:(65/150): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-04-20T10:16:58.332 INFO:teuthology.orchestra.run.smithi042.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 21 MB/s | 4.1 MB 00:00 2024-04-20T10:16:58.465 INFO:teuthology.orchestra.run.smithi042.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-20T10:16:58.574 INFO:teuthology.orchestra.run.smithi042.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-20T10:16:58.641 INFO:teuthology.orchestra.run.smithi042.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-20T10:16:58.874 INFO:teuthology.orchestra.run.smithi042.stdout:(70/150): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-04-20T10:16:58.983 INFO:teuthology.orchestra.run.smithi042.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 9.4 MB/s | 1.0 MB 00:00 2024-04-20T10:16:59.042 INFO:teuthology.orchestra.run.smithi042.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-20T10:16:59.275 INFO:teuthology.orchestra.run.smithi042.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-04-20T10:16:59.351 INFO:teuthology.orchestra.run.smithi042.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-20T10:16:59.426 INFO:teuthology.orchestra.run.smithi042.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.3 MB/s | 248 kB 00:00 2024-04-20T10:16:59.510 INFO:teuthology.orchestra.run.smithi042.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 309 kB/s | 26 kB 00:00 2024-04-20T10:16:59.576 INFO:teuthology.orchestra.run.smithi042.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 290 kB/s | 19 kB 00:00 2024-04-20T10:16:59.652 INFO:teuthology.orchestra.run.smithi042.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-04-20T10:16:59.719 INFO:teuthology.orchestra.run.smithi042.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 526 kB/s | 35 kB 00:00 2024-04-20T10:16:59.936 INFO:teuthology.orchestra.run.smithi042.stdout:(80/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-04-20T10:17:00.011 INFO:teuthology.orchestra.run.smithi042.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-20T10:17:00.078 INFO:teuthology.orchestra.run.smithi042.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-20T10:17:00.145 INFO:teuthology.orchestra.run.smithi042.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-20T10:17:00.212 INFO:teuthology.orchestra.run.smithi042.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 642 kB/s | 42 kB 00:00 2024-04-20T10:17:00.287 INFO:teuthology.orchestra.run.smithi042.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-20T10:17:00.355 INFO:teuthology.orchestra.run.smithi042.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-20T10:17:00.430 INFO:teuthology.orchestra.run.smithi042.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-20T10:17:00.522 INFO:teuthology.orchestra.run.smithi042.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 560 kB/s | 51 kB 00:00 2024-04-20T10:17:00.589 INFO:teuthology.orchestra.run.smithi042.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 813 kB/s | 54 kB 00:00 2024-04-20T10:17:00.688 INFO:teuthology.orchestra.run.smithi046.stdout:(30/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 179 kB/s | 5.4 MB 00:30 2024-04-20T10:17:01.039 INFO:teuthology.orchestra.run.smithi042.stdout:(90/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-04-20T10:17:01.106 INFO:teuthology.orchestra.run.smithi042.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 635 kB/s | 42 kB 00:00 2024-04-20T10:17:01.173 INFO:teuthology.orchestra.run.smithi042.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 669 kB/s | 44 kB 00:00 2024-04-20T10:17:01.248 INFO:teuthology.orchestra.run.smithi042.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-20T10:17:01.315 INFO:teuthology.orchestra.run.smithi042.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 546 kB/s | 36 kB 00:00 2024-04-20T10:17:01.382 INFO:teuthology.orchestra.run.smithi042.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 717 kB/s | 47 kB 00:00 2024-04-20T10:17:01.458 INFO:teuthology.orchestra.run.smithi042.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-20T10:17:01.525 INFO:teuthology.orchestra.run.smithi042.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 879 kB/s | 58 kB 00:00 2024-04-20T10:17:01.591 INFO:teuthology.orchestra.run.smithi042.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-20T10:17:01.658 INFO:teuthology.orchestra.run.smithi042.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 964 kB/s | 64 kB 00:00 2024-04-20T10:17:01.717 INFO:teuthology.orchestra.run.smithi042.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-04-20T10:17:01.784 INFO:teuthology.orchestra.run.smithi042.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 337 kB/s | 22 kB 00:00 2024-04-20T10:17:01.893 INFO:teuthology.orchestra.run.smithi042.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-04-20T10:17:02.451 INFO:teuthology.orchestra.run.smithi042.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 983 kB/s | 548 kB 00:00 2024-04-20T10:17:02.568 INFO:teuthology.orchestra.run.smithi042.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-04-20T10:17:02.652 INFO:teuthology.orchestra.run.smithi042.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 236 kB/s | 19 kB 00:00 2024-04-20T10:17:02.977 INFO:teuthology.orchestra.run.smithi042.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-20T10:17:03.070 INFO:teuthology.orchestra.run.smithi042.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 284 kB/s | 26 kB 00:00 2024-04-20T10:17:03.145 INFO:teuthology.orchestra.run.smithi042.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 659 kB/s | 49 kB 00:00 2024-04-20T10:17:03.203 INFO:teuthology.orchestra.run.smithi042.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-20T10:17:03.278 INFO:teuthology.orchestra.run.smithi042.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-20T10:17:03.379 INFO:teuthology.orchestra.run.smithi042.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.3 MB/s | 838 kB 00:00 2024-04-20T10:17:03.462 INFO:teuthology.orchestra.run.smithi042.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 6.5 MB/s | 548 kB 00:00 2024-04-20T10:17:03.538 INFO:teuthology.orchestra.run.smithi042.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 580 kB/s | 43 kB 00:00 2024-04-20T10:17:03.605 INFO:teuthology.orchestra.run.smithi042.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 487 kB/s | 32 kB 00:00 2024-04-20T10:17:03.672 INFO:teuthology.orchestra.run.smithi042.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 226 kB/s | 15 kB 00:00 2024-04-20T10:17:03.747 INFO:teuthology.orchestra.run.smithi042.stdout:(116/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.3 MB/s | 172 kB 00:00 2024-04-20T10:17:03.839 INFO:teuthology.orchestra.run.smithi042.stdout:(117/150): python3-cherrypy-18.6.1-2.el9.noarch 3.9 MB/s | 358 kB 00:00 2024-04-20T10:17:03.914 INFO:teuthology.orchestra.run.smithi042.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-20T10:17:03.981 INFO:teuthology.orchestra.run.smithi042.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 534 kB/s | 35 kB 00:00 2024-04-20T10:17:04.065 INFO:teuthology.orchestra.run.smithi042.stdout:(120/150): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-20T10:17:04.207 INFO:teuthology.orchestra.run.smithi042.stdout:(121/150): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-20T10:17:04.291 INFO:teuthology.orchestra.run.smithi042.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-20T10:17:04.357 INFO:teuthology.orchestra.run.smithi042.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 162 kB/s | 11 kB 00:00 2024-04-20T10:17:04.425 INFO:teuthology.orchestra.run.smithi042.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 267 kB/s | 18 kB 00:00 2024-04-20T10:17:04.491 INFO:teuthology.orchestra.run.smithi042.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 351 kB/s | 23 kB 00:00 2024-04-20T10:17:04.558 INFO:teuthology.orchestra.run.smithi042.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 295 kB/s | 19 kB 00:00 2024-04-20T10:17:04.634 INFO:teuthology.orchestra.run.smithi042.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 264 kB/s | 20 kB 00:00 2024-04-20T10:17:04.750 INFO:teuthology.orchestra.run.smithi042.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-04-20T10:17:04.818 INFO:teuthology.orchestra.run.smithi042.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 705 kB/s | 46 kB 00:00 2024-04-20T10:17:04.885 INFO:teuthology.orchestra.run.smithi042.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-20T10:17:04.951 INFO:teuthology.orchestra.run.smithi042.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 877 kB/s | 58 kB 00:00 2024-04-20T10:17:05.035 INFO:teuthology.orchestra.run.smithi042.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-20T10:17:05.102 INFO:teuthology.orchestra.run.smithi042.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 249 kB/s | 16 kB 00:00 2024-04-20T10:17:05.130 INFO:teuthology.orchestra.run.smithi162.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 203 kB/s | 3.5 MB 00:17 2024-04-20T10:17:05.177 INFO:teuthology.orchestra.run.smithi042.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-20T10:17:05.244 INFO:teuthology.orchestra.run.smithi042.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 428 kB/s | 28 kB 00:00 2024-04-20T10:17:05.311 INFO:teuthology.orchestra.run.smithi042.stdout:(136/150): python3-repoze-lru-0.7-16.el9.noarch 467 kB/s | 31 kB 00:00 2024-04-20T10:17:05.403 INFO:teuthology.orchestra.run.smithi042.stdout:(137/150): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-20T10:17:05.470 INFO:teuthology.orchestra.run.smithi042.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 896 kB/s | 59 kB 00:00 2024-04-20T10:17:05.537 INFO:teuthology.orchestra.run.smithi042.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 541 kB/s | 36 kB 00:00 2024-04-20T10:17:05.652 INFO:teuthology.orchestra.run.smithi042.stdout:(140/150): python3-typing-extensions-4.4.0-2.el 449 kB/s | 51 kB 00:00 2024-04-20T10:17:05.746 INFO:teuthology.orchestra.run.smithi042.stdout:(141/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-20T10:17:05.821 INFO:teuthology.orchestra.run.smithi042.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-20T10:17:05.905 INFO:teuthology.orchestra.run.smithi042.stdout:(143/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-20T10:17:05.972 INFO:teuthology.orchestra.run.smithi042.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 302 kB/s | 20 kB 00:00 2024-04-20T10:17:06.047 INFO:teuthology.orchestra.run.smithi042.stdout:(145/150): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-20T10:17:06.097 INFO:teuthology.orchestra.run.smithi162.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 257 kB/s | 249 kB 00:00 2024-04-20T10:17:06.173 INFO:teuthology.orchestra.run.smithi042.stdout:(146/150): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-20T10:17:06.305 INFO:teuthology.orchestra.run.smithi162.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 237 kB/s | 49 kB 00:00 2024-04-20T10:17:06.422 INFO:teuthology.orchestra.run.smithi162.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 141 kB/s | 16 kB 00:00 2024-04-20T10:17:07.289 INFO:teuthology.orchestra.run.smithi162.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 315 kB/s | 273 kB 00:00 2024-04-20T10:17:09.847 INFO:teuthology.orchestra.run.smithi162.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 298 kB/s | 763 kB 00:02 2024-04-20T10:17:10.647 INFO:teuthology.orchestra.run.smithi162.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 23 kB/s | 18 kB 00:00 2024-04-20T10:17:10.755 INFO:teuthology.orchestra.run.smithi162.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-20T10:17:10.856 INFO:teuthology.orchestra.run.smithi162.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 7.8 MB/s | 803 kB 00:00 2024-04-20T10:17:10.898 INFO:teuthology.orchestra.run.smithi162.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-20T10:17:10.940 INFO:teuthology.orchestra.run.smithi162.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-04-20T10:17:11.007 INFO:teuthology.orchestra.run.smithi162.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-20T10:17:11.049 INFO:teuthology.orchestra.run.smithi162.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-20T10:17:11.091 INFO:teuthology.orchestra.run.smithi162.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-20T10:17:11.132 INFO:teuthology.orchestra.run.smithi162.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-20T10:17:11.200 INFO:teuthology.orchestra.run.smithi162.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-04-20T10:17:11.250 INFO:teuthology.orchestra.run.smithi162.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-20T10:17:11.308 INFO:teuthology.orchestra.run.smithi162.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-20T10:17:11.350 INFO:teuthology.orchestra.run.smithi162.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-20T10:17:11.384 INFO:teuthology.orchestra.run.smithi162.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 642 kB/s | 21 kB 00:00 2024-04-20T10:17:11.426 INFO:teuthology.orchestra.run.smithi162.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-20T10:17:11.459 INFO:teuthology.orchestra.run.smithi162.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 979 kB/s | 33 kB 00:00 2024-04-20T10:17:11.551 INFO:teuthology.orchestra.run.smithi162.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-04-20T10:17:11.585 INFO:teuthology.orchestra.run.smithi162.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 549 kB/s | 18 kB 00:00 2024-04-20T10:17:11.619 INFO:teuthology.orchestra.run.smithi162.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 334 kB/s | 11 kB 00:00 2024-04-20T10:17:11.661 INFO:teuthology.orchestra.run.smithi162.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-20T10:17:11.702 INFO:teuthology.orchestra.run.smithi162.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-20T10:17:11.753 INFO:teuthology.orchestra.run.smithi162.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-20T10:17:11.795 INFO:teuthology.orchestra.run.smithi162.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 582 kB 00:00 2024-04-20T10:17:11.837 INFO:teuthology.orchestra.run.smithi162.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 611 kB/s | 25 kB 00:00 2024-04-20T10:17:11.879 INFO:teuthology.orchestra.run.smithi162.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-20T10:17:11.921 INFO:teuthology.orchestra.run.smithi162.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-04-20T10:17:11.962 INFO:teuthology.orchestra.run.smithi162.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-04-20T10:17:12.005 INFO:teuthology.orchestra.run.smithi162.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 845 kB/s | 35 kB 00:00 2024-04-20T10:17:12.105 INFO:teuthology.orchestra.run.smithi162.stdout:(65/150): openblas-openmp-0.3.21-2.el9.x86_64.r 47 MB/s | 4.7 MB 00:00 2024-04-20T10:17:12.205 INFO:teuthology.orchestra.run.smithi162.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 41 MB/s | 4.1 MB 00:00 2024-04-20T10:17:12.272 INFO:teuthology.orchestra.run.smithi162.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-04-20T10:17:12.322 INFO:teuthology.orchestra.run.smithi162.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 18 MB/s | 911 kB 00:00 2024-04-20T10:17:12.356 INFO:teuthology.orchestra.run.smithi162.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 3.9 MB/s | 133 kB 00:00 2024-04-20T10:17:12.473 INFO:teuthology.orchestra.run.smithi162.stdout:(70/150): postgresql-server-13.14-1.el9.x86_64. 50 MB/s | 5.8 MB 00:00 2024-04-20T10:17:12.523 INFO:teuthology.orchestra.run.smithi162.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-04-20T10:17:12.565 INFO:teuthology.orchestra.run.smithi162.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 2.0 MB/s | 87 kB 00:00 2024-04-20T10:17:12.691 INFO:teuthology.orchestra.run.smithi162.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 48 MB/s | 6.0 MB 00:00 2024-04-20T10:17:12.732 INFO:teuthology.orchestra.run.smithi162.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 5.8 MB/s | 244 kB 00:00 2024-04-20T10:17:12.774 INFO:teuthology.orchestra.run.smithi162.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-04-20T10:17:12.816 INFO:teuthology.orchestra.run.smithi162.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 618 kB/s | 26 kB 00:00 2024-04-20T10:17:12.858 INFO:teuthology.orchestra.run.smithi162.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 461 kB/s | 19 kB 00:00 2024-04-20T10:17:12.900 INFO:teuthology.orchestra.run.smithi162.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 4.1 MB/s | 172 kB 00:00 2024-04-20T10:17:12.942 INFO:teuthology.orchestra.run.smithi162.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 838 kB/s | 35 kB 00:00 2024-04-20T10:17:13.068 INFO:teuthology.orchestra.run.smithi162.stdout:(80/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-04-20T10:17:13.109 INFO:teuthology.orchestra.run.smithi162.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-04-20T10:17:13.151 INFO:teuthology.orchestra.run.smithi162.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-04-20T10:17:13.193 INFO:teuthology.orchestra.run.smithi162.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-20T10:17:13.227 INFO:teuthology.orchestra.run.smithi162.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 42 kB 00:00 2024-04-20T10:17:13.269 INFO:teuthology.orchestra.run.smithi162.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 6.4 MB/s | 269 kB 00:00 2024-04-20T10:17:13.311 INFO:teuthology.orchestra.run.smithi162.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-04-20T10:17:13.353 INFO:teuthology.orchestra.run.smithi162.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 6.6 MB/s | 279 kB 00:00 2024-04-20T10:17:13.386 INFO:teuthology.orchestra.run.smithi162.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-04-20T10:17:13.420 INFO:teuthology.orchestra.run.smithi162.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-04-20T10:17:13.679 INFO:teuthology.orchestra.run.smithi162.stdout:(90/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 63 MB/s | 16 MB 00:00 2024-04-20T10:17:13.712 INFO:teuthology.orchestra.run.smithi162.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-04-20T10:17:13.745 INFO:teuthology.orchestra.run.smithi162.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-04-20T10:17:13.787 INFO:teuthology.orchestra.run.smithi162.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-20T10:17:13.821 INFO:teuthology.orchestra.run.smithi162.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-04-20T10:17:13.854 INFO:teuthology.orchestra.run.smithi162.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-20T10:17:13.896 INFO:teuthology.orchestra.run.smithi162.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 7.2 MB/s | 305 kB 00:00 2024-04-20T10:17:13.912 INFO:teuthology.orchestra.run.smithi046.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 197 kB/s | 3.5 MB 00:18 2024-04-20T10:17:13.939 INFO:teuthology.orchestra.run.smithi162.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-04-20T10:17:13.972 INFO:teuthology.orchestra.run.smithi162.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.7 MB/s | 94 kB 00:00 2024-04-20T10:17:14.014 INFO:teuthology.orchestra.run.smithi162.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-04-20T10:17:14.056 INFO:teuthology.orchestra.run.smithi162.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-20T10:17:14.181 INFO:teuthology.orchestra.run.smithi162.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-04-20T10:17:14.424 INFO:teuthology.orchestra.run.smithi162.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-04-20T10:17:14.938 INFO:teuthology.orchestra.run.smithi042.stdout:(147/150): ceph-mgr-diskprediction-local-19.0.0 233 kB/s | 7.4 MB 00:32 2024-04-20T10:17:15.040 INFO:teuthology.orchestra.run.smithi162.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 890 kB/s | 548 kB 00:00 2024-04-20T10:17:15.240 INFO:teuthology.orchestra.run.smithi162.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 1.5 MB/s | 309 kB 00:00 2024-04-20T10:17:15.349 INFO:teuthology.orchestra.run.smithi162.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-04-20T10:17:15.439 INFO:teuthology.orchestra.run.smithi042.stdout:(148/150): librados2-19.0.0-3133.ga8c84f59.el9. 375 kB/s | 3.4 MB 00:09 2024-04-20T10:17:15.512 INFO:teuthology.orchestra.run.smithi046.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 155 kB/s | 249 kB 00:01 2024-04-20T10:17:15.608 INFO:teuthology.orchestra.run.smithi162.stdout:(106/150): ceph-mgr-diskprediction-local-19.0.0 285 kB/s | 7.4 MB 00:26 2024-04-20T10:17:15.883 INFO:teuthology.orchestra.run.smithi162.stdout:(107/150): libarrow-9.0.0-10.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2024-04-20T10:17:15.908 INFO:teuthology.orchestra.run.smithi162.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 86 kB/s | 26 kB 00:00 2024-04-20T10:17:16.008 INFO:teuthology.orchestra.run.smithi162.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 389 kB/s | 49 kB 00:00 2024-04-20T10:17:16.117 INFO:teuthology.orchestra.run.smithi162.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-20T10:17:16.159 INFO:teuthology.orchestra.run.smithi162.stdout:(111/150): libunwind-1.6.2-1.el9.x86_64.rpm 269 kB/s | 67 kB 00:00 2024-04-20T10:17:16.234 INFO:teuthology.orchestra.run.smithi162.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.0 MB/s | 838 kB 00:00 2024-04-20T10:17:16.343 INFO:teuthology.orchestra.run.smithi162.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-04-20T10:17:16.443 INFO:teuthology.orchestra.run.smithi162.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-04-20T10:17:16.502 INFO:teuthology.orchestra.run.smithi162.stdout:(115/150): python3-asyncssh-2.13.2-5.el9.noarch 1.6 MB/s | 548 kB 00:00 2024-04-20T10:17:16.512 INFO:teuthology.orchestra.run.smithi046.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 49 kB/s | 49 kB 00:00 2024-04-20T10:17:16.543 INFO:teuthology.orchestra.run.smithi162.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-20T10:17:16.611 INFO:teuthology.orchestra.run.smithi162.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-20T10:17:16.644 INFO:teuthology.orchestra.run.smithi162.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-20T10:17:16.729 INFO:teuthology.orchestra.run.smithi046.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 76 kB/s | 16 kB 00:00 2024-04-20T10:17:16.736 INFO:teuthology.orchestra.run.smithi162.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-20T10:17:16.761 INFO:teuthology.orchestra.run.smithi162.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-20T10:17:16.903 INFO:teuthology.orchestra.run.smithi162.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 1.2 MB/s | 203 kB 00:00 2024-04-20T10:17:16.970 INFO:teuthology.orchestra.run.smithi162.stdout:(122/150): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-04-20T10:17:17.020 INFO:teuthology.orchestra.run.smithi162.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-20T10:17:17.071 INFO:teuthology.orchestra.run.smithi162.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-04-20T10:17:17.113 INFO:teuthology.orchestra.run.smithi162.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-20T10:17:17.163 INFO:teuthology.orchestra.run.smithi162.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 253 kB/s | 23 kB 00:00 2024-04-20T10:17:17.205 INFO:teuthology.orchestra.run.smithi162.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-20T10:17:17.255 INFO:teuthology.orchestra.run.smithi162.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-20T10:17:17.355 INFO:teuthology.orchestra.run.smithi162.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-04-20T10:17:17.414 INFO:teuthology.orchestra.run.smithi162.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-04-20T10:17:17.456 INFO:teuthology.orchestra.run.smithi162.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 789 kB/s | 79 kB 00:00 2024-04-20T10:17:17.515 INFO:teuthology.orchestra.run.smithi162.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-04-20T10:17:17.565 INFO:teuthology.orchestra.run.smithi162.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-20T10:17:17.615 INFO:teuthology.orchestra.run.smithi162.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-20T10:17:17.665 INFO:teuthology.orchestra.run.smithi162.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 897 kB/s | 90 kB 00:00 2024-04-20T10:17:17.724 INFO:teuthology.orchestra.run.smithi162.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-20T10:17:17.766 INFO:teuthology.orchestra.run.smithi162.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-04-20T10:17:17.833 INFO:teuthology.orchestra.run.smithi162.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-20T10:17:17.883 INFO:teuthology.orchestra.run.smithi162.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 505 kB/s | 59 kB 00:00 2024-04-20T10:17:17.950 INFO:teuthology.orchestra.run.smithi162.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 304 kB/s | 36 kB 00:00 2024-04-20T10:17:17.984 INFO:teuthology.orchestra.run.smithi162.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-20T10:17:18.084 INFO:teuthology.orchestra.run.smithi162.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-20T10:17:18.110 INFO:teuthology.orchestra.run.smithi162.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 716 kB/s | 90 kB 00:00 2024-04-20T10:17:18.202 INFO:teuthology.orchestra.run.smithi162.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-20T10:17:18.227 INFO:teuthology.orchestra.run.smithi162.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-04-20T10:17:18.319 INFO:teuthology.orchestra.run.smithi162.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-20T10:17:18.402 INFO:teuthology.orchestra.run.smithi162.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-04-20T10:17:18.529 INFO:teuthology.orchestra.run.smithi046.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 152 kB/s | 273 kB 00:01 2024-04-20T10:17:21.704 INFO:teuthology.orchestra.run.smithi046.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 240 kB/s | 763 kB 00:03 2024-04-20T10:17:21.921 INFO:teuthology.orchestra.run.smithi046.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 84 kB/s | 18 kB 00:00 2024-04-20T10:17:22.338 INFO:teuthology.orchestra.run.smithi046.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 193 kB/s | 80 kB 00:00 2024-04-20T10:17:23.222 INFO:teuthology.orchestra.run.smithi046.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 909 kB/s | 803 kB 00:00 2024-04-20T10:17:23.897 INFO:teuthology.orchestra.run.smithi046.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 289 kB/s | 195 kB 00:00 2024-04-20T10:17:24.630 INFO:teuthology.orchestra.run.smithi046.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 345 kB/s | 253 kB 00:00 2024-04-20T10:17:26.247 INFO:teuthology.orchestra.run.smithi046.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 777 kB/s | 1.2 MB 00:01 2024-04-20T10:17:26.664 INFO:teuthology.orchestra.run.smithi046.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 256 kB/s | 106 kB 00:00 2024-04-20T10:17:26.955 INFO:teuthology.orchestra.run.smithi042.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 268 kB/s | 3.1 MB 00:12 2024-04-20T10:17:27.198 INFO:teuthology.orchestra.run.smithi046.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 253 kB/s | 135 kB 00:00 2024-04-20T10:17:27.623 INFO:teuthology.orchestra.run.smithi046.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 294 kB/s | 125 kB 00:00 2024-04-20T10:17:28.207 INFO:teuthology.orchestra.run.smithi046.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 368 kB/s | 215 kB 00:00 2024-04-20T10:17:28.307 INFO:teuthology.orchestra.run.smithi046.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-04-20T10:17:28.407 INFO:teuthology.orchestra.run.smithi046.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 9.2 MB/s | 943 kB 00:00 2024-04-20T10:17:28.474 INFO:teuthology.orchestra.run.smithi046.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-20T10:17:28.516 INFO:teuthology.orchestra.run.smithi046.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 518 kB/s | 21 kB 00:00 2024-04-20T10:17:28.591 INFO:teuthology.orchestra.run.smithi046.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-20T10:17:28.650 INFO:teuthology.orchestra.run.smithi046.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 563 kB/s | 33 kB 00:00 2024-04-20T10:17:28.808 INFO:teuthology.orchestra.run.smithi046.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-20T10:17:28.850 INFO:teuthology.orchestra.run.smithi046.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-04-20T10:17:28.884 INFO:teuthology.orchestra.run.smithi046.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 334 kB/s | 11 kB 00:00 2024-04-20T10:17:28.951 INFO:teuthology.orchestra.run.smithi046.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-20T10:17:29.001 INFO:teuthology.orchestra.run.smithi046.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-04-20T10:17:29.093 INFO:teuthology.orchestra.run.smithi046.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-20T10:17:29.185 INFO:teuthology.orchestra.run.smithi046.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-04-20T10:17:29.236 INFO:teuthology.orchestra.run.smithi046.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 508 kB/s | 25 kB 00:00 2024-04-20T10:17:29.294 INFO:teuthology.orchestra.run.smithi046.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 815 kB/s | 47 kB 00:00 2024-04-20T10:17:29.370 INFO:teuthology.orchestra.run.smithi046.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-20T10:17:29.445 INFO:teuthology.orchestra.run.smithi046.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-20T10:17:29.495 INFO:teuthology.orchestra.run.smithi046.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-04-20T10:17:29.762 INFO:teuthology.orchestra.run.smithi046.stdout:(65/150): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-20T10:17:29.925 INFO:teuthology.orchestra.run.smithi162.stdout:(148/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 279 kB/s | 3.1 MB 00:11 2024-04-20T10:17:29.945 INFO:teuthology.orchestra.run.smithi046.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 23 MB/s | 4.1 MB 00:00 2024-04-20T10:17:30.063 INFO:teuthology.orchestra.run.smithi046.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-20T10:17:30.163 INFO:teuthology.orchestra.run.smithi046.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-20T10:17:30.238 INFO:teuthology.orchestra.run.smithi046.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-04-20T10:17:30.480 INFO:teuthology.orchestra.run.smithi046.stdout:(70/150): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-04-20T10:17:30.580 INFO:teuthology.orchestra.run.smithi046.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-20T10:17:30.639 INFO:teuthology.orchestra.run.smithi046.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-20T10:17:30.881 INFO:teuthology.orchestra.run.smithi046.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 6.0 MB 00:00 2024-04-20T10:17:30.956 INFO:teuthology.orchestra.run.smithi046.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-20T10:17:31.031 INFO:teuthology.orchestra.run.smithi046.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-20T10:17:31.081 INFO:teuthology.orchestra.run.smithi046.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-04-20T10:17:31.132 INFO:teuthology.orchestra.run.smithi046.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-20T10:17:31.199 INFO:teuthology.orchestra.run.smithi046.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-20T10:17:31.257 INFO:teuthology.orchestra.run.smithi046.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 596 kB/s | 35 kB 00:00 2024-04-20T10:17:31.466 INFO:teuthology.orchestra.run.smithi046.stdout:(80/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-04-20T10:17:31.549 INFO:teuthology.orchestra.run.smithi046.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-04-20T10:17:31.625 INFO:teuthology.orchestra.run.smithi046.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-20T10:17:31.684 INFO:teuthology.orchestra.run.smithi046.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-20T10:17:31.734 INFO:teuthology.orchestra.run.smithi046.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-20T10:17:31.809 INFO:teuthology.orchestra.run.smithi046.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-20T10:17:31.876 INFO:teuthology.orchestra.run.smithi046.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-20T10:17:31.951 INFO:teuthology.orchestra.run.smithi046.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-20T10:17:32.010 INFO:teuthology.orchestra.run.smithi046.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-04-20T10:17:32.060 INFO:teuthology.orchestra.run.smithi046.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-20T10:17:32.794 INFO:teuthology.orchestra.run.smithi046.stdout:(90/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-20T10:17:32.853 INFO:teuthology.orchestra.run.smithi046.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 716 kB/s | 42 kB 00:00 2024-04-20T10:17:32.903 INFO:teuthology.orchestra.run.smithi046.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-04-20T10:17:32.995 INFO:teuthology.orchestra.run.smithi046.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-04-20T10:17:33.037 INFO:teuthology.orchestra.run.smithi046.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 867 kB/s | 36 kB 00:00 2024-04-20T10:17:33.129 INFO:teuthology.orchestra.run.smithi046.stdout:(95/150): ceph-mgr-diskprediction-local-19.0.0- 234 kB/s | 7.4 MB 00:32 2024-04-20T10:17:33.158 INFO:teuthology.orchestra.run.smithi162.stdout:(149/150): librados2-19.0.0-3133.ga8c84f59.el9. 234 kB/s | 3.4 MB 00:14 2024-04-20T10:17:33.179 INFO:teuthology.orchestra.run.smithi046.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 334 kB/s | 47 kB 00:00 2024-04-20T10:17:33.230 INFO:teuthology.orchestra.run.smithi046.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-20T10:17:33.262 INFO:teuthology.orchestra.run.smithi046.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 768 kB/s | 58 kB 00:00 2024-04-20T10:17:33.313 INFO:teuthology.orchestra.run.smithi046.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-20T10:17:33.338 INFO:teuthology.orchestra.run.smithi046.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-04-20T10:17:33.389 INFO:teuthology.orchestra.run.smithi046.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-04-20T10:17:33.422 INFO:teuthology.orchestra.run.smithi046.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-04-20T10:17:33.681 INFO:teuthology.orchestra.run.smithi046.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-04-20T10:17:33.773 INFO:teuthology.orchestra.run.smithi046.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-04-20T10:17:33.831 INFO:teuthology.orchestra.run.smithi046.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-04-20T10:17:34.023 INFO:teuthology.orchestra.run.smithi046.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 903 kB/s | 309 kB 00:00 2024-04-20T10:17:34.090 INFO:teuthology.orchestra.run.smithi046.stdout:(107/150): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-20T10:17:34.116 INFO:teuthology.orchestra.run.smithi046.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-20T10:17:34.149 INFO:teuthology.orchestra.run.smithi046.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 830 kB/s | 49 kB 00:00 2024-04-20T10:17:34.183 INFO:teuthology.orchestra.run.smithi046.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-20T10:17:34.208 INFO:teuthology.orchestra.run.smithi046.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-20T10:17:34.275 INFO:teuthology.orchestra.run.smithi046.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-04-20T10:17:34.309 INFO:teuthology.orchestra.run.smithi046.stdout:(113/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 838 kB 00:00 2024-04-20T10:17:34.334 INFO:teuthology.orchestra.run.smithi046.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-04-20T10:17:34.367 INFO:teuthology.orchestra.run.smithi046.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 552 kB/s | 32 kB 00:00 2024-04-20T10:17:34.393 INFO:teuthology.orchestra.run.smithi046.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 254 kB/s | 15 kB 00:00 2024-04-20T10:17:34.535 INFO:teuthology.orchestra.run.smithi046.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-04-20T10:17:34.643 INFO:teuthology.orchestra.run.smithi046.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-04-20T10:17:34.669 INFO:teuthology.orchestra.run.smithi046.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-20T10:17:34.719 INFO:teuthology.orchestra.run.smithi046.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 469 kB/s | 35 kB 00:00 2024-04-20T10:17:34.744 INFO:teuthology.orchestra.run.smithi046.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-04-20T10:17:34.811 INFO:teuthology.orchestra.run.smithi046.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-20T10:17:34.886 INFO:teuthology.orchestra.run.smithi046.stdout:(123/150): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-20T10:17:34.912 INFO:teuthology.orchestra.run.smithi046.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-20T10:17:34.945 INFO:teuthology.orchestra.run.smithi046.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-04-20T10:17:34.970 INFO:teuthology.orchestra.run.smithi046.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-04-20T10:17:35.004 INFO:teuthology.orchestra.run.smithi046.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-04-20T10:17:35.029 INFO:teuthology.orchestra.run.smithi046.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-04-20T10:17:35.088 INFO:teuthology.orchestra.run.smithi046.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-04-20T10:17:35.121 INFO:teuthology.orchestra.run.smithi046.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-04-20T10:17:35.147 INFO:teuthology.orchestra.run.smithi046.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-20T10:17:35.180 INFO:teuthology.orchestra.run.smithi046.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-04-20T10:17:35.206 INFO:teuthology.orchestra.run.smithi046.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-20T10:17:35.239 INFO:teuthology.orchestra.run.smithi046.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-20T10:17:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-20T10:17:35.298 INFO:teuthology.orchestra.run.smithi046.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-04-20T10:17:35.323 INFO:teuthology.orchestra.run.smithi046.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-04-20T10:17:35.357 INFO:teuthology.orchestra.run.smithi046.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-20T10:17:35.382 INFO:teuthology.orchestra.run.smithi046.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-20T10:17:35.424 INFO:teuthology.orchestra.run.smithi046.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 534 kB/s | 36 kB 00:00 2024-04-20T10:17:35.449 INFO:teuthology.orchestra.run.smithi046.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 764 kB/s | 51 kB 00:00 2024-04-20T10:17:35.483 INFO:teuthology.orchestra.run.smithi046.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-20T10:17:35.508 INFO:teuthology.orchestra.run.smithi046.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-20T10:17:35.559 INFO:teuthology.orchestra.run.smithi046.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-20T10:17:35.584 INFO:teuthology.orchestra.run.smithi046.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-04-20T10:17:35.617 INFO:teuthology.orchestra.run.smithi046.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-20T10:17:35.676 INFO:teuthology.orchestra.run.smithi046.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-20T10:17:47.209 INFO:teuthology.orchestra.run.smithi046.stdout:(148/150): librados2-19.0.0-3133.ga8c84f59.el9. 300 kB/s | 3.4 MB 00:11 2024-04-20T10:17:50.175 INFO:teuthology.orchestra.run.smithi046.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 222 kB/s | 3.1 MB 00:14 2024-04-20T10:18:24.698 INFO:teuthology.orchestra.run.smithi046.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 363 kB/s | 49 MB 02:18 2024-04-20T10:18:24.702 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:18:24.702 INFO:teuthology.orchestra.run.smithi046.stdout:Total 1.2 MB/s | 218 MB 03:00 2024-04-20T10:18:25.723 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-04-20T10:18:25.861 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-04-20T10:18:25.861 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-04-20T10:18:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-04-20T10:18:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-04-20T10:18:33.007 INFO:teuthology.orchestra.run.smithi042.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 366 kB/s | 49 MB 02:17 2024-04-20T10:18:33.012 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:18:33.012 INFO:teuthology.orchestra.run.smithi042.stdout:Total 1.1 MB/s | 218 MB 03:19 2024-04-20T10:18:33.096 INFO:teuthology.orchestra.run.smithi162.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 377 kB/s | 49 MB 02:13 2024-04-20T10:18:33.100 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:18:33.100 INFO:teuthology.orchestra.run.smithi162.stdout:Total 1.1 MB/s | 218 MB 03:12 2024-04-20T10:18:33.598 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-04-20T10:18:33.659 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T10:18:33.683 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T10:18:33.718 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T10:18:34.064 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T10:18:34.084 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-04-20T10:18:34.167 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:18:34.210 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-04-20T10:18:34.210 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-04-20T10:18:34.292 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:18:34.318 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:18:34.411 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:18:34.452 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:18:34.481 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T10:18:34.507 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T10:18:34.669 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T10:18:34.729 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T10:18:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:18:34.862 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:18:34.890 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:18:34.956 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:18:34.975 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:18:34.996 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:18:35.271 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:18:35.308 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:18:35.418 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T10:18:35.633 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T10:18:35.674 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T10:18:35.710 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T10:18:35.787 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T10:18:35.833 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T10:18:35.849 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-20T10:18:35.878 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T10:18:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-04-20T10:18:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-04-20T10:18:36.074 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-20T10:18:36.074 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-20T10:18:36.076 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T10:18:36.128 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T10:18:36.201 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T10:18:36.253 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T10:18:36.281 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T10:18:36.325 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T10:18:36.405 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T10:18:36.444 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T10:18:36.490 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T10:18:36.518 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T10:18:36.584 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T10:18:36.633 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T10:18:36.712 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T10:18:36.858 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T10:18:36.912 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T10:18:37.476 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T10:18:37.560 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T10:18:37.592 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T10:18:37.636 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T10:18:37.674 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T10:18:37.707 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T10:18:37.764 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T10:18:37.837 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T10:18:37.874 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T10:18:37.911 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T10:18:37.976 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T10:18:38.010 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T10:18:38.054 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T10:18:38.096 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T10:18:38.242 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T10:18:38.284 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T10:18:38.314 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T10:18:38.356 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T10:18:38.455 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T10:18:38.497 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T10:18:38.537 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T10:18:38.571 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T10:18:38.595 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T10:18:38.674 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T10:18:38.805 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T10:18:38.836 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-20T10:18:38.836 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-20T10:18:38.853 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T10:18:38.881 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T10:18:38.916 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T10:18:38.961 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T10:18:38.998 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T10:18:39.037 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T10:18:39.067 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T10:18:39.101 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T10:18:39.134 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T10:18:39.177 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T10:18:39.210 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T10:18:39.301 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T10:18:39.342 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T10:18:39.372 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T10:18:39.421 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T10:18:39.532 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T10:18:39.584 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T10:18:40.368 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T10:18:40.420 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T10:18:40.454 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T10:18:40.559 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T10:18:40.613 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T10:18:40.706 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T10:18:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T10:18:40.924 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T10:18:40.953 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T10:18:41.033 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T10:18:41.740 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T10:18:41.742 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-04-20T10:18:41.823 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T10:18:41.848 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T10:18:41.856 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T10:18:41.891 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T10:18:42.219 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T10:18:42.341 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:18:42.462 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:18:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:18:42.583 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:18:42.624 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:18:42.654 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T10:18:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T10:18:42.831 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T10:18:42.885 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T10:18:42.913 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:18:43.000 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:18:43.022 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:18:43.068 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:18:43.090 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:18:43.114 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:18:43.187 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T10:18:43.273 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T10:18:43.301 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T10:18:43.334 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T10:18:43.415 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:18:43.456 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:18:43.580 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T10:18:43.662 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T10:18:43.677 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T10:18:43.705 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T10:18:43.705 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:18:43.740 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T10:18:43.780 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:18:43.804 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T10:18:43.810 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T10:18:43.846 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T10:18:43.849 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T10:18:43.892 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T10:18:43.933 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T10:18:43.962 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T10:18:44.030 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T10:18:44.084 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T10:18:44.114 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T10:18:44.157 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T10:18:44.234 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T10:18:44.268 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T10:18:44.312 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T10:18:44.341 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T10:18:44.351 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T10:18:44.391 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:18:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T10:18:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T10:18:44.463 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:18:44.493 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T10:18:44.530 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T10:18:44.664 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T10:18:44.725 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T10:18:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-20T10:18:45.223 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T10:18:45.253 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T10:18:45.253 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T10:18:45.328 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T10:18:45.365 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T10:18:45.375 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T10:18:45.408 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T10:18:45.446 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T10:18:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T10:18:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T10:18:45.608 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T10:18:45.647 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T10:18:45.683 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T10:18:45.731 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T10:18:45.766 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T10:18:45.809 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T10:18:45.828 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T10:18:45.851 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T10:18:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:18:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T10:18:46.040 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T10:18:46.070 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T10:18:46.111 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T10:18:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:18:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:18:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T10:18:46.236 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T10:18:46.277 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T10:18:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:18:46.302 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T10:18:46.327 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T10:18:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:18:46.394 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T10:18:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T10:18:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T10:18:46.522 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T10:18:46.559 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T10:18:46.587 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T10:18:46.621 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T10:18:46.665 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T10:18:46.704 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T10:18:46.743 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T10:18:46.773 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T10:18:46.807 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T10:18:46.841 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T10:18:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T10:18:46.874 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T10:18:46.908 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T10:18:46.914 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:18:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T10:18:46.947 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:18:46.976 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:18:46.996 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T10:18:47.047 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T10:18:47.078 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T10:18:47.125 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T10:18:47.130 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:18:47.160 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:18:47.225 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T10:18:47.248 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:18:47.272 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T10:18:47.277 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:18:47.350 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:18:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:18:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:18:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:18:47.643 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T10:18:47.643 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T10:18:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:18:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:18:47.811 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:18:47.862 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:18:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T10:18:48.062 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T10:18:48.068 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T10:18:48.102 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T10:18:48.187 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T10:18:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T10:18:48.243 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T10:18:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T10:18:48.304 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T10:18:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T10:18:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T10:18:48.413 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T10:18:48.459 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T10:18:48.523 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T10:18:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T10:18:48.581 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T10:18:48.601 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T10:18:48.617 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T10:18:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T10:18:48.769 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T10:18:48.837 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T10:18:48.871 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T10:18:48.928 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T10:18:49.038 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T10:18:49.083 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T10:18:49.138 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T10:18:49.173 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T10:18:49.297 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T10:18:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T10:18:49.350 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T10:18:49.457 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T10:18:49.459 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T10:18:49.652 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T10:18:49.724 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T10:18:50.477 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T10:18:50.591 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T10:18:50.639 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T10:18:50.686 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T10:18:50.718 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T10:18:50.744 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T10:18:50.796 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T10:18:50.821 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T10:18:50.849 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T10:18:50.882 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T10:18:50.896 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T10:18:51.006 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T10:18:51.048 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T10:18:51.088 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T10:18:51.159 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T10:18:51.196 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T10:18:51.204 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T10:18:51.236 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:18:51.268 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T10:18:51.311 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:18:51.325 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T10:18:51.341 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T10:18:51.377 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T10:18:51.521 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T10:18:51.585 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T10:18:51.622 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T10:18:51.670 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T10:18:51.804 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T10:18:51.856 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T10:18:51.865 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T10:18:51.889 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:18:51.916 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T10:18:51.956 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T10:18:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:18:51.982 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T10:18:51.989 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T10:18:52.188 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T10:18:52.377 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T10:18:52.428 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T10:18:52.475 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T10:18:52.545 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T10:18:52.644 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T10:18:52.708 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T10:18:52.770 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T10:18:52.817 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T10:18:52.872 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T10:18:52.923 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T10:18:53.000 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T10:18:53.061 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T10:18:53.178 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T10:18:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T10:18:53.283 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T10:18:53.342 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T10:18:53.498 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T10:18:53.575 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T10:18:54.232 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:18:54.270 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:18:54.601 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T10:18:54.656 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T10:18:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T10:18:54.808 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:18:54.825 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T10:18:54.857 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:18:54.894 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T10:18:54.946 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:18:54.946 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T10:18:54.947 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T10:18:54.947 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:18:54.975 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:18:55.013 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T10:18:55.057 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T10:18:55.303 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T10:18:55.337 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T10:18:55.479 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T10:18:56.387 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T10:18:56.544 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T10:18:58.309 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T10:18:58.412 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T10:18:58.450 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T10:18:58.488 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T10:18:58.884 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T10:18:58.932 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:18:59.059 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:18:59.090 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T10:18:59.150 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T10:18:59.403 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:18:59.403 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-04-20T10:18:59.403 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-04-20T10:18:59.403 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-04-20T10:18:59.404 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-04-20T10:18:59.404 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-04-20T10:18:59.404 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /root 2024-04-20T10:18:59.404 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-04-20T10:18:59.404 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:18:59.597 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:18:59.612 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T10:18:59.613 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:18:59.725 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T10:18:59.760 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:18:59.844 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:18:59.870 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T10:19:01.140 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:19:01.166 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:19:01.167 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:01.167 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T10:19:01.167 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:19:01.167 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:19:01.167 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:19:01.407 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:19:01.430 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:19:01.430 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:01.430 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T10:19:01.430 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:19:01.431 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:19:01.431 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:19:01.899 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:19:01.922 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:19:01.922 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:01.923 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T10:19:01.923 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:19:01.923 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:19:01.923 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:19:02.038 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T10:19:02.097 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T10:19:02.160 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T10:19:02.356 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T10:19:02.409 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T10:19:02.489 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T10:19:02.578 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T10:19:02.684 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:19:02.730 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T10:19:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:19:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T10:19:02.807 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T10:19:02.834 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T10:19:03.290 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:19:03.316 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:19:03.349 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T10:19:03.404 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:19:03.414 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T10:19:03.439 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:19:03.454 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:19:03.610 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:19:03.610 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T10:19:03.610 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T10:19:03.611 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:03.655 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:19:04.534 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:19:04.566 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:19:05.582 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:19:05.608 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:19:05.716 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:19:05.887 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T10:19:05.918 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:19:05.942 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:19:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T10:19:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:19:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:19:05.942 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:19:05.994 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:19:06.005 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:19:06.124 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T10:19:06.176 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T10:19:06.312 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:19:06.312 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /sys 2024-04-20T10:19:06.312 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /proc 2024-04-20T10:19:06.312 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /mnt 2024-04-20T10:19:06.312 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /var/tmp 2024-04-20T10:19:06.312 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /home 2024-04-20T10:19:06.312 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /root 2024-04-20T10:19:06.312 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /tmp 2024-04-20T10:19:06.313 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:19:06.484 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:19:06.499 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:19:06.500 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:06.500 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T10:19:06.500 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:19:07.150 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T10:19:07.188 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:19:07.211 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:19:07.211 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:07.211 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T10:19:07.211 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:19:07.211 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:19:07.212 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:19:07.255 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:19:07.278 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:19:07.278 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:07.278 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T10:19:07.278 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:19:07.323 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:19:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:19:07.346 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:07.346 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T10:19:07.346 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:19:07.346 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:19:07.346 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:19:07.673 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T10:19:07.696 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:19:07.697 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:19:07.987 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:19:08.012 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:19:08.012 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:08.012 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T10:19:08.012 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:19:08.012 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:19:08.012 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:19:08.238 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:19:08.260 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:19:08.260 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:08.260 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T10:19:08.260 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:19:08.260 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:19:08.260 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:19:08.697 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:19:08.718 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:19:08.719 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:08.719 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T10:19:08.719 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:19:08.719 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:19:08.719 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:19:08.827 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T10:19:08.875 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T10:19:08.931 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T10:19:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T10:19:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T10:19:09.242 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T10:19:09.339 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T10:19:09.477 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T10:19:09.514 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T10:19:09.546 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T10:19:09.582 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T10:19:10.042 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:19:10.072 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:19:10.104 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T10:19:10.165 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T10:19:10.194 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:19:11.226 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:19:11.263 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:19:12.225 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:19:12.255 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:19:12.358 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:19:12.518 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T10:19:12.552 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T10:19:12.557 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:19:12.579 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:19:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T10:19:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:19:12.579 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:19:12.579 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:19:12.594 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T10:19:12.624 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:19:12.635 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T10:19:12.635 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:19:12.666 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T10:19:12.694 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T10:19:12.733 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T10:19:12.781 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T10:19:12.887 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T10:19:12.940 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T10:19:12.969 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T10:19:13.502 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:13.542 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:13.562 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:13.704 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T10:19:13.757 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:19:13.780 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:19:13.780 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:13.780 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T10:19:13.780 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:19:13.780 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:19:13.780 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:19:13.827 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:19:13.849 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:19:13.849 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:13.849 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T10:19:13.849 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:19:13.894 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:19:13.916 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:19:13.916 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:13.916 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T10:19:13.916 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:19:13.916 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:19:13.916 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:19:14.220 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:19:14.242 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:19:14.242 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:14.242 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T10:19:14.242 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:19:14.243 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:19:14.243 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:19:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T10:19:14.496 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T10:19:14.537 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T10:19:14.622 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T10:19:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T10:19:14.658 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:19:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:19:14.742 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:19:17.129 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2024-04-20T10:19:17.129 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2024-04-20T10:19:17.129 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2024-04-20T10:19:17.129 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2024-04-20T10:19:17.129 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2024-04-20T10:19:17.129 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /root 2024-04-20T10:19:17.129 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2024-04-20T10:19:17.129 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:17.371 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:19:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:19:17.421 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:17.421 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T10:19:17.421 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:18.815 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T10:19:18.861 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T10:19:18.897 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T10:19:18.920 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T10:19:18.948 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T10:19:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:19:19.129 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T10:19:19.169 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T10:19:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T10:19:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:19:19.212 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:19.212 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T10:19:19.212 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:19:19.212 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:19:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:19:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:19:19.561 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:19.561 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T10:19:19.561 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:19:19.561 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:19:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:19.778 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:19.821 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:19.840 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:20.066 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:19:20.142 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:19:20.143 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:20.143 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T10:19:20.143 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:19:20.143 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:19:20.143 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:20.277 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T10:19:20.338 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T10:19:20.423 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T10:19:20.623 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T10:19:20.675 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T10:19:20.708 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T10:19:20.731 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T10:19:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T10:19:20.764 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:20.764 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T10:19:20.764 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T10:19:20.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T10:19:20.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T10:19:20.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T10:19:20.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:19:20.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:19:20.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T10:19:20.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T10:19:20.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T10:19:20.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T10:19:20.765 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:19:20.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T10:19:20.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T10:19:20.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T10:19:20.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T10:19:20.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T10:19:20.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T10:19:20.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T10:19:20.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T10:19:20.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T10:19:20.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T10:19:20.766 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T10:19:20.767 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T10:19:20.767 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T10:19:20.767 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T10:19:20.767 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T10:19:20.780 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T10:19:20.780 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T10:19:20.780 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T10:19:20.780 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T10:19:20.780 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T10:19:20.780 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T10:19:20.780 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T10:19:20.780 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T10:19:20.780 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T10:19:20.780 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T10:19:20.780 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T10:19:20.780 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T10:19:20.781 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T10:19:20.781 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T10:19:20.781 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T10:19:20.781 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T10:19:20.781 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T10:19:20.781 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T10:19:20.781 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T10:19:20.781 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T10:19:20.781 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T10:19:20.781 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T10:19:20.781 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T10:19:20.781 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T10:19:20.782 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T10:19:20.782 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T10:19:20.782 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T10:19:20.782 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T10:19:20.782 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T10:19:20.782 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T10:19:20.782 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T10:19:20.782 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T10:19:20.782 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T10:19:20.782 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T10:19:20.783 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T10:19:20.784 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T10:19:20.785 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T10:19:20.786 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T10:19:20.787 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T10:19:20.788 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T10:19:20.789 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T10:19:20.789 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T10:19:20.789 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T10:19:20.789 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T10:19:20.789 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T10:19:20.789 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T10:19:20.822 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T10:19:20.830 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T10:19:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T10:19:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:19:20.940 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T10:19:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:19:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:21.122 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T10:19:21.172 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T10:19:21.212 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T10:19:21.244 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T10:19:21.862 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:19:21.898 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:19:21.955 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T10:19:22.069 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T10:19:22.100 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:19:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:19:22.878 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-04-20T10:19:22.878 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.879 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.880 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.881 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T10:19:22.882 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T10:19:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T10:19:22.884 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T10:19:22.885 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T10:19:22.886 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T10:19:22.887 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T10:19:22.888 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T10:19:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T10:19:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T10:19:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T10:19:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T10:19:22.889 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-20T10:19:22.889 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-04-20T10:19:23.270 DEBUG:teuthology.parallel:result is None 2024-04-20T10:19:23.486 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:19:23.535 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:19:24.710 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:19:24.744 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:19:24.899 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:19:25.108 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T10:19:25.137 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:19:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:19:25.221 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:25.221 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T10:19:25.221 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:19:25.221 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:19:25.221 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:25.288 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:19:25.328 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:19:25.445 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T10:19:25.517 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T10:19:26.593 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T10:19:26.625 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:19:26.695 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:19:26.695 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:26.695 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T10:19:26.696 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:19:26.696 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:19:26.696 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:26.750 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:19:26.825 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:19:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T10:19:26.825 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:26.901 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:19:26.946 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:26.947 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T10:19:26.947 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T10:19:26.947 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T10:19:26.947 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T10:19:26.947 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T10:19:26.947 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:19:26.947 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:19:26.947 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T10:19:26.947 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T10:19:26.947 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T10:19:26.947 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T10:19:26.947 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:19:26.948 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T10:19:26.948 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T10:19:26.948 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T10:19:26.948 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T10:19:26.948 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T10:19:26.948 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T10:19:26.948 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T10:19:26.948 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T10:19:26.948 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T10:19:26.948 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T10:19:26.948 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T10:19:26.948 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T10:19:26.949 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T10:19:26.949 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T10:19:26.949 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T10:19:26.949 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T10:19:26.949 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T10:19:26.949 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T10:19:26.962 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T10:19:26.962 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T10:19:26.962 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T10:19:26.962 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T10:19:26.963 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T10:19:26.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T10:19:26.965 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T10:19:26.966 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T10:19:26.966 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T10:19:26.966 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T10:19:26.966 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T10:19:26.966 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T10:19:26.966 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T10:19:26.966 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T10:19:26.966 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T10:19:26.966 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T10:19:26.966 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T10:19:26.966 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T10:19:26.967 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T10:19:26.967 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T10:19:26.967 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T10:19:26.967 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T10:19:26.967 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T10:19:26.967 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T10:19:26.967 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T10:19:26.967 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T10:19:26.967 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T10:19:26.968 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T10:19:26.968 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T10:19:26.968 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T10:19:26.968 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T10:19:26.968 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T10:19:26.968 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T10:19:26.968 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T10:19:26.968 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T10:19:26.968 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T10:19:26.968 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T10:19:26.969 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T10:19:26.969 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T10:19:26.969 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T10:19:26.969 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T10:19:26.969 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T10:19:26.969 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T10:19:26.969 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T10:19:26.969 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T10:19:26.969 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T10:19:26.969 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T10:19:26.969 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T10:19:26.970 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T10:19:26.971 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T10:19:26.972 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T10:19:26.972 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T10:19:26.972 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T10:19:26.972 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T10:19:26.972 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T10:19:26.972 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T10:19:26.972 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T10:19:26.972 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T10:19:26.981 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:19:26.982 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:26.982 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T10:19:26.982 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:19:26.982 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:19:26.982 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:27.360 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:19:27.435 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:19:27.435 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:27.435 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T10:19:27.435 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:19:27.435 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:19:27.435 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:29.054 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:29.054 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:19:29.054 INFO:teuthology.orchestra.run.smithi162.stdout:Upgraded: 2024-04-20T10:19:29.054 INFO:teuthology.orchestra.run.smithi162.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.054 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.054 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-04-20T10:19:29.054 INFO:teuthology.orchestra.run.smithi162.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T10:19:29.054 INFO:teuthology.orchestra.run.smithi162.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T10:19:29.054 INFO:teuthology.orchestra.run.smithi162.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T10:19:29.054 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T10:19:29.056 INFO:teuthology.orchestra.run.smithi162.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.057 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T10:19:29.058 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T10:19:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T10:19:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T10:19:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T10:19:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T10:19:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T10:19:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T10:19:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T10:19:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T10:19:29.060 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T10:19:29.061 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T10:19:29.062 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.063 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T10:19:29.064 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T10:19:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T10:19:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T10:19:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T10:19:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T10:19:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T10:19:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T10:19:29.065 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:19:29.066 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-04-20T10:19:29.442 DEBUG:teuthology.parallel:result is None 2024-04-20T10:19:32.766 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T10:19:32.812 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T10:19:32.891 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T10:19:32.942 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T10:19:32.986 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T10:19:33.335 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T10:19:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T10:19:33.415 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T10:19:34.381 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:34.431 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:34.490 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:35.880 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T10:19:35.971 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T10:19:36.063 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T10:19:36.279 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T10:19:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T10:19:36.321 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:19:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:19:36.445 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:36.955 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:36.955 INFO:teuthology.orchestra.run.smithi042.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T10:19:36.955 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T10:19:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T10:19:42.819 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T10:19:42.820 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T10:19:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T10:19:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T10:19:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T10:19:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T10:19:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T10:19:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T10:19:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T10:19:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T10:19:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T10:19:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T10:19:42.821 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T10:19:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T10:19:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T10:19:42.832 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T10:19:42.833 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T10:19:42.834 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T10:19:42.835 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T10:19:42.836 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T10:19:42.837 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T10:19:42.838 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T10:19:42.839 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T10:19:42.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T10:19:45.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:45.338 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:45.338 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-04-20T10:19:45.338 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.338 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T10:19:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T10:19:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T10:19:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T10:19:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T10:19:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T10:19:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T10:19:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T10:19:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T10:19:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T10:19:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T10:19:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T10:19:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T10:19:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T10:19:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T10:19:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T10:19:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T10:19:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T10:19:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T10:19:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T10:19:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:19:45.349 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-20T10:19:45.863 DEBUG:teuthology.parallel:result is None 2024-04-20T10:19:45.864 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:19:46.016 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T10:19:46.068 INFO:teuthology.orchestra.run.smithi042.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T10:19:46.068 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T10:19:46.069 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T10:19:46.071 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:19:46.213 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T10:19:46.239 INFO:teuthology.orchestra.run.smithi046.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T10:19:46.239 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T10:19:46.239 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T10:19:46.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:19:46.460 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T10:19:46.485 INFO:teuthology.orchestra.run.smithi162.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T10:19:46.485 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T10:19:46.485 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T10:19:46.487 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T10:19:46.487 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:19:46.487 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T10:19:46.559 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-20T10:19:46.559 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T10:19:46.591 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-20T10:19:46.591 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T10:19:46.621 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T10:19:46.621 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:19:46.621 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T10:19:46.694 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T10:19:46.798 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-20T10:19:46.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T10:19:46.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T10:19:46.898 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-20T10:19:46.898 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T10:19:46.926 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T10:19:46.992 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T10:19:46.992 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:19:46.992 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T10:19:47.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T10:19:47.173 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-20T10:19:47.173 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T10:19:47.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T10:19:47.262 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-20T10:19:47.262 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T10:19:47.287 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T10:19:47.351 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T10:19:47.351 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:19:47.351 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T10:19:47.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T10:19:47.531 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-20T10:19:47.531 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T10:19:47.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T10:19:47.621 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-20T10:19:47.621 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T10:19:47.645 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T10:19:47.709 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-20T10:19:47.825 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'subvols': {'create': 2}} 2024-04-20T10:19:47.825 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:19:47.826 INFO:tasks.cephadm:Cluster fsid is 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:19:47.826 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-20T10:19:47.826 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.42', 'mon.b': '172.21.15.46', 'mon.c': '172.21.15.162'} 2024-04-20T10:19:47.826 INFO:tasks.cephadm:First mon is mon.a on smithi042 2024-04-20T10:19:47.826 INFO:tasks.cephadm:First mgr is x 2024-04-20T10:19:47.827 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-20T10:19:47.827 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname $(hostname -s) 2024-04-20T10:19:47.892 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname $(hostname -s) 2024-04-20T10:19:47.917 DEBUG:teuthology.orchestra.run.smithi162:> sudo hostname $(hostname -s) 2024-04-20T10:19:47.941 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-20T10:19:47.941 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:19:48.155 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78637/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3133-ga8c84f59', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3133.ga8c84f59'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-16 21:41:56.138585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240416.193735', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-20T10:19:48.299 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-rishabh-testing-20240416.193735, sha1 a8c84f5997950fd7d008204839253291d52e6eda from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:19:48.301 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm 2024-04-20T10:19:48.301 DEBUG:teuthology.orchestra.run.smithi042:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T10:19:48.868 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 20 10:19 /home/ubuntu/cephtest/cephadm 2024-04-20T10:19:48.868 DEBUG:teuthology.orchestra.run.smithi046:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T10:19:49.423 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 20 10:19 /home/ubuntu/cephtest/cephadm 2024-04-20T10:19:49.424 DEBUG:teuthology.orchestra.run.smithi162:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T10:19:50.239 INFO:teuthology.orchestra.run.smithi162.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 20 10:19 /home/ubuntu/cephtest/cephadm 2024-04-20T10:19:50.240 DEBUG:teuthology.orchestra.run.smithi042:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T10:19:50.297 DEBUG:teuthology.orchestra.run.smithi046:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T10:19:50.317 DEBUG:teuthology.orchestra.run.smithi162:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T10:19:50.352 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda on all hosts... 2024-04-20T10:19:50.353 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-20T10:19:50.358 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-20T10:19:50.361 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-20T10:19:51.932 INFO:teuthology.orchestra.run.smithi162.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T10:19:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T10:19:52.616 INFO:teuthology.orchestra.run.smithi042.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T10:21:08.458 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-20T10:21:08.459 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:21:08.459 INFO:teuthology.orchestra.run.smithi162.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-20T10:21:08.459 INFO:teuthology.orchestra.run.smithi162.stdout: "repo_digests": [ 2024-04-20T10:21:08.459 INFO:teuthology.orchestra.run.smithi162.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-20T10:21:08.459 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-04-20T10:21:08.459 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-20T10:21:25.838 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-20T10:21:25.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:21:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-20T10:21:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "repo_digests": [ 2024-04-20T10:21:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-20T10:21:25.839 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-20T10:21:25.839 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-20T10:21:59.030 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:21:59.031 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:21:59.031 INFO:teuthology.orchestra.run.smithi042.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-20T10:21:59.031 INFO:teuthology.orchestra.run.smithi042.stdout: "repo_digests": [ 2024-04-20T10:21:59.031 INFO:teuthology.orchestra.run.smithi042.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-20T10:21:59.031 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-20T10:21:59.031 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:21:59.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph 2024-04-20T10:21:59.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph 2024-04-20T10:21:59.227 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph 2024-04-20T10:21:59.260 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /etc/ceph 2024-04-20T10:21:59.353 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /etc/ceph 2024-04-20T10:21:59.383 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /etc/ceph 2024-04-20T10:21:59.411 INFO:tasks.cephadm:Writing seed config... 2024-04-20T10:21:59.412 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-20T10:21:59.412 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-20T10:21:59.412 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-20T10:21:59.412 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-20T10:21:59.412 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-20T10:21:59.412 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-20T10:21:59.413 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-20T10:21:59.413 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-20T10:21:59.413 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-20T10:21:59.413 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-20T10:21:59.413 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-20T10:21:59.413 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-20T10:21:59.413 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-20T10:21:59.413 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-20T10:21:59.413 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-20T10:21:59.413 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-20T10:21:59.413 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-20T10:21:59.413 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-20T10:21:59.414 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-20T10:21:59.415 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-20T10:21:59.416 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:21:59.416 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-20T10:21:59.479 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 83e8f766-feff-11ee-bc92-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-20T10:21:59.479 DEBUG:teuthology.orchestra.run.smithi042:mon.a> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.a.service 2024-04-20T10:21:59.538 DEBUG:teuthology.orchestra.run.smithi042:mgr.x> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.x.service 2024-04-20T10:21:59.589 INFO:tasks.cephadm:Bootstrapping... 2024-04-20T10:21:59.589 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda -v bootstrap --fsid 83e8f766-feff-11ee-bc92-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.42 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-20T10:21:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:21:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda', '-v', 'bootstrap', '--fsid', '83e8f766-feff-11ee-bc92-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.42', '--skip-admin-label'] 2024-04-20T10:21:59.883 INFO:teuthology.orchestra.run.smithi042.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-20T10:21:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying podman|docker is present... 2024-04-20T10:21:59.943 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-20T10:21:59.943 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying lvm2 is present... 2024-04-20T10:21:59.943 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying time synchronization is in place... 2024-04-20T10:21:59.960 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-20T10:21:59.960 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-20T10:21:59.975 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-20T10:21:59.975 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-04-20T10:21:59.993 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2024-04-20T10:22:00.010 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2024-04-20T10:22:00.010 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2024-04-20T10:22:00.010 INFO:teuthology.orchestra.run.smithi042.stdout:Repeating the final host check... 2024-04-20T10:22:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-20T10:22:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-20T10:22:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl is present 2024-04-20T10:22:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:lvcreate is present 2024-04-20T10:22:00.086 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-20T10:22:00.087 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-20T10:22:00.108 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-20T10:22:00.109 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-04-20T10:22:00.135 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2024-04-20T10:22:00.157 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2024-04-20T10:22:00.157 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2024-04-20T10:22:00.157 INFO:teuthology.orchestra.run.smithi042.stdout:Host looks OK 2024-04-20T10:22:00.158 INFO:teuthology.orchestra.run.smithi042.stdout:Cluster fsid: 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:22:00.158 INFO:teuthology.orchestra.run.smithi042.stdout:Acquiring lock 140289772313760 on /run/cephadm/83e8f766-feff-11ee-bc92-c7b262605968.lock 2024-04-20T10:22:00.159 INFO:teuthology.orchestra.run.smithi042.stdout:Lock 140289772313760 acquired on /run/cephadm/83e8f766-feff-11ee-bc92-c7b262605968.lock 2024-04-20T10:22:00.160 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 3300 ... 2024-04-20T10:22:00.161 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 6789 ... 2024-04-20T10:22:00.162 INFO:teuthology.orchestra.run.smithi042.stdout:Base mon IP(s) is [172.21.15.42:3300, 172.21.15.42:6789], mon addrv is [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-04-20T10:22:00.177 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.42 metric 100 2024-04-20T10:22:00.177 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.42 metric 100 2024-04-20T10:22:00.191 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-20T10:22:00.191 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-20T10:22:00.191 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-20T10:22:00.202 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-20T10:22:00.203 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-20T10:22:00.203 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-20T10:22:00.203 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-20T10:22:00.203 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-04-20T10:22:00.203 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-20T10:22:00.206 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2024-04-20T10:22:00.206 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2024-04-20T10:22:00.206 INFO:teuthology.orchestra.run.smithi042.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-20T10:22:00.209 INFO:teuthology.orchestra.run.smithi042.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-20T10:22:00.212 INFO:teuthology.orchestra.run.smithi042.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T10:22:01.065 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-20T10:22:01.065 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T10:22:01.066 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Getting image source signatures 2024-04-20T10:22:01.066 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:9473d3fd79377e8e2904ab0f292b04da81cb2f2bfccde15ec927b481a725f6f4 2024-04-20T10:22:01.066 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-20T10:22:01.066 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying config sha256:76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-20T10:22:01.066 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-20T10:22:02.475 INFO:teuthology.orchestra.run.smithi042.stdout:ceph: stdout ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T10:22:02.475 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph version: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T10:22:02.475 INFO:teuthology.orchestra.run.smithi042.stdout:Extracting ceph user uid/gid from container image... 2024-04-20T10:22:12.642 INFO:teuthology.orchestra.run.smithi042.stdout:stat: stdout 167 167 2024-04-20T10:22:12.642 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial keys... 2024-04-20T10:22:13.831 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQBVlyNmQDLxCBAA/vs6RCIGThC3PqxcxMC0kg== 2024-04-20T10:22:14.926 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQBWlyNmxqwoEhAA4/jKIRAQWwQzxILZet56DA== 2024-04-20T10:22:16.062 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQBXlyNmUmyLFxAA13kHwg+RJWyxZxP4WCxGIQ== 2024-04-20T10:22:16.065 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial monmap... 2024-04-20T10:22:17.216 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-20T10:22:17.216 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-20T10:22:17.216 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:22:17.216 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-20T10:22:17.217 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool for a [v2:172.21.15.42:3300,v1:172.21.15.42:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-20T10:22:17.217 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = quincy 2024-04-20T10:22:17.217 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: set fsid to 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:22:17.217 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-20T10:22:17.217 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:22:17.217 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mon... 2024-04-20T10:22:18.862 INFO:teuthology.orchestra.run.smithi042.stdout:create mon.a on 2024-04-20T10:22:19.181 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-20T10:22:19.427 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-20T10:22:19.712 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-83e8f766-feff-11ee-bc92-c7b262605968.target → /etc/systemd/system/ceph-83e8f766-feff-11ee-bc92-c7b262605968.target. 2024-04-20T10:22:19.712 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-83e8f766-feff-11ee-bc92-c7b262605968.target → /etc/systemd/system/ceph-83e8f766-feff-11ee-bc92-c7b262605968.target. 2024-04-20T10:22:20.114 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.a 2024-04-20T10:22:20.114 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.a.service: Unit ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.a.service not loaded. 2024-04-20T10:22:20.408 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-83e8f766-feff-11ee-bc92-c7b262605968.target.wants/ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.a.service → /etc/systemd/system/ceph-83e8f766-feff-11ee-bc92-c7b262605968@.service. 2024-04-20T10:22:21.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:20 smithi042 podman[16692]: 2024-04-20 10:22:20.843896843 +0000 UTC m=+0.136961697 container create b2fd4ee85b4c2872489284de36d3508a9ca7c2ec21674d63379325f2305c8029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-20T10:22:21.254 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T10:22:21.254 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-04-20T10:22:21.274 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T10:22:21.274 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-04-20T10:22:21.274 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-04-20T10:22:21.274 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2024-04-20T10:22:21.276 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon to start... 2024-04-20T10:22:21.276 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon... 2024-04-20T10:22:21.428 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:21 smithi042 podman[16692]: 2024-04-20 10:22:21.131338096 +0000 UTC m=+0.424402951 container init b2fd4ee85b4c2872489284de36d3508a9ca7c2ec21674d63379325f2305c8029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-20T10:22:21.428 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:21 smithi042 podman[16692]: 2024-04-20 10:22:21.142512167 +0000 UTC m=+0.435577018 container start b2fd4ee85b4c2872489284de36d3508a9ca7c2ec21674d63379325f2305c8029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-20T10:22:21.428 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:21 smithi042 bash[16692]: b2fd4ee85b4c2872489284de36d3508a9ca7c2ec21674d63379325f2305c8029 2024-04-20T10:22:21.429 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:21 smithi042 ceph-mon[16710]: mkfs 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:22:21.429 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:21 smithi042 systemd[1]: Started Ceph mon.a for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:22:21.429 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:21 smithi042 ceph-mon[16710]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-20T10:22:22.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:22 smithi042 ceph-mon[16710]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-20T10:22:22.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:22 smithi042 ceph-mon[16710]: monmap epoch 1 2024-04-20T10:22:22.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:22 smithi042 ceph-mon[16710]: fsid 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:22:22.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:22 smithi042 ceph-mon[16710]: last_changed 2024-04-20T10:22:16.558137+0000 2024-04-20T10:22:22.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:22 smithi042 ceph-mon[16710]: created 2024-04-20T10:22:16.558137+0000 2024-04-20T10:22:22.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:22 smithi042 ceph-mon[16710]: min_mon_release 19 (squid) 2024-04-20T10:22:22.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:22 smithi042 ceph-mon[16710]: election_strategy: 1 2024-04-20T10:22:22.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:22 smithi042 ceph-mon[16710]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-20T10:22:22.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:22 smithi042 ceph-mon[16710]: fsmap 2024-04-20T10:22:22.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:22 smithi042 ceph-mon[16710]: osdmap e1: 0 total, 0 up, 0 in 2024-04-20T10:22:22.275 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:22 smithi042 ceph-mon[16710]: mgrmap e1: no daemons active 2024-04-20T10:22:22.275 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:22 smithi042 ceph-mon[16710]: from='client.? 172.21.15.42:0/2791861497' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-20T10:22:22.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout cluster: 2024-04-20T10:22:22.944 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout id: 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:22:22.944 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-20T10:22:22.944 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:22.944 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout services: 2024-04-20T10:22:22.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.76921s) 2024-04-20T10:22:22.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-20T10:22:22.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-20T10:22:22.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:22.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout data: 2024-04-20T10:22:22.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-20T10:22:22.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-20T10:22:22.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-20T10:22:22.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pgs: 2024-04-20T10:22:22.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:22.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.896+0000 7f456b200700 1 Processor -- start 2024-04-20T10:22:22.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.896+0000 7f456b200700 1 -- start start 2024-04-20T10:22:22.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.896+0000 7f456b200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 0x7f456410aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.897+0000 7f456b200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f456410b060 con 0x7f456410a670 2024-04-20T10:22:22.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.897+0000 7f4563e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 0x7f456410aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.897+0000 7f4563e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 0x7f456410aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44922/0 (socket says 172.21.15.42:44922) 2024-04-20T10:22:22.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.897+0000 7f4563e00700 1 -- 172.21.15.42:0/496831487 learned_addr learned my addr 172.21.15.42:0/496831487 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:22.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.898+0000 7f4563e00700 1 -- 172.21.15.42:0/496831487 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456410b870 con 0x7f456410a670 2024-04-20T10:22:22.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.898+0000 7f4563e00700 1 --2- 172.21.15.42:0/496831487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 0x7f456410aa90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f45540098a0 tx=0x7f4554009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6286f3730997e58d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:22.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.899+0000 7f4563400700 1 -- 172.21.15.42:0/496831487 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4554004160 con 0x7f456410a670 2024-04-20T10:22:22.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.899+0000 7f4563400700 1 -- 172.21.15.42:0/496831487 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4554004300 con 0x7f456410a670 2024-04-20T10:22:22.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.899+0000 7f4563400700 1 -- 172.21.15.42:0/496831487 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45540036a0 con 0x7f456410a670 2024-04-20T10:22:22.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.899+0000 7f456b200700 1 -- 172.21.15.42:0/496831487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 msgr2=0x7f456410aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.899+0000 7f456b200700 1 --2- 172.21.15.42:0/496831487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 0x7f456410aa90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f45540098a0 tx=0x7f4554009bb0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.899+0000 7f456b200700 1 -- 172.21.15.42:0/496831487 shutdown_connections 2024-04-20T10:22:22.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.899+0000 7f456b200700 1 --2- 172.21.15.42:0/496831487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 0x7f456410aa90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.900+0000 7f456b200700 1 -- 172.21.15.42:0/496831487 >> 172.21.15.42:0/496831487 conn(0x7f456407db60 msgr2=0x7f45641084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:22.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.900+0000 7f456b200700 1 -- 172.21.15.42:0/496831487 shutdown_connections 2024-04-20T10:22:22.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.900+0000 7f456b200700 1 -- 172.21.15.42:0/496831487 wait complete. 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.901+0000 7f456b200700 1 Processor -- start 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.901+0000 7f456b200700 1 -- start start 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.901+0000 7f456b200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 0x7f456419dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.901+0000 7f456b200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f456410bda0 con 0x7f456410a670 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.902+0000 7f4563e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 0x7f456419dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.902+0000 7f4563e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 0x7f456419dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44934/0 (socket says 172.21.15.42:44934) 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.902+0000 7f4563e00700 1 -- 172.21.15.42:0/2791861497 learned_addr learned my addr 172.21.15.42:0/2791861497 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.902+0000 7f4563e00700 1 -- 172.21.15.42:0/2791861497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456419e2a0 con 0x7f456410a670 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.903+0000 7f4563e00700 1 --2- 172.21.15.42:0/2791861497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 0x7f456419dd60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f456410b300 tx=0x7f4554004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.903+0000 7f4561600700 1 -- 172.21.15.42:0/2791861497 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4554003db0 con 0x7f456410a670 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.903+0000 7f4561600700 1 -- 172.21.15.42:0/2791861497 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4554011630 con 0x7f456410a670 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.903+0000 7f456b200700 1 -- 172.21.15.42:0/2791861497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456419e530 con 0x7f456410a670 2024-04-20T10:22:22.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.903+0000 7f4561600700 1 -- 172.21.15.42:0/2791861497 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4554011930 con 0x7f456410a670 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.903+0000 7f456b200700 1 -- 172.21.15.42:0/2791861497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45641a1220 con 0x7f456410a670 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.904+0000 7f4561600700 1 -- 172.21.15.42:0/2791861497 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4554024480 con 0x7f456410a670 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.904+0000 7f4561600700 1 -- 172.21.15.42:0/2791861497 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f455401a7a0 con 0x7f456410a670 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.905+0000 7f456b200700 1 -- 172.21.15.42:0/2791861497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45640640b0 con 0x7f456410a670 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.908+0000 7f4561600700 1 -- 172.21.15.42:0/2791861497 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f455401aa40 con 0x7f456410a670 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.988+0000 7f456b200700 1 -- 172.21.15.42:0/2791861497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f456402e1e0 con 0x7f456410a670 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.989+0000 7f4561600700 1 -- 172.21.15.42:0/2791861497 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7f455401ac20 con 0x7f456410a670 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.990+0000 7f456b200700 1 -- 172.21.15.42:0/2791861497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 msgr2=0x7f456419dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.990+0000 7f456b200700 1 --2- 172.21.15.42:0/2791861497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 0x7f456419dd60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f456410b300 tx=0x7f4554004750 comp rx=0 tx=0).stop 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.990+0000 7f456b200700 1 -- 172.21.15.42:0/2791861497 shutdown_connections 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.990+0000 7f456b200700 1 --2- 172.21.15.42:0/2791861497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f456410a670 0x7f456419dd60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.990+0000 7f456b200700 1 -- 172.21.15.42:0/2791861497 >> 172.21.15.42:0/2791861497 conn(0x7f456407db60 msgr2=0x7f4564194a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:22.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.991+0000 7f456b200700 1 -- 172.21.15.42:0/2791861497 shutdown_connections 2024-04-20T10:22:22.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:21.991+0000 7f456b200700 1 -- 172.21.15.42:0/2791861497 wait complete. 2024-04-20T10:22:22.956 INFO:teuthology.orchestra.run.smithi042.stdout:mon is available 2024-04-20T10:22:22.956 INFO:teuthology.orchestra.run.smithi042.stdout:Assimilating anything we can from ceph.conf... 2024-04-20T10:22:24.295 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:24 smithi042 ceph-mon[16710]: from='client.? 172.21.15.42:0/2513022124' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-20T10:22:24.295 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:24 smithi042 ceph-mon[16710]: from='client.? 172.21.15.42:0/2513022124' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-20T10:22:24.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:24.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2024-04-20T10:22:24.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:22:24.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-04-20T10:22:24.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-20T10:22:24.596 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-20T10:22:24.596 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-20T10:22:24.597 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-20T10:22:24.597 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:24.597 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-20T10:22:24.597 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-20T10:22:24.597 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:24.597 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2024-04-20T10:22:24.597 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-20T10:22:24.597 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-20T10:22:24.597 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-20T10:22:24.599 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-20T10:22:24.599 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-20T10:22:24.599 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.894+0000 7fd253200700 1 Processor -- start 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.894+0000 7fd253200700 1 -- start start 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.894+0000 7fd253200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 0x7fd24c10a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.895+0000 7fd253200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c10af80 con 0x7fd24c10a590 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.895+0000 7fd251e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 0x7fd24c10a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.895+0000 7fd251e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 0x7fd24c10a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44938/0 (socket says 172.21.15.42:44938) 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.895+0000 7fd251e00700 1 -- 172.21.15.42:0/2823164162 learned_addr learned my addr 172.21.15.42:0/2823164162 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.896+0000 7fd251e00700 1 -- 172.21.15.42:0/2823164162 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c10b740 con 0x7fd24c10a590 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.896+0000 7fd251e00700 1 --2- 172.21.15.42:0/2823164162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 0x7fd24c10a9b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd2400098a0 tx=0x7fd240009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c5b7b9e22dc060 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.896+0000 7fd250a00700 1 -- 172.21.15.42:0/2823164162 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd240004160 con 0x7fd24c10a590 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.896+0000 7fd250a00700 1 -- 172.21.15.42:0/2823164162 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd240004300 con 0x7fd24c10a590 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.896+0000 7fd250a00700 1 -- 172.21.15.42:0/2823164162 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2400036a0 con 0x7fd24c10a590 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.897+0000 7fd253200700 1 -- 172.21.15.42:0/2823164162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 msgr2=0x7fd24c10a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.897+0000 7fd253200700 1 --2- 172.21.15.42:0/2823164162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 0x7fd24c10a9b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd2400098a0 tx=0x7fd240009bb0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.600 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.897+0000 7fd253200700 1 -- 172.21.15.42:0/2823164162 shutdown_connections 2024-04-20T10:22:24.601 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.897+0000 7fd253200700 1 --2- 172.21.15.42:0/2823164162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 0x7fd24c10a9b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.601 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.897+0000 7fd253200700 1 -- 172.21.15.42:0/2823164162 >> 172.21.15.42:0/2823164162 conn(0x7fd24c07dc10 msgr2=0x7fd24c1084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:24.601 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.897+0000 7fd253200700 1 -- 172.21.15.42:0/2823164162 shutdown_connections 2024-04-20T10:22:24.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.897+0000 7fd253200700 1 -- 172.21.15.42:0/2823164162 wait complete. 2024-04-20T10:22:24.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.898+0000 7fd253200700 1 Processor -- start 2024-04-20T10:22:24.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.899+0000 7fd253200700 1 -- start start 2024-04-20T10:22:24.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.899+0000 7fd253200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 0x7fd24c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.899+0000 7fd253200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c10bc70 con 0x7fd24c10a590 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.899+0000 7fd251e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 0x7fd24c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.899+0000 7fd251e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 0x7fd24c19dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44946/0 (socket says 172.21.15.42:44946) 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.899+0000 7fd251e00700 1 -- 172.21.15.42:0/2513022124 learned_addr learned my addr 172.21.15.42:0/2513022124 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.900+0000 7fd251e00700 1 -- 172.21.15.42:0/2513022124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c19e1c0 con 0x7fd24c10a590 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.900+0000 7fd251e00700 1 --2- 172.21.15.42:0/2513022124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 0x7fd24c19dc80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd240000c00 tx=0x7fd240004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.900+0000 7fd246a00700 1 -- 172.21.15.42:0/2513022124 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd240003db0 con 0x7fd24c10a590 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.900+0000 7fd246a00700 1 -- 172.21.15.42:0/2513022124 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd240011630 con 0x7fd24c10a590 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.900+0000 7fd253200700 1 -- 172.21.15.42:0/2513022124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24c19e450 con 0x7fd24c10a590 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.900+0000 7fd246a00700 1 -- 172.21.15.42:0/2513022124 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd240011950 con 0x7fd24c10a590 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.900+0000 7fd253200700 1 -- 172.21.15.42:0/2513022124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24c1a1140 con 0x7fd24c10a590 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.901+0000 7fd246a00700 1 -- 172.21.15.42:0/2513022124 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd24001a460 con 0x7fd24c10a590 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.901+0000 7fd246a00700 1 -- 172.21.15.42:0/2513022124 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd24001b780 con 0x7fd24c10a590 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.902+0000 7fd253200700 1 -- 172.21.15.42:0/2513022124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24c0640b0 con 0x7fd24c10a590 2024-04-20T10:22:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.904+0000 7fd246a00700 1 -- 172.21.15.42:0/2513022124 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fd24001b980 con 0x7fd24c10a590 2024-04-20T10:22:24.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.982+0000 7fd253200700 1 -- 172.21.15.42:0/2513022124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd24c19e930 con 0x7fd24c10a590 2024-04-20T10:22:24.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.993+0000 7fd246a00700 1 -- 172.21.15.42:0/2513022124 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fd24c19e930 con 0x7fd24c10a590 2024-04-20T10:22:24.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.994+0000 7fd246a00700 1 -- 172.21.15.42:0/2513022124 <== mon.0 v2:172.21.15.42:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd24001ada0 con 0x7fd24c10a590 2024-04-20T10:22:24.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.996+0000 7fd253200700 1 -- 172.21.15.42:0/2513022124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 msgr2=0x7fd24c19dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.996+0000 7fd253200700 1 --2- 172.21.15.42:0/2513022124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 0x7fd24c19dc80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd240000c00 tx=0x7fd240004750 comp rx=0 tx=0).stop 2024-04-20T10:22:24.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.996+0000 7fd253200700 1 -- 172.21.15.42:0/2513022124 shutdown_connections 2024-04-20T10:22:24.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.996+0000 7fd253200700 1 --2- 172.21.15.42:0/2513022124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24c10a590 0x7fd24c19dc80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.996+0000 7fd253200700 1 -- 172.21.15.42:0/2513022124 >> 172.21.15.42:0/2513022124 conn(0x7fd24c07dc10 msgr2=0x7fd24c1948f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:24.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.996+0000 7fd253200700 1 -- 172.21.15.42:0/2513022124 shutdown_connections 2024-04-20T10:22:24.606 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:23.997+0000 7fd253200700 1 -- 172.21.15.42:0/2513022124 wait complete. 2024-04-20T10:22:24.606 INFO:teuthology.orchestra.run.smithi042.stdout:Generating new minimal ceph.conf... 2024-04-20T10:22:25.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:25 smithi042 ceph-mon[16710]: from='client.? 172.21.15.42:0/4192011813' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:22:25.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.195+0000 7f5411c00700 1 Processor -- start 2024-04-20T10:22:25.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.196+0000 7f5411c00700 1 -- start start 2024-04-20T10:22:25.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.196+0000 7f5411c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 0x7f540c07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.196+0000 7f5411c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f540c07c820 con 0x7f540c07de30 2024-04-20T10:22:25.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.197+0000 7f540aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 0x7f540c07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.197+0000 7f540aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 0x7f540c07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44954/0 (socket says 172.21.15.42:44954) 2024-04-20T10:22:25.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.197+0000 7f540aa00700 1 -- 172.21.15.42:0/2943741223 learned_addr learned my addr 172.21.15.42:0/2943741223 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:25.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.197+0000 7f540aa00700 1 -- 172.21.15.42:0/2943741223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c07c9a0 con 0x7f540c07de30 2024-04-20T10:22:25.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.198+0000 7f540aa00700 1 --2- 172.21.15.42:0/2943741223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 0x7f540c07c2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009e80 tx=0x7f53fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf96f5c8419f964 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:25.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.198+0000 7f540a000700 1 -- 172.21.15.42:0/2943741223 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53fc017070 con 0x7f540c07de30 2024-04-20T10:22:25.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.198+0000 7f540a000700 1 -- 172.21.15.42:0/2943741223 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f53fc00b8c0 con 0x7f540c07de30 2024-04-20T10:22:25.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.198+0000 7f540a000700 1 -- 172.21.15.42:0/2943741223 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53fc0045c0 con 0x7f540c07de30 2024-04-20T10:22:25.879 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.199+0000 7f5411c00700 1 -- 172.21.15.42:0/2943741223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 msgr2=0x7f540c07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.879 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.199+0000 7f5411c00700 1 --2- 172.21.15.42:0/2943741223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 0x7f540c07c2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009e80 tx=0x7f53fc00b330 comp rx=0 tx=0).stop 2024-04-20T10:22:25.879 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.199+0000 7f5411c00700 1 -- 172.21.15.42:0/2943741223 shutdown_connections 2024-04-20T10:22:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.199+0000 7f5411c00700 1 --2- 172.21.15.42:0/2943741223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 0x7f540c07c2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.199+0000 7f5411c00700 1 -- 172.21.15.42:0/2943741223 >> 172.21.15.42:0/2943741223 conn(0x7f540c103a60 msgr2=0x7f540c105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.199+0000 7f5411c00700 1 -- 172.21.15.42:0/2943741223 shutdown_connections 2024-04-20T10:22:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.199+0000 7f5411c00700 1 -- 172.21.15.42:0/2943741223 wait complete. 2024-04-20T10:22:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.200+0000 7f5411c00700 1 Processor -- start 2024-04-20T10:22:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.200+0000 7f5411c00700 1 -- start start 2024-04-20T10:22:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.201+0000 7f5411c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 0x7f540c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.201+0000 7f5411c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f540c10ea70 con 0x7f540c07de30 2024-04-20T10:22:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.201+0000 7f540aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 0x7f540c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.201+0000 7f540aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 0x7f540c19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44958/0 (socket says 172.21.15.42:44958) 2024-04-20T10:22:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.201+0000 7f540aa00700 1 -- 172.21.15.42:0/4192011813 learned_addr learned my addr 172.21.15.42:0/4192011813 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.201+0000 7f540aa00700 1 -- 172.21.15.42:0/4192011813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c19eb20 con 0x7f540c07de30 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.202+0000 7f540aa00700 1 --2- 172.21.15.42:0/4192011813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 0x7f540c19e5e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009530 tx=0x7f53fc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.202+0000 7f5403e00700 1 -- 172.21.15.42:0/4192011813 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53fc020020 con 0x7f540c07de30 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.202+0000 7f5403e00700 1 -- 172.21.15.42:0/4192011813 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f53fc012430 con 0x7f540c07de30 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.202+0000 7f5403e00700 1 -- 172.21.15.42:0/4192011813 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53fc017040 con 0x7f540c07de30 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.202+0000 7f5411c00700 1 -- 172.21.15.42:0/4192011813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c19edb0 con 0x7f540c07de30 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.202+0000 7f5411c00700 1 -- 172.21.15.42:0/4192011813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c1a1aa0 con 0x7f540c07de30 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.202+0000 7f5403e00700 1 -- 172.21.15.42:0/4192011813 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53fc023020 con 0x7f540c07de30 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.202+0000 7f5403e00700 1 -- 172.21.15.42:0/4192011813 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f53fc00f050 con 0x7f540c07de30 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.203+0000 7f5411c00700 1 -- 172.21.15.42:0/4192011813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540c0640b0 con 0x7f540c07de30 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.206+0000 7f5403e00700 1 -- 172.21.15.42:0/4192011813 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f53fc02a020 con 0x7f540c07de30 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.286+0000 7f5411c00700 1 -- 172.21.15.42:0/4192011813 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f540c1a1c70 con 0x7f540c07de30 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.286+0000 7f5403e00700 1 -- 172.21.15.42:0/4192011813 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f53fc00b860 con 0x7f540c07de30 2024-04-20T10:22:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.288+0000 7f5411c00700 1 -- 172.21.15.42:0/4192011813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 msgr2=0x7f540c19e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.288+0000 7f5411c00700 1 --2- 172.21.15.42:0/4192011813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 0x7f540c19e5e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009530 tx=0x7f53fc00bf60 comp rx=0 tx=0).stop 2024-04-20T10:22:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.288+0000 7f5411c00700 1 -- 172.21.15.42:0/4192011813 shutdown_connections 2024-04-20T10:22:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.288+0000 7f5411c00700 1 --2- 172.21.15.42:0/4192011813 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f540c07de30 0x7f540c19e5e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.288+0000 7f5411c00700 1 -- 172.21.15.42:0/4192011813 >> 172.21.15.42:0/4192011813 conn(0x7f540c103a60 msgr2=0x7f540c104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.288+0000 7f5411c00700 1 -- 172.21.15.42:0/4192011813 shutdown_connections 2024-04-20T10:22:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:25.288+0000 7f5411c00700 1 -- 172.21.15.42:0/4192011813 wait complete. 2024-04-20T10:22:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:Restarting the monitor... 2024-04-20T10:22:26.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:25 smithi042 systemd[1]: Stopping Ceph mon.a for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:22:26.780 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:26 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a[16706]: 2024-04-20T10:22:26.457+0000 7fe6b6a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:22:26.780 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:26 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a[16706]: 2024-04-20T10:22:26.457+0000 7fe6b6a00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-20T10:22:26.780 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:26 smithi042 podman[16954]: 2024-04-20 10:22:26.540476675 +0000 UTC m=+0.215298897 container died b2fd4ee85b4c2872489284de36d3508a9ca7c2ec21674d63379325f2305c8029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True) 2024-04-20T10:22:27.076 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:26 smithi042 podman[16954]: 2024-04-20 10:22:26.883283041 +0000 UTC m=+0.558105279 container cleanup b2fd4ee85b4c2872489284de36d3508a9ca7c2ec21674d63379325f2305c8029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-20T10:22:27.076 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:26 smithi042 bash[16954]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a 2024-04-20T10:22:27.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:27 smithi042 podman[16968]: 2024-04-20 10:22:27.075676815 +0000 UTC m=+0.511436745 container remove b2fd4ee85b4c2872489284de36d3508a9ca7c2ec21674d63379325f2305c8029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True) 2024-04-20T10:22:27.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:27 smithi042 systemd[1]: ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-20T10:22:27.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:27 smithi042 systemd[1]: Stopped Ceph mon.a for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:22:27.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:27 smithi042 systemd[1]: ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.a.service: Consumed 2.032s CPU time. 2024-04-20T10:22:27.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:27 smithi042 systemd[1]: Starting Ceph mon.a for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:22:28.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:27 smithi042 podman[17059]: 2024-04-20 10:22:27.807180421 +0000 UTC m=+0.036153278 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:22:28.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:27 smithi042 podman[17059]: 2024-04-20 10:22:27.918268689 +0000 UTC m=+0.147241571 container create f39180ca248a36dae9b7cdcfbb967e6ac1f0ebbf0a7d992b6adf38d8f337c158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T10:22:28.756 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 podman[17059]: 2024-04-20 10:22:28.4739493 +0000 UTC m=+0.702922189 container init f39180ca248a36dae9b7cdcfbb967e6ac1f0ebbf0a7d992b6adf38d8f337c158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-20T10:22:28.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 podman[17059]: 2024-04-20 10:22:28.484964125 +0000 UTC m=+0.713937014 container start f39180ca248a36dae9b7cdcfbb967e6ac1f0ebbf0a7d992b6adf38d8f337c158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:22:28.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: set uid:gid to 167:167 (ceph:ceph) 2024-04-20T10:22:28.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-20T10:22:28.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: pidfile_write: ignore empty --pid-file 2024-04-20T10:22:28.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: load: jerasure load: lrc 2024-04-20T10:22:28.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: RocksDB version: 7.9.2 2024-04-20T10:22:28.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Git sha 0 2024-04-20T10:22:28.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-20T10:22:28.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: DB SUMMARY 2024-04-20T10:22:28.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: DB Session ID: Z9HXB58J12LRMZKRCMXY 2024-04-20T10:22:28.757 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: CURRENT file: CURRENT 2024-04-20T10:22:28.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: IDENTITY file: IDENTITY 2024-04-20T10:22:28.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-20T10:22:28.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-20T10:22:28.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105090 ; 2024-04-20T10:22:28.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.error_if_exists: 0 2024-04-20T10:22:28.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.create_if_missing: 0 2024-04-20T10:22:28.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.paranoid_checks: 1 2024-04-20T10:22:28.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-20T10:22:28.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-20T10:22:28.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-20T10:22:28.758 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.env: 0x55c4308a4cc0 2024-04-20T10:22:28.759 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.fs: PosixFileSystem 2024-04-20T10:22:28.759 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.info_log: 0x55c456ea9c60 2024-04-20T10:22:28.759 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_file_opening_threads: 16 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.statistics: (nil) 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.use_fsync: 0 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_log_file_size: 0 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.keep_log_file_num: 1000 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.recycle_log_file_num: 0 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.allow_fallocate: 1 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.allow_mmap_reads: 0 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.allow_mmap_writes: 0 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.use_direct_reads: 0 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.create_missing_column_families: 0 2024-04-20T10:22:28.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.db_log_dir: 2024-04-20T10:22:28.762 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.wal_dir: 2024-04-20T10:22:28.762 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-20T10:22:28.762 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-20T10:22:28.762 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-20T10:22:28.762 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-20T10:22:28.762 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-20T10:22:28.762 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-20T10:22:28.762 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.advise_random_on_open: 1 2024-04-20T10:22:28.762 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.db_write_buffer_size: 0 2024-04-20T10:22:28.762 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.write_buffer_manager: 0x55c456e781e0 2024-04-20T10:22:28.762 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-20T10:22:28.762 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.rate_limiter: (nil) 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.wal_recovery_mode: 2 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.enable_thread_tracking: 0 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.enable_pipelined_write: 0 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.unordered_write: 0 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.row_cache: None 2024-04-20T10:22:28.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.wal_filter: None 2024-04-20T10:22:28.764 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-20T10:22:28.764 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.allow_ingest_behind: 0 2024-04-20T10:22:28.764 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.two_write_queues: 0 2024-04-20T10:22:28.764 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.manual_wal_flush: 0 2024-04-20T10:22:28.764 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.wal_compression: 0 2024-04-20T10:22:28.764 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.atomic_flush: 0 2024-04-20T10:22:28.764 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-20T10:22:28.764 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-20T10:22:28.764 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-20T10:22:28.764 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.log_readahead_size: 0 2024-04-20T10:22:28.764 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-20T10:22:28.764 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.best_efforts_recovery: 0 2024-04-20T10:22:28.765 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-20T10:22:28.765 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-20T10:22:28.765 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.allow_data_in_errors: 0 2024-04-20T10:22:28.765 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.db_host_id: __hostname__ 2024-04-20T10:22:28.765 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-20T10:22:28.765 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_background_jobs: 2 2024-04-20T10:22:28.765 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_background_compactions: -1 2024-04-20T10:22:28.765 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_subcompactions: 1 2024-04-20T10:22:28.765 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-20T10:22:28.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-20T10:22:28.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-20T10:22:28.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_total_wal_size: 0 2024-04-20T10:22:28.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-20T10:22:28.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-20T10:22:28.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-20T10:22:28.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-20T10:22:28.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_open_files: -1 2024-04-20T10:22:28.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bytes_per_sync: 0 2024-04-20T10:22:28.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-20T10:22:28.766 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-20T10:22:28.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_readahead_size: 0 2024-04-20T10:22:28.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_background_flushes: -1 2024-04-20T10:22:28.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Compression algorithms supported: 2024-04-20T10:22:28.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-20T10:22:28.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: kZSTD supported: 0 2024-04-20T10:22:28.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: kXpressCompression supported: 0 2024-04-20T10:22:28.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: kLZ4HCCompression supported: 1 2024-04-20T10:22:28.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: kZlibCompression supported: 1 2024-04-20T10:22:28.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: kSnappyCompression supported: 1 2024-04-20T10:22:28.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: kLZ4Compression supported: 1 2024-04-20T10:22:28.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: kBZip2Compression supported: 0 2024-04-20T10:22:28.767 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-20T10:22:28.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-20T10:22:28.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T10:22:28.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-20T10:22:28.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-20T10:22:28.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.merge_operator: 2024-04-20T10:22:28.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_filter: None 2024-04-20T10:22:28.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_filter_factory: None 2024-04-20T10:22:28.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.sst_partitioner_factory: None 2024-04-20T10:22:28.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-20T10:22:28.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-20T10:22:28.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c456ea9e20) 2024-04-20T10:22:28.769 INFO:journalctl@ceph.mon.a.smithi042.stdout: cache_index_and_filter_blocks: 1 2024-04-20T10:22:28.769 INFO:journalctl@ceph.mon.a.smithi042.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-20T10:22:28.769 INFO:journalctl@ceph.mon.a.smithi042.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-20T10:22:28.769 INFO:journalctl@ceph.mon.a.smithi042.stdout: pin_top_level_index_and_filter: 1 2024-04-20T10:22:28.769 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_type: 0 2024-04-20T10:22:28.769 INFO:journalctl@ceph.mon.a.smithi042.stdout: data_block_index_type: 0 2024-04-20T10:22:28.769 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_shortening: 1 2024-04-20T10:22:28.769 INFO:journalctl@ceph.mon.a.smithi042.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-20T10:22:28.769 INFO:journalctl@ceph.mon.a.smithi042.stdout: checksum: 4 2024-04-20T10:22:28.769 INFO:journalctl@ceph.mon.a.smithi042.stdout: no_block_cache: 0 2024-04-20T10:22:28.769 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache: 0x55c456eab090 2024-04-20T10:22:28.770 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_name: BinnedLRUCache 2024-04-20T10:22:28.770 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_options: 2024-04-20T10:22:28.770 INFO:journalctl@ceph.mon.a.smithi042.stdout: capacity : 536870912 2024-04-20T10:22:28.770 INFO:journalctl@ceph.mon.a.smithi042.stdout: num_shard_bits : 4 2024-04-20T10:22:28.770 INFO:journalctl@ceph.mon.a.smithi042.stdout: strict_capacity_limit : 0 2024-04-20T10:22:28.770 INFO:journalctl@ceph.mon.a.smithi042.stdout: high_pri_pool_ratio: 0.000 2024-04-20T10:22:28.770 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_compressed: (nil) 2024-04-20T10:22:28.770 INFO:journalctl@ceph.mon.a.smithi042.stdout: persistent_cache: (nil) 2024-04-20T10:22:28.770 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_size: 4096 2024-04-20T10:22:28.770 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_size_deviation: 10 2024-04-20T10:22:28.770 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_restart_interval: 16 2024-04-20T10:22:28.771 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_block_restart_interval: 1 2024-04-20T10:22:28.771 INFO:journalctl@ceph.mon.a.smithi042.stdout: metadata_block_size: 4096 2024-04-20T10:22:28.771 INFO:journalctl@ceph.mon.a.smithi042.stdout: partition_filters: 0 2024-04-20T10:22:28.771 INFO:journalctl@ceph.mon.a.smithi042.stdout: use_delta_encoding: 1 2024-04-20T10:22:28.771 INFO:journalctl@ceph.mon.a.smithi042.stdout: filter_policy: bloomfilter 2024-04-20T10:22:28.771 INFO:journalctl@ceph.mon.a.smithi042.stdout: whole_key_filtering: 1 2024-04-20T10:22:28.771 INFO:journalctl@ceph.mon.a.smithi042.stdout: verify_compression: 0 2024-04-20T10:22:28.771 INFO:journalctl@ceph.mon.a.smithi042.stdout: read_amp_bytes_per_bit: 0 2024-04-20T10:22:28.771 INFO:journalctl@ceph.mon.a.smithi042.stdout: format_version: 5 2024-04-20T10:22:28.771 INFO:journalctl@ceph.mon.a.smithi042.stdout: enable_index_compression: 1 2024-04-20T10:22:28.771 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_align: 0 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout: max_auto_readahead_size: 262144 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout: prepopulate_block_cache: 0 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout: initial_auto_readahead_size: 8192 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout: num_file_reads_for_auto_readahead: 2 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.write_buffer_size: 33554432 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_write_buffer_number: 2 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compression: NoCompression 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bottommost_compression: Disabled 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.prefix_extractor: nullptr 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.num_levels: 7 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-20T10:22:28.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compression_opts.level: 32767 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compression_opts.strategy: 0 2024-04-20T10:22:28.773 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compression_opts.enabled: false 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.target_file_size_base: 67108864 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-20T10:22:28.774 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.arena_block_size: 1048576 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-20T10:22:28.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.disable_auto_compactions: 0 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.inplace_update_support: 0 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-20T10:22:28.776 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-20T10:22:28.777 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-20T10:22:28.777 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-20T10:22:28.777 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.bloom_locality: 0 2024-04-20T10:22:28.777 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.max_successive_merges: 0 2024-04-20T10:22:28.777 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-20T10:22:28.777 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.paranoid_file_checks: 0 2024-04-20T10:22:28.777 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.force_consistency_checks: 1 2024-04-20T10:22:28.777 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.report_bg_io_stats: 0 2024-04-20T10:22:28.777 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.ttl: 2592000 2024-04-20T10:22:28.777 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-20T10:22:28.777 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.enable_blob_files: false 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.min_blob_size: 0 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.blob_file_size: 268435456 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.blob_file_starting_level: 0 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-20T10:22:28.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c05c8f18-cdda-4778-8e68-0607a424eb18 2024-04-20T10:22:28.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713608548541546, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-20T10:22:28.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-20T10:22:28.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713608548543555, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100208, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98096, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14165, "raw_average_key_size": 49, "raw_value_size": 90430, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713608548, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c05c8f18-cdda-4778-8e68-0607a424eb18", "db_session_id": "Z9HXB58J12LRMZKRCMXY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-20T10:22:28.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713608548543704, "job": 1, "event": "recovery_finished"} 2024-04-20T10:22:28.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-20T10:22:28.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-20T10:22:28.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c456f98000 2024-04-20T10:22:28.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: DB pointer 0x55c456f82000 2024-04-20T10:22:28.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-20T10:22:28.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-20T10:22:28.779 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** DB Stats ** 2024-04-20T10:22:28.780 INFO:journalctl@ceph.mon.a.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T10:22:28.780 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-20T10:22:28.780 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T10:22:28.780 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T10:22:28.780 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-20T10:22:28.780 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T10:22:28.780 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T10:22:28.780 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-04-20T10:22:28.780 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** Compaction Stats [default] ** 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: L0 2/0 99.72 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 68.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: Sum 2/0 99.72 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 68.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 68.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** Compaction Stats [default] ** 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 68.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-20T10:22:28.781 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-04-20T10:22:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T10:22:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-20T10:22:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-20T10:22:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-20T10:22:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-20T10:22:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-20T10:22:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative compaction: 0.00 GB write, 14.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T10:22:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval compaction: 0.00 GB write, 14.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T10:22:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-20T10:22:28.782 INFO:journalctl@ceph.mon.a.smithi042.stdout: Block cache BinnedLRUCache@0x55c456eab090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-20T10:22:28.783 INFO:journalctl@ceph.mon.a.smithi042.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-20T10:22:28.783 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-04-20T10:22:28.783 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-20T10:22:28.783 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: starting mon.a rank 0 at public addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] at bind addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:22:28.783 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: mon.a@-1(???) e1 preinit fsid 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:22:28.783 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: mon.a@-1(???).mds e1 new map 2024-04-20T10:22:28.783 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: mon.a@-1(???).mds e1 print_map 2024-04-20T10:22:28.783 INFO:journalctl@ceph.mon.a.smithi042.stdout: e1 2024-04-20T10:22:28.783 INFO:journalctl@ceph.mon.a.smithi042.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T10:22:28.783 INFO:journalctl@ceph.mon.a.smithi042.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T10:22:28.783 INFO:journalctl@ceph.mon.a.smithi042.stdout: legacy client fscid: -1 2024-04-20T10:22:28.783 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-04-20T10:22:28.784 INFO:journalctl@ceph.mon.a.smithi042.stdout: No filesystems configured 2024-04-20T10:22:28.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-20T10:22:28.784 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T10:22:28.785 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T10:22:28.785 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T10:22:28.785 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-20T10:22:28.785 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-20T10:22:28.785 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: monmap epoch 1 2024-04-20T10:22:28.785 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: fsid 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:22:28.785 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: last_changed 2024-04-20T10:22:16.558137+0000 2024-04-20T10:22:28.785 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: created 2024-04-20T10:22:16.558137+0000 2024-04-20T10:22:28.785 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: min_mon_release 19 (squid) 2024-04-20T10:22:28.786 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: election_strategy: 1 2024-04-20T10:22:28.786 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-20T10:22:28.786 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: fsmap 2024-04-20T10:22:28.786 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: osdmap e1: 0 total, 0 up, 0 in 2024-04-20T10:22:28.786 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 ceph-mon[17078]: mgrmap e1: no daemons active 2024-04-20T10:22:28.786 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 bash[17059]: f39180ca248a36dae9b7cdcfbb967e6ac1f0ebbf0a7d992b6adf38d8f337c158 2024-04-20T10:22:28.786 INFO:teuthology.orchestra.run.smithi042.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-20T10:22:29.058 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:28 smithi042 systemd[1]: Started Ceph mon.a for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:22:30.327 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.374+0000 7f9751600700 1 Processor -- start 2024-04-20T10:22:30.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.374+0000 7f9751600700 1 -- start start 2024-04-20T10:22:30.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.374+0000 7f9751600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 0x7f974c10b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.374+0000 7f9751600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f974c07beb0 con 0x7f974c108f00 2024-04-20T10:22:30.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.375+0000 7f974aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 0x7f974c10b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:30.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.375+0000 7f974aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 0x7f974c10b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34860/0 (socket says 172.21.15.42:34860) 2024-04-20T10:22:30.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.375+0000 7f974aa00700 1 -- 172.21.15.42:0/1114701341 learned_addr learned my addr 172.21.15.42:0/1114701341 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:30.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.376+0000 7f974aa00700 1 -- 172.21.15.42:0/1114701341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c10b830 con 0x7f974c108f00 2024-04-20T10:22:30.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.376+0000 7f974aa00700 1 --2- 172.21.15.42:0/1114701341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 0x7f974c10b2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f973c0098a0 tx=0x7f973c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72535d38b4ce9958 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:30.357 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.376+0000 7f9749600700 1 -- 172.21.15.42:0/1114701341 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f973c004160 con 0x7f974c108f00 2024-04-20T10:22:30.357 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.376+0000 7f9749600700 1 -- 172.21.15.42:0/1114701341 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f973c004300 con 0x7f974c108f00 2024-04-20T10:22:30.357 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.376+0000 7f9749600700 1 -- 172.21.15.42:0/1114701341 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f973c011b50 con 0x7f974c108f00 2024-04-20T10:22:30.357 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.377+0000 7f9751600700 1 -- 172.21.15.42:0/1114701341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 msgr2=0x7f974c10b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.358 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.377+0000 7f9751600700 1 --2- 172.21.15.42:0/1114701341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 0x7f974c10b2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f973c0098a0 tx=0x7f973c009bb0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.358 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.377+0000 7f9751600700 1 -- 172.21.15.42:0/1114701341 shutdown_connections 2024-04-20T10:22:30.358 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.377+0000 7f9751600700 1 --2- 172.21.15.42:0/1114701341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 0x7f974c10b2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.359 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.377+0000 7f9751600700 1 -- 172.21.15.42:0/1114701341 >> 172.21.15.42:0/1114701341 conn(0x7f974c102910 msgr2=0x7f974c104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:30.359 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.377+0000 7f9751600700 1 -- 172.21.15.42:0/1114701341 shutdown_connections 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.377+0000 7f9751600700 1 -- 172.21.15.42:0/1114701341 wait complete. 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.378+0000 7f9751600700 1 Processor -- start 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.378+0000 7f9751600700 1 -- start start 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.379+0000 7f9751600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 0x7f974c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.379+0000 7f9751600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f974c10c860 con 0x7f974c108f00 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.379+0000 7f974aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 0x7f974c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.379+0000 7f974aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 0x7f974c19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34866/0 (socket says 172.21.15.42:34866) 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.379+0000 7f974aa00700 1 -- 172.21.15.42:0/872586949 learned_addr learned my addr 172.21.15.42:0/872586949 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.379+0000 7f974aa00700 1 -- 172.21.15.42:0/872586949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c19eba0 con 0x7f974c108f00 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.380+0000 7f974aa00700 1 --2- 172.21.15.42:0/872586949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 0x7f974c19e660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f973c011b60 tx=0x7f973c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.380+0000 7f9743400700 1 -- 172.21.15.42:0/872586949 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f973c003970 con 0x7f974c108f00 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.380+0000 7f9743400700 1 -- 172.21.15.42:0/872586949 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f973c003b10 con 0x7f974c108f00 2024-04-20T10:22:30.360 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.380+0000 7f9743400700 1 -- 172.21.15.42:0/872586949 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f973c024610 con 0x7f974c108f00 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.380+0000 7f9751600700 1 -- 172.21.15.42:0/872586949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974c19ee30 con 0x7f974c108f00 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.380+0000 7f9751600700 1 -- 172.21.15.42:0/872586949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974c1a1ad0 con 0x7f974c108f00 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.381+0000 7f9743400700 1 -- 172.21.15.42:0/872586949 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f973c024ca0 con 0x7f974c108f00 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.381+0000 7f9743400700 1 -- 172.21.15.42:0/872586949 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f973c011e00 con 0x7f974c108f00 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.381+0000 7f9751600700 1 -- 172.21.15.42:0/872586949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f974c0640b0 con 0x7f974c108f00 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.385+0000 7f9743400700 1 -- 172.21.15.42:0/872586949 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f973c043e50 con 0x7f974c108f00 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.460+0000 7f9751600700 1 -- 172.21.15.42:0/872586949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f974c1a1ca0 con 0x7f974c108f00 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.463+0000 7f9743400700 1 -- 172.21.15.42:0/872586949 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f973c032030 con 0x7f974c108f00 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.464+0000 7f9751600700 1 -- 172.21.15.42:0/872586949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 msgr2=0x7f974c19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.464+0000 7f9751600700 1 --2- 172.21.15.42:0/872586949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 0x7f974c19e660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f973c011b60 tx=0x7f973c011c40 comp rx=0 tx=0).stop 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.464+0000 7f9751600700 1 -- 172.21.15.42:0/872586949 shutdown_connections 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.464+0000 7f9751600700 1 --2- 172.21.15.42:0/872586949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f974c108f00 0x7f974c19e660 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.464+0000 7f9751600700 1 -- 172.21.15.42:0/872586949 >> 172.21.15.42:0/872586949 conn(0x7f974c102910 msgr2=0x7f974c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:30.361 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.464+0000 7f9751600700 1 -- 172.21.15.42:0/872586949 shutdown_connections 2024-04-20T10:22:30.362 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:29.465+0000 7f9751600700 1 -- 172.21.15.42:0/872586949 wait complete. 2024-04-20T10:22:30.362 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-20T10:22:30.363 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-20T10:22:30.363 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mgr... 2024-04-20T10:22:30.363 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-20T10:22:30.363 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-20T10:22:30.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:30 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/872586949' entity='client.admin' 2024-04-20T10:22:30.727 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.x 2024-04-20T10:22:30.727 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.x.service: Unit ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.x.service not loaded. 2024-04-20T10:22:30.989 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-83e8f766-feff-11ee-bc92-c7b262605968.target.wants/ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-83e8f766-feff-11ee-bc92-c7b262605968@.service. 2024-04-20T10:22:31.354 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:31 smithi042 systemd[1]: Starting Ceph mgr.x for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:22:31.683 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:31 smithi042 podman[17301]: 2024-04-20 10:22:31.294876256 +0000 UTC m=+0.035926543 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:22:31.683 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:31 smithi042 podman[17301]: 2024-04-20 10:22:31.39754929 +0000 UTC m=+0.138599598 container create 5bd72d6b593f6a53bcb340da07515b7a463d86bd77fe122b46b8eea14806f53c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:22:31.683 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:31 smithi042 podman[17301]: 2024-04-20 10:22:31.617912452 +0000 UTC m=+0.358962767 container init 5bd72d6b593f6a53bcb340da07515b7a463d86bd77fe122b46b8eea14806f53c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:22:31.683 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:31 smithi042 podman[17301]: 2024-04-20 10:22:31.627755951 +0000 UTC m=+0.368806266 container start 5bd72d6b593f6a53bcb340da07515b7a463d86bd77fe122b46b8eea14806f53c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-20T10:22:31.683 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:31 smithi042 bash[17301]: 5bd72d6b593f6a53bcb340da07515b7a463d86bd77fe122b46b8eea14806f53c 2024-04-20T10:22:31.717 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T10:22:31.718 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-04-20T10:22:31.736 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T10:22:31.736 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-04-20T10:22:31.737 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-04-20T10:22:31.737 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2024-04-20T10:22:31.765 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T10:22:31.766 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-04-20T10:22:31.787 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T10:22:31.787 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-04-20T10:22:31.787 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-04-20T10:22:31.788 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-20T10:22:31.788 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr to start... 2024-04-20T10:22:31.788 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr... 2024-04-20T10:22:31.944 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:31 smithi042 systemd[1]: Started Ceph mgr.x for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:22:31.944 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:31 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:31.897+0000 7f86a3e7a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:22:32.200 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:32 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:32.166+0000 7f86a3e7a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:22:32.537 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:32 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:32.305+0000 7f86a3e7a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:22:32.847 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:32 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3106588015' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T10:22:32.847 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:32 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:32.729+0000 7f86a3e7a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:22:32.847 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:32 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:32.847+0000 7f86a3e7a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:22:33.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:33.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:22:33.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "83e8f766-feff-11ee-bc92-c7b262605968", 2024-04-20T10:22:33.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T10:22:33.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T10:22:33.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T10:22:33.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T10:22:33.254 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:33.254 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T10:22:33.254 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T10:22:33.254 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-04-20T10:22:33.254 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:22:33.254 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T10:22:33.254 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T10:22:33.255 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:22:33.255 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-20T10:22:33.255 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T10:22:33.255 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:33.255 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T10:22:33.255 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T10:22:33.255 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:33.255 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T10:22:33.255 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:33.255 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T10:22:33.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T10:22:33.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T10:22:33.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T10:22:33.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T10:22:33.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T10:22:33.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:33.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T10:22:33.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T10:22:33.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T10:22:33.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T10:22:33.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T10:22:33.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T10:22:33.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T10:22:33.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T10:22:33.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T10:22:33.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:33.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T10:22:33.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:33.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T10:22:33.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T10:22:33.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:33.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T10:22:33.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T10:22:33.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T10:22:21.222435+0000", 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T10:22:33.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.447+0000 7f7ec1800700 1 Processor -- start 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.447+0000 7f7ec1800700 1 -- start start 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.448+0000 7f7ec1800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 0x7f7ebc10b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.448+0000 7f7ec1800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc07be30 con 0x7f7ebc108dc0 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.448+0000 7f7ebbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 0x7f7ebc10b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.449+0000 7f7ebbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 0x7f7ebc10b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34886/0 (socket says 172.21.15.42:34886) 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.449+0000 7f7ebbe00700 1 -- 172.21.15.42:0/865389043 learned_addr learned my addr 172.21.15.42:0/865389043 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.449+0000 7f7ebbe00700 1 -- 172.21.15.42:0/865389043 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc10b6f0 con 0x7f7ebc108dc0 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.449+0000 7f7ebbe00700 1 --2- 172.21.15.42:0/865389043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 0x7f7ebc10b1b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0098a0 tx=0x7f7eac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64c2227877fd448 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.450+0000 7f7ebaa00700 1 -- 172.21.15.42:0/865389043 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eac004160 con 0x7f7ebc108dc0 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.450+0000 7f7ebaa00700 1 -- 172.21.15.42:0/865389043 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7eac004300 con 0x7f7ebc108dc0 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.450+0000 7f7ebaa00700 1 -- 172.21.15.42:0/865389043 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eac011b50 con 0x7f7ebc108dc0 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.450+0000 7f7ec1800700 1 -- 172.21.15.42:0/865389043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 msgr2=0x7f7ebc10b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.450+0000 7f7ec1800700 1 --2- 172.21.15.42:0/865389043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 0x7f7ebc10b1b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0098a0 tx=0x7f7eac009bb0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.450+0000 7f7ec1800700 1 -- 172.21.15.42:0/865389043 shutdown_connections 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.451+0000 7f7ec1800700 1 --2- 172.21.15.42:0/865389043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 0x7f7ebc10b1b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.451+0000 7f7ec1800700 1 -- 172.21.15.42:0/865389043 >> 172.21.15.42:0/865389043 conn(0x7f7ebc102790 msgr2=0x7f7ebc104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.451+0000 7f7ec1800700 1 -- 172.21.15.42:0/865389043 shutdown_connections 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.451+0000 7f7ec1800700 1 -- 172.21.15.42:0/865389043 wait complete. 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.452+0000 7f7ec1800700 1 Processor -- start 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.452+0000 7f7ec1800700 1 -- start start 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.452+0000 7f7ec1800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 0x7f7ebc19e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.452+0000 7f7ec1800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc10c6b0 con 0x7f7ebc108dc0 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.452+0000 7f7ebbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 0x7f7ebc19e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.452+0000 7f7ebbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 0x7f7ebc19e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34890/0 (socket says 172.21.15.42:34890) 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.452+0000 7f7ebbe00700 1 -- 172.21.15.42:0/3106588015 learned_addr learned my addr 172.21.15.42:0/3106588015 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:33.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.453+0000 7f7ebbe00700 1 -- 172.21.15.42:0/3106588015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc19e9f0 con 0x7f7ebc108dc0 2024-04-20T10:22:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.453+0000 7f7ebbe00700 1 --2- 172.21.15.42:0/3106588015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 0x7f7ebc19e4b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009fb0 tx=0x7f7eac011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.453+0000 7f7eb8c00700 1 -- 172.21.15.42:0/3106588015 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eac0038b0 con 0x7f7ebc108dc0 2024-04-20T10:22:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.453+0000 7f7eb8c00700 1 -- 172.21.15.42:0/3106588015 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7eac011e50 con 0x7f7ebc108dc0 2024-04-20T10:22:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.453+0000 7f7eb8c00700 1 -- 172.21.15.42:0/3106588015 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eac019420 con 0x7f7ebc108dc0 2024-04-20T10:22:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.453+0000 7f7ec1800700 1 -- 172.21.15.42:0/3106588015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc19ec80 con 0x7f7ebc108dc0 2024-04-20T10:22:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.453+0000 7f7ec1800700 1 -- 172.21.15.42:0/3106588015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc1a1970 con 0x7f7ebc108dc0 2024-04-20T10:22:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.454+0000 7f7eb8c00700 1 -- 172.21.15.42:0/3106588015 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7eac0196d0 con 0x7f7ebc108dc0 2024-04-20T10:22:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.454+0000 7f7eb8c00700 1 -- 172.21.15.42:0/3106588015 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7eac02dc20 con 0x7f7ebc108dc0 2024-04-20T10:22:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.455+0000 7f7ec1800700 1 -- 172.21.15.42:0/3106588015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ebc0640b0 con 0x7f7ebc108dc0 2024-04-20T10:22:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.457+0000 7f7eb8c00700 1 -- 172.21.15.42:0/3106588015 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f7eac043e50 con 0x7f7ebc108dc0 2024-04-20T10:22:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.537+0000 7f7ec1800700 1 -- 172.21.15.42:0/3106588015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7ebc1a1ec0 con 0x7f7ebc108dc0 2024-04-20T10:22:33.269 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.537+0000 7f7eb8c00700 1 -- 172.21.15.42:0/3106588015 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7eac032020 con 0x7f7ebc108dc0 2024-04-20T10:22:33.270 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.539+0000 7f7ec1800700 1 -- 172.21.15.42:0/3106588015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 msgr2=0x7f7ebc19e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.270 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.539+0000 7f7ec1800700 1 --2- 172.21.15.42:0/3106588015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 0x7f7ebc19e4b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009fb0 tx=0x7f7eac011bf0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.270 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.539+0000 7f7ec1800700 1 -- 172.21.15.42:0/3106588015 shutdown_connections 2024-04-20T10:22:33.270 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.539+0000 7f7ec1800700 1 --2- 172.21.15.42:0/3106588015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ebc108dc0 0x7f7ebc19e4b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.270 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.539+0000 7f7ec1800700 1 -- 172.21.15.42:0/3106588015 >> 172.21.15.42:0/3106588015 conn(0x7f7ebc102790 msgr2=0x7f7ebc103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:33.270 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.539+0000 7f7ec1800700 1 -- 172.21.15.42:0/3106588015 shutdown_connections 2024-04-20T10:22:33.270 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:32.540+0000 7f7ec1800700 1 -- 172.21.15.42:0/3106588015 wait complete. 2024-04-20T10:22:33.270 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (1/15)... 2024-04-20T10:22:34.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:34.479+0000 7f86a3e7a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:22:34.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:34.714+0000 7f86a3e7a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:22:35.330 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:34.931+0000 7f86a3e7a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:22:35.588 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:35 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:35.330+0000 7f86a3e7a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:22:35.588 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:35 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:35.455+0000 7f86a3e7a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:22:35.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:35 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:35.715+0000 7f86a3e7a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:22:36.294 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:36 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4103813715' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T10:22:36.294 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:35 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:35.978+0000 7f86a3e7a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:22:36.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:36.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "83e8f766-feff-11ee-bc92-c7b262605968", 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T10:22:36.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:22:36.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-20T10:22:36.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T10:22:36.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:36.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T10:22:36.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T10:22:36.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:36.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T10:22:36.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:36.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T10:22:36.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T10:22:36.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T10:22:36.656 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T10:22:36.656 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T10:22:36.656 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T10:22:36.656 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:36.656 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T10:22:36.656 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T10:22:36.656 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T10:22:36.656 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T10:22:36.656 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T10:22:36.656 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T10:22:36.656 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T10:22:36.656 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T10:22:36.657 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T10:22:21.222435+0000", 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.882+0000 7fabd6c00700 1 Processor -- start 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.882+0000 7fabd6c00700 1 -- start start 2024-04-20T10:22:36.658 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.883+0000 7fabd6c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 0x7fabd007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.883+0000 7fabd6c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabd007c820 con 0x7fabd007de30 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.883+0000 7fabd5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 0x7fabd007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.883+0000 7fabd5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 0x7fabd007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34896/0 (socket says 172.21.15.42:34896) 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.883+0000 7fabd5800700 1 -- 172.21.15.42:0/2672090766 learned_addr learned my addr 172.21.15.42:0/2672090766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.884+0000 7fabd5800700 1 -- 172.21.15.42:0/2672090766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd007c9a0 con 0x7fabd007de30 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.884+0000 7fabd5800700 1 --2- 172.21.15.42:0/2672090766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 0x7fabd007c2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fabc40098a0 tx=0x7fabc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5e4b9ac9f5adb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.885+0000 7fabcfe00700 1 -- 172.21.15.42:0/2672090766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabc4004160 con 0x7fabd007de30 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.885+0000 7fabcfe00700 1 -- 172.21.15.42:0/2672090766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fabc4004300 con 0x7fabd007de30 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.885+0000 7fabcfe00700 1 -- 172.21.15.42:0/2672090766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabc4011b50 con 0x7fabd007de30 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.885+0000 7fabd6c00700 1 -- 172.21.15.42:0/2672090766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 msgr2=0x7fabd007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.885+0000 7fabd6c00700 1 --2- 172.21.15.42:0/2672090766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 0x7fabd007c2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fabc40098a0 tx=0x7fabc4009bb0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.885+0000 7fabd6c00700 1 -- 172.21.15.42:0/2672090766 shutdown_connections 2024-04-20T10:22:36.659 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.885+0000 7fabd6c00700 1 --2- 172.21.15.42:0/2672090766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 0x7fabd007c2e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.885+0000 7fabd6c00700 1 -- 172.21.15.42:0/2672090766 >> 172.21.15.42:0/2672090766 conn(0x7fabd01038b0 msgr2=0x7fabd0105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.886+0000 7fabd6c00700 1 -- 172.21.15.42:0/2672090766 shutdown_connections 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.886+0000 7fabd6c00700 1 -- 172.21.15.42:0/2672090766 wait complete. 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.887+0000 7fabd6c00700 1 Processor -- start 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.887+0000 7fabd6c00700 1 -- start start 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.887+0000 7fabd6c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 0x7fabd019e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.887+0000 7fabd6c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabd010e8c0 con 0x7fabd007de30 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.887+0000 7fabd5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 0x7fabd019e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.887+0000 7fabd5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 0x7fabd019e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34898/0 (socket says 172.21.15.42:34898) 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.887+0000 7fabd5800700 1 -- 172.21.15.42:0/4103813715 learned_addr learned my addr 172.21.15.42:0/4103813715 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.888+0000 7fabd5800700 1 -- 172.21.15.42:0/4103813715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd019e9e0 con 0x7fabd007de30 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.888+0000 7fabd5800700 1 --2- 172.21.15.42:0/4103813715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 0x7fabd019e4a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fabc4011b60 tx=0x7fabc4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:36.660 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.888+0000 7fabce000700 1 -- 172.21.15.42:0/4103813715 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabc4003970 con 0x7fabd007de30 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.888+0000 7fabce000700 1 -- 172.21.15.42:0/4103813715 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fabc4003b10 con 0x7fabd007de30 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.888+0000 7fabd6c00700 1 -- 172.21.15.42:0/4103813715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd019ec70 con 0x7fabd007de30 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.888+0000 7fabce000700 1 -- 172.21.15.42:0/4103813715 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabc4024610 con 0x7fabd007de30 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.889+0000 7fabd6c00700 1 -- 172.21.15.42:0/4103813715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd01a1960 con 0x7fabd007de30 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.889+0000 7fabce000700 1 -- 172.21.15.42:0/4103813715 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fabc4024ca0 con 0x7fabd007de30 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.889+0000 7fabce000700 1 -- 172.21.15.42:0/4103813715 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fabc4011e00 con 0x7fabd007de30 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.890+0000 7fabd6c00700 1 -- 172.21.15.42:0/4103813715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd00640b0 con 0x7fabd007de30 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.892+0000 7fabce000700 1 -- 172.21.15.42:0/4103813715 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fabc401a9f0 con 0x7fabd007de30 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.972+0000 7fabd6c00700 1 -- 172.21.15.42:0/4103813715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fabd01a1b30 con 0x7fabd007de30 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.973+0000 7fabce000700 1 -- 172.21.15.42:0/4103813715 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fabc401a430 con 0x7fabd007de30 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.974+0000 7fabd6c00700 1 -- 172.21.15.42:0/4103813715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 msgr2=0x7fabd019e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.974+0000 7fabd6c00700 1 --2- 172.21.15.42:0/4103813715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 0x7fabd019e4a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fabc4011b60 tx=0x7fabc4011c40 comp rx=0 tx=0).stop 2024-04-20T10:22:36.661 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.974+0000 7fabd6c00700 1 -- 172.21.15.42:0/4103813715 shutdown_connections 2024-04-20T10:22:36.662 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.974+0000 7fabd6c00700 1 --2- 172.21.15.42:0/4103813715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd007de30 0x7fabd019e4a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.662 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.974+0000 7fabd6c00700 1 -- 172.21.15.42:0/4103813715 >> 172.21.15.42:0/4103813715 conn(0x7fabd01038b0 msgr2=0x7fabd0104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:36.662 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.975+0000 7fabd6c00700 1 -- 172.21.15.42:0/4103813715 shutdown_connections 2024-04-20T10:22:36.662 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:35.975+0000 7fabd6c00700 1 -- 172.21.15.42:0/4103813715 wait complete. 2024-04-20T10:22:36.662 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (2/15)... 2024-04-20T10:22:37.103 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:36 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:36.811+0000 7f86a3e7a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:22:37.783 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:37.379+0000 7f86a3e7a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:22:37.783 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:37.510+0000 7f86a3e7a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:22:37.783 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:37.783+0000 7f86a3e7a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:22:38.803 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:38 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:38.447+0000 7f86a3e7a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:22:38.803 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:38 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:38.573+0000 7f86a3e7a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:22:39.059 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:38 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:38.908+0000 7f86a3e7a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:22:39.308 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:39.095+0000 7f86a3e7a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:22:39.309 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:39.247+0000 7f86a3e7a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:22:39.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:39 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/754525034' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T10:22:39.603 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:39.408+0000 7f86a3e7a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:22:40.028 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:39.790+0000 7f86a3e7a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:22:40.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:40.045 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:22:40.045 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "83e8f766-feff-11ee-bc92-c7b262605968", 2024-04-20T10:22:40.046 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T10:22:40.046 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T10:22:40.046 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T10:22:40.046 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T10:22:40.046 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:40.046 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T10:22:40.046 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T10:22:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-04-20T10:22:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:22:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T10:22:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T10:22:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:22:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-20T10:22:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T10:22:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T10:22:40.048 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T10:22:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T10:22:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T10:22:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T10:22:40.049 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T10:22:40.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T10:22:40.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T10:22:40.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T10:22:40.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:40.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T10:22:40.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T10:22:40.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T10:22:40.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T10:22:40.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T10:22:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T10:22:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T10:22:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T10:22:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T10:22:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T10:22:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T10:22:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T10:22:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T10:22:40.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T10:22:40.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T10:22:40.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T10:22:40.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T10:22:40.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T10:22:40.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T10:22:40.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:22:40.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:22:40.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:40.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T10:22:40.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T10:22:21.222435+0000", 2024-04-20T10:22:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:22:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T10:22:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:22:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.215+0000 7f5637a00700 1 Processor -- start 2024-04-20T10:22:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.215+0000 7f5637a00700 1 -- start start 2024-04-20T10:22:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.215+0000 7f5637a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 0x7f56301088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.215+0000 7f5637a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5630108eb0 con 0x7f56301084c0 2024-04-20T10:22:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.216+0000 7f5635000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 0x7f56301088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:40.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.216+0000 7f5635000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 0x7f56301088e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39824/0 (socket says 172.21.15.42:39824) 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.216+0000 7f5635000700 1 -- 172.21.15.42:0/1600671924 learned_addr learned my addr 172.21.15.42:0/1600671924 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.216+0000 7f5635000700 1 -- 172.21.15.42:0/1600671924 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5630109670 con 0x7f56301084c0 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.216+0000 7f5635000700 1 --2- 172.21.15.42:0/1600671924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 0x7f56301088e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56240098a0 tx=0x7f5624009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa9d8777edf0c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.217+0000 7f562f400700 1 -- 172.21.15.42:0/1600671924 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5624004160 con 0x7f56301084c0 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.217+0000 7f562f400700 1 -- 172.21.15.42:0/1600671924 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5624004300 con 0x7f56301084c0 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.217+0000 7f562f400700 1 -- 172.21.15.42:0/1600671924 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5624011b50 con 0x7f56301084c0 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.217+0000 7f5637a00700 1 -- 172.21.15.42:0/1600671924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 msgr2=0x7f56301088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.217+0000 7f5637a00700 1 --2- 172.21.15.42:0/1600671924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 0x7f56301088e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56240098a0 tx=0x7f5624009bb0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.218+0000 7f5637a00700 1 -- 172.21.15.42:0/1600671924 shutdown_connections 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.218+0000 7f5637a00700 1 --2- 172.21.15.42:0/1600671924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 0x7f56301088e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.218+0000 7f5637a00700 1 -- 172.21.15.42:0/1600671924 >> 172.21.15.42:0/1600671924 conn(0x7f5630103aa0 msgr2=0x7f5630105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.218+0000 7f5637a00700 1 -- 172.21.15.42:0/1600671924 shutdown_connections 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.218+0000 7f5637a00700 1 -- 172.21.15.42:0/1600671924 wait complete. 2024-04-20T10:22:40.059 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.219+0000 7f5637a00700 1 Processor -- start 2024-04-20T10:22:40.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.219+0000 7f5637a00700 1 -- start start 2024-04-20T10:22:40.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.219+0000 7f5637a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 0x7f563019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:40.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.219+0000 7f5637a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f563010ca20 con 0x7f56301084c0 2024-04-20T10:22:40.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.220+0000 7f5635000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 0x7f563019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:40.060 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.220+0000 7f5635000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 0x7f563019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39834/0 (socket says 172.21.15.42:39834) 2024-04-20T10:22:40.062 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.220+0000 7f5635000700 1 -- 172.21.15.42:0/754525034 learned_addr learned my addr 172.21.15.42:0/754525034 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:40.062 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.220+0000 7f5635000700 1 -- 172.21.15.42:0/754525034 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563019eba0 con 0x7f56301084c0 2024-04-20T10:22:40.062 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.220+0000 7f5635000700 1 --2- 172.21.15.42:0/754525034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 0x7f563019e660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5624011850 tx=0x7f5624011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:40.062 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.221+0000 7f562d600700 1 -- 172.21.15.42:0/754525034 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5624003af0 con 0x7f56301084c0 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.221+0000 7f562d600700 1 -- 172.21.15.42:0/754525034 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5624020070 con 0x7f56301084c0 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.221+0000 7f5637a00700 1 -- 172.21.15.42:0/754525034 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563019ee30 con 0x7f56301084c0 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.221+0000 7f562d600700 1 -- 172.21.15.42:0/754525034 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56240195a0 con 0x7f56301084c0 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.221+0000 7f5637a00700 1 -- 172.21.15.42:0/754525034 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56301a1b20 con 0x7f56301084c0 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.221+0000 7f562d600700 1 -- 172.21.15.42:0/754525034 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5624019c30 con 0x7f56301084c0 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.221+0000 7f562d600700 1 -- 172.21.15.42:0/754525034 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5624011e00 con 0x7f56301084c0 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.222+0000 7f5637a00700 1 -- 172.21.15.42:0/754525034 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56300640b0 con 0x7f56301084c0 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.225+0000 7f562d600700 1 -- 172.21.15.42:0/754525034 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f5624003c90 con 0x7f56301084c0 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.305+0000 7f5637a00700 1 -- 172.21.15.42:0/754525034 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f56301a1cf0 con 0x7f56301084c0 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.306+0000 7f562d600700 1 -- 172.21.15.42:0/754525034 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f562401a430 con 0x7f56301084c0 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.307+0000 7f5637a00700 1 -- 172.21.15.42:0/754525034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 msgr2=0x7f563019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.307+0000 7f5637a00700 1 --2- 172.21.15.42:0/754525034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 0x7f563019e660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5624011850 tx=0x7f5624011c10 comp rx=0 tx=0).stop 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.307+0000 7f5637a00700 1 -- 172.21.15.42:0/754525034 shutdown_connections 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.307+0000 7f5637a00700 1 --2- 172.21.15.42:0/754525034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56301084c0 0x7f563019e660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.307+0000 7f5637a00700 1 -- 172.21.15.42:0/754525034 >> 172.21.15.42:0/754525034 conn(0x7f5630103aa0 msgr2=0x7f5630104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:40.063 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.307+0000 7f5637a00700 1 -- 172.21.15.42:0/754525034 shutdown_connections 2024-04-20T10:22:40.064 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:39.307+0000 7f5637a00700 1 -- 172.21.15.42:0/754525034 wait complete. 2024-04-20T10:22:40.064 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (3/15)... 2024-04-20T10:22:40.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:40.590+0000 7f86a3e7a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:22:41.353 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:40.885+0000 7f86a3e7a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:22:41.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: Activating manager daemon x 2024-04-20T10:22:41.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: mgrmap e2: x(active, starting, since 0.00605129s) 2024-04-20T10:22:41.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: from='mgr.14100 172.21.15.42:0/387813682' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T10:22:41.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: from='mgr.14100 172.21.15.42:0/387813682' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T10:22:41.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: from='mgr.14100 172.21.15.42:0/387813682' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T10:22:41.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: from='mgr.14100 172.21.15.42:0/387813682' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:22:41.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: from='mgr.14100 172.21.15.42:0/387813682' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T10:22:41.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: Manager daemon x is now available 2024-04-20T10:22:41.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: from='mgr.14100 172.21.15.42:0/387813682' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:22:41.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: from='mgr.14100 172.21.15.42:0/387813682' entity='mgr.x' 2024-04-20T10:22:41.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: from='mgr.14100 172.21.15.42:0/387813682' entity='mgr.x' 2024-04-20T10:22:41.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: from='mgr.14100 172.21.15.42:0/387813682' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T10:22:41.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:40 smithi042 ceph-mon[17078]: from='mgr.14100 172.21.15.42:0/387813682' entity='mgr.x' 2024-04-20T10:22:43.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:42 smithi042 ceph-mon[17078]: mgrmap e3: x(active, since 1.02233s) 2024-04-20T10:22:43.757 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:43.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:22:43.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "83e8f766-feff-11ee-bc92-c7b262605968", 2024-04-20T10:22:43.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T10:22:43.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T10:22:43.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T10:22:43.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T10:22:43.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:43.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T10:22:43.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T10:22:43.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-04-20T10:22:43.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:22:43.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T10:22:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:43.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:43.761 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T10:22:21.222435+0000", 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.698+0000 7f1c3e800700 1 Processor -- start 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.698+0000 7f1c3e800700 1 -- start start 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.699+0000 7f1c3e800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 0x7f1c38110490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.699+0000 7f1c3e800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1c38110a60 con 0x7f1c38110070 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.699+0000 7f1c37400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 0x7f1c38110490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.699+0000 7f1c37400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 0x7f1c38110490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39902/0 (socket says 172.21.15.42:39902) 2024-04-20T10:22:43.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.699+0000 7f1c37400700 1 -- 172.21.15.42:0/1139932899 learned_addr learned my addr 172.21.15.42:0/1139932899 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.700+0000 7f1c37400700 1 -- 172.21.15.42:0/1139932899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3810ecd0 con 0x7f1c38110070 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.700+0000 7f1c37400700 1 --2- 172.21.15.42:0/1139932899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 0x7f1c38110490 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1c280098a0 tx=0x7f1c28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53c714fb1772629 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.700+0000 7f1c36a00700 1 -- 172.21.15.42:0/1139932899 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c28004160 con 0x7f1c38110070 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.700+0000 7f1c36a00700 1 -- 172.21.15.42:0/1139932899 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c28004300 con 0x7f1c38110070 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.700+0000 7f1c36a00700 1 -- 172.21.15.42:0/1139932899 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c28011b50 con 0x7f1c38110070 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.701+0000 7f1c3e800700 1 -- 172.21.15.42:0/1139932899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 msgr2=0x7f1c38110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.701+0000 7f1c3e800700 1 --2- 172.21.15.42:0/1139932899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 0x7f1c38110490 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1c280098a0 tx=0x7f1c28009bb0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.701+0000 7f1c3e800700 1 -- 172.21.15.42:0/1139932899 shutdown_connections 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.701+0000 7f1c3e800700 1 --2- 172.21.15.42:0/1139932899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 0x7f1c38110490 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.701+0000 7f1c3e800700 1 -- 172.21.15.42:0/1139932899 >> 172.21.15.42:0/1139932899 conn(0x7f1c3807db90 msgr2=0x7f1c3807fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.701+0000 7f1c3e800700 1 -- 172.21.15.42:0/1139932899 shutdown_connections 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.701+0000 7f1c3e800700 1 -- 172.21.15.42:0/1139932899 wait complete. 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.702+0000 7f1c3e800700 1 Processor -- start 2024-04-20T10:22:43.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.703+0000 7f1c3e800700 1 -- start start 2024-04-20T10:22:43.764 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.703+0000 7f1c3e800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 0x7f1c381a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:43.764 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.703+0000 7f1c3e800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1c38113090 con 0x7f1c38110070 2024-04-20T10:22:43.764 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.703+0000 7f1c37400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 0x7f1c381a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:43.764 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.703+0000 7f1c37400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 0x7f1c381a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39908/0 (socket says 172.21.15.42:39908) 2024-04-20T10:22:43.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.703+0000 7f1c37400700 1 -- 172.21.15.42:0/1972254634 learned_addr learned my addr 172.21.15.42:0/1972254634 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:43.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.704+0000 7f1c37400700 1 -- 172.21.15.42:0/1972254634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c381a2f90 con 0x7f1c38110070 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.704+0000 7f1c37400700 1 --2- 172.21.15.42:0/1972254634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 0x7f1c381a2a50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1c28011850 tx=0x7f1c28011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.704+0000 7f1c34c00700 1 -- 172.21.15.42:0/1972254634 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c28003af0 con 0x7f1c38110070 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.704+0000 7f1c34c00700 1 -- 172.21.15.42:0/1972254634 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c28020070 con 0x7f1c38110070 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.704+0000 7f1c3e800700 1 -- 172.21.15.42:0/1972254634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c381a3220 con 0x7f1c38110070 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.704+0000 7f1c34c00700 1 -- 172.21.15.42:0/1972254634 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c280195a0 con 0x7f1c38110070 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.705+0000 7f1c3e800700 1 -- 172.21.15.42:0/1972254634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c381a5f10 con 0x7f1c38110070 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.706+0000 7f1c34c00700 1 -- 172.21.15.42:0/1972254634 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7f1c280198a0 con 0x7f1c38110070 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.706+0000 7f1c3e800700 1 -- 172.21.15.42:0/1972254634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c380640b0 con 0x7f1c38110070 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.706+0000 7f1c34c00700 1 --2- 172.21.15.42:0/1972254634 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f1c1c039440 0x7f1c1c03b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.706+0000 7f1c34c00700 1 -- 172.21.15.42:0/1972254634 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1c28011e00 con 0x7f1c38110070 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.706+0000 7f1c2fe00700 1 --2- 172.21.15.42:0/1972254634 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f1c1c039440 0x7f1c1c03b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.707+0000 7f1c2fe00700 1 --2- 172.21.15.42:0/1972254634 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f1c1c039440 0x7f1c1c03b900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1c20006fd0 tx=0x7f1c20006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:43.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.713+0000 7f1c34c00700 1 -- 172.21.15.42:0/1972254634 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1c2800e260 con 0x7f1c38110070 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:42.905+0000 7f1c34c00700 1 -- 172.21.15.42:0/1972254634 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f1c2801a430 con 0x7f1c38110070 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:43.032+0000 7f1c3e800700 1 -- 172.21.15.42:0/1972254634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1c381a60e0 con 0x7f1c38110070 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:43.033+0000 7f1c34c00700 1 -- 172.21.15.42:0/1972254634 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1c381a60e0 con 0x7f1c38110070 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:43.036+0000 7f1c3e800700 1 -- 172.21.15.42:0/1972254634 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f1c1c039440 msgr2=0x7f1c1c03b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:43.036+0000 7f1c3e800700 1 --2- 172.21.15.42:0/1972254634 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f1c1c039440 0x7f1c1c03b900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1c20006fd0 tx=0x7f1c20006e40 comp rx=0 tx=0).stop 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:43.036+0000 7f1c3e800700 1 -- 172.21.15.42:0/1972254634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 msgr2=0x7f1c381a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:43.036+0000 7f1c3e800700 1 --2- 172.21.15.42:0/1972254634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 0x7f1c381a2a50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1c28011850 tx=0x7f1c28011c10 comp rx=0 tx=0).stop 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:43.036+0000 7f1c3e800700 1 -- 172.21.15.42:0/1972254634 shutdown_connections 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:43.036+0000 7f1c3e800700 1 --2- 172.21.15.42:0/1972254634 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f1c1c039440 0x7f1c1c03b900 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:43.036+0000 7f1c3e800700 1 --2- 172.21.15.42:0/1972254634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c38110070 0x7f1c381a2a50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:43.036+0000 7f1c3e800700 1 -- 172.21.15.42:0/1972254634 >> 172.21.15.42:0/1972254634 conn(0x7f1c3807db90 msgr2=0x7f1c3807f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:43.036+0000 7f1c3e800700 1 -- 172.21.15.42:0/1972254634 shutdown_connections 2024-04-20T10:22:43.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:43.036+0000 7f1c3e800700 1 -- 172.21.15.42:0/1972254634 wait complete. 2024-04-20T10:22:43.771 INFO:teuthology.orchestra.run.smithi042.stdout:mgr is available 2024-04-20T10:22:44.068 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:43 smithi042 ceph-mon[17078]: mgrmap e4: x(active, since 2s) 2024-04-20T10:22:44.068 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:43 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1972254634' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T10:22:45.328 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:44 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2555774055' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-20T10:22:45.337 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2024-04-20T10:22:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:22:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-20T10:22:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-20T10:22:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-20T10:22:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-20T10:22:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-20T10:22:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-20T10:22:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-20T10:22:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2024-04-20T10:22:45.339 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-20T10:22:45.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-20T10:22:45.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.374+0000 7fdaba400700 1 Processor -- start 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.375+0000 7fdaba400700 1 -- start start 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.375+0000 7fdaba400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 0x7fdab410aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.375+0000 7fdaba400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdab410b0c0 con 0x7fdab410a6d0 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.376+0000 7fdab3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 0x7fdab410aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.376+0000 7fdab3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 0x7fdab410aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39918/0 (socket says 172.21.15.42:39918) 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.376+0000 7fdab3400700 1 -- 172.21.15.42:0/3397233833 learned_addr learned my addr 172.21.15.42:0/3397233833 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.376+0000 7fdab3400700 1 -- 172.21.15.42:0/3397233833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab410b880 con 0x7fdab410a6d0 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.377+0000 7fdab3400700 1 --2- 172.21.15.42:0/3397233833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 0x7fdab410aaf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40098a0 tx=0x7fdaa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a965c66a0bf65397 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.377+0000 7fdab2000700 1 -- 172.21.15.42:0/3397233833 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaa4004160 con 0x7fdab410a6d0 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.377+0000 7fdab2000700 1 -- 172.21.15.42:0/3397233833 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdaa4004300 con 0x7fdab410a6d0 2024-04-20T10:22:45.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.377+0000 7fdab2000700 1 -- 172.21.15.42:0/3397233833 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaa4011b50 con 0x7fdab410a6d0 2024-04-20T10:22:45.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.378+0000 7fdaba400700 1 -- 172.21.15.42:0/3397233833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 msgr2=0x7fdab410aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.378+0000 7fdaba400700 1 --2- 172.21.15.42:0/3397233833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 0x7fdab410aaf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40098a0 tx=0x7fdaa4009bb0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.378+0000 7fdaba400700 1 -- 172.21.15.42:0/3397233833 shutdown_connections 2024-04-20T10:22:45.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.378+0000 7fdaba400700 1 --2- 172.21.15.42:0/3397233833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 0x7fdab410aaf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.378+0000 7fdaba400700 1 -- 172.21.15.42:0/3397233833 >> 172.21.15.42:0/3397233833 conn(0x7fdab407de10 msgr2=0x7fdab41084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:45.349 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.378+0000 7fdaba400700 1 -- 172.21.15.42:0/3397233833 shutdown_connections 2024-04-20T10:22:45.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.378+0000 7fdaba400700 1 -- 172.21.15.42:0/3397233833 wait complete. 2024-04-20T10:22:45.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.379+0000 7fdaba400700 1 Processor -- start 2024-04-20T10:22:45.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.379+0000 7fdaba400700 1 -- start start 2024-04-20T10:22:45.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.380+0000 7fdaba400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 0x7fdab419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.380+0000 7fdaba400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdab410ec30 con 0x7fdab410a6d0 2024-04-20T10:22:45.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.380+0000 7fdab3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 0x7fdab419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.350 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.380+0000 7fdab3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 0x7fdab419e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39922/0 (socket says 172.21.15.42:39922) 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.380+0000 7fdab3400700 1 -- 172.21.15.42:0/2555774055 learned_addr learned my addr 172.21.15.42:0/2555774055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.380+0000 7fdab3400700 1 -- 172.21.15.42:0/2555774055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab419ebb0 con 0x7fdab410a6d0 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.381+0000 7fdab3400700 1 --2- 172.21.15.42:0/2555774055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 0x7fdab419e670 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4011850 tx=0x7fdaa4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.381+0000 7fdaabe00700 1 -- 172.21.15.42:0/2555774055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaa4003af0 con 0x7fdab410a6d0 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.381+0000 7fdaabe00700 1 -- 172.21.15.42:0/2555774055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdaa4020070 con 0x7fdab410a6d0 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.381+0000 7fdaabe00700 1 -- 172.21.15.42:0/2555774055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaa40195a0 con 0x7fdab410a6d0 2024-04-20T10:22:45.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.381+0000 7fdaba400700 1 -- 172.21.15.42:0/2555774055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab419ee40 con 0x7fdab410a6d0 2024-04-20T10:22:45.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.381+0000 7fdaba400700 1 -- 172.21.15.42:0/2555774055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab41a1b30 con 0x7fdab410a6d0 2024-04-20T10:22:45.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.382+0000 7fdaabe00700 1 -- 172.21.15.42:0/2555774055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fdaa4019850 con 0x7fdab410a6d0 2024-04-20T10:22:45.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.382+0000 7fdaba400700 1 -- 172.21.15.42:0/2555774055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab40640b0 con 0x7fdab410a6d0 2024-04-20T10:22:45.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.383+0000 7fdaabe00700 1 --2- 172.21.15.42:0/2555774055 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7fda980395b0 0x7fda9803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.383+0000 7fdaabe00700 1 -- 172.21.15.42:0/2555774055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdaa4011e00 con 0x7fdab410a6d0 2024-04-20T10:22:45.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.383+0000 7fdab2a00700 1 --2- 172.21.15.42:0/2555774055 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7fda980395b0 0x7fda9803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.384+0000 7fdab2a00700 1 --2- 172.21.15.42:0/2555774055 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7fda980395b0 0x7fda9803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0006fd0 tx=0x7fdaa0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:45.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.389+0000 7fdaabe00700 1 -- 172.21.15.42:0/2555774055 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdaa400e260 con 0x7fdab410a6d0 2024-04-20T10:22:45.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.617+0000 7fdaba400700 1 -- 172.21.15.42:0/2555774055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fdab41a1d00 con 0x7fdab410a6d0 2024-04-20T10:22:45.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.619+0000 7fdaabe00700 1 -- 172.21.15.42:0/2555774055 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fdab41a1d00 con 0x7fdab410a6d0 2024-04-20T10:22:45.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.622+0000 7fdaba400700 1 -- 172.21.15.42:0/2555774055 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7fda980395b0 msgr2=0x7fda9803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.622+0000 7fdaba400700 1 --2- 172.21.15.42:0/2555774055 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7fda980395b0 0x7fda9803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0006fd0 tx=0x7fdaa0006e40 comp rx=0 tx=0).stop 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.622+0000 7fdaba400700 1 -- 172.21.15.42:0/2555774055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 msgr2=0x7fdab419e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.622+0000 7fdaba400700 1 --2- 172.21.15.42:0/2555774055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 0x7fdab419e670 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4011850 tx=0x7fdaa4011c10 comp rx=0 tx=0).stop 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.622+0000 7fdaba400700 1 -- 172.21.15.42:0/2555774055 shutdown_connections 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.622+0000 7fdaba400700 1 --2- 172.21.15.42:0/2555774055 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7fda980395b0 0x7fda9803ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.622+0000 7fdaba400700 1 --2- 172.21.15.42:0/2555774055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdab410a6d0 0x7fdab419e670 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.622+0000 7fdaba400700 1 -- 172.21.15.42:0/2555774055 >> 172.21.15.42:0/2555774055 conn(0x7fdab407de10 msgr2=0x7fdab4107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.622+0000 7fdaba400700 1 -- 172.21.15.42:0/2555774055 shutdown_connections 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:44.622+0000 7fdaba400700 1 -- 172.21.15.42:0/2555774055 wait complete. 2024-04-20T10:22:45.355 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling cephadm module... 2024-04-20T10:22:46.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:46 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3137253655' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-20T10:22:46.603 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:46 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: ignoring --setuser ceph since I am not root 2024-04-20T10:22:46.603 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:46 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: ignoring --setgroup ceph since I am not root 2024-04-20T10:22:46.904 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:46 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:46.652+0000 7fc7234ee200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:22:46.904 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:46 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:46.902+0000 7fc7234ee200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:22:47.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.982+0000 7ff44e600700 1 Processor -- start 2024-04-20T10:22:47.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.982+0000 7ff44e600700 1 -- start start 2024-04-20T10:22:47.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.982+0000 7ff44e600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 0x7ff44807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.983+0000 7ff44e600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff44807c8f0 con 0x7ff44807de70 2024-04-20T10:22:47.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.983+0000 7ff44d200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 0x7ff44807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.983+0000 7ff44d200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 0x7ff44807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39932/0 (socket says 172.21.15.42:39932) 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.983+0000 7ff44d200700 1 -- 172.21.15.42:0/1599231826 learned_addr learned my addr 172.21.15.42:0/1599231826 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.983+0000 7ff44d200700 1 -- 172.21.15.42:0/1599231826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44807ca70 con 0x7ff44807de70 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.984+0000 7ff44d200700 1 --2- 172.21.15.42:0/1599231826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 0x7ff44807c320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0098a0 tx=0x7ff43c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ff7ffab05a81e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.984+0000 7ff447400700 1 -- 172.21.15.42:0/1599231826 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff43c004160 con 0x7ff44807de70 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.984+0000 7ff447400700 1 -- 172.21.15.42:0/1599231826 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff43c004300 con 0x7ff44807de70 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.984+0000 7ff447400700 1 -- 172.21.15.42:0/1599231826 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff43c011b50 con 0x7ff44807de70 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.985+0000 7ff44e600700 1 -- 172.21.15.42:0/1599231826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 msgr2=0x7ff44807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.985+0000 7ff44e600700 1 --2- 172.21.15.42:0/1599231826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 0x7ff44807c320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0098a0 tx=0x7ff43c009bb0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.985+0000 7ff44e600700 1 -- 172.21.15.42:0/1599231826 shutdown_connections 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.985+0000 7ff44e600700 1 --2- 172.21.15.42:0/1599231826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 0x7ff44807c320 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.985+0000 7ff44e600700 1 -- 172.21.15.42:0/1599231826 >> 172.21.15.42:0/1599231826 conn(0x7ff4481038b0 msgr2=0x7ff448105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.985+0000 7ff44e600700 1 -- 172.21.15.42:0/1599231826 shutdown_connections 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.985+0000 7ff44e600700 1 -- 172.21.15.42:0/1599231826 wait complete. 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.986+0000 7ff44e600700 1 Processor -- start 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.986+0000 7ff44e600700 1 -- start start 2024-04-20T10:22:47.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.987+0000 7ff44e600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 0x7ff44819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.987+0000 7ff44e600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff44810e880 con 0x7ff44807de70 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.987+0000 7ff44d200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 0x7ff44819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.987+0000 7ff44d200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 0x7ff44819e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39944/0 (socket says 172.21.15.42:39944) 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.987+0000 7ff44d200700 1 -- 172.21.15.42:0/3137253655 learned_addr learned my addr 172.21.15.42:0/3137253655 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.987+0000 7ff44d200700 1 -- 172.21.15.42:0/3137253655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44819e9d0 con 0x7ff44807de70 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.988+0000 7ff44d200700 1 --2- 172.21.15.42:0/3137253655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 0x7ff44819e490 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff43c011b60 tx=0x7ff43c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.988+0000 7ff445600700 1 -- 172.21.15.42:0/3137253655 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff43c003970 con 0x7ff44807de70 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.988+0000 7ff445600700 1 -- 172.21.15.42:0/3137253655 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff43c003b10 con 0x7ff44807de70 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.988+0000 7ff445600700 1 -- 172.21.15.42:0/3137253655 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff43c024610 con 0x7ff44807de70 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.988+0000 7ff44e600700 1 -- 172.21.15.42:0/3137253655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44819ec60 con 0x7ff44807de70 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.988+0000 7ff44e600700 1 -- 172.21.15.42:0/3137253655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4481a1950 con 0x7ff44807de70 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.989+0000 7ff44e600700 1 -- 172.21.15.42:0/3137253655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4480640b0 con 0x7ff44807de70 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.989+0000 7ff445600700 1 -- 172.21.15.42:0/3137253655 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ff43c01a9f0 con 0x7ff44807de70 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.990+0000 7ff445600700 1 --2- 172.21.15.42:0/3137253655 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7ff4300395b0 0x7ff43003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.990+0000 7ff445600700 1 -- 172.21.15.42:0/3137253655 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff43c011e00 con 0x7ff44807de70 2024-04-20T10:22:47.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.990+0000 7ff447e00700 1 --2- 172.21.15.42:0/3137253655 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7ff4300395b0 0x7ff43003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.991+0000 7ff447e00700 1 --2- 172.21.15.42:0/3137253655 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7ff4300395b0 0x7ff43003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff434006fd0 tx=0x7ff434006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:47.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:45.996+0000 7ff445600700 1 -- 172.21.15.42:0/3137253655 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff43c016260 con 0x7ff44807de70 2024-04-20T10:22:47.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.287+0000 7ff44e600700 1 -- 172.21.15.42:0/3137253655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff4481a1b20 con 0x7ff44807de70 2024-04-20T10:22:47.074 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.341+0000 7ff445600700 1 -- 172.21.15.42:0/3137253655 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff4481a1b20 con 0x7ff44807de70 2024-04-20T10:22:47.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.345+0000 7ff44e600700 1 -- 172.21.15.42:0/3137253655 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7ff4300395b0 msgr2=0x7ff43003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.345+0000 7ff44e600700 1 --2- 172.21.15.42:0/3137253655 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7ff4300395b0 0x7ff43003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff434006fd0 tx=0x7ff434006e40 comp rx=0 tx=0).stop 2024-04-20T10:22:47.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.345+0000 7ff44e600700 1 -- 172.21.15.42:0/3137253655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 msgr2=0x7ff44819e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.345+0000 7ff445600700 1 -- 172.21.15.42:0/3137253655 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff43c024c40 con 0x7ff44807de70 2024-04-20T10:22:47.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.346+0000 7ff44e600700 1 --2- 172.21.15.42:0/3137253655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 0x7ff44819e490 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff43c011b60 tx=0x7ff43c011c40 comp rx=0 tx=0).stop 2024-04-20T10:22:47.075 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.346+0000 7ff44e600700 1 -- 172.21.15.42:0/3137253655 shutdown_connections 2024-04-20T10:22:47.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.346+0000 7ff44e600700 1 --2- 172.21.15.42:0/3137253655 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7ff4300395b0 0x7ff43003ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.346+0000 7ff44e600700 1 --2- 172.21.15.42:0/3137253655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff44807de70 0x7ff44819e490 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.346+0000 7ff44e600700 1 -- 172.21.15.42:0/3137253655 >> 172.21.15.42:0/3137253655 conn(0x7ff4481038b0 msgr2=0x7ff448104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:47.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.346+0000 7ff44e600700 1 -- 172.21.15.42:0/3137253655 shutdown_connections 2024-04-20T10:22:47.076 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:46.347+0000 7ff44e600700 1 -- 172.21.15.42:0/3137253655 wait complete. 2024-04-20T10:22:47.223 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:47 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:47.038+0000 7fc7234ee200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:22:47.486 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:47 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3137253655' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-20T10:22:47.486 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:47 smithi042 ceph-mon[17078]: mgrmap e5: x(active, since 5s) 2024-04-20T10:22:47.487 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:47 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:47.460+0000 7fc7234ee200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:22:47.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:47 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:47.578+0000 7fc7234ee200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:22:48.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:48 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/882876836' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-20T10:22:48.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:22:48.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-20T10:22:48.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T10:22:48.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-20T10:22:48.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-20T10:22:48.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:22:48.645 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.643+0000 7f8a71e00700 1 Processor -- start 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.643+0000 7f8a71e00700 1 -- start start 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.644+0000 7f8a71e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 0x7f8a6c10b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.644+0000 7f8a71e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c07be90 con 0x7f8a6c108d60 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.644+0000 7f8a70a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 0x7f8a6c10b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.644+0000 7f8a70a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 0x7f8a6c10b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39970/0 (socket says 172.21.15.42:39970) 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.644+0000 7f8a70a00700 1 -- 172.21.15.42:0/1093708347 learned_addr learned my addr 172.21.15.42:0/1093708347 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.645+0000 7f8a70a00700 1 -- 172.21.15.42:0/1093708347 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6c10b690 con 0x7f8a6c108d60 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.645+0000 7f8a70a00700 1 --2- 172.21.15.42:0/1093708347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 0x7f8a6c10b150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8a600098a0 tx=0x7f8a60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332cf9044e6e0098 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.645+0000 7f8a6b400700 1 -- 172.21.15.42:0/1093708347 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a60004160 con 0x7f8a6c108d60 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.645+0000 7f8a6b400700 1 -- 172.21.15.42:0/1093708347 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8a60004300 con 0x7f8a6c108d60 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.645+0000 7f8a6b400700 1 -- 172.21.15.42:0/1093708347 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a60011b50 con 0x7f8a6c108d60 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.646+0000 7f8a71e00700 1 -- 172.21.15.42:0/1093708347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 msgr2=0x7f8a6c10b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:48.646 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.646+0000 7f8a71e00700 1 --2- 172.21.15.42:0/1093708347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 0x7f8a6c10b150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8a600098a0 tx=0x7f8a60009bb0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.646+0000 7f8a71e00700 1 -- 172.21.15.42:0/1093708347 shutdown_connections 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.646+0000 7f8a71e00700 1 --2- 172.21.15.42:0/1093708347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 0x7f8a6c10b150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.646+0000 7f8a71e00700 1 -- 172.21.15.42:0/1093708347 >> 172.21.15.42:0/1093708347 conn(0x7f8a6c102790 msgr2=0x7f8a6c104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.646+0000 7f8a71e00700 1 -- 172.21.15.42:0/1093708347 shutdown_connections 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.647+0000 7f8a71e00700 1 -- 172.21.15.42:0/1093708347 wait complete. 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.647+0000 7f8a71e00700 1 Processor -- start 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.648+0000 7f8a71e00700 1 -- start start 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.648+0000 7f8a71e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 0x7f8a6c1085e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.648+0000 7f8a71e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c10c640 con 0x7f8a6c108d60 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.648+0000 7f8a70a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 0x7f8a6c1085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.648+0000 7f8a70a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 0x7f8a6c1085e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39982/0 (socket says 172.21.15.42:39982) 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.648+0000 7f8a70a00700 1 -- 172.21.15.42:0/882876836 learned_addr learned my addr 172.21.15.42:0/882876836 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.649+0000 7f8a70a00700 1 -- 172.21.15.42:0/882876836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6c108b20 con 0x7f8a6c108d60 2024-04-20T10:22:48.647 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.649+0000 7f8a70a00700 1 --2- 172.21.15.42:0/882876836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 0x7f8a6c1085e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8a60011b60 tx=0x7f8a60011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.649+0000 7f8a69600700 1 -- 172.21.15.42:0/882876836 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a60003970 con 0x7f8a6c108d60 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.649+0000 7f8a69600700 1 -- 172.21.15.42:0/882876836 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8a60003b10 con 0x7f8a6c108d60 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.649+0000 7f8a71e00700 1 -- 172.21.15.42:0/882876836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6c106d10 con 0x7f8a6c108d60 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.649+0000 7f8a69600700 1 -- 172.21.15.42:0/882876836 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a60024610 con 0x7f8a6c108d60 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.649+0000 7f8a71e00700 1 -- 172.21.15.42:0/882876836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6c107250 con 0x7f8a6c108d60 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.650+0000 7f8a69600700 1 -- 172.21.15.42:0/882876836 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f8a600248c0 con 0x7f8a6c108d60 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.651+0000 7f8a71e00700 1 -- 172.21.15.42:0/882876836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a6c0640b0 con 0x7f8a6c108d60 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.651+0000 7f8a69600700 1 --2- 172.21.15.42:0/882876836 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f8a54039600 0x7f8a5403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.651+0000 7f8a69600700 1 -- 172.21.15.42:0/882876836 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8a60011e00 con 0x7f8a6c108d60 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.651+0000 7f8a6be00700 1 -- 172.21.15.42:0/882876836 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f8a54039600 msgr2=0x7f8a5403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2556893432 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.651+0000 7f8a6be00700 1 --2- 172.21.15.42:0/882876836 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f8a54039600 0x7f8a5403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.656+0000 7f8a69600700 1 -- 172.21.15.42:0/882876836 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8a6004d390 con 0x7f8a6c108d60 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.852+0000 7f8a6be00700 1 -- 172.21.15.42:0/882876836 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f8a54039600 msgr2=0x7f8a5403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2556893432 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.852+0000 7f8a6be00700 1 --2- 172.21.15.42:0/882876836 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f8a54039600 0x7f8a5403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:22:48.648 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.945+0000 7f8a71e00700 1 -- 172.21.15.42:0/882876836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f8a6c1a2270 con 0x7f8a6c108d60 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.946+0000 7f8a69600700 1 -- 172.21.15.42:0/882876836 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f8a6002d910 con 0x7f8a6c108d60 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.950+0000 7f8a71e00700 1 -- 172.21.15.42:0/882876836 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f8a54039600 msgr2=0x7f8a5403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.951+0000 7f8a71e00700 1 --2- 172.21.15.42:0/882876836 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f8a54039600 0x7f8a5403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.951+0000 7f8a71e00700 1 -- 172.21.15.42:0/882876836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 msgr2=0x7f8a6c1085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.951+0000 7f8a71e00700 1 --2- 172.21.15.42:0/882876836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 0x7f8a6c1085e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8a60011b60 tx=0x7f8a60011c40 comp rx=0 tx=0).stop 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.951+0000 7f8a71e00700 1 -- 172.21.15.42:0/882876836 shutdown_connections 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.951+0000 7f8a71e00700 1 --2- 172.21.15.42:0/882876836 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f8a54039600 0x7f8a5403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.951+0000 7f8a71e00700 1 --2- 172.21.15.42:0/882876836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6c108d60 0x7f8a6c1085e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.951+0000 7f8a71e00700 1 -- 172.21.15.42:0/882876836 >> 172.21.15.42:0/882876836 conn(0x7f8a6c102790 msgr2=0x7f8a6c103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.951+0000 7f8a71e00700 1 -- 172.21.15.42:0/882876836 shutdown_connections 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:47.952+0000 7f8a71e00700 1 -- 172.21.15.42:0/882876836 wait complete. 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2024-04-20T10:22:48.649 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 5... 2024-04-20T10:22:49.603 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:49 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:49.205+0000 7fc7234ee200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:22:49.603 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:49 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:49.443+0000 7fc7234ee200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:22:50.061 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:49 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:49.663+0000 7fc7234ee200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:22:50.353 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:50.059+0000 7fc7234ee200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:22:50.353 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:50.186+0000 7fc7234ee200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:22:50.711 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:50.444+0000 7fc7234ee200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:22:50.711 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:50.709+0000 7fc7234ee200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:22:51.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:51 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:51.540+0000 7fc7234ee200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:22:52.523 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:52 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:52.113+0000 7fc7234ee200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:22:52.523 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:52 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:52.243+0000 7fc7234ee200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:22:52.524 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:52 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:52.521+0000 7fc7234ee200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:22:53.603 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:53 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:53.187+0000 7fc7234ee200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:22:53.603 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:53 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:53.317+0000 7fc7234ee200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:22:53.965 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:53 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:53.648+0000 7fc7234ee200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:22:53.966 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:53 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:53.833+0000 7fc7234ee200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:22:53.966 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:53 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:53.964+0000 7fc7234ee200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:22:54.353 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:54.121+0000 7fc7234ee200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:22:54.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:54.503+0000 7fc7234ee200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:22:55.590 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:55.292+0000 7fc7234ee200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:22:55.855 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:22:55.589+0000 7fc7234ee200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:22:55.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-mon[17078]: Active manager daemon x restarted 2024-04-20T10:22:55.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-mon[17078]: Activating manager daemon x 2024-04-20T10:22:55.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-mon[17078]: osdmap e2: 0 total, 0 up, 0 in 2024-04-20T10:22:55.856 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-mon[17078]: mgrmap e6: x(active, starting, since 0.00653781s) 2024-04-20T10:22:55.856 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:22:55.856 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T10:22:55.856 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T10:22:55.856 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T10:22:55.856 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T10:22:55.856 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-mon[17078]: Manager daemon x is now available 2024-04-20T10:22:55.856 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:22:55.856 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:55 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:22:57.007 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:56 smithi042 ceph-mon[17078]: Found migration_current of "None". Setting to last migration. 2024-04-20T10:22:57.007 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:56 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:22:57.007 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:56 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:22:57.007 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:56 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:22:57.007 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:56 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T10:22:57.007 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:56 smithi042 ceph-mon[17078]: mgrmap e7: x(active, since 1.01817s) 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.206+0000 7f38ada00700 1 Processor -- start 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.206+0000 7f38ada00700 1 -- start start 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.207+0000 7f38ada00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 0x7f38a810b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.207+0000 7f38ada00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f38a807be50 con 0x7f38a8108d90 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.207+0000 7f38a7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 0x7f38a810b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.207+0000 7f38a7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 0x7f38a810b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55482/0 (socket says 172.21.15.42:55482) 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.207+0000 7f38a7e00700 1 -- 172.21.15.42:0/3833970380 learned_addr learned my addr 172.21.15.42:0/3833970380 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.208+0000 7f38a7e00700 1 -- 172.21.15.42:0/3833970380 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a810b6c0 con 0x7f38a8108d90 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.208+0000 7f38a7e00700 1 --2- 172.21.15.42:0/3833970380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 0x7f38a810b180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38980098a0 tx=0x7f3898009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5417b0047148e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.208+0000 7f38a6a00700 1 -- 172.21.15.42:0/3833970380 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3898004160 con 0x7f38a8108d90 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.208+0000 7f38a6a00700 1 -- 172.21.15.42:0/3833970380 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3898004300 con 0x7f38a8108d90 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.208+0000 7f38a6a00700 1 -- 172.21.15.42:0/3833970380 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3898011b50 con 0x7f38a8108d90 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.209+0000 7f38ada00700 1 -- 172.21.15.42:0/3833970380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 msgr2=0x7f38a810b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.209+0000 7f38ada00700 1 --2- 172.21.15.42:0/3833970380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 0x7f38a810b180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38980098a0 tx=0x7f3898009bb0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.209+0000 7f38ada00700 1 -- 172.21.15.42:0/3833970380 shutdown_connections 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.209+0000 7f38ada00700 1 --2- 172.21.15.42:0/3833970380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 0x7f38a810b180 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.209+0000 7f38ada00700 1 -- 172.21.15.42:0/3833970380 >> 172.21.15.42:0/3833970380 conn(0x7f38a8102740 msgr2=0x7f38a8104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.209+0000 7f38ada00700 1 -- 172.21.15.42:0/3833970380 shutdown_connections 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.209+0000 7f38ada00700 1 -- 172.21.15.42:0/3833970380 wait complete. 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.210+0000 7f38ada00700 1 Processor -- start 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.211+0000 7f38ada00700 1 -- start start 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.211+0000 7f38ada00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 0x7f38a8108610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.211+0000 7f38ada00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f38a810c690 con 0x7f38a8108d90 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.211+0000 7f38a7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 0x7f38a8108610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.211+0000 7f38a7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 0x7f38a8108610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55484/0 (socket says 172.21.15.42:55484) 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.211+0000 7f38a7e00700 1 -- 172.21.15.42:0/3509764326 learned_addr learned my addr 172.21.15.42:0/3509764326 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.212+0000 7f38a7e00700 1 -- 172.21.15.42:0/3509764326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a8108b50 con 0x7f38a8108d90 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.212+0000 7f38a7e00700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 0x7f38a8108610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3898011b60 tx=0x7f3898011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.212+0000 7f38a4c00700 1 -- 172.21.15.42:0/3509764326 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3898003970 con 0x7f38a8108d90 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.212+0000 7f38a4c00700 1 -- 172.21.15.42:0/3509764326 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3898003b10 con 0x7f38a8108d90 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.212+0000 7f38a4c00700 1 -- 172.21.15.42:0/3509764326 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3898024610 con 0x7f38a8108d90 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.212+0000 7f38ada00700 1 -- 172.21.15.42:0/3509764326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a8106d40 con 0x7f38a8108d90 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.212+0000 7f38ada00700 1 -- 172.21.15.42:0/3509764326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a8107280 con 0x7f38a8108d90 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.213+0000 7f38a4c00700 1 -- 172.21.15.42:0/3509764326 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f389801a430 con 0x7f38a8108d90 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.214+0000 7f38a4c00700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 0x7f388c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.214+0000 7f38a4c00700 1 -- 172.21.15.42:0/3509764326 --> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f388c03c1d0 con 0x7f388c039600 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.214+0000 7f38a4c00700 1 -- 172.21.15.42:0/3509764326 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3898011e00 con 0x7f38a8108d90 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.214+0000 7f38a7400700 1 -- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 msgr2=0x7f388c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2556893432 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.214+0000 7f38a7400700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 0x7f388c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.415+0000 7f38a7400700 1 -- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 msgr2=0x7f388c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2556893432 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.415+0000 7f38a7400700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 0x7f388c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.816+0000 7f38a7400700 1 -- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 msgr2=0x7f388c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2556893432 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:49.816+0000 7f38a7400700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 0x7f388c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:50.617+0000 7f38a7400700 1 -- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 msgr2=0x7f388c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2556893432 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:50.617+0000 7f38a7400700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 0x7f388c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:52.219+0000 7f38a7400700 1 -- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 msgr2=0x7f388c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2556893432 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:52.219+0000 7f38a7400700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 0x7f388c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:55.422+0000 7f38a7400700 1 -- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 msgr2=0x7f388c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2556893432 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:55.422+0000 7f38a7400700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 0x7f388c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:55.597+0000 7f38a4c00700 1 -- 172.21.15.42:0/3509764326 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f389801a9f0 con 0x7f38a8108d90 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:55.598+0000 7f38a4c00700 1 -- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 msgr2=0x7f388c03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:55.598+0000 7f38a4c00700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 0x7f388c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.605+0000 7f38a4c00700 1 -- 172.21.15.42:0/3509764326 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f389804f250 con 0x7f38a8108d90 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.605+0000 7f38a4c00700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f388c03cef0 0x7f388c03f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.605+0000 7f38a4c00700 1 -- 172.21.15.42:0/3509764326 --> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f388c03c1d0 con 0x7f388c03cef0 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.606+0000 7f38a7400700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f388c03cef0 0x7f388c03f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.609+0000 7f38a7400700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f388c03cef0 0x7f388c03f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3894003a10 tx=0x7f38940092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.610+0000 7f38a4c00700 1 -- 172.21.15.42:0/3509764326 <== mgr.14120 v2:172.21.15.42:6800/2743527860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f388c03c1d0 con 0x7f388c03cef0 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.619+0000 7f389e000700 1 -- 172.21.15.42:0/3509764326 --> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3880002600 con 0x7f388c03cef0 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.620+0000 7f38a4c00700 1 -- 172.21.15.42:0/3509764326 <== mgr.14120 v2:172.21.15.42:6800/2743527860 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f3880002600 con 0x7f388c03cef0 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.620+0000 7f389e000700 1 -- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f388c03cef0 msgr2=0x7f388c03f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:57.332 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.620+0000 7f389e000700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f388c03cef0 0x7f388c03f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3894003a10 tx=0x7f38940092b0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.620+0000 7f389e000700 1 -- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 msgr2=0x7f38a8108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.620+0000 7f389e000700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 0x7f38a8108610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3898011b60 tx=0x7f3898011c40 comp rx=0 tx=0).stop 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.621+0000 7f389e000700 1 -- 172.21.15.42:0/3509764326 shutdown_connections 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.621+0000 7f389e000700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f388c03cef0 0x7f388c03f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.621+0000 7f389e000700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38a8108d90 0x7f38a8108610 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.621+0000 7f389e000700 1 --2- 172.21.15.42:0/3509764326 >> [v2:172.21.15.42:6800/2556893432,v1:172.21.15.42:6801/2556893432] conn(0x7f388c039600 0x7f388c03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.621+0000 7f389e000700 1 -- 172.21.15.42:0/3509764326 >> 172.21.15.42:0/3509764326 conn(0x7f38a8102740 msgr2=0x7f38a8103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.621+0000 7f389e000700 1 -- 172.21.15.42:0/3509764326 shutdown_connections 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:56.621+0000 7f389e000700 1 -- 172.21.15.42:0/3509764326 wait complete. 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 5 is available 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi042.stdout:Setting orchestrator backend to cephadm... 2024-04-20T10:22:58.180 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:57 smithi042 ceph-mon[17078]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-20T10:22:58.180 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:57 smithi042 ceph-mon[17078]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-20T10:22:58.180 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:57 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:22:58.180 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:57 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:22:58.180 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:57 smithi042 ceph-mon[17078]: [20/Apr/2024:10:22:57] ENGINE Bus STARTING 2024-04-20T10:22:58.180 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:57 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:22:58.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.925+0000 7f4d0ae00700 1 Processor -- start 2024-04-20T10:22:58.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.926+0000 7f4d0ae00700 1 -- start start 2024-04-20T10:22:58.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.926+0000 7f4d0ae00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 0x7f4d0410aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:58.850 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.926+0000 7f4d0ae00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0410b060 con 0x7f4d0410a670 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.927+0000 7f4d03e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 0x7f4d0410aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.927+0000 7f4d03e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 0x7f4d0410aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55550/0 (socket says 172.21.15.42:55550) 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.927+0000 7f4d03e00700 1 -- 172.21.15.42:0/307738330 learned_addr learned my addr 172.21.15.42:0/307738330 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.927+0000 7f4d03e00700 1 -- 172.21.15.42:0/307738330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0410b870 con 0x7f4d0410a670 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.927+0000 7f4d03e00700 1 --2- 172.21.15.42:0/307738330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 0x7f4d0410aa90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40098a0 tx=0x7f4cf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc360f0d1158173f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.928+0000 7f4d02a00700 1 -- 172.21.15.42:0/307738330 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cf4004160 con 0x7f4d0410a670 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.928+0000 7f4d02a00700 1 -- 172.21.15.42:0/307738330 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4cf4004300 con 0x7f4d0410a670 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.928+0000 7f4d02a00700 1 -- 172.21.15.42:0/307738330 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cf4011b50 con 0x7f4d0410a670 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.928+0000 7f4d0ae00700 1 -- 172.21.15.42:0/307738330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 msgr2=0x7f4d0410aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.928+0000 7f4d0ae00700 1 --2- 172.21.15.42:0/307738330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 0x7f4d0410aa90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40098a0 tx=0x7f4cf4009bb0 comp rx=0 tx=0).stop 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.929+0000 7f4d0ae00700 1 -- 172.21.15.42:0/307738330 shutdown_connections 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.929+0000 7f4d0ae00700 1 --2- 172.21.15.42:0/307738330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 0x7f4d0410aa90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.929+0000 7f4d0ae00700 1 -- 172.21.15.42:0/307738330 >> 172.21.15.42:0/307738330 conn(0x7f4d0407db60 msgr2=0x7f4d041084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:58.851 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.929+0000 7f4d0ae00700 1 -- 172.21.15.42:0/307738330 shutdown_connections 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.929+0000 7f4d0ae00700 1 -- 172.21.15.42:0/307738330 wait complete. 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.930+0000 7f4d0ae00700 1 Processor -- start 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.930+0000 7f4d0ae00700 1 -- start start 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.930+0000 7f4d0ae00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 0x7f4d0419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.930+0000 7f4d0ae00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0410ec20 con 0x7f4d0410a670 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.930+0000 7f4d03e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 0x7f4d0419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.930+0000 7f4d03e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 0x7f4d0419e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55566/0 (socket says 172.21.15.42:55566) 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.930+0000 7f4d03e00700 1 -- 172.21.15.42:0/2605724126 learned_addr learned my addr 172.21.15.42:0/2605724126 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.931+0000 7f4d03e00700 1 -- 172.21.15.42:0/2605724126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0419eb80 con 0x7f4d0410a670 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.931+0000 7f4d03e00700 1 --2- 172.21.15.42:0/2605724126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 0x7f4d0419e640 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4011850 tx=0x7f4cf4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.931+0000 7f4d00c00700 1 -- 172.21.15.42:0/2605724126 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cf4003af0 con 0x7f4d0410a670 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.931+0000 7f4d00c00700 1 -- 172.21.15.42:0/2605724126 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4cf4020070 con 0x7f4d0410a670 2024-04-20T10:22:58.852 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.931+0000 7f4d0ae00700 1 -- 172.21.15.42:0/2605724126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0419ee10 con 0x7f4d0410a670 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.931+0000 7f4d00c00700 1 -- 172.21.15.42:0/2605724126 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cf40195a0 con 0x7f4d0410a670 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.932+0000 7f4d0ae00700 1 -- 172.21.15.42:0/2605724126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d041a1b00 con 0x7f4d0410a670 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.933+0000 7f4d00c00700 1 -- 172.21.15.42:0/2605724126 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f4cf401a430 con 0x7f4d0410a670 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.933+0000 7f4d0ae00700 1 -- 172.21.15.42:0/2605724126 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d040640b0 con 0x7f4d0410a670 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.933+0000 7f4d00c00700 1 --2- 172.21.15.42:0/2605724126 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f4ce80394e0 0x7f4ce803b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.933+0000 7f4d00c00700 1 -- 172.21.15.42:0/2605724126 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4cf4011e00 con 0x7f4d0410a670 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.933+0000 7f4d03400700 1 --2- 172.21.15.42:0/2605724126 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f4ce80394e0 0x7f4ce803b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.934+0000 7f4d03400700 1 --2- 172.21.15.42:0/2605724126 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f4ce80394e0 0x7f4ce803b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0006fd0 tx=0x7f4cf0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.939+0000 7f4d00c00700 1 -- 172.21.15.42:0/2605724126 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4cf400e260 con 0x7f4d0410a670 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:57.995+0000 7f4d00c00700 1 -- 172.21.15.42:0/2605724126 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4cf4019850 con 0x7f4d0410a670 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:58.178+0000 7f4d0ae00700 1 -- 172.21.15.42:0/2605724126 --> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f4d041083a0 con 0x7f4ce80394e0 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:58.190+0000 7f4d00c00700 1 -- 172.21.15.42:0/2605724126 <== mgr.14120 v2:172.21.15.42:6800/2743527860 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4d041083a0 con 0x7f4ce80394e0 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:58.193+0000 7f4d0ae00700 1 -- 172.21.15.42:0/2605724126 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f4ce80394e0 msgr2=0x7f4ce803b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:58.853 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:58.193+0000 7f4d0ae00700 1 --2- 172.21.15.42:0/2605724126 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f4ce80394e0 0x7f4ce803b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0006fd0 tx=0x7f4cf0006e40 comp rx=0 tx=0).stop 2024-04-20T10:22:58.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:58.193+0000 7f4d0ae00700 1 -- 172.21.15.42:0/2605724126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 msgr2=0x7f4d0419e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:58.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:58.193+0000 7f4d0ae00700 1 --2- 172.21.15.42:0/2605724126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 0x7f4d0419e640 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4011850 tx=0x7f4cf4011c10 comp rx=0 tx=0).stop 2024-04-20T10:22:58.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:58.193+0000 7f4d0ae00700 1 -- 172.21.15.42:0/2605724126 shutdown_connections 2024-04-20T10:22:58.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:58.193+0000 7f4d0ae00700 1 --2- 172.21.15.42:0/2605724126 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f4ce80394e0 0x7f4ce803b9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:58.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:58.193+0000 7f4d0ae00700 1 --2- 172.21.15.42:0/2605724126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d0410a670 0x7f4d0419e640 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:58.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:58.193+0000 7f4d0ae00700 1 -- 172.21.15.42:0/2605724126 >> 172.21.15.42:0/2605724126 conn(0x7f4d0407db60 msgr2=0x7f4d04107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:58.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:58.193+0000 7f4d0ae00700 1 -- 172.21.15.42:0/2605724126 shutdown_connections 2024-04-20T10:22:58.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:58.194+0000 7f4d0ae00700 1 -- 172.21.15.42:0/2605724126 wait complete. 2024-04-20T10:22:59.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:58 smithi042 ceph-mon[17078]: [20/Apr/2024:10:22:57] ENGINE Serving on http://172.21.15.42:8765 2024-04-20T10:22:59.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:58 smithi042 ceph-mon[17078]: [20/Apr/2024:10:22:57] ENGINE Serving on https://172.21.15.42:7150 2024-04-20T10:22:59.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:58 smithi042 ceph-mon[17078]: [20/Apr/2024:10:22:57] ENGINE Bus STARTED 2024-04-20T10:22:59.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:58 smithi042 ceph-mon[17078]: mgrmap e8: x(active, since 2s) 2024-04-20T10:22:59.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:58 smithi042 ceph-mon[17078]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:22:59.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:58 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:22:59.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:22:58 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:23:00.471 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-20T10:23:00.472 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.539+0000 7f05f6c00700 1 Processor -- start 2024-04-20T10:23:00.472 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.539+0000 7f05f6c00700 1 -- start start 2024-04-20T10:23:00.472 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.539+0000 7f05f6c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 0x7f05f010b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:00.473 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.540+0000 7f05f6c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f05f007beb0 con 0x7f05f0108d40 2024-04-20T10:23:00.473 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.540+0000 7f05f5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 0x7f05f010b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:00.473 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.540+0000 7f05f5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 0x7f05f010b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44734/0 (socket says 172.21.15.42:44734) 2024-04-20T10:23:00.474 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.540+0000 7f05f5800700 1 -- 172.21.15.42:0/2600363732 learned_addr learned my addr 172.21.15.42:0/2600363732 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:00.474 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.540+0000 7f05f5800700 1 -- 172.21.15.42:0/2600363732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f010b670 con 0x7f05f0108d40 2024-04-20T10:23:00.474 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.541+0000 7f05f5800700 1 --2- 172.21.15.42:0/2600363732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 0x7f05f010b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f05e40098a0 tx=0x7f05e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a13ac299ac99e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:00.474 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.541+0000 7f05efe00700 1 -- 172.21.15.42:0/2600363732 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4004160 con 0x7f05f0108d40 2024-04-20T10:23:00.475 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.541+0000 7f05efe00700 1 -- 172.21.15.42:0/2600363732 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05e4004300 con 0x7f05f0108d40 2024-04-20T10:23:00.475 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.541+0000 7f05efe00700 1 -- 172.21.15.42:0/2600363732 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4011b50 con 0x7f05f0108d40 2024-04-20T10:23:00.475 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.541+0000 7f05f6c00700 1 -- 172.21.15.42:0/2600363732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 msgr2=0x7f05f010b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:00.475 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.541+0000 7f05f6c00700 1 --2- 172.21.15.42:0/2600363732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 0x7f05f010b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f05e40098a0 tx=0x7f05e4009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:00.475 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.542+0000 7f05f6c00700 1 -- 172.21.15.42:0/2600363732 shutdown_connections 2024-04-20T10:23:00.476 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.542+0000 7f05f6c00700 1 --2- 172.21.15.42:0/2600363732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 0x7f05f010b130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:00.476 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.542+0000 7f05f6c00700 1 -- 172.21.15.42:0/2600363732 >> 172.21.15.42:0/2600363732 conn(0x7f05f0102790 msgr2=0x7f05f0104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:00.476 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.542+0000 7f05f6c00700 1 -- 172.21.15.42:0/2600363732 shutdown_connections 2024-04-20T10:23:00.476 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.542+0000 7f05f6c00700 1 -- 172.21.15.42:0/2600363732 wait complete. 2024-04-20T10:23:00.477 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.543+0000 7f05f6c00700 1 Processor -- start 2024-04-20T10:23:00.477 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.543+0000 7f05f6c00700 1 -- start start 2024-04-20T10:23:00.477 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.543+0000 7f05f6c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 0x7f05f019e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:00.477 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.543+0000 7f05f6c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f05f010c6a0 con 0x7f05f0108d40 2024-04-20T10:23:00.477 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.543+0000 7f05f5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 0x7f05f019e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:00.477 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.544+0000 7f05f5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 0x7f05f019e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44744/0 (socket says 172.21.15.42:44744) 2024-04-20T10:23:00.477 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.544+0000 7f05f5800700 1 -- 172.21.15.42:0/18268866 learned_addr learned my addr 172.21.15.42:0/18268866 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:00.479 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.544+0000 7f05f5800700 1 -- 172.21.15.42:0/18268866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f019e9e0 con 0x7f05f0108d40 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.544+0000 7f05f5800700 1 --2- 172.21.15.42:0/18268866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 0x7f05f019e4a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f05e4011b60 tx=0x7f05e4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.545+0000 7f05ee000700 1 -- 172.21.15.42:0/18268866 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4003970 con 0x7f05f0108d40 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.545+0000 7f05ee000700 1 -- 172.21.15.42:0/18268866 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05e4003b10 con 0x7f05f0108d40 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.545+0000 7f05f6c00700 1 -- 172.21.15.42:0/18268866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f019ec70 con 0x7f05f0108d40 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.545+0000 7f05ee000700 1 -- 172.21.15.42:0/18268866 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4024610 con 0x7f05f0108d40 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.545+0000 7f05f6c00700 1 -- 172.21.15.42:0/18268866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01a1960 con 0x7f05f0108d40 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.546+0000 7f05ee000700 1 -- 172.21.15.42:0/18268866 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f05e401a9f0 con 0x7f05f0108d40 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.546+0000 7f05f6c00700 1 -- 172.21.15.42:0/18268866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f00640b0 con 0x7f05f0108d40 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.546+0000 7f05ee000700 1 --2- 172.21.15.42:0/18268866 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f05d80395b0 0x7f05d803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.546+0000 7f05ee000700 1 -- 172.21.15.42:0/18268866 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f05e4011e00 con 0x7f05f0108d40 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.547+0000 7f05f4e00700 1 --2- 172.21.15.42:0/18268866 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f05d80395b0 0x7f05d803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.547+0000 7f05f4e00700 1 --2- 172.21.15.42:0/18268866 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f05d80395b0 0x7f05d803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f05dc006fd0 tx=0x7f05dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.557+0000 7f05ee000700 1 -- 172.21.15.42:0/18268866 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f05e400b3e0 con 0x7f05f0108d40 2024-04-20T10:23:00.480 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.785+0000 7f05f6c00700 1 -- 172.21.15.42:0/18268866 --> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f05f0103920 con 0x7f05d80395b0 2024-04-20T10:23:00.481 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.787+0000 7f05ee000700 1 -- 172.21.15.42:0/18268866 <== mgr.14120 v2:172.21.15.42:6800/2743527860 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f05f0103920 con 0x7f05d80395b0 2024-04-20T10:23:00.481 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.790+0000 7f05f6c00700 1 -- 172.21.15.42:0/18268866 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f05d80395b0 msgr2=0x7f05d803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:00.481 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.790+0000 7f05f6c00700 1 --2- 172.21.15.42:0/18268866 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f05d80395b0 0x7f05d803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f05dc006fd0 tx=0x7f05dc006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:00.481 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.790+0000 7f05f6c00700 1 -- 172.21.15.42:0/18268866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 msgr2=0x7f05f019e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:00.481 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.790+0000 7f05f6c00700 1 --2- 172.21.15.42:0/18268866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 0x7f05f019e4a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f05e4011b60 tx=0x7f05e4011c40 comp rx=0 tx=0).stop 2024-04-20T10:23:00.481 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.790+0000 7f05f6c00700 1 -- 172.21.15.42:0/18268866 shutdown_connections 2024-04-20T10:23:00.481 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.790+0000 7f05f6c00700 1 --2- 172.21.15.42:0/18268866 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f05d80395b0 0x7f05d803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:00.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.790+0000 7f05f6c00700 1 --2- 172.21.15.42:0/18268866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0108d40 0x7f05f019e4a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:00.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.790+0000 7f05f6c00700 1 -- 172.21.15.42:0/18268866 >> 172.21.15.42:0/18268866 conn(0x7f05f0102790 msgr2=0x7f05f0103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:00.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.790+0000 7f05f6c00700 1 -- 172.21.15.42:0/18268866 shutdown_connections 2024-04-20T10:23:00.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:22:59.790+0000 7f05f6c00700 1 -- 172.21.15.42:0/18268866 wait complete. 2024-04-20T10:23:00.482 INFO:teuthology.orchestra.run.smithi042.stdout:Generating ssh key... 2024-04-20T10:23:00.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:00 smithi042 ceph-mon[17078]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:23:01.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-mon[17078]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:23:01.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-mon[17078]: Generating ssh key... 2024-04-20T10:23:01.854 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: Generating public/private rsa key pair. 2024-04-20T10:23:01.854 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: Your identification has been saved in /tmp/tmpmn1hqcdg/key. 2024-04-20T10:23:01.855 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: Your public key has been saved in /tmp/tmpmn1hqcdg/key.pub. 2024-04-20T10:23:01.855 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: The key fingerprint is: 2024-04-20T10:23:01.855 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: SHA256:vAhbqS/aXIpGH+12M5OmKaYWBFbhGlLJc8FxkpCMrbs ceph-83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:23:01.855 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: The key's randomart image is: 2024-04-20T10:23:01.855 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: +---[RSA 3072]----+ 2024-04-20T10:23:01.855 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: | =+B=+. | 2024-04-20T10:23:01.855 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: |ooO +o | 2024-04-20T10:23:01.855 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: |o+ + | 2024-04-20T10:23:01.855 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: |o + o | 2024-04-20T10:23:01.856 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: | + ..o S | 2024-04-20T10:23:01.856 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: |. o .=.. . | 2024-04-20T10:23:01.856 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: | o oooo .. | 2024-04-20T10:23:01.856 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: |E o+=+o.B | 2024-04-20T10:23:01.856 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: | o++++++ + | 2024-04-20T10:23:01.856 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: +----[SHA256]-----+ 2024-04-20T10:23:02.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.115+0000 7f5cca600700 1 Processor -- start 2024-04-20T10:23:02.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.116+0000 7f5cca600700 1 -- start start 2024-04-20T10:23:02.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.116+0000 7f5cca600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 0x7f5cc4110490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.116+0000 7f5cca600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc4110a60 con 0x7f5cc4110070 2024-04-20T10:23:02.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.117+0000 7f5cc3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 0x7f5cc4110490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.117+0000 7f5cc3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 0x7f5cc4110490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44758/0 (socket says 172.21.15.42:44758) 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.117+0000 7f5cc3400700 1 -- 172.21.15.42:0/2112816341 learned_addr learned my addr 172.21.15.42:0/2112816341 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.117+0000 7f5cc3400700 1 -- 172.21.15.42:0/2112816341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc410ecd0 con 0x7f5cc4110070 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.118+0000 7f5cc3400700 1 --2- 172.21.15.42:0/2112816341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 0x7f5cc4110490 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40098a0 tx=0x7f5cb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71b45f602b277d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.118+0000 7f5cc2000700 1 -- 172.21.15.42:0/2112816341 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cb4004160 con 0x7f5cc4110070 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.118+0000 7f5cc2000700 1 -- 172.21.15.42:0/2112816341 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5cb4004300 con 0x7f5cc4110070 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.118+0000 7f5cc2000700 1 -- 172.21.15.42:0/2112816341 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cb4011b50 con 0x7f5cc4110070 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.119+0000 7f5cca600700 1 -- 172.21.15.42:0/2112816341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 msgr2=0x7f5cc4110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.119+0000 7f5cca600700 1 --2- 172.21.15.42:0/2112816341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 0x7f5cc4110490 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40098a0 tx=0x7f5cb4009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.119+0000 7f5cca600700 1 -- 172.21.15.42:0/2112816341 shutdown_connections 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.119+0000 7f5cca600700 1 --2- 172.21.15.42:0/2112816341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 0x7f5cc4110490 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.119+0000 7f5cca600700 1 -- 172.21.15.42:0/2112816341 >> 172.21.15.42:0/2112816341 conn(0x7f5cc407db90 msgr2=0x7f5cc407fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:02.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.119+0000 7f5cca600700 1 -- 172.21.15.42:0/2112816341 shutdown_connections 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.119+0000 7f5cca600700 1 -- 172.21.15.42:0/2112816341 wait complete. 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.120+0000 7f5cca600700 1 Processor -- start 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.120+0000 7f5cca600700 1 -- start start 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.120+0000 7f5cca600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 0x7f5cc41a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.120+0000 7f5cca600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc4113090 con 0x7f5cc4110070 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.121+0000 7f5cc3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 0x7f5cc41a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.121+0000 7f5cc3400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 0x7f5cc41a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44766/0 (socket says 172.21.15.42:44766) 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.121+0000 7f5cc3400700 1 -- 172.21.15.42:0/1221973071 learned_addr learned my addr 172.21.15.42:0/1221973071 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.121+0000 7f5cc3400700 1 -- 172.21.15.42:0/1221973071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc41a2f90 con 0x7f5cc4110070 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.121+0000 7f5cc3400700 1 --2- 172.21.15.42:0/1221973071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 0x7f5cc41a2a50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4011850 tx=0x7f5cb4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.122+0000 7f5cbbe00700 1 -- 172.21.15.42:0/1221973071 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cb4003af0 con 0x7f5cc4110070 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.122+0000 7f5cbbe00700 1 -- 172.21.15.42:0/1221973071 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5cb4020070 con 0x7f5cc4110070 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.122+0000 7f5cbbe00700 1 -- 172.21.15.42:0/1221973071 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cb40195a0 con 0x7f5cc4110070 2024-04-20T10:23:02.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.122+0000 7f5cca600700 1 -- 172.21.15.42:0/1221973071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc41a3220 con 0x7f5cc4110070 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.122+0000 7f5cca600700 1 -- 172.21.15.42:0/1221973071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc41a5f10 con 0x7f5cc4110070 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.123+0000 7f5cbbe00700 1 -- 172.21.15.42:0/1221973071 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5cb4019850 con 0x7f5cc4110070 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.123+0000 7f5cca600700 1 -- 172.21.15.42:0/1221973071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc40640b0 con 0x7f5cc4110070 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.123+0000 7f5cbbe00700 1 --2- 172.21.15.42:0/1221973071 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f5ca8039600 0x7f5ca803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.123+0000 7f5cbbe00700 1 -- 172.21.15.42:0/1221973071 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5cb4011e00 con 0x7f5cc4110070 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.124+0000 7f5cc2a00700 1 --2- 172.21.15.42:0/1221973071 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f5ca8039600 0x7f5ca803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.124+0000 7f5cc2a00700 1 --2- 172.21.15.42:0/1221973071 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f5ca8039600 0x7f5ca803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0006fd0 tx=0x7f5cb0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.129+0000 7f5cbbe00700 1 -- 172.21.15.42:0/1221973071 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5cb400e260 con 0x7f5cc4110070 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.359+0000 7f5cca600700 1 -- 172.21.15.42:0/1221973071 --> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f5cc407fe80 con 0x7f5ca8039600 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.708+0000 7f5cbbe00700 1 -- 172.21.15.42:0/1221973071 <== mgr.14120 v2:172.21.15.42:6800/2743527860 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5cc407fe80 con 0x7f5ca8039600 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.711+0000 7f5cca600700 1 -- 172.21.15.42:0/1221973071 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f5ca8039600 msgr2=0x7f5ca803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.711+0000 7f5cca600700 1 --2- 172.21.15.42:0/1221973071 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f5ca8039600 0x7f5ca803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0006fd0 tx=0x7f5cb0006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:02.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.711+0000 7f5cca600700 1 -- 172.21.15.42:0/1221973071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 msgr2=0x7f5cc41a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.711+0000 7f5cca600700 1 --2- 172.21.15.42:0/1221973071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 0x7f5cc41a2a50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4011850 tx=0x7f5cb4011c10 comp rx=0 tx=0).stop 2024-04-20T10:23:02.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.711+0000 7f5cca600700 1 -- 172.21.15.42:0/1221973071 shutdown_connections 2024-04-20T10:23:02.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.711+0000 7f5cca600700 1 --2- 172.21.15.42:0/1221973071 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f5ca8039600 0x7f5ca803bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.711+0000 7f5cca600700 1 --2- 172.21.15.42:0/1221973071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cc4110070 0x7f5cc41a2a50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.711+0000 7f5cca600700 1 -- 172.21.15.42:0/1221973071 >> 172.21.15.42:0/1221973071 conn(0x7f5cc407db90 msgr2=0x7f5cc407f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:02.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.711+0000 7f5cca600700 1 -- 172.21.15.42:0/1221973071 shutdown_connections 2024-04-20T10:23:02.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:01.712+0000 7f5cca600700 1 -- 172.21.15.42:0/1221973071 wait complete. 2024-04-20T10:23:02.966 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:02 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:23:02.967 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:02 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:23:03.929 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:03 smithi042 ceph-mon[17078]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:23:03.944 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt7OM2VL3w2UDxYjuVc1OV+b2aO3dcNv0hShbO9f5CHa7UlxQkepH6SvCCWMX6fsd8ZDNG0notHbBAjcOvIHRT6KdsN8ZkYd50aYCrYKGk1ZzUstmsXV82Z+yqPtkAAqqiaDXwhFDwYOPJQlH9lc/oFJgQZ1yfwNgYJCE/UUWijv8o8avFiY8cAafFWi2FdJKGNt/HfWP93txBXaoiQZL+Ws+/lGfnZqska7Svi59McJQPql0yqHEpEphTkMyoqmDrIZERNm3hic7neCAJm3dc0t9aBzrZulUWcAEqcV95Xtt3bRhc1kajWwf/atrv8MpVOtrE59OT1hiUarvlVHP1GXY5fFVU091GSAuS/YpLKNCaOwGbvN49Rxutu+VAdZ6bwn/wRAqj/reOY6Oi9r7dZdvmdo1rkp+qvXD5iS2hQ/KHUOJtpzWOiVfn04N6spnED8S8oyZOKyXdBPXoPCPVFas1uD9aE1UCpOwup7uHugXUqqWw9L+BdR55KlRURus= ceph-83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:23:03.944 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.034+0000 7f7765a00700 1 Processor -- start 2024-04-20T10:23:03.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.035+0000 7f7765a00700 1 -- start start 2024-04-20T10:23:03.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.035+0000 7f7765a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 0x7f776010a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:03.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.035+0000 7f7765a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f776010aef0 con 0x7f776010a500 2024-04-20T10:23:03.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.036+0000 7f775fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 0x7f776010a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:03.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.036+0000 7f775fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 0x7f776010a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44770/0 (socket says 172.21.15.42:44770) 2024-04-20T10:23:03.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.036+0000 7f775fe00700 1 -- 172.21.15.42:0/1583977071 learned_addr learned my addr 172.21.15.42:0/1583977071 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:03.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.036+0000 7f775fe00700 1 -- 172.21.15.42:0/1583977071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776010b6b0 con 0x7f776010a500 2024-04-20T10:23:03.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.036+0000 7f775fe00700 1 --2- 172.21.15.42:0/1583977071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 0x7f776010a920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77500098a0 tx=0x7f7750009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9336bfab91ea8c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:03.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.037+0000 7f775ea00700 1 -- 172.21.15.42:0/1583977071 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7750004160 con 0x7f776010a500 2024-04-20T10:23:03.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.037+0000 7f775ea00700 1 -- 172.21.15.42:0/1583977071 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7750004300 con 0x7f776010a500 2024-04-20T10:23:03.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.037+0000 7f775ea00700 1 -- 172.21.15.42:0/1583977071 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7750011b50 con 0x7f776010a500 2024-04-20T10:23:03.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.037+0000 7f7765a00700 1 -- 172.21.15.42:0/1583977071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 msgr2=0x7f776010a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:03.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.037+0000 7f7765a00700 1 --2- 172.21.15.42:0/1583977071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 0x7f776010a920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77500098a0 tx=0x7f7750009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:03.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.038+0000 7f7765a00700 1 -- 172.21.15.42:0/1583977071 shutdown_connections 2024-04-20T10:23:03.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.038+0000 7f7765a00700 1 --2- 172.21.15.42:0/1583977071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 0x7f776010a920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:03.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.038+0000 7f7765a00700 1 -- 172.21.15.42:0/1583977071 >> 172.21.15.42:0/1583977071 conn(0x7f776007d9b0 msgr2=0x7f77601084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:03.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.038+0000 7f7765a00700 1 -- 172.21.15.42:0/1583977071 shutdown_connections 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.038+0000 7f7765a00700 1 -- 172.21.15.42:0/1583977071 wait complete. 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.039+0000 7f7765a00700 1 Processor -- start 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.039+0000 7f7765a00700 1 -- start start 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.039+0000 7f7765a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 0x7f776019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.039+0000 7f7765a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f776010ea60 con 0x7f776010a500 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.039+0000 7f775fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 0x7f776019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.040+0000 7f775fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 0x7f776019e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44780/0 (socket says 172.21.15.42:44780) 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.040+0000 7f775fe00700 1 -- 172.21.15.42:0/3851002482 learned_addr learned my addr 172.21.15.42:0/3851002482 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.040+0000 7f775fe00700 1 -- 172.21.15.42:0/3851002482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776019ea10 con 0x7f776010a500 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.040+0000 7f775fe00700 1 --2- 172.21.15.42:0/3851002482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 0x7f776019e4d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7750011850 tx=0x7f7750011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.040+0000 7f775cc00700 1 -- 172.21.15.42:0/3851002482 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7750003af0 con 0x7f776010a500 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.041+0000 7f775cc00700 1 -- 172.21.15.42:0/3851002482 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7750020070 con 0x7f776010a500 2024-04-20T10:23:03.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.041+0000 7f775cc00700 1 -- 172.21.15.42:0/3851002482 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77500195a0 con 0x7f776010a500 2024-04-20T10:23:03.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.041+0000 7f7765a00700 1 -- 172.21.15.42:0/3851002482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776019eca0 con 0x7f776010a500 2024-04-20T10:23:03.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.041+0000 7f7765a00700 1 -- 172.21.15.42:0/3851002482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77601a1990 con 0x7f776010a500 2024-04-20T10:23:03.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.042+0000 7f7765a00700 1 -- 172.21.15.42:0/3851002482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77600640b0 con 0x7f776010a500 2024-04-20T10:23:03.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.042+0000 7f775cc00700 1 -- 172.21.15.42:0/3851002482 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f775001a430 con 0x7f776010a500 2024-04-20T10:23:03.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.043+0000 7f775cc00700 1 --2- 172.21.15.42:0/3851002482 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f7744039600 0x7f774403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:03.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.043+0000 7f775cc00700 1 -- 172.21.15.42:0/3851002482 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7750011e00 con 0x7f776010a500 2024-04-20T10:23:03.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.043+0000 7f775f400700 1 --2- 172.21.15.42:0/3851002482 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f7744039600 0x7f774403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:03.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.044+0000 7f775f400700 1 --2- 172.21.15.42:0/3851002482 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f7744039600 0x7f774403bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f774c006fd0 tx=0x7f774c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:03.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.048+0000 7f775cc00700 1 -- 172.21.15.42:0/3851002482 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f775000e260 con 0x7f776010a500 2024-04-20T10:23:03.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.281+0000 7f7765a00700 1 -- 172.21.15.42:0/3851002482 --> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f77601083e0 con 0x7f7744039600 2024-04-20T10:23:03.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.282+0000 7f775cc00700 1 -- 172.21.15.42:0/3851002482 <== mgr.14120 v2:172.21.15.42:6800/2743527860 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f77601083e0 con 0x7f7744039600 2024-04-20T10:23:03.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.286+0000 7f7765a00700 1 -- 172.21.15.42:0/3851002482 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f7744039600 msgr2=0x7f774403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:03.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.286+0000 7f7765a00700 1 --2- 172.21.15.42:0/3851002482 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f7744039600 0x7f774403bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f774c006fd0 tx=0x7f774c006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:03.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.286+0000 7f7765a00700 1 -- 172.21.15.42:0/3851002482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 msgr2=0x7f776019e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:03.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.286+0000 7f7765a00700 1 --2- 172.21.15.42:0/3851002482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 0x7f776019e4d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7750011850 tx=0x7f7750011c10 comp rx=0 tx=0).stop 2024-04-20T10:23:03.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.286+0000 7f7765a00700 1 -- 172.21.15.42:0/3851002482 shutdown_connections 2024-04-20T10:23:03.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.286+0000 7f7765a00700 1 --2- 172.21.15.42:0/3851002482 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f7744039600 0x7f774403bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:03.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.286+0000 7f7765a00700 1 --2- 172.21.15.42:0/3851002482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f776010a500 0x7f776019e4d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:03.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.286+0000 7f7765a00700 1 -- 172.21.15.42:0/3851002482 >> 172.21.15.42:0/3851002482 conn(0x7f776007d9b0 msgr2=0x7f7760107cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:03.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.286+0000 7f7765a00700 1 -- 172.21.15.42:0/3851002482 shutdown_connections 2024-04-20T10:23:03.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:03.286+0000 7f7765a00700 1 -- 172.21.15.42:0/3851002482 wait complete. 2024-04-20T10:23:03.956 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-20T10:23:03.956 INFO:teuthology.orchestra.run.smithi042.stdout:Adding key to root@localhost authorized_keys... 2024-04-20T10:23:03.962 INFO:teuthology.orchestra.run.smithi042.stdout:Adding host smithi042... 2024-04-20T10:23:05.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:05 smithi042 ceph-mon[17078]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:23:06.799 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:06 smithi042 ceph-mon[17078]: Deploying cephadm binary to smithi042 2024-04-20T10:23:11.916 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Added host 'smithi042' with addr '172.21.15.42' 2024-04-20T10:23:11.916 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.569+0000 7f4470200700 1 Processor -- start 2024-04-20T10:23:11.916 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.569+0000 7f4470200700 1 -- start start 2024-04-20T10:23:11.916 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.570+0000 7f4470200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 0x7f44681088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:11.916 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.570+0000 7f4470200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4468108ea0 con 0x7f44681084b0 2024-04-20T10:23:11.916 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.570+0000 7f446d800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 0x7f44681088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:11.916 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.570+0000 7f446d800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 0x7f44681088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44790/0 (socket says 172.21.15.42:44790) 2024-04-20T10:23:11.916 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.570+0000 7f446d800700 1 -- 172.21.15.42:0/1280498456 learned_addr learned my addr 172.21.15.42:0/1280498456 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:11.916 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.571+0000 7f446d800700 1 -- 172.21.15.42:0/1280498456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4468109660 con 0x7f44681084b0 2024-04-20T10:23:11.916 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.571+0000 7f446d800700 1 --2- 172.21.15.42:0/1280498456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 0x7f44681088d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f445c0098a0 tx=0x7f445c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efd72bba6fbccf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.572+0000 7f4467e00700 1 -- 172.21.15.42:0/1280498456 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f445c004160 con 0x7f44681084b0 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.572+0000 7f4467e00700 1 -- 172.21.15.42:0/1280498456 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f445c004300 con 0x7f44681084b0 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.572+0000 7f4467e00700 1 -- 172.21.15.42:0/1280498456 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f445c011b50 con 0x7f44681084b0 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.572+0000 7f4470200700 1 -- 172.21.15.42:0/1280498456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 msgr2=0x7f44681088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.572+0000 7f4470200700 1 --2- 172.21.15.42:0/1280498456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 0x7f44681088d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f445c0098a0 tx=0x7f445c009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.572+0000 7f4470200700 1 -- 172.21.15.42:0/1280498456 shutdown_connections 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.572+0000 7f4470200700 1 --2- 172.21.15.42:0/1280498456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 0x7f44681088d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.572+0000 7f4470200700 1 -- 172.21.15.42:0/1280498456 >> 172.21.15.42:0/1280498456 conn(0x7f4468103ad0 msgr2=0x7f4468105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.573+0000 7f4470200700 1 -- 172.21.15.42:0/1280498456 shutdown_connections 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.573+0000 7f4470200700 1 -- 172.21.15.42:0/1280498456 wait complete. 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.574+0000 7f4470200700 1 Processor -- start 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.574+0000 7f4470200700 1 -- start start 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.574+0000 7f4470200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 0x7f446807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:11.917 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.574+0000 7f4470200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f446810ca10 con 0x7f44681084b0 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.574+0000 7f446d800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 0x7f446807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.574+0000 7f446d800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 0x7f446807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44792/0 (socket says 172.21.15.42:44792) 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.575+0000 7f446d800700 1 -- 172.21.15.42:0/3421485602 learned_addr learned my addr 172.21.15.42:0/3421485602 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.575+0000 7f446d800700 1 -- 172.21.15.42:0/3421485602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446807e0b0 con 0x7f44681084b0 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.575+0000 7f446d800700 1 --2- 172.21.15.42:0/3421485602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 0x7f446807db70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f445c011850 tx=0x7f445c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.575+0000 7f4466000700 1 -- 172.21.15.42:0/3421485602 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f445c003af0 con 0x7f44681084b0 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.576+0000 7f4466000700 1 -- 172.21.15.42:0/3421485602 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f445c020070 con 0x7f44681084b0 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.576+0000 7f4466000700 1 -- 172.21.15.42:0/3421485602 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f445c0195a0 con 0x7f44681084b0 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.576+0000 7f4470200700 1 -- 172.21.15.42:0/3421485602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446807c2a0 con 0x7f44681084b0 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.576+0000 7f4470200700 1 -- 172.21.15.42:0/3421485602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446807c7e0 con 0x7f44681084b0 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.577+0000 7f4470200700 1 -- 172.21.15.42:0/3421485602 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44680640b0 con 0x7f44681084b0 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.577+0000 7f4466000700 1 -- 172.21.15.42:0/3421485602 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f445c01a430 con 0x7f44681084b0 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.577+0000 7f4466000700 1 --2- 172.21.15.42:0/3421485602 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f44500395b0 0x7f445003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:11.918 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.578+0000 7f4466000700 1 -- 172.21.15.42:0/3421485602 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f445c011e00 con 0x7f44681084b0 2024-04-20T10:23:11.919 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.578+0000 7f446ce00700 1 --2- 172.21.15.42:0/3421485602 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f44500395b0 0x7f445003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:11.919 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.579+0000 7f446ce00700 1 --2- 172.21.15.42:0/3421485602 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f44500395b0 0x7f445003ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4454006fd0 tx=0x7f4454006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:11.919 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.583+0000 7f4466000700 1 -- 172.21.15.42:0/3421485602 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f445c00e260 con 0x7f44681084b0 2024-04-20T10:23:11.919 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:04.818+0000 7f4470200700 1 -- 172.21.15.42:0/3421485602 --> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}) -- 0x7f446807d040 con 0x7f44500395b0 2024-04-20T10:23:11.919 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:11.157+0000 7f4466000700 1 -- 172.21.15.42:0/3421485602 <== mgr.14120 v2:172.21.15.42:6800/2743527860 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f446807d040 con 0x7f44500395b0 2024-04-20T10:23:11.919 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:11.161+0000 7f4470200700 1 -- 172.21.15.42:0/3421485602 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f44500395b0 msgr2=0x7f445003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:11.919 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:11.161+0000 7f4470200700 1 --2- 172.21.15.42:0/3421485602 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f44500395b0 0x7f445003ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4454006fd0 tx=0x7f4454006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:11.919 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:11.161+0000 7f4470200700 1 -- 172.21.15.42:0/3421485602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 msgr2=0x7f446807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:11.919 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:11.161+0000 7f4470200700 1 --2- 172.21.15.42:0/3421485602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 0x7f446807db70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f445c011850 tx=0x7f445c011c10 comp rx=0 tx=0).stop 2024-04-20T10:23:11.919 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:11.161+0000 7f4470200700 1 -- 172.21.15.42:0/3421485602 shutdown_connections 2024-04-20T10:23:11.919 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:11.161+0000 7f4470200700 1 --2- 172.21.15.42:0/3421485602 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f44500395b0 0x7f445003ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:11.919 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:11.161+0000 7f4470200700 1 --2- 172.21.15.42:0/3421485602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44681084b0 0x7f446807db70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:11.920 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:11.161+0000 7f4470200700 1 -- 172.21.15.42:0/3421485602 >> 172.21.15.42:0/3421485602 conn(0x7f4468103ad0 msgr2=0x7f4468104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:11.920 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:11.161+0000 7f4470200700 1 -- 172.21.15.42:0/3421485602 shutdown_connections 2024-04-20T10:23:11.920 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:11.162+0000 7f4470200700 1 -- 172.21.15.42:0/3421485602 wait complete. 2024-04-20T10:23:11.920 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying unmanaged mon service... 2024-04-20T10:23:12.449 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:12 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:23:12.449 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:12 smithi042 ceph-mon[17078]: Added host smithi042 2024-04-20T10:23:12.449 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:12 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:23:14.031 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:13 smithi042 ceph-mon[17078]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:23:14.032 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:13 smithi042 ceph-mon[17078]: Saving service mon spec with placement count:5 2024-04-20T10:23:14.032 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:13 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:23:14.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-20T10:23:14.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.622+0000 7fa153200700 1 Processor -- start 2024-04-20T10:23:14.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.623+0000 7fa153200700 1 -- start start 2024-04-20T10:23:14.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.623+0000 7fa153200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 0x7fa154108750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:14.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.623+0000 7fa153200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa154108d20 con 0x7fa154108330 2024-04-20T10:23:14.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.624+0000 7fa151e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 0x7fa154108750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:14.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.624+0000 7fa151e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 0x7fa154108750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45126/0 (socket says 172.21.15.42:45126) 2024-04-20T10:23:14.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.624+0000 7fa151e00700 1 -- 172.21.15.42:0/3552788766 learned_addr learned my addr 172.21.15.42:0/3552788766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:14.587 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.625+0000 7fa151e00700 1 -- 172.21.15.42:0/3552788766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa154109530 con 0x7fa154108330 2024-04-20T10:23:14.587 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.625+0000 7fa151e00700 1 --2- 172.21.15.42:0/3552788766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 0x7fa154108750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa1440098a0 tx=0x7fa144009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf10244a7d3f6e35 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:14.587 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.625+0000 7fa150a00700 1 -- 172.21.15.42:0/3552788766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa144004160 con 0x7fa154108330 2024-04-20T10:23:14.587 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.625+0000 7fa150a00700 1 -- 172.21.15.42:0/3552788766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa144004300 con 0x7fa154108330 2024-04-20T10:23:14.587 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.625+0000 7fa150a00700 1 -- 172.21.15.42:0/3552788766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa144011b50 con 0x7fa154108330 2024-04-20T10:23:14.588 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.626+0000 7fa153200700 1 -- 172.21.15.42:0/3552788766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 msgr2=0x7fa154108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:14.588 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.626+0000 7fa153200700 1 --2- 172.21.15.42:0/3552788766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 0x7fa154108750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa1440098a0 tx=0x7fa144009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:14.588 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.626+0000 7fa153200700 1 -- 172.21.15.42:0/3552788766 shutdown_connections 2024-04-20T10:23:14.588 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.626+0000 7fa153200700 1 --2- 172.21.15.42:0/3552788766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 0x7fa154108750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:14.588 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.626+0000 7fa153200700 1 -- 172.21.15.42:0/3552788766 >> 172.21.15.42:0/3552788766 conn(0x7fa154103950 msgr2=0x7fa154105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:14.588 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.626+0000 7fa153200700 1 -- 172.21.15.42:0/3552788766 shutdown_connections 2024-04-20T10:23:14.588 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.626+0000 7fa153200700 1 -- 172.21.15.42:0/3552788766 wait complete. 2024-04-20T10:23:14.589 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.627+0000 7fa153200700 1 Processor -- start 2024-04-20T10:23:14.590 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.627+0000 7fa153200700 1 -- start start 2024-04-20T10:23:14.590 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.628+0000 7fa153200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 0x7fa15419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:14.590 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.628+0000 7fa153200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa15410c8e0 con 0x7fa154108330 2024-04-20T10:23:14.590 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.628+0000 7fa151e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 0x7fa15419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:14.590 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.628+0000 7fa151e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 0x7fa15419c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45136/0 (socket says 172.21.15.42:45136) 2024-04-20T10:23:14.590 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.628+0000 7fa151e00700 1 -- 172.21.15.42:0/4155568248 learned_addr learned my addr 172.21.15.42:0/4155568248 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:14.590 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.628+0000 7fa151e00700 1 -- 172.21.15.42:0/4155568248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15419c800 con 0x7fa154108330 2024-04-20T10:23:14.590 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.629+0000 7fa151e00700 1 --2- 172.21.15.42:0/4155568248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 0x7fa15419c2c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa144000c00 tx=0x7fa144011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:14.591 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.629+0000 7fa14aa00700 1 -- 172.21.15.42:0/4155568248 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa144003b20 con 0x7fa154108330 2024-04-20T10:23:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.629+0000 7fa14aa00700 1 -- 172.21.15.42:0/4155568248 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa144003cc0 con 0x7fa154108330 2024-04-20T10:23:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.629+0000 7fa153200700 1 -- 172.21.15.42:0/4155568248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15419ca90 con 0x7fa154108330 2024-04-20T10:23:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.629+0000 7fa14aa00700 1 -- 172.21.15.42:0/4155568248 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa144024910 con 0x7fa154108330 2024-04-20T10:23:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.629+0000 7fa153200700 1 -- 172.21.15.42:0/4155568248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15419f780 con 0x7fa154108330 2024-04-20T10:23:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.630+0000 7fa14aa00700 1 -- 172.21.15.42:0/4155568248 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa144022020 con 0x7fa154108330 2024-04-20T10:23:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.631+0000 7fa153200700 1 -- 172.21.15.42:0/4155568248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1540640b0 con 0x7fa154108330 2024-04-20T10:23:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.631+0000 7fa14aa00700 1 --2- 172.21.15.42:0/4155568248 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fa138039600 0x7fa13803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.631+0000 7fa14aa00700 1 -- 172.21.15.42:0/4155568248 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa144011e00 con 0x7fa154108330 2024-04-20T10:23:14.594 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.631+0000 7fa151400700 1 --2- 172.21.15.42:0/4155568248 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fa138039600 0x7fa13803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:14.594 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.632+0000 7fa151400700 1 --2- 172.21.15.42:0/4155568248 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fa138039600 0x7fa13803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa140006fd0 tx=0x7fa140006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:14.594 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.637+0000 7fa14aa00700 1 -- 172.21.15.42:0/4155568248 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa14404d130 con 0x7fa154108330 2024-04-20T10:23:14.594 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.908+0000 7fa153200700 1 -- 172.21.15.42:0/4155568248 --> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa154104b70 con 0x7fa138039600 2024-04-20T10:23:14.594 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.920+0000 7fa14aa00700 1 -- 172.21.15.42:0/4155568248 <== mgr.14120 v2:172.21.15.42:6800/2743527860 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa154104b70 con 0x7fa138039600 2024-04-20T10:23:14.594 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.923+0000 7fa153200700 1 -- 172.21.15.42:0/4155568248 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fa138039600 msgr2=0x7fa13803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:14.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.923+0000 7fa153200700 1 --2- 172.21.15.42:0/4155568248 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fa138039600 0x7fa13803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa140006fd0 tx=0x7fa140006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:14.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.923+0000 7fa153200700 1 -- 172.21.15.42:0/4155568248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 msgr2=0x7fa15419c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:14.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.923+0000 7fa153200700 1 --2- 172.21.15.42:0/4155568248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 0x7fa15419c2c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa144000c00 tx=0x7fa144011c10 comp rx=0 tx=0).stop 2024-04-20T10:23:14.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.923+0000 7fa153200700 1 -- 172.21.15.42:0/4155568248 shutdown_connections 2024-04-20T10:23:14.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.923+0000 7fa153200700 1 --2- 172.21.15.42:0/4155568248 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fa138039600 0x7fa13803bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:14.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.923+0000 7fa153200700 1 --2- 172.21.15.42:0/4155568248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa154108330 0x7fa15419c2c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:14.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.923+0000 7fa153200700 1 -- 172.21.15.42:0/4155568248 >> 172.21.15.42:0/4155568248 conn(0x7fa154103950 msgr2=0x7fa154104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:14.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.923+0000 7fa153200700 1 -- 172.21.15.42:0/4155568248 shutdown_connections 2024-04-20T10:23:14.595 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:12.923+0000 7fa153200700 1 -- 172.21.15.42:0/4155568248 wait complete. 2024-04-20T10:23:14.595 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying unmanaged mgr service... 2024-04-20T10:23:15.447 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:15 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:23:15.448 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:15 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:23:16.142 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-20T10:23:16.142 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.188+0000 7f011f200700 1 Processor -- start 2024-04-20T10:23:16.142 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.188+0000 7f011f200700 1 -- start start 2024-04-20T10:23:16.142 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.189+0000 7f011f200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 0x7f011807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:16.142 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.189+0000 7f011f200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f011807c820 con 0x7f011807de30 2024-04-20T10:23:16.142 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.189+0000 7f0117e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 0x7f011807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:16.142 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.189+0000 7f0117e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 0x7f011807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45144/0 (socket says 172.21.15.42:45144) 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.189+0000 7f0117e00700 1 -- 172.21.15.42:0/3628901292 learned_addr learned my addr 172.21.15.42:0/3628901292 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.190+0000 7f0117e00700 1 -- 172.21.15.42:0/3628901292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011807c9a0 con 0x7f011807de30 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.190+0000 7f0117e00700 1 --2- 172.21.15.42:0/3628901292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 0x7f011807c2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01080098a0 tx=0x7f0108009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152408720cbd1e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.190+0000 7f0116a00700 1 -- 172.21.15.42:0/3628901292 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0108004160 con 0x7f011807de30 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.190+0000 7f0116a00700 1 -- 172.21.15.42:0/3628901292 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0108004300 con 0x7f011807de30 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.190+0000 7f0116a00700 1 -- 172.21.15.42:0/3628901292 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0108011b20 con 0x7f011807de30 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.191+0000 7f011f200700 1 -- 172.21.15.42:0/3628901292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 msgr2=0x7f011807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.191+0000 7f011f200700 1 --2- 172.21.15.42:0/3628901292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 0x7f011807c2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01080098a0 tx=0x7f0108009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.191+0000 7f011f200700 1 -- 172.21.15.42:0/3628901292 shutdown_connections 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.191+0000 7f011f200700 1 --2- 172.21.15.42:0/3628901292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 0x7f011807c2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.191+0000 7f011f200700 1 -- 172.21.15.42:0/3628901292 >> 172.21.15.42:0/3628901292 conn(0x7f0118103a60 msgr2=0x7f0118105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.192+0000 7f011f200700 1 -- 172.21.15.42:0/3628901292 shutdown_connections 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.192+0000 7f011f200700 1 -- 172.21.15.42:0/3628901292 wait complete. 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.193+0000 7f011f200700 1 Processor -- start 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.193+0000 7f011f200700 1 -- start start 2024-04-20T10:23:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.194+0000 7f011f200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 0x7f011819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.194+0000 7f011f200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f011810ea60 con 0x7f011807de30 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.194+0000 7f0117e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 0x7f011819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.194+0000 7f0117e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 0x7f011819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45150/0 (socket says 172.21.15.42:45150) 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.194+0000 7f0117e00700 1 -- 172.21.15.42:0/1334773877 learned_addr learned my addr 172.21.15.42:0/1334773877 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.194+0000 7f0117e00700 1 -- 172.21.15.42:0/1334773877 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011819eaf0 con 0x7f011807de30 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.195+0000 7f0117e00700 1 --2- 172.21.15.42:0/1334773877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 0x7f011819e5b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0108004660 tx=0x7f0108011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.195+0000 7f0114c00700 1 -- 172.21.15.42:0/1334773877 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0108003ca0 con 0x7f011807de30 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.195+0000 7f0114c00700 1 -- 172.21.15.42:0/1334773877 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010801a910 con 0x7f011807de30 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.195+0000 7f011f200700 1 -- 172.21.15.42:0/1334773877 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011819ed80 con 0x7f011807de30 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.195+0000 7f0114c00700 1 -- 172.21.15.42:0/1334773877 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0108024970 con 0x7f011807de30 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.195+0000 7f011f200700 1 -- 172.21.15.42:0/1334773877 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011819f1e0 con 0x7f011807de30 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.196+0000 7f011f200700 1 -- 172.21.15.42:0/1334773877 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f8005360 con 0x7f011807de30 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.197+0000 7f0114c00700 1 -- 172.21.15.42:0/1334773877 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0108022020 con 0x7f011807de30 2024-04-20T10:23:16.147 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.197+0000 7f0114c00700 1 --2- 172.21.15.42:0/1334773877 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f00fc0397a0 0x7f00fc03bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.197+0000 7f0114c00700 1 -- 172.21.15.42:0/1334773877 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f010804db80 con 0x7f011807de30 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.198+0000 7f0117400700 1 --2- 172.21.15.42:0/1334773877 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f00fc0397a0 0x7f00fc03bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.199+0000 7f0117400700 1 --2- 172.21.15.42:0/1334773877 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f00fc0397a0 0x7f00fc03bc60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0104006fd0 tx=0x7f0104006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.203+0000 7f0114c00700 1 -- 172.21.15.42:0/1334773877 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0108019c70 con 0x7f011807de30 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.447+0000 7f011f200700 1 -- 172.21.15.42:0/1334773877 --> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f00f8000bf0 con 0x7f00fc0397a0 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.456+0000 7f0114c00700 1 -- 172.21.15.42:0/1334773877 <== mgr.14120 v2:172.21.15.42:6800/2743527860 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f00f8000bf0 con 0x7f00fc0397a0 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.459+0000 7f011f200700 1 -- 172.21.15.42:0/1334773877 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f00fc0397a0 msgr2=0x7f00fc03bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.459+0000 7f011f200700 1 --2- 172.21.15.42:0/1334773877 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f00fc0397a0 0x7f00fc03bc60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0104006fd0 tx=0x7f0104006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.460+0000 7f011f200700 1 -- 172.21.15.42:0/1334773877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 msgr2=0x7f011819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.460+0000 7f011f200700 1 --2- 172.21.15.42:0/1334773877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 0x7f011819e5b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0108004660 tx=0x7f0108011ce0 comp rx=0 tx=0).stop 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.460+0000 7f011f200700 1 -- 172.21.15.42:0/1334773877 shutdown_connections 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.460+0000 7f011f200700 1 --2- 172.21.15.42:0/1334773877 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f00fc0397a0 0x7f00fc03bc60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.460+0000 7f011f200700 1 --2- 172.21.15.42:0/1334773877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011807de30 0x7f011819e5b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.460+0000 7f011f200700 1 -- 172.21.15.42:0/1334773877 >> 172.21.15.42:0/1334773877 conn(0x7f0118103a60 msgr2=0x7f0118104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:16.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.460+0000 7f011f200700 1 -- 172.21.15.42:0/1334773877 shutdown_connections 2024-04-20T10:23:16.149 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:15.460+0000 7f011f200700 1 -- 172.21.15.42:0/1334773877 wait complete. 2024-04-20T10:23:16.720 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:16 smithi042 ceph-mon[17078]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:23:16.720 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:16 smithi042 ceph-mon[17078]: Saving service mgr spec with placement count:2 2024-04-20T10:23:16.721 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:16 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:23:17.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:17 smithi042 ceph-mon[17078]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:23:17.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:17 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2619338311' entity='client.admin' 2024-04-20T10:23:18.025 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.719+0000 7fe060000700 1 Processor -- start 2024-04-20T10:23:18.025 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.719+0000 7fe060000700 1 -- start start 2024-04-20T10:23:18.025 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.720+0000 7fe060000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 0x7fe0581088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:18.025 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.720+0000 7fe060000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe058108e90 con 0x7fe0581084a0 2024-04-20T10:23:18.025 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.720+0000 7fe05d600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 0x7fe0581088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:18.025 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.720+0000 7fe05d600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 0x7fe0581088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45162/0 (socket says 172.21.15.42:45162) 2024-04-20T10:23:18.025 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.720+0000 7fe05d600700 1 -- 172.21.15.42:0/1266110944 learned_addr learned my addr 172.21.15.42:0/1266110944 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:18.025 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.721+0000 7fe05d600700 1 -- 172.21.15.42:0/1266110944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe058109650 con 0x7fe0581084a0 2024-04-20T10:23:18.025 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.721+0000 7fe05d600700 1 --2- 172.21.15.42:0/1266110944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 0x7fe0581088c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009e80 tx=0x7fe04c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e0807272a109f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.722+0000 7fe057e00700 1 -- 172.21.15.42:0/1266110944 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe04c017070 con 0x7fe0581084a0 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.722+0000 7fe057e00700 1 -- 172.21.15.42:0/1266110944 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe04c00b8c0 con 0x7fe0581084a0 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.722+0000 7fe057e00700 1 -- 172.21.15.42:0/1266110944 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe04c0045c0 con 0x7fe0581084a0 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.722+0000 7fe060000700 1 -- 172.21.15.42:0/1266110944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 msgr2=0x7fe0581088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.722+0000 7fe060000700 1 --2- 172.21.15.42:0/1266110944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 0x7fe0581088c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009e80 tx=0x7fe04c00b330 comp rx=0 tx=0).stop 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.722+0000 7fe060000700 1 -- 172.21.15.42:0/1266110944 shutdown_connections 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.722+0000 7fe060000700 1 --2- 172.21.15.42:0/1266110944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 0x7fe0581088c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.722+0000 7fe060000700 1 -- 172.21.15.42:0/1266110944 >> 172.21.15.42:0/1266110944 conn(0x7fe058103a60 msgr2=0x7fe058105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.722+0000 7fe060000700 1 -- 172.21.15.42:0/1266110944 shutdown_connections 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.723+0000 7fe060000700 1 -- 172.21.15.42:0/1266110944 wait complete. 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.724+0000 7fe060000700 1 Processor -- start 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.724+0000 7fe060000700 1 -- start start 2024-04-20T10:23:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.724+0000 7fe060000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 0x7fe05819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.724+0000 7fe060000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe05810ca00 con 0x7fe0581084a0 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.724+0000 7fe05d600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 0x7fe05819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.724+0000 7fe05d600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 0x7fe05819c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45164/0 (socket says 172.21.15.42:45164) 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.725+0000 7fe05d600700 1 -- 172.21.15.42:0/2619338311 learned_addr learned my addr 172.21.15.42:0/2619338311 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.725+0000 7fe05d600700 1 -- 172.21.15.42:0/2619338311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05819c920 con 0x7fe0581084a0 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.725+0000 7fe05d600700 1 --2- 172.21.15.42:0/2619338311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 0x7fe05819c3e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe04c003fa0 tx=0x7fe04c004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.726+0000 7fe056000700 1 -- 172.21.15.42:0/2619338311 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe04c020020 con 0x7fe0581084a0 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.726+0000 7fe056000700 1 -- 172.21.15.42:0/2619338311 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe04c00bba0 con 0x7fe0581084a0 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.726+0000 7fe056000700 1 -- 172.21.15.42:0/2619338311 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe04c017040 con 0x7fe0581084a0 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.726+0000 7fe060000700 1 -- 172.21.15.42:0/2619338311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05819cbb0 con 0x7fe0581084a0 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.726+0000 7fe060000700 1 -- 172.21.15.42:0/2619338311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe05819f8a0 con 0x7fe0581084a0 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.727+0000 7fe060000700 1 -- 172.21.15.42:0/2619338311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0580640b0 con 0x7fe0581084a0 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.727+0000 7fe056000700 1 -- 172.21.15.42:0/2619338311 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe04c023020 con 0x7fe0581084a0 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.728+0000 7fe056000700 1 --2- 172.21.15.42:0/2619338311 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe0400395b0 0x7fe04003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:18.027 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.728+0000 7fe056000700 1 -- 172.21.15.42:0/2619338311 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe04c00f050 con 0x7fe0581084a0 2024-04-20T10:23:18.028 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.728+0000 7fe05cc00700 1 --2- 172.21.15.42:0/2619338311 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe0400395b0 0x7fe04003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:18.028 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.729+0000 7fe05cc00700 1 --2- 172.21.15.42:0/2619338311 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe0400395b0 0x7fe04003ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe044006fd0 tx=0x7fe044006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:18.028 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.733+0000 7fe056000700 1 -- 172.21.15.42:0/2619338311 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe04c02a020 con 0x7fe0581084a0 2024-04-20T10:23:18.028 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.965+0000 7fe060000700 1 -- 172.21.15.42:0/2619338311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fe05819fe70 con 0x7fe0581084a0 2024-04-20T10:23:18.028 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.969+0000 7fe056000700 1 -- 172.21.15.42:0/2619338311 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fe04c02d350 con 0x7fe0581084a0 2024-04-20T10:23:18.028 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.972+0000 7fe060000700 1 -- 172.21.15.42:0/2619338311 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe0400395b0 msgr2=0x7fe04003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:18.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.972+0000 7fe060000700 1 --2- 172.21.15.42:0/2619338311 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe0400395b0 0x7fe04003ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe044006fd0 tx=0x7fe044006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:18.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.972+0000 7fe060000700 1 -- 172.21.15.42:0/2619338311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 msgr2=0x7fe05819c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:18.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.973+0000 7fe060000700 1 --2- 172.21.15.42:0/2619338311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 0x7fe05819c3e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe04c003fa0 tx=0x7fe04c004080 comp rx=0 tx=0).stop 2024-04-20T10:23:18.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.973+0000 7fe060000700 1 -- 172.21.15.42:0/2619338311 shutdown_connections 2024-04-20T10:23:18.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.973+0000 7fe060000700 1 --2- 172.21.15.42:0/2619338311 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe0400395b0 0x7fe04003ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:18.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.973+0000 7fe060000700 1 --2- 172.21.15.42:0/2619338311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0581084a0 0x7fe05819c3e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:18.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.973+0000 7fe060000700 1 -- 172.21.15.42:0/2619338311 >> 172.21.15.42:0/2619338311 conn(0x7fe058103a60 msgr2=0x7fe0581045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:18.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.973+0000 7fe060000700 1 -- 172.21.15.42:0/2619338311 shutdown_connections 2024-04-20T10:23:18.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:16.973+0000 7fe060000700 1 -- 172.21.15.42:0/2619338311 wait complete. 2024-04-20T10:23:18.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:18 smithi042 ceph-mon[17078]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:23:18.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:18 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:23:19.582 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.639+0000 7ff1b2e00700 1 Processor -- start 2024-04-20T10:23:19.582 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.639+0000 7ff1b2e00700 1 -- start start 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.640+0000 7ff1b2e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 0x7ff1ac07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.640+0000 7ff1b2e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac07c8f0 con 0x7ff1ac07de70 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.640+0000 7ff1abe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 0x7ff1ac07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.640+0000 7ff1abe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 0x7ff1ac07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45168/0 (socket says 172.21.15.42:45168) 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.640+0000 7ff1abe00700 1 -- 172.21.15.42:0/2862631525 learned_addr learned my addr 172.21.15.42:0/2862631525 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.641+0000 7ff1abe00700 1 -- 172.21.15.42:0/2862631525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac07ca70 con 0x7ff1ac07de70 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.641+0000 7ff1abe00700 1 --2- 172.21.15.42:0/2862631525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 0x7ff1ac07c320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0098a0 tx=0x7ff19c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8871964590e3cf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.641+0000 7ff1aaa00700 1 -- 172.21.15.42:0/2862631525 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff19c004160 con 0x7ff1ac07de70 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.641+0000 7ff1aaa00700 1 -- 172.21.15.42:0/2862631525 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff19c004300 con 0x7ff1ac07de70 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.641+0000 7ff1aaa00700 1 -- 172.21.15.42:0/2862631525 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff19c011b20 con 0x7ff1ac07de70 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.642+0000 7ff1b2e00700 1 -- 172.21.15.42:0/2862631525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 msgr2=0x7ff1ac07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.642+0000 7ff1b2e00700 1 --2- 172.21.15.42:0/2862631525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 0x7ff1ac07c320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0098a0 tx=0x7ff19c009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:19.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.642+0000 7ff1b2e00700 1 -- 172.21.15.42:0/2862631525 shutdown_connections 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.642+0000 7ff1b2e00700 1 --2- 172.21.15.42:0/2862631525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 0x7ff1ac07c320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.642+0000 7ff1b2e00700 1 -- 172.21.15.42:0/2862631525 >> 172.21.15.42:0/2862631525 conn(0x7ff1ac103a60 msgr2=0x7ff1ac105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.642+0000 7ff1b2e00700 1 -- 172.21.15.42:0/2862631525 shutdown_connections 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.642+0000 7ff1b2e00700 1 -- 172.21.15.42:0/2862631525 wait complete. 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.643+0000 7ff1b2e00700 1 Processor -- start 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.644+0000 7ff1b2e00700 1 -- start start 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.644+0000 7ff1b2e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 0x7ff1ac19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.644+0000 7ff1b2e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac10ea80 con 0x7ff1ac07de70 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.644+0000 7ff1abe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 0x7ff1ac19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.644+0000 7ff1abe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 0x7ff1ac19e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45184/0 (socket says 172.21.15.42:45184) 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.644+0000 7ff1abe00700 1 -- 172.21.15.42:0/1039483153 learned_addr learned my addr 172.21.15.42:0/1039483153 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.645+0000 7ff1abe00700 1 -- 172.21.15.42:0/1039483153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac19eae0 con 0x7ff1ac07de70 2024-04-20T10:23:19.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.645+0000 7ff1abe00700 1 --2- 172.21.15.42:0/1039483153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 0x7ff1ac19e5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff19c011b60 tx=0x7ff19c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.645+0000 7ff1a8c00700 1 -- 172.21.15.42:0/1039483153 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff19c003b60 con 0x7ff1ac07de70 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.645+0000 7ff1a8c00700 1 -- 172.21.15.42:0/1039483153 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff19c01a910 con 0x7ff1ac07de70 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.645+0000 7ff1a8c00700 1 -- 172.21.15.42:0/1039483153 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff19c024930 con 0x7ff1ac07de70 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.645+0000 7ff1b2e00700 1 -- 172.21.15.42:0/1039483153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ac19ed70 con 0x7ff1ac07de70 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.645+0000 7ff1b2e00700 1 -- 172.21.15.42:0/1039483153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ac19f1d0 con 0x7ff1ac07de70 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.647+0000 7ff1a8c00700 1 -- 172.21.15.42:0/1039483153 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff19c022020 con 0x7ff1ac07de70 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.647+0000 7ff1b2e00700 1 -- 172.21.15.42:0/1039483153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ac0640b0 con 0x7ff1ac07de70 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.647+0000 7ff1a8c00700 1 --2- 172.21.15.42:0/1039483153 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7ff190039600 0x7ff19003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.647+0000 7ff1a8c00700 1 -- 172.21.15.42:0/1039483153 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff19c04d960 con 0x7ff1ac07de70 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.648+0000 7ff1ab400700 1 --2- 172.21.15.42:0/1039483153 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7ff190039600 0x7ff19003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.648+0000 7ff1ab400700 1 --2- 172.21.15.42:0/1039483153 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7ff190039600 0x7ff19003bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff198006fd0 tx=0x7ff198006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.653+0000 7ff1a8c00700 1 -- 172.21.15.42:0/1039483153 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff19c00b3e0 con 0x7ff1ac07de70 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.912+0000 7ff1b2e00700 1 -- 172.21.15.42:0/1039483153 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7ff1ac1a1ba0 con 0x7ff1ac07de70 2024-04-20T10:23:19.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.916+0000 7ff1a8c00700 1 -- 172.21.15.42:0/1039483153 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7ff19c02dde0 con 0x7ff1ac07de70 2024-04-20T10:23:19.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.919+0000 7ff1b2e00700 1 -- 172.21.15.42:0/1039483153 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7ff190039600 msgr2=0x7ff19003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:19.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.919+0000 7ff1b2e00700 1 --2- 172.21.15.42:0/1039483153 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7ff190039600 0x7ff19003bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff198006fd0 tx=0x7ff198006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:19.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.919+0000 7ff1b2e00700 1 -- 172.21.15.42:0/1039483153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 msgr2=0x7ff1ac19e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:19.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.919+0000 7ff1b2e00700 1 --2- 172.21.15.42:0/1039483153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 0x7ff1ac19e5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff19c011b60 tx=0x7ff19c011c40 comp rx=0 tx=0).stop 2024-04-20T10:23:19.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.919+0000 7ff1b2e00700 1 -- 172.21.15.42:0/1039483153 shutdown_connections 2024-04-20T10:23:19.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.919+0000 7ff1b2e00700 1 --2- 172.21.15.42:0/1039483153 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7ff190039600 0x7ff19003bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:19.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.919+0000 7ff1b2e00700 1 --2- 172.21.15.42:0/1039483153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1ac07de70 0x7ff1ac19e5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:19.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.919+0000 7ff1b2e00700 1 -- 172.21.15.42:0/1039483153 >> 172.21.15.42:0/1039483153 conn(0x7ff1ac103a60 msgr2=0x7ff1ac104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:19.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.919+0000 7ff1b2e00700 1 -- 172.21.15.42:0/1039483153 shutdown_connections 2024-04-20T10:23:19.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:18.919+0000 7ff1b2e00700 1 -- 172.21.15.42:0/1039483153 wait complete. 2024-04-20T10:23:19.586 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling the dashboard module... 2024-04-20T10:23:20.208 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:19 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1039483153' entity='client.admin' 2024-04-20T10:23:21.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:21 smithi042 ceph-mon[17078]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:23:21.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:21 smithi042 ceph-mon[17078]: from='mgr.14120 172.21.15.42:0/4206822574' entity='mgr.x' 2024-04-20T10:23:21.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:21 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2292347706' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-20T10:23:21.326 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:21 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: ignoring --setuser ceph since I am not root 2024-04-20T10:23:21.327 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:21 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: ignoring --setgroup ceph since I am not root 2024-04-20T10:23:21.327 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:21 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:21.324+0000 7f3110575200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:23:21.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:21 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:21.596+0000 7f3110575200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:23:21.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:21 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:21.739+0000 7f3110575200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:23:21.911 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.270+0000 7f3d91c00700 1 Processor -- start 2024-04-20T10:23:21.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.270+0000 7f3d91c00700 1 -- start start 2024-04-20T10:23:21.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.271+0000 7f3d91c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 0x7f3d8c10aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:21.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.271+0000 7f3d91c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c10b0c0 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.272+0000 7f3d8aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 0x7f3d8c10aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:21.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.272+0000 7f3d8aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 0x7f3d8c10aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38226/0 (socket says 172.21.15.42:38226) 2024-04-20T10:23:21.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.272+0000 7f3d8aa00700 1 -- 172.21.15.42:0/2255689132 learned_addr learned my addr 172.21.15.42:0/2255689132 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:21.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.272+0000 7f3d8aa00700 1 -- 172.21.15.42:0/2255689132 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c10b8d0 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.273+0000 7f3d8aa00700 1 --2- 172.21.15.42:0/2255689132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 0x7f3d8c10aaf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0098a0 tx=0x7f3d7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79db0ff1d27d3731 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:21.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.273+0000 7f3d89600700 1 -- 172.21.15.42:0/2255689132 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d7c004160 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.273+0000 7f3d89600700 1 -- 172.21.15.42:0/2255689132 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d7c004300 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.273+0000 7f3d89600700 1 -- 172.21.15.42:0/2255689132 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d7c011b20 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.274+0000 7f3d91c00700 1 -- 172.21.15.42:0/2255689132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 msgr2=0x7f3d8c10aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:21.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.274+0000 7f3d91c00700 1 --2- 172.21.15.42:0/2255689132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 0x7f3d8c10aaf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0098a0 tx=0x7f3d7c009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:21.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.274+0000 7f3d91c00700 1 -- 172.21.15.42:0/2255689132 shutdown_connections 2024-04-20T10:23:21.925 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.274+0000 7f3d91c00700 1 --2- 172.21.15.42:0/2255689132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 0x7f3d8c10aaf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:21.925 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.274+0000 7f3d91c00700 1 -- 172.21.15.42:0/2255689132 >> 172.21.15.42:0/2255689132 conn(0x7f3d8c07dbf0 msgr2=0x7f3d8c1084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:21.925 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.274+0000 7f3d91c00700 1 -- 172.21.15.42:0/2255689132 shutdown_connections 2024-04-20T10:23:21.925 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.274+0000 7f3d91c00700 1 -- 172.21.15.42:0/2255689132 wait complete. 2024-04-20T10:23:21.925 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.275+0000 7f3d91c00700 1 Processor -- start 2024-04-20T10:23:21.925 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.275+0000 7f3d91c00700 1 -- start start 2024-04-20T10:23:21.925 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.275+0000 7f3d91c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 0x7f3d8c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:21.925 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.275+0000 7f3d91c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c10ec80 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.926 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.276+0000 7f3d8aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 0x7f3d8c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:21.926 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.276+0000 7f3d8aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 0x7f3d8c19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38228/0 (socket says 172.21.15.42:38228) 2024-04-20T10:23:21.926 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.276+0000 7f3d8aa00700 1 -- 172.21.15.42:0/2292347706 learned_addr learned my addr 172.21.15.42:0/2292347706 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:21.926 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.276+0000 7f3d8aa00700 1 -- 172.21.15.42:0/2292347706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c19eb00 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.926 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.276+0000 7f3d8aa00700 1 --2- 172.21.15.42:0/2292347706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 0x7f3d8c19e5c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c000c00 tx=0x7f3d7c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:21.926 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.277+0000 7f3d83400700 1 -- 172.21.15.42:0/2292347706 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d7c003b60 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.926 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.277+0000 7f3d83400700 1 -- 172.21.15.42:0/2292347706 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d7c01a910 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.926 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.277+0000 7f3d91c00700 1 -- 172.21.15.42:0/2292347706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8c19ed90 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.927 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.277+0000 7f3d83400700 1 -- 172.21.15.42:0/2292347706 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d7c024930 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.927 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.277+0000 7f3d91c00700 1 -- 172.21.15.42:0/2292347706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8c19f1f0 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.927 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.278+0000 7f3d83400700 1 -- 172.21.15.42:0/2292347706 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3d7c022020 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.927 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.278+0000 7f3d91c00700 1 -- 172.21.15.42:0/2292347706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8c0640b0 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.927 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.279+0000 7f3d83400700 1 --2- 172.21.15.42:0/2292347706 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f3d70039600 0x7f3d7003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:21.927 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.279+0000 7f3d83400700 1 -- 172.21.15.42:0/2292347706 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3d7c04d940 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.928 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.279+0000 7f3d8a000700 1 --2- 172.21.15.42:0/2292347706 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f3d70039600 0x7f3d7003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:21.928 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.280+0000 7f3d8a000700 1 --2- 172.21.15.42:0/2292347706 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f3d70039600 0x7f3d7003bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3d78006fd0 tx=0x7f3d78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:21.928 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.285+0000 7f3d83400700 1 -- 172.21.15.42:0/2292347706 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3d7c00f2e0 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.928 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:20.581+0000 7f3d91c00700 1 -- 172.21.15.42:0/2292347706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f3d8c1a1bc0 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.928 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:21.002+0000 7f3d83400700 1 -- 172.21.15.42:0/2292347706 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f3d7c04d290 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.928 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:21.007+0000 7f3d83400700 1 -- 172.21.15.42:0/2292347706 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f3d7c02d3f0 con 0x7f3d8c10a6d0 2024-04-20T10:23:21.928 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:21.008+0000 7f3d91c00700 1 -- 172.21.15.42:0/2292347706 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f3d70039600 msgr2=0x7f3d7003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:21.929 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:21.008+0000 7f3d91c00700 1 --2- 172.21.15.42:0/2292347706 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f3d70039600 0x7f3d7003bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3d78006fd0 tx=0x7f3d78006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:21.929 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:21.008+0000 7f3d91c00700 1 -- 172.21.15.42:0/2292347706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 msgr2=0x7f3d8c19e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:21.929 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:21.008+0000 7f3d91c00700 1 --2- 172.21.15.42:0/2292347706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 0x7f3d8c19e5c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c000c00 tx=0x7f3d7c011c10 comp rx=0 tx=0).stop 2024-04-20T10:23:21.929 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:21.009+0000 7f3d91c00700 1 -- 172.21.15.42:0/2292347706 shutdown_connections 2024-04-20T10:23:21.929 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:21.009+0000 7f3d91c00700 1 --2- 172.21.15.42:0/2292347706 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f3d70039600 0x7f3d7003bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:21.929 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:21.009+0000 7f3d91c00700 1 --2- 172.21.15.42:0/2292347706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d8c10a6d0 0x7f3d8c19e5c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:21.929 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:21.009+0000 7f3d91c00700 1 -- 172.21.15.42:0/2292347706 >> 172.21.15.42:0/2292347706 conn(0x7f3d8c07dbf0 msgr2=0x7f3d8c107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:21.929 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:21.009+0000 7f3d91c00700 1 -- 172.21.15.42:0/2292347706 shutdown_connections 2024-04-20T10:23:21.930 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:21.009+0000 7f3d91c00700 1 -- 172.21.15.42:0/2292347706 wait complete. 2024-04-20T10:23:22.162 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:22 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2292347706' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-20T10:23:22.162 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:22 smithi042 ceph-mon[17078]: mgrmap e9: x(active, since 25s) 2024-04-20T10:23:22.510 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:22 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:22.161+0000 7f3110575200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:23:22.510 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:22 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:22.276+0000 7f3110575200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:23:23.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:23 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/759795899' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-20T10:23:24.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:23:24.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-20T10:23:24.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T10:23:24.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-20T10:23:24.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-20T10:23:24.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:23:24.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.915+0000 7fe848000700 1 Processor -- start 2024-04-20T10:23:24.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.915+0000 7fe848000700 1 -- start start 2024-04-20T10:23:24.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.916+0000 7fe848000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 0x7fe8401088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.916+0000 7fe848000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe840108e90 con 0x7fe8401084a0 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.916+0000 7fe845600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 0x7fe8401088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.916+0000 7fe845600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 0x7fe8401088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38266/0 (socket says 172.21.15.42:38266) 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.916+0000 7fe845600700 1 -- 172.21.15.42:0/3230111132 learned_addr learned my addr 172.21.15.42:0/3230111132 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.917+0000 7fe845600700 1 -- 172.21.15.42:0/3230111132 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe840109650 con 0x7fe8401084a0 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.917+0000 7fe845600700 1 --2- 172.21.15.42:0/3230111132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 0x7fe8401088c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8340098a0 tx=0x7fe834009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8905a3df7e7f1533 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.917+0000 7fe83fe00700 1 -- 172.21.15.42:0/3230111132 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe834004160 con 0x7fe8401084a0 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.917+0000 7fe83fe00700 1 -- 172.21.15.42:0/3230111132 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe834004300 con 0x7fe8401084a0 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.917+0000 7fe83fe00700 1 -- 172.21.15.42:0/3230111132 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe834011b20 con 0x7fe8401084a0 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.918+0000 7fe848000700 1 -- 172.21.15.42:0/3230111132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 msgr2=0x7fe8401088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.918+0000 7fe848000700 1 --2- 172.21.15.42:0/3230111132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 0x7fe8401088c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8340098a0 tx=0x7fe834009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.918+0000 7fe848000700 1 -- 172.21.15.42:0/3230111132 shutdown_connections 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.918+0000 7fe848000700 1 --2- 172.21.15.42:0/3230111132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 0x7fe8401088c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:24.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.918+0000 7fe848000700 1 -- 172.21.15.42:0/3230111132 >> 172.21.15.42:0/3230111132 conn(0x7fe840103a60 msgr2=0x7fe840105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.918+0000 7fe848000700 1 -- 172.21.15.42:0/3230111132 shutdown_connections 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.918+0000 7fe848000700 1 -- 172.21.15.42:0/3230111132 wait complete. 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.919+0000 7fe848000700 1 Processor -- start 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.919+0000 7fe848000700 1 -- start start 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.920+0000 7fe848000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 0x7fe84019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.920+0000 7fe848000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe84010ca00 con 0x7fe8401084a0 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.920+0000 7fe845600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 0x7fe84019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.920+0000 7fe845600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 0x7fe84019e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38274/0 (socket says 172.21.15.42:38274) 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.920+0000 7fe845600700 1 -- 172.21.15.42:0/759795899 learned_addr learned my addr 172.21.15.42:0/759795899 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.920+0000 7fe845600700 1 -- 172.21.15.42:0/759795899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84019eb20 con 0x7fe8401084a0 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.921+0000 7fe845600700 1 --2- 172.21.15.42:0/759795899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 0x7fe84019e5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8340116a0 tx=0x7fe834011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.921+0000 7fe83e000700 1 -- 172.21.15.42:0/759795899 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe834003a40 con 0x7fe8401084a0 2024-04-20T10:23:24.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.921+0000 7fe83e000700 1 -- 172.21.15.42:0/759795899 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe834003be0 con 0x7fe8401084a0 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.921+0000 7fe848000700 1 -- 172.21.15.42:0/759795899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe84019edb0 con 0x7fe8401084a0 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.921+0000 7fe83e000700 1 -- 172.21.15.42:0/759795899 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe83401a720 con 0x7fe8401084a0 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.921+0000 7fe848000700 1 -- 172.21.15.42:0/759795899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe84019f210 con 0x7fe8401084a0 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.922+0000 7fe83e000700 1 -- 172.21.15.42:0/759795899 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fe834024430 con 0x7fe8401084a0 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.922+0000 7fe848000700 1 -- 172.21.15.42:0/759795899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8400640b0 con 0x7fe8401084a0 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.923+0000 7fe83e000700 1 --2- 172.21.15.42:0/759795899 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe828039610 0x7fe82803bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.923+0000 7fe83e000700 1 -- 172.21.15.42:0/759795899 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe83404dd00 con 0x7fe8401084a0 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.923+0000 7fe844c00700 1 -- 172.21.15.42:0/759795899 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe828039610 msgr2=0x7fe82803bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2743527860 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.923+0000 7fe844c00700 1 --2- 172.21.15.42:0/759795899 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe828039610 0x7fe82803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:22.928+0000 7fe83e000700 1 -- 172.21.15.42:0/759795899 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe834011e20 con 0x7fe8401084a0 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.124+0000 7fe844c00700 1 -- 172.21.15.42:0/759795899 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe828039610 msgr2=0x7fe82803bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2743527860 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.124+0000 7fe844c00700 1 --2- 172.21.15.42:0/759795899 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe828039610 0x7fe82803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:23:24.019 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.218+0000 7fe848000700 1 -- 172.21.15.42:0/759795899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe8401a1bc0 con 0x7fe8401084a0 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.219+0000 7fe83e000700 1 -- 172.21.15.42:0/759795899 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fe83404d650 con 0x7fe8401084a0 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.222+0000 7fe848000700 1 -- 172.21.15.42:0/759795899 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe828039610 msgr2=0x7fe82803bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.222+0000 7fe848000700 1 --2- 172.21.15.42:0/759795899 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe828039610 0x7fe82803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.222+0000 7fe848000700 1 -- 172.21.15.42:0/759795899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 msgr2=0x7fe84019e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.222+0000 7fe848000700 1 --2- 172.21.15.42:0/759795899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 0x7fe84019e5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8340116a0 tx=0x7fe834011ba0 comp rx=0 tx=0).stop 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.222+0000 7fe848000700 1 -- 172.21.15.42:0/759795899 shutdown_connections 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.222+0000 7fe848000700 1 --2- 172.21.15.42:0/759795899 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7fe828039610 0x7fe82803bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.222+0000 7fe848000700 1 --2- 172.21.15.42:0/759795899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8401084a0 0x7fe84019e5e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.222+0000 7fe848000700 1 -- 172.21.15.42:0/759795899 >> 172.21.15.42:0/759795899 conn(0x7fe840103a60 msgr2=0x7fe8401044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.222+0000 7fe848000700 1 -- 172.21.15.42:0/759795899 shutdown_connections 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:23.223+0000 7fe848000700 1 -- 172.21.15.42:0/759795899 wait complete. 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2024-04-20T10:23:24.020 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 9... 2024-04-20T10:23:24.177 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:23 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:23.916+0000 7f3110575200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:23:24.177 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:24 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:24.158+0000 7f3110575200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:23:24.436 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:24 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:24.384+0000 7f3110575200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:23:25.103 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:24 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:24.785+0000 7f3110575200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:23:25.103 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:24 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:24.914+0000 7f3110575200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:23:25.438 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:25 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:25.173+0000 7f3110575200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:23:25.438 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:25 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:25.437+0000 7f3110575200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:23:26.353 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:26 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:26.257+0000 7f3110575200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:23:27.103 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:26 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:26.834+0000 7f3110575200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:23:27.103 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:26 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:26.967+0000 7f3110575200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:23:27.486 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:27 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:27.249+0000 7f3110575200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:23:28.064 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:27 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:27.930+0000 7f3110575200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:23:28.064 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:28 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:28.063+0000 7f3110575200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:23:28.723 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:28 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:28.400+0000 7f3110575200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:23:28.723 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:28 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:28.588+0000 7f3110575200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:23:28.723 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:28 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:28.722+0000 7f3110575200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:23:29.103 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:28 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:28.880+0000 7f3110575200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:23:29.603 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:29 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:29.265+0000 7f3110575200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:23:30.353 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:30 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:30.062+0000 7f3110575200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:23:30.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:23:30 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:23:30.361+0000 7f3110575200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:23:30.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:30 smithi042 ceph-mon[17078]: Active manager daemon x restarted 2024-04-20T10:23:30.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:30 smithi042 ceph-mon[17078]: Activating manager daemon x 2024-04-20T10:23:30.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:30 smithi042 ceph-mon[17078]: osdmap e3: 0 total, 0 up, 0 in 2024-04-20T10:23:30.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:30 smithi042 ceph-mon[17078]: mgrmap e10: x(active, starting, since 0.00363773s) 2024-04-20T10:23:30.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:23:30.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T10:23:30.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T10:23:30.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T10:23:30.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T10:23:30.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:30 smithi042 ceph-mon[17078]: Manager daemon x is now available 2024-04-20T10:23:31.753 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:23:31.754 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:23:31.754 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T10:23:31.754 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:31 smithi042 ceph-mon[17078]: mgrmap e11: x(active, since 1.00883s) 2024-04-20T10:23:32.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-20T10:23:32.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-20T10:23:32.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-20T10:23:32.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-20T10:23:32.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.596+0000 7f1361800700 1 Processor -- start 2024-04-20T10:23:32.041 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.596+0000 7f1361800700 1 -- start start 2024-04-20T10:23:32.041 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.597+0000 7f1361800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 0x7f135c1077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:32.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.597+0000 7f1361800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f135c107da0 con 0x7f135c1073b0 2024-04-20T10:23:32.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.597+0000 7f135aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 0x7f135c1077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:32.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.597+0000 7f135aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 0x7f135c1077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38286/0 (socket says 172.21.15.42:38286) 2024-04-20T10:23:32.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.597+0000 7f135aa00700 1 -- 172.21.15.42:0/3020911772 learned_addr learned my addr 172.21.15.42:0/3020911772 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:32.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.598+0000 7f135aa00700 1 -- 172.21.15.42:0/3020911772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c1085b0 con 0x7f135c1073b0 2024-04-20T10:23:32.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.598+0000 7f135aa00700 1 --2- 172.21.15.42:0/3020911772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 0x7f135c1077d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f134c0098a0 tx=0x7f134c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f1625c2b1e03b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:32.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.598+0000 7f1359600700 1 -- 172.21.15.42:0/3020911772 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f134c004160 con 0x7f135c1073b0 2024-04-20T10:23:32.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.598+0000 7f1359600700 1 -- 172.21.15.42:0/3020911772 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134c004300 con 0x7f135c1073b0 2024-04-20T10:23:32.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.598+0000 7f1359600700 1 -- 172.21.15.42:0/3020911772 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f134c011b20 con 0x7f135c1073b0 2024-04-20T10:23:32.043 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.599+0000 7f1361800700 1 -- 172.21.15.42:0/3020911772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 msgr2=0x7f135c1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:32.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.599+0000 7f1361800700 1 --2- 172.21.15.42:0/3020911772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 0x7f135c1077d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f134c0098a0 tx=0x7f134c009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:32.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.599+0000 7f1361800700 1 -- 172.21.15.42:0/3020911772 shutdown_connections 2024-04-20T10:23:32.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.599+0000 7f1361800700 1 --2- 172.21.15.42:0/3020911772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 0x7f135c1077d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:32.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.599+0000 7f1361800700 1 -- 172.21.15.42:0/3020911772 >> 172.21.15.42:0/3020911772 conn(0x7f135c102950 msgr2=0x7f135c104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:32.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.599+0000 7f1361800700 1 -- 172.21.15.42:0/3020911772 shutdown_connections 2024-04-20T10:23:32.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.600+0000 7f1361800700 1 -- 172.21.15.42:0/3020911772 wait complete. 2024-04-20T10:23:32.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.601+0000 7f1361800700 1 Processor -- start 2024-04-20T10:23:32.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.601+0000 7f1361800700 1 -- start start 2024-04-20T10:23:32.044 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.601+0000 7f1361800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 0x7f135c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:32.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.601+0000 7f1361800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f135c10b960 con 0x7f135c1073b0 2024-04-20T10:23:32.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.601+0000 7f135aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 0x7f135c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:32.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.601+0000 7f135aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 0x7f135c19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38298/0 (socket says 172.21.15.42:38298) 2024-04-20T10:23:32.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.601+0000 7f135aa00700 1 -- 172.21.15.42:0/3206558623 learned_addr learned my addr 172.21.15.42:0/3206558623 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:32.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.602+0000 7f135aa00700 1 -- 172.21.15.42:0/3206558623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c19c980 con 0x7f135c1073b0 2024-04-20T10:23:32.050 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.602+0000 7f135aa00700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 0x7f135c19c440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f134c000c00 tx=0x7f134c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.602+0000 7f1353400700 1 -- 172.21.15.42:0/3206558623 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f134c003b60 con 0x7f135c1073b0 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.602+0000 7f1353400700 1 -- 172.21.15.42:0/3206558623 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134c01a910 con 0x7f135c1073b0 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.602+0000 7f1361800700 1 -- 172.21.15.42:0/3206558623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135c19cc10 con 0x7f135c1073b0 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.602+0000 7f1353400700 1 -- 172.21.15.42:0/3206558623 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f134c024930 con 0x7f135c1073b0 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.603+0000 7f1361800700 1 -- 172.21.15.42:0/3206558623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135c19d070 con 0x7f135c1073b0 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.604+0000 7f1353400700 1 -- 172.21.15.42:0/3206558623 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f134c022020 con 0x7f135c1073b0 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.605+0000 7f1353400700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 0x7f134003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.605+0000 7f1353400700 1 -- 172.21.15.42:0/3206558623 --> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f134003c280 con 0x7f13400396b0 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.605+0000 7f1353400700 1 -- 172.21.15.42:0/3206558623 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f134c04d9a0 con 0x7f135c1073b0 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.605+0000 7f135a000700 1 -- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 msgr2=0x7f134003bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2743527860 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.605+0000 7f135a000700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 0x7f134003bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.806+0000 7f135a000700 1 -- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 msgr2=0x7f134003bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2743527860 2024-04-20T10:23:32.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:24.806+0000 7f135a000700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 0x7f134003bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:25.207+0000 7f135a000700 1 -- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 msgr2=0x7f134003bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2743527860 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:25.207+0000 7f135a000700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 0x7f134003bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:26.008+0000 7f135a000700 1 -- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 msgr2=0x7f134003bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2743527860 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:26.008+0000 7f135a000700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 0x7f134003bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:27.610+0000 7f135a000700 1 -- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 msgr2=0x7f134003bb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2743527860 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:27.610+0000 7f135a000700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 0x7f134003bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:30.369+0000 7f1353400700 1 -- 172.21.15.42:0/3206558623 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f134c02d3f0 con 0x7f135c1073b0 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:30.369+0000 7f1353400700 1 -- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 msgr2=0x7f134003bb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:30.369+0000 7f1353400700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 0x7f134003bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.370+0000 7f1353400700 1 -- 172.21.15.42:0/3206558623 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f134c04f690 con 0x7f135c1073b0 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.370+0000 7f1353400700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f134003cff0 0x7f134003f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.371+0000 7f1353400700 1 -- 172.21.15.42:0/3206558623 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f134c00af30 con 0x7f134003cff0 2024-04-20T10:23:32.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.372+0000 7f135a000700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f134003cff0 0x7f134003f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.374+0000 7f135a000700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f134003cff0 0x7f134003f3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1348003a10 tx=0x7f13480092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.375+0000 7f1353400700 1 -- 172.21.15.42:0/3206558623 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f134c00af30 con 0x7f134003cff0 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.387+0000 7f1361800700 1 -- 172.21.15.42:0/3206558623 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f135c0640b0 con 0x7f134003cff0 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.388+0000 7f1353400700 1 -- 172.21.15.42:0/3206558623 <== mgr.14152 v2:172.21.15.42:6800/3610568749 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f135c0640b0 con 0x7f134003cff0 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.388+0000 7f1361800700 1 -- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f134003cff0 msgr2=0x7f134003f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.388+0000 7f1361800700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f134003cff0 0x7f134003f3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1348003a10 tx=0x7f13480092b0 comp rx=0 tx=0).stop 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.388+0000 7f1361800700 1 -- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 msgr2=0x7f135c19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.388+0000 7f1361800700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 0x7f135c19c440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f134c000c00 tx=0x7f134c011c10 comp rx=0 tx=0).stop 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.389+0000 7f1361800700 1 -- 172.21.15.42:0/3206558623 shutdown_connections 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.389+0000 7f1361800700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f134003cff0 0x7f134003f3e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.389+0000 7f1361800700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f135c1073b0 0x7f135c19c440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.389+0000 7f1361800700 1 --2- 172.21.15.42:0/3206558623 >> [v2:172.21.15.42:6800/2743527860,v1:172.21.15.42:6801/2743527860] conn(0x7f13400396b0 0x7f134003bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:32.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.389+0000 7f1361800700 1 -- 172.21.15.42:0/3206558623 >> 172.21.15.42:0/3206558623 conn(0x7f135c102950 msgr2=0x7f135c192f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:32.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.389+0000 7f1361800700 1 -- 172.21.15.42:0/3206558623 shutdown_connections 2024-04-20T10:23:32.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:31.389+0000 7f1361800700 1 -- 172.21.15.42:0/3206558623 wait complete. 2024-04-20T10:23:32.054 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 9 is available 2024-04-20T10:23:32.054 INFO:teuthology.orchestra.run.smithi042.stdout:Generating a dashboard self-signed certificate... 2024-04-20T10:23:32.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:32 smithi042 ceph-mon[17078]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-20T10:23:32.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:32 smithi042 ceph-mon[17078]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-20T10:23:33.725 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:33 smithi042 ceph-mon[17078]: [20/Apr/2024:10:23:32] ENGINE Bus STARTING 2024-04-20T10:23:33.725 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:33 smithi042 ceph-mon[17078]: [20/Apr/2024:10:23:32] ENGINE Serving on http://172.21.15.42:8765 2024-04-20T10:23:33.725 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:33 smithi042 ceph-mon[17078]: mgrmap e12: x(active, since 2s) 2024-04-20T10:23:33.725 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:33 smithi042 ceph-mon[17078]: [20/Apr/2024:10:23:32] ENGINE Serving on https://172.21.15.42:7150 2024-04-20T10:23:33.725 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:33 smithi042 ceph-mon[17078]: [20/Apr/2024:10:23:32] ENGINE Bus STARTED 2024-04-20T10:23:33.725 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:33 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:34.342 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-20T10:23:34.342 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.256+0000 7f92e2c00700 1 Processor -- start 2024-04-20T10:23:34.342 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.256+0000 7f92e2c00700 1 -- start start 2024-04-20T10:23:34.342 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.257+0000 7f92e2c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 0x7f92dc10c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:34.342 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.257+0000 7f92e2c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc10c650 con 0x7f92dc10bc60 2024-04-20T10:23:34.342 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.257+0000 7f92dbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 0x7f92dc10c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:34.342 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.257+0000 7f92dbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 0x7f92dc10c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50092/0 (socket says 172.21.15.42:50092) 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.257+0000 7f92dbe00700 1 -- 172.21.15.42:0/2509884147 learned_addr learned my addr 172.21.15.42:0/2509884147 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.258+0000 7f92dbe00700 1 -- 172.21.15.42:0/2509884147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc10a2f0 con 0x7f92dc10bc60 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.258+0000 7f92dbe00700 1 --2- 172.21.15.42:0/2509884147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 0x7f92dc10c080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0098a0 tx=0x7f92cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff73c46e14ff2b9c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.258+0000 7f92daa00700 1 -- 172.21.15.42:0/2509884147 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92cc004160 con 0x7f92dc10bc60 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.258+0000 7f92daa00700 1 -- 172.21.15.42:0/2509884147 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92cc004300 con 0x7f92dc10bc60 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.259+0000 7f92daa00700 1 -- 172.21.15.42:0/2509884147 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92cc011b20 con 0x7f92dc10bc60 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.259+0000 7f92e2c00700 1 -- 172.21.15.42:0/2509884147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 msgr2=0x7f92dc10c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.259+0000 7f92e2c00700 1 --2- 172.21.15.42:0/2509884147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 0x7f92dc10c080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0098a0 tx=0x7f92cc009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.259+0000 7f92e2c00700 1 -- 172.21.15.42:0/2509884147 shutdown_connections 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.259+0000 7f92e2c00700 1 --2- 172.21.15.42:0/2509884147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 0x7f92dc10c080 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.259+0000 7f92e2c00700 1 -- 172.21.15.42:0/2509884147 >> 172.21.15.42:0/2509884147 conn(0x7f92dc07dc10 msgr2=0x7f92dc1084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.259+0000 7f92e2c00700 1 -- 172.21.15.42:0/2509884147 shutdown_connections 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.259+0000 7f92e2c00700 1 -- 172.21.15.42:0/2509884147 wait complete. 2024-04-20T10:23:34.343 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.260+0000 7f92e2c00700 1 Processor -- start 2024-04-20T10:23:34.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.261+0000 7f92e2c00700 1 -- start start 2024-04-20T10:23:34.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.261+0000 7f92e2c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 0x7f92dc1a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:34.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.261+0000 7f92e2c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc080480 con 0x7f92dc10bc60 2024-04-20T10:23:34.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.261+0000 7f92dbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 0x7f92dc1a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:34.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.261+0000 7f92dbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 0x7f92dc1a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50104/0 (socket says 172.21.15.42:50104) 2024-04-20T10:23:34.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.261+0000 7f92dbe00700 1 -- 172.21.15.42:0/594646571 learned_addr learned my addr 172.21.15.42:0/594646571 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:34.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.262+0000 7f92dbe00700 1 -- 172.21.15.42:0/594646571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc1a2ff0 con 0x7f92dc10bc60 2024-04-20T10:23:34.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.262+0000 7f92dbe00700 1 --2- 172.21.15.42:0/594646571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 0x7f92dc1a2ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0116a0 tx=0x7f92cc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:34.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.262+0000 7f92d8c00700 1 -- 172.21.15.42:0/594646571 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92cc003a40 con 0x7f92dc10bc60 2024-04-20T10:23:34.344 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.262+0000 7f92d8c00700 1 -- 172.21.15.42:0/594646571 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92cc003be0 con 0x7f92dc10bc60 2024-04-20T10:23:34.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.262+0000 7f92d8c00700 1 -- 172.21.15.42:0/594646571 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92cc01a720 con 0x7f92dc10bc60 2024-04-20T10:23:34.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.263+0000 7f92e2c00700 1 -- 172.21.15.42:0/594646571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92dc1a3280 con 0x7f92dc10bc60 2024-04-20T10:23:34.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.263+0000 7f92e2c00700 1 -- 172.21.15.42:0/594646571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92dc1a36e0 con 0x7f92dc10bc60 2024-04-20T10:23:34.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.264+0000 7f92d8c00700 1 -- 172.21.15.42:0/594646571 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f92cc024aa0 con 0x7f92dc10bc60 2024-04-20T10:23:34.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.265+0000 7f92e2c00700 1 -- 172.21.15.42:0/594646571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92dc0640b0 con 0x7f92dc10bc60 2024-04-20T10:23:34.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.265+0000 7f92d8c00700 1 --2- 172.21.15.42:0/594646571 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f92c0039610 0x7f92c003bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:34.345 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.265+0000 7f92d8c00700 1 -- 172.21.15.42:0/594646571 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f92cc04e2d0 con 0x7f92dc10bc60 2024-04-20T10:23:34.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.266+0000 7f92db400700 1 --2- 172.21.15.42:0/594646571 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f92c0039610 0x7f92c003bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.266+0000 7f92db400700 1 --2- 172.21.15.42:0/594646571 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f92c0039610 0x7f92c003bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92c8006fd0 tx=0x7f92c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.271+0000 7f92d8c00700 1 -- 172.21.15.42:0/594646571 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f92cc029060 con 0x7f92dc10bc60 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.517+0000 7f92e2c00700 1 -- 172.21.15.42:0/594646571 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f92dc062e90 con 0x7f92c0039610 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.723+0000 7f92d8c00700 1 -- 172.21.15.42:0/594646571 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f92dc062e90 con 0x7f92c0039610 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.727+0000 7f92e2c00700 1 -- 172.21.15.42:0/594646571 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f92c0039610 msgr2=0x7f92c003bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.727+0000 7f92e2c00700 1 --2- 172.21.15.42:0/594646571 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f92c0039610 0x7f92c003bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92c8006fd0 tx=0x7f92c8006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.727+0000 7f92e2c00700 1 -- 172.21.15.42:0/594646571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 msgr2=0x7f92dc1a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.727+0000 7f92e2c00700 1 --2- 172.21.15.42:0/594646571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 0x7f92dc1a2ab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0116a0 tx=0x7f92cc011ba0 comp rx=0 tx=0).stop 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.727+0000 7f92e2c00700 1 -- 172.21.15.42:0/594646571 shutdown_connections 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.727+0000 7f92e2c00700 1 --2- 172.21.15.42:0/594646571 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f92c0039610 0x7f92c003bad0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.727+0000 7f92e2c00700 1 --2- 172.21.15.42:0/594646571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92dc10bc60 0x7f92dc1a2ab0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.727+0000 7f92e2c00700 1 -- 172.21.15.42:0/594646571 >> 172.21.15.42:0/594646571 conn(0x7f92dc07dc10 msgr2=0x7f92dc106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.727+0000 7f92e2c00700 1 -- 172.21.15.42:0/594646571 shutdown_connections 2024-04-20T10:23:34.348 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:33.727+0000 7f92e2c00700 1 -- 172.21.15.42:0/594646571 wait complete. 2024-04-20T10:23:34.349 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial admin user... 2024-04-20T10:23:34.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:34 smithi042 ceph-mon[17078]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:23:34.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:34.935 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:36.248 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$6Z.Xm8D1GavdBGUz05tR6ed1h7vtlkgQ90M1WzLjCd52cyHZ2xQ8G", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713608615, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-20T10:23:36.249 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.007+0000 7fd245200700 1 Processor -- start 2024-04-20T10:23:36.249 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.007+0000 7fd245200700 1 -- start start 2024-04-20T10:23:36.249 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.008+0000 7fd245200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 0x7fd24007c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:36.249 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.008+0000 7fd245200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd24007c910 con 0x7fd24007de90 2024-04-20T10:23:36.249 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.008+0000 7fd23f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 0x7fd24007c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:36.249 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.008+0000 7fd23f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 0x7fd24007c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50112/0 (socket says 172.21.15.42:50112) 2024-04-20T10:23:36.249 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.008+0000 7fd23f400700 1 -- 172.21.15.42:0/1102551136 learned_addr learned my addr 172.21.15.42:0/1102551136 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:36.249 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.009+0000 7fd23f400700 1 -- 172.21.15.42:0/1102551136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24007ca90 con 0x7fd24007de90 2024-04-20T10:23:36.249 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.009+0000 7fd23f400700 1 --2- 172.21.15.42:0/1102551136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 0x7fd24007c340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2300098a0 tx=0x7fd230009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2714a025d9ea26b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:36.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.010+0000 7fd23ea00700 1 -- 172.21.15.42:0/1102551136 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd230004160 con 0x7fd24007de90 2024-04-20T10:23:36.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.010+0000 7fd23ea00700 1 -- 172.21.15.42:0/1102551136 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd230004300 con 0x7fd24007de90 2024-04-20T10:23:36.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.010+0000 7fd23ea00700 1 -- 172.21.15.42:0/1102551136 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd230011b20 con 0x7fd24007de90 2024-04-20T10:23:36.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.010+0000 7fd245200700 1 -- 172.21.15.42:0/1102551136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 msgr2=0x7fd24007c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:36.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.010+0000 7fd245200700 1 --2- 172.21.15.42:0/1102551136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 0x7fd24007c340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2300098a0 tx=0x7fd230009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:36.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.010+0000 7fd245200700 1 -- 172.21.15.42:0/1102551136 shutdown_connections 2024-04-20T10:23:36.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.010+0000 7fd245200700 1 --2- 172.21.15.42:0/1102551136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 0x7fd24007c340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:36.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.011+0000 7fd245200700 1 -- 172.21.15.42:0/1102551136 >> 172.21.15.42:0/1102551136 conn(0x7fd2401038b0 msgr2=0x7fd240105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:36.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.011+0000 7fd245200700 1 -- 172.21.15.42:0/1102551136 shutdown_connections 2024-04-20T10:23:36.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.011+0000 7fd245200700 1 -- 172.21.15.42:0/1102551136 wait complete. 2024-04-20T10:23:36.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.012+0000 7fd245200700 1 Processor -- start 2024-04-20T10:23:36.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.012+0000 7fd245200700 1 -- start start 2024-04-20T10:23:36.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.012+0000 7fd245200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 0x7fd24019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:36.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.012+0000 7fd245200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd24010e910 con 0x7fd24007de90 2024-04-20T10:23:36.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.012+0000 7fd23f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 0x7fd24019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:36.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.013+0000 7fd23f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 0x7fd24019e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50122/0 (socket says 172.21.15.42:50122) 2024-04-20T10:23:36.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.013+0000 7fd23f400700 1 -- 172.21.15.42:0/1188271424 learned_addr learned my addr 172.21.15.42:0/1188271424 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:36.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.013+0000 7fd23f400700 1 -- 172.21.15.42:0/1188271424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24019e9a0 con 0x7fd24007de90 2024-04-20T10:23:36.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.013+0000 7fd23f400700 1 --2- 172.21.15.42:0/1188271424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 0x7fd24019e460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2300116a0 tx=0x7fd230011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:36.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.014+0000 7fd23cc00700 1 -- 172.21.15.42:0/1188271424 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd230003a40 con 0x7fd24007de90 2024-04-20T10:23:36.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.014+0000 7fd23cc00700 1 -- 172.21.15.42:0/1188271424 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd230003be0 con 0x7fd24007de90 2024-04-20T10:23:36.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.014+0000 7fd23cc00700 1 -- 172.21.15.42:0/1188271424 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd23001a750 con 0x7fd24007de90 2024-04-20T10:23:36.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.014+0000 7fd245200700 1 -- 172.21.15.42:0/1188271424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24019ec30 con 0x7fd24007de90 2024-04-20T10:23:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.014+0000 7fd245200700 1 -- 172.21.15.42:0/1188271424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24019f090 con 0x7fd24007de90 2024-04-20T10:23:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.015+0000 7fd245200700 1 -- 172.21.15.42:0/1188271424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2400640b0 con 0x7fd24007de90 2024-04-20T10:23:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.016+0000 7fd23cc00700 1 -- 172.21.15.42:0/1188271424 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fd230024aa0 con 0x7fd24007de90 2024-04-20T10:23:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.017+0000 7fd23cc00700 1 --2- 172.21.15.42:0/1188271424 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd2240395b0 0x7fd22403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.017+0000 7fd23cc00700 1 -- 172.21.15.42:0/1188271424 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd23004dfd0 con 0x7fd24007de90 2024-04-20T10:23:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.017+0000 7fd236a00700 1 --2- 172.21.15.42:0/1188271424 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd2240395b0 0x7fd22403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.018+0000 7fd236a00700 1 --2- 172.21.15.42:0/1188271424 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd2240395b0 0x7fd22403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd228006fd0 tx=0x7fd228006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.023+0000 7fd23cc00700 1 -- 172.21.15.42:0/1188271424 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd230011e20 con 0x7fd24007de90 2024-04-20T10:23:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.269+0000 7fd245200700 1 -- 172.21.15.42:0/1188271424 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fd2401a19f0 con 0x7fd2240395b0 2024-04-20T10:23:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.524+0000 7fd23cc00700 1 -- 172.21.15.42:0/1188271424 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fd2401a19f0 con 0x7fd2240395b0 2024-04-20T10:23:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.527+0000 7fd245200700 1 -- 172.21.15.42:0/1188271424 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd2240395b0 msgr2=0x7fd22403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:36.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.527+0000 7fd245200700 1 --2- 172.21.15.42:0/1188271424 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd2240395b0 0x7fd22403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd228006fd0 tx=0x7fd228006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:36.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.527+0000 7fd245200700 1 -- 172.21.15.42:0/1188271424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 msgr2=0x7fd24019e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:36.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.527+0000 7fd245200700 1 --2- 172.21.15.42:0/1188271424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 0x7fd24019e460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2300116a0 tx=0x7fd230011ba0 comp rx=0 tx=0).stop 2024-04-20T10:23:36.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.527+0000 7fd245200700 1 -- 172.21.15.42:0/1188271424 shutdown_connections 2024-04-20T10:23:36.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.527+0000 7fd245200700 1 --2- 172.21.15.42:0/1188271424 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd2240395b0 0x7fd22403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:36.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.527+0000 7fd245200700 1 --2- 172.21.15.42:0/1188271424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd24007de90 0x7fd24019e460 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:36.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.527+0000 7fd245200700 1 -- 172.21.15.42:0/1188271424 >> 172.21.15.42:0/1188271424 conn(0x7fd2401038b0 msgr2=0x7fd2401043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:36.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.527+0000 7fd245200700 1 -- 172.21.15.42:0/1188271424 shutdown_connections 2024-04-20T10:23:36.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:35.527+0000 7fd245200700 1 -- 172.21.15.42:0/1188271424 wait complete. 2024-04-20T10:23:36.253 INFO:teuthology.orchestra.run.smithi042.stdout:Fetching dashboard port number... 2024-04-20T10:23:36.576 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:36 smithi042 ceph-mon[17078]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:23:36.576 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:36 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:36.576 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:36 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:37.505 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:37 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4233928834' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-20T10:23:38.442 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:38 smithi042 ceph-mon[17078]: mgrmap e13: x(active, since 7s) 2024-04-20T10:23:38.698 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 8443 2024-04-20T10:23:38.698 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.891+0000 7fda8b800700 1 Processor -- start 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.891+0000 7fda8b800700 1 -- start start 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.892+0000 7fda8b800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 0x7fda841088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.892+0000 7fda8b800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fda84108e90 con 0x7fda841084a0 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.892+0000 7fda88e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 0x7fda841088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.892+0000 7fda88e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 0x7fda841088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50128/0 (socket says 172.21.15.42:50128) 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.892+0000 7fda88e00700 1 -- 172.21.15.42:0/474684231 learned_addr learned my addr 172.21.15.42:0/474684231 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.893+0000 7fda88e00700 1 -- 172.21.15.42:0/474684231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda84109650 con 0x7fda841084a0 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.893+0000 7fda88e00700 1 --2- 172.21.15.42:0/474684231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 0x7fda841088c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda780098a0 tx=0x7fda78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a5f770a599d7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.893+0000 7fda83400700 1 -- 172.21.15.42:0/474684231 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda78004160 con 0x7fda841084a0 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.893+0000 7fda83400700 1 -- 172.21.15.42:0/474684231 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda78004300 con 0x7fda841084a0 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.893+0000 7fda83400700 1 -- 172.21.15.42:0/474684231 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda78011b20 con 0x7fda841084a0 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.894+0000 7fda8b800700 1 -- 172.21.15.42:0/474684231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 msgr2=0x7fda841088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:38.699 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.894+0000 7fda8b800700 1 --2- 172.21.15.42:0/474684231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 0x7fda841088c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda780098a0 tx=0x7fda78009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.894+0000 7fda8b800700 1 -- 172.21.15.42:0/474684231 shutdown_connections 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.894+0000 7fda8b800700 1 --2- 172.21.15.42:0/474684231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 0x7fda841088c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.894+0000 7fda8b800700 1 -- 172.21.15.42:0/474684231 >> 172.21.15.42:0/474684231 conn(0x7fda84103a60 msgr2=0x7fda84105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.894+0000 7fda8b800700 1 -- 172.21.15.42:0/474684231 shutdown_connections 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.894+0000 7fda8b800700 1 -- 172.21.15.42:0/474684231 wait complete. 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.896+0000 7fda8b800700 1 Processor -- start 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.896+0000 7fda8b800700 1 -- start start 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.896+0000 7fda8b800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 0x7fda841a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.897+0000 7fda8b800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fda8410ca00 con 0x7fda841084a0 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.897+0000 7fda88e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 0x7fda841a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.897+0000 7fda88e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 0x7fda841a2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50136/0 (socket says 172.21.15.42:50136) 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.897+0000 7fda88e00700 1 -- 172.21.15.42:0/4233928834 learned_addr learned my addr 172.21.15.42:0/4233928834 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.897+0000 7fda88e00700 1 -- 172.21.15.42:0/4233928834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda841a2f50 con 0x7fda841084a0 2024-04-20T10:23:38.700 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.898+0000 7fda88e00700 1 --2- 172.21.15.42:0/4233928834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 0x7fda841a2a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fda78000c00 tx=0x7fda78011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.898+0000 7fda81600700 1 -- 172.21.15.42:0/4233928834 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda78003b60 con 0x7fda841084a0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.898+0000 7fda81600700 1 -- 172.21.15.42:0/4233928834 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda7801a910 con 0x7fda841084a0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.898+0000 7fda81600700 1 -- 172.21.15.42:0/4233928834 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda78024930 con 0x7fda841084a0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.898+0000 7fda8b800700 1 -- 172.21.15.42:0/4233928834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda841a31e0 con 0x7fda841084a0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.898+0000 7fda8b800700 1 -- 172.21.15.42:0/4233928834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda841a3640 con 0x7fda841084a0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.899+0000 7fda81600700 1 -- 172.21.15.42:0/4233928834 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fda78022020 con 0x7fda841084a0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.900+0000 7fda8b800700 1 -- 172.21.15.42:0/4233928834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda840640b0 con 0x7fda841084a0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.900+0000 7fda81600700 1 --2- 172.21.15.42:0/4233928834 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fda6c0396b0 0x7fda6c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.900+0000 7fda81600700 1 -- 172.21.15.42:0/4233928834 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fda7804dbe0 con 0x7fda841084a0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.901+0000 7fda83e00700 1 --2- 172.21.15.42:0/4233928834 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fda6c0396b0 0x7fda6c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.901+0000 7fda83e00700 1 --2- 172.21.15.42:0/4233928834 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fda6c0396b0 0x7fda6c03bb70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fda70006fd0 tx=0x7fda70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:36.906+0000 7fda81600700 1 -- 172.21.15.42:0/4233928834 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fda7801ab30 con 0x7fda841084a0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:37.128+0000 7fda8b800700 1 -- 172.21.15.42:0/4233928834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fda841a6010 con 0x7fda841084a0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:37.129+0000 7fda81600700 1 -- 172.21.15.42:0/4233928834 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fda7802d770 con 0x7fda841084a0 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:37.132+0000 7fda8b800700 1 -- 172.21.15.42:0/4233928834 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fda6c0396b0 msgr2=0x7fda6c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:38.701 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:37.132+0000 7fda8b800700 1 --2- 172.21.15.42:0/4233928834 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fda6c0396b0 0x7fda6c03bb70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fda70006fd0 tx=0x7fda70006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:37.132+0000 7fda8b800700 1 -- 172.21.15.42:0/4233928834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 msgr2=0x7fda841a2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:37.132+0000 7fda8b800700 1 --2- 172.21.15.42:0/4233928834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 0x7fda841a2a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fda78000c00 tx=0x7fda78011c10 comp rx=0 tx=0).stop 2024-04-20T10:23:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:37.132+0000 7fda8b800700 1 -- 172.21.15.42:0/4233928834 shutdown_connections 2024-04-20T10:23:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:37.132+0000 7fda8b800700 1 --2- 172.21.15.42:0/4233928834 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fda6c0396b0 0x7fda6c03bb70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:37.132+0000 7fda8b800700 1 --2- 172.21.15.42:0/4233928834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda841084a0 0x7fda841a2a10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:37.132+0000 7fda8b800700 1 -- 172.21.15.42:0/4233928834 >> 172.21.15.42:0/4233928834 conn(0x7fda84103a60 msgr2=0x7fda841044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:37.132+0000 7fda8b800700 1 -- 172.21.15.42:0/4233928834 shutdown_connections 2024-04-20T10:23:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:37.132+0000 7fda8b800700 1 -- 172.21.15.42:0/4233928834 wait complete. 2024-04-20T10:23:38.728 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T10:23:38.729 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-04-20T10:23:38.753 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T10:23:38.753 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-04-20T10:23:38.754 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-04-20T10:23:38.754 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-20T10:23:38.761 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph Dashboard is now available at: 2024-04-20T10:23:38.761 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:23:38.761 INFO:teuthology.orchestra.run.smithi042.stdout: URL: https://smithi042.front.sepia.ceph.com:8443/ 2024-04-20T10:23:38.761 INFO:teuthology.orchestra.run.smithi042.stdout: User: admin 2024-04-20T10:23:38.761 INFO:teuthology.orchestra.run.smithi042.stdout: Password: v9wgqddjd0 2024-04-20T10:23:38.762 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:23:38.762 INFO:teuthology.orchestra.run.smithi042.stdout:Saving cluster configuration to /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config directory 2024-04-20T10:23:38.764 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling autotune for osd_memory_target 2024-04-20T10:23:40.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.384+0000 7f0427200700 1 Processor -- start 2024-04-20T10:23:40.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.384+0000 7f0427200700 1 -- start start 2024-04-20T10:23:40.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.384+0000 7f0427200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 0x7f042007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:40.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.384+0000 7f0427200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f042007c820 con 0x7f042007de30 2024-04-20T10:23:40.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.385+0000 7f041fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 0x7f042007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:40.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.385+0000 7f041fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 0x7f042007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36130/0 (socket says 172.21.15.42:36130) 2024-04-20T10:23:40.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.385+0000 7f041fe00700 1 -- 172.21.15.42:0/2844220619 learned_addr learned my addr 172.21.15.42:0/2844220619 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:40.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.385+0000 7f041fe00700 1 -- 172.21.15.42:0/2844220619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042007c9a0 con 0x7f042007de30 2024-04-20T10:23:40.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.386+0000 7f041fe00700 1 --2- 172.21.15.42:0/2844220619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 0x7f042007c2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0410009e80 tx=0x7f041000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cf5c18c012d83b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:40.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.386+0000 7f041f400700 1 -- 172.21.15.42:0/2844220619 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0410017070 con 0x7f042007de30 2024-04-20T10:23:40.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.386+0000 7f041f400700 1 -- 172.21.15.42:0/2844220619 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f041000b8c0 con 0x7f042007de30 2024-04-20T10:23:40.235 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.386+0000 7f041f400700 1 -- 172.21.15.42:0/2844220619 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04100045c0 con 0x7f042007de30 2024-04-20T10:23:40.236 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.387+0000 7f0427200700 1 -- 172.21.15.42:0/2844220619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 msgr2=0x7f042007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:40.236 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.387+0000 7f0427200700 1 --2- 172.21.15.42:0/2844220619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 0x7f042007c2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0410009e80 tx=0x7f041000b330 comp rx=0 tx=0).stop 2024-04-20T10:23:40.236 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.387+0000 7f0427200700 1 -- 172.21.15.42:0/2844220619 shutdown_connections 2024-04-20T10:23:40.236 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.387+0000 7f0427200700 1 --2- 172.21.15.42:0/2844220619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 0x7f042007c2e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:40.236 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.387+0000 7f0427200700 1 -- 172.21.15.42:0/2844220619 >> 172.21.15.42:0/2844220619 conn(0x7f0420103a60 msgr2=0x7f0420105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:40.236 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.387+0000 7f0427200700 1 -- 172.21.15.42:0/2844220619 shutdown_connections 2024-04-20T10:23:40.236 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.387+0000 7f0427200700 1 -- 172.21.15.42:0/2844220619 wait complete. 2024-04-20T10:23:40.239 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.388+0000 7f0427200700 1 Processor -- start 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.389+0000 7f0427200700 1 -- start start 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.389+0000 7f0427200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 0x7f042019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.389+0000 7f0427200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f042010ea60 con 0x7f042007de30 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.389+0000 7f041fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 0x7f042019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.389+0000 7f041fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 0x7f042019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36132/0 (socket says 172.21.15.42:36132) 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.389+0000 7f041fe00700 1 -- 172.21.15.42:0/1699859024 learned_addr learned my addr 172.21.15.42:0/1699859024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.390+0000 7f041fe00700 1 -- 172.21.15.42:0/1699859024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042019eb40 con 0x7f042007de30 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.390+0000 7f041fe00700 1 --2- 172.21.15.42:0/1699859024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 0x7f042019e600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0410009fd0 tx=0x7f0410009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.390+0000 7f041d600700 1 -- 172.21.15.42:0/1699859024 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0410020020 con 0x7f042007de30 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.390+0000 7f041d600700 1 -- 172.21.15.42:0/1699859024 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0410012430 con 0x7f042007de30 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.390+0000 7f041d600700 1 -- 172.21.15.42:0/1699859024 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0410017040 con 0x7f042007de30 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.390+0000 7f0427200700 1 -- 172.21.15.42:0/1699859024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042019edd0 con 0x7f042007de30 2024-04-20T10:23:40.240 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.391+0000 7f0427200700 1 -- 172.21.15.42:0/1699859024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04201a1ac0 con 0x7f042007de30 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.392+0000 7f0427200700 1 -- 172.21.15.42:0/1699859024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04200640b0 con 0x7f042007de30 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.392+0000 7f041d600700 1 -- 172.21.15.42:0/1699859024 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0410023020 con 0x7f042007de30 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.392+0000 7f041d600700 1 --2- 172.21.15.42:0/1699859024 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0404039730 0x7f040403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.392+0000 7f041d600700 1 -- 172.21.15.42:0/1699859024 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f041000f050 con 0x7f042007de30 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.393+0000 7f0417400700 1 --2- 172.21.15.42:0/1699859024 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0404039730 0x7f040403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.393+0000 7f0417400700 1 --2- 172.21.15.42:0/1699859024 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0404039730 0x7f040403bbf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0408006fd0 tx=0x7f0408006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.400+0000 7f041d600700 1 -- 172.21.15.42:0/1699859024 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f041002a020 con 0x7f042007de30 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.631+0000 7f0427200700 1 -- 172.21.15.42:0/1699859024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f04201a1c90 con 0x7f042007de30 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.632+0000 7f041d600700 1 -- 172.21.15.42:0/1699859024 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f04201a1c90 con 0x7f042007de30 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.636+0000 7f0427200700 1 -- 172.21.15.42:0/1699859024 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0404039730 msgr2=0x7f040403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.636+0000 7f0427200700 1 --2- 172.21.15.42:0/1699859024 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0404039730 0x7f040403bbf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0408006fd0 tx=0x7f0408006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.636+0000 7f0427200700 1 -- 172.21.15.42:0/1699859024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 msgr2=0x7f042019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:40.241 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.636+0000 7f0427200700 1 --2- 172.21.15.42:0/1699859024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 0x7f042019e600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0410009fd0 tx=0x7f0410009390 comp rx=0 tx=0).stop 2024-04-20T10:23:40.242 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.636+0000 7f0427200700 1 -- 172.21.15.42:0/1699859024 shutdown_connections 2024-04-20T10:23:40.242 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.636+0000 7f0427200700 1 --2- 172.21.15.42:0/1699859024 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0404039730 0x7f040403bbf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:40.242 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.636+0000 7f0427200700 1 --2- 172.21.15.42:0/1699859024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f042007de30 0x7f042019e600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:40.242 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.636+0000 7f0427200700 1 -- 172.21.15.42:0/1699859024 >> 172.21.15.42:0/1699859024 conn(0x7f0420103a60 msgr2=0x7f0420104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:40.242 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.636+0000 7f0427200700 1 -- 172.21.15.42:0/1699859024 shutdown_connections 2024-04-20T10:23:40.242 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:39.637+0000 7f0427200700 1 -- 172.21.15.42:0/1699859024 wait complete. 2024-04-20T10:23:41.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.925+0000 7f1e5de00700 1 Processor -- start 2024-04-20T10:23:41.945 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.925+0000 7f1e5de00700 1 -- start start 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.925+0000 7f1e5de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 0x7f1e58108860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.926+0000 7f1e5de00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58108e30 con 0x7f1e58108440 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.926+0000 7f1e56a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 0x7f1e58108860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.926+0000 7f1e56a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 0x7f1e58108860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36138/0 (socket says 172.21.15.42:36138) 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.926+0000 7f1e56a00700 1 -- 172.21.15.42:0/863785499 learned_addr learned my addr 172.21.15.42:0/863785499 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.926+0000 7f1e56a00700 1 -- 172.21.15.42:0/863785499 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e58109640 con 0x7f1e58108440 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.927+0000 7f1e56a00700 1 --2- 172.21.15.42:0/863785499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 0x7f1e58108860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1e480098a0 tx=0x7f1e48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1461ba0be6a74a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.927+0000 7f1e56000700 1 -- 172.21.15.42:0/863785499 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e48004160 con 0x7f1e58108440 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.927+0000 7f1e56000700 1 -- 172.21.15.42:0/863785499 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e48004300 con 0x7f1e58108440 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.927+0000 7f1e56000700 1 -- 172.21.15.42:0/863785499 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e48011b20 con 0x7f1e58108440 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.928+0000 7f1e5de00700 1 -- 172.21.15.42:0/863785499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 msgr2=0x7f1e58108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.928+0000 7f1e5de00700 1 --2- 172.21.15.42:0/863785499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 0x7f1e58108860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1e480098a0 tx=0x7f1e48009bb0 comp rx=0 tx=0).stop 2024-04-20T10:23:41.946 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.928+0000 7f1e5de00700 1 -- 172.21.15.42:0/863785499 shutdown_connections 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.928+0000 7f1e5de00700 1 --2- 172.21.15.42:0/863785499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 0x7f1e58108860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.928+0000 7f1e5de00700 1 -- 172.21.15.42:0/863785499 >> 172.21.15.42:0/863785499 conn(0x7f1e58103a60 msgr2=0x7f1e58105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.928+0000 7f1e5de00700 1 -- 172.21.15.42:0/863785499 shutdown_connections 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.928+0000 7f1e5de00700 1 -- 172.21.15.42:0/863785499 wait complete. 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.929+0000 7f1e5de00700 1 Processor -- start 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.930+0000 7f1e5de00700 1 -- start start 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.930+0000 7f1e5de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 0x7f1e5819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.930+0000 7f1e5de00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5810c9f0 con 0x7f1e58108440 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.930+0000 7f1e56a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 0x7f1e5819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.930+0000 7f1e56a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 0x7f1e5819c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36144/0 (socket says 172.21.15.42:36144) 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.930+0000 7f1e56a00700 1 -- 172.21.15.42:0/903275329 learned_addr learned my addr 172.21.15.42:0/903275329 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.931+0000 7f1e56a00700 1 -- 172.21.15.42:0/903275329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5819c930 con 0x7f1e58108440 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.931+0000 7f1e56a00700 1 --2- 172.21.15.42:0/903275329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 0x7f1e5819c3f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1e48000c00 tx=0x7f1e48011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.931+0000 7f1e5ca00700 1 -- 172.21.15.42:0/903275329 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e48003b60 con 0x7f1e58108440 2024-04-20T10:23:41.947 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.931+0000 7f1e5ca00700 1 -- 172.21.15.42:0/903275329 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e4801a910 con 0x7f1e58108440 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.931+0000 7f1e5ca00700 1 -- 172.21.15.42:0/903275329 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e48024930 con 0x7f1e58108440 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.931+0000 7f1e5de00700 1 -- 172.21.15.42:0/903275329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5819cbc0 con 0x7f1e58108440 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.931+0000 7f1e5de00700 1 -- 172.21.15.42:0/903275329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5819d020 con 0x7f1e58108440 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.933+0000 7f1e5ca00700 1 -- 172.21.15.42:0/903275329 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1e48022020 con 0x7f1e58108440 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.933+0000 7f1e5de00700 1 -- 172.21.15.42:0/903275329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e580640b0 con 0x7f1e58108440 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.933+0000 7f1e5ca00700 1 --2- 172.21.15.42:0/903275329 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e3c039700 0x7f1e3c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.933+0000 7f1e5ca00700 1 -- 172.21.15.42:0/903275329 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1e4804da90 con 0x7f1e58108440 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.933+0000 7f1e4e000700 1 --2- 172.21.15.42:0/903275329 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e3c039700 0x7f1e3c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.934+0000 7f1e4e000700 1 --2- 172.21.15.42:0/903275329 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e3c039700 0x7f1e3c03bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1e40006fd0 tx=0x7f1e40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:40.939+0000 7f1e5ca00700 1 -- 172.21.15.42:0/903275329 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1e48016260 con 0x7f1e58108440 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:41.297+0000 7f1e5de00700 1 -- 172.21.15.42:0/903275329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f1e5819fab0 con 0x7f1e58108440 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:41.301+0000 7f1e5ca00700 1 -- 172.21.15.42:0/903275329 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f1e4802dde0 con 0x7f1e58108440 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:41.303+0000 7f1e5de00700 1 -- 172.21.15.42:0/903275329 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e3c039700 msgr2=0x7f1e3c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:41.303+0000 7f1e5de00700 1 --2- 172.21.15.42:0/903275329 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e3c039700 0x7f1e3c03bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1e40006fd0 tx=0x7f1e40006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:41.303+0000 7f1e5de00700 1 -- 172.21.15.42:0/903275329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 msgr2=0x7f1e5819c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:41.303+0000 7f1e5de00700 1 --2- 172.21.15.42:0/903275329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 0x7f1e5819c3f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1e48000c00 tx=0x7f1e48011c10 comp rx=0 tx=0).stop 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:41.304+0000 7f1e5de00700 1 -- 172.21.15.42:0/903275329 shutdown_connections 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:41.304+0000 7f1e5de00700 1 --2- 172.21.15.42:0/903275329 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e3c039700 0x7f1e3c03bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:41.304+0000 7f1e5de00700 1 --2- 172.21.15.42:0/903275329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e58108440 0x7f1e5819c3f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:41.304+0000 7f1e5de00700 1 -- 172.21.15.42:0/903275329 >> 172.21.15.42:0/903275329 conn(0x7f1e58103a60 msgr2=0x7f1e581045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:41.304+0000 7f1e5de00700 1 -- 172.21.15.42:0/903275329 shutdown_connections 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-20T10:23:41.304+0000 7f1e5de00700 1 -- 172.21.15.42:0/903275329 wait complete. 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout:Or, if you are only running a single cluster on this host: 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:23:41.949 INFO:teuthology.orchestra.run.smithi042.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-20T10:23:41.950 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:23:41.950 INFO:teuthology.orchestra.run.smithi042.stdout: ceph telemetry on 2024-04-20T10:23:41.950 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:23:41.950 INFO:teuthology.orchestra.run.smithi042.stdout:For more information see: 2024-04-20T10:23:41.950 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:23:41.950 INFO:teuthology.orchestra.run.smithi042.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-20T10:23:41.950 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:23:41.950 INFO:teuthology.orchestra.run.smithi042.stdout:Bootstrap complete. 2024-04-20T10:23:42.024 INFO:tasks.cephadm:Fetching config... 2024-04-20T10:23:42.024 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:23:42.024 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-20T10:23:42.067 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-20T10:23:42.067 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:23:42.067 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-20T10:23:42.155 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-20T10:23:42.155 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:23:42.155 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-20T10:23:42.268 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-20T10:23:42.269 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:23:42.269 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-20T10:23:42.364 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-20T10:23:42.364 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt7OM2VL3w2UDxYjuVc1OV+b2aO3dcNv0hShbO9f5CHa7UlxQkepH6SvCCWMX6fsd8ZDNG0notHbBAjcOvIHRT6KdsN8ZkYd50aYCrYKGk1ZzUstmsXV82Z+yqPtkAAqqiaDXwhFDwYOPJQlH9lc/oFJgQZ1yfwNgYJCE/UUWijv8o8avFiY8cAafFWi2FdJKGNt/HfWP93txBXaoiQZL+Ws+/lGfnZqska7Svi59McJQPql0yqHEpEphTkMyoqmDrIZERNm3hic7neCAJm3dc0t9aBzrZulUWcAEqcV95Xtt3bRhc1kajWwf/atrv8MpVOtrE59OT1hiUarvlVHP1GXY5fFVU091GSAuS/YpLKNCaOwGbvN49Rxutu+VAdZ6bwn/wRAqj/reOY6Oi9r7dZdvmdo1rkp+qvXD5iS2hQ/KHUOJtpzWOiVfn04N6spnED8S8oyZOKyXdBPXoPCPVFas1uD9aE1UCpOwup7uHugXUqqWw9L+BdR55KlRURus= ceph-83e8f766-feff-11ee-bc92-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T10:23:42.498 INFO:teuthology.orchestra.run.smithi042.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt7OM2VL3w2UDxYjuVc1OV+b2aO3dcNv0hShbO9f5CHa7UlxQkepH6SvCCWMX6fsd8ZDNG0notHbBAjcOvIHRT6KdsN8ZkYd50aYCrYKGk1ZzUstmsXV82Z+yqPtkAAqqiaDXwhFDwYOPJQlH9lc/oFJgQZ1yfwNgYJCE/UUWijv8o8avFiY8cAafFWi2FdJKGNt/HfWP93txBXaoiQZL+Ws+/lGfnZqska7Svi59McJQPql0yqHEpEphTkMyoqmDrIZERNm3hic7neCAJm3dc0t9aBzrZulUWcAEqcV95Xtt3bRhc1kajWwf/atrv8MpVOtrE59OT1hiUarvlVHP1GXY5fFVU091GSAuS/YpLKNCaOwGbvN49Rxutu+VAdZ6bwn/wRAqj/reOY6Oi9r7dZdvmdo1rkp+qvXD5iS2hQ/KHUOJtpzWOiVfn04N6spnED8S8oyZOKyXdBPXoPCPVFas1uD9aE1UCpOwup7uHugXUqqWw9L+BdR55KlRURus= ceph-83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:23:42.521 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:42 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/903275329' entity='client.admin' 2024-04-20T10:23:42.533 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt7OM2VL3w2UDxYjuVc1OV+b2aO3dcNv0hShbO9f5CHa7UlxQkepH6SvCCWMX6fsd8ZDNG0notHbBAjcOvIHRT6KdsN8ZkYd50aYCrYKGk1ZzUstmsXV82Z+yqPtkAAqqiaDXwhFDwYOPJQlH9lc/oFJgQZ1yfwNgYJCE/UUWijv8o8avFiY8cAafFWi2FdJKGNt/HfWP93txBXaoiQZL+Ws+/lGfnZqska7Svi59McJQPql0yqHEpEphTkMyoqmDrIZERNm3hic7neCAJm3dc0t9aBzrZulUWcAEqcV95Xtt3bRhc1kajWwf/atrv8MpVOtrE59OT1hiUarvlVHP1GXY5fFVU091GSAuS/YpLKNCaOwGbvN49Rxutu+VAdZ6bwn/wRAqj/reOY6Oi9r7dZdvmdo1rkp+qvXD5iS2hQ/KHUOJtpzWOiVfn04N6spnED8S8oyZOKyXdBPXoPCPVFas1uD9aE1UCpOwup7uHugXUqqWw9L+BdR55KlRURus= ceph-83e8f766-feff-11ee-bc92-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T10:23:42.574 INFO:teuthology.orchestra.run.smithi046.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt7OM2VL3w2UDxYjuVc1OV+b2aO3dcNv0hShbO9f5CHa7UlxQkepH6SvCCWMX6fsd8ZDNG0notHbBAjcOvIHRT6KdsN8ZkYd50aYCrYKGk1ZzUstmsXV82Z+yqPtkAAqqiaDXwhFDwYOPJQlH9lc/oFJgQZ1yfwNgYJCE/UUWijv8o8avFiY8cAafFWi2FdJKGNt/HfWP93txBXaoiQZL+Ws+/lGfnZqska7Svi59McJQPql0yqHEpEphTkMyoqmDrIZERNm3hic7neCAJm3dc0t9aBzrZulUWcAEqcV95Xtt3bRhc1kajWwf/atrv8MpVOtrE59OT1hiUarvlVHP1GXY5fFVU091GSAuS/YpLKNCaOwGbvN49Rxutu+VAdZ6bwn/wRAqj/reOY6Oi9r7dZdvmdo1rkp+qvXD5iS2hQ/KHUOJtpzWOiVfn04N6spnED8S8oyZOKyXdBPXoPCPVFas1uD9aE1UCpOwup7uHugXUqqWw9L+BdR55KlRURus= ceph-83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:23:42.587 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt7OM2VL3w2UDxYjuVc1OV+b2aO3dcNv0hShbO9f5CHa7UlxQkepH6SvCCWMX6fsd8ZDNG0notHbBAjcOvIHRT6KdsN8ZkYd50aYCrYKGk1ZzUstmsXV82Z+yqPtkAAqqiaDXwhFDwYOPJQlH9lc/oFJgQZ1yfwNgYJCE/UUWijv8o8avFiY8cAafFWi2FdJKGNt/HfWP93txBXaoiQZL+Ws+/lGfnZqska7Svi59McJQPql0yqHEpEphTkMyoqmDrIZERNm3hic7neCAJm3dc0t9aBzrZulUWcAEqcV95Xtt3bRhc1kajWwf/atrv8MpVOtrE59OT1hiUarvlVHP1GXY5fFVU091GSAuS/YpLKNCaOwGbvN49Rxutu+VAdZ6bwn/wRAqj/reOY6Oi9r7dZdvmdo1rkp+qvXD5iS2hQ/KHUOJtpzWOiVfn04N6spnED8S8oyZOKyXdBPXoPCPVFas1uD9aE1UCpOwup7uHugXUqqWw9L+BdR55KlRURus= ceph-83e8f766-feff-11ee-bc92-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T10:23:42.627 INFO:teuthology.orchestra.run.smithi162.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt7OM2VL3w2UDxYjuVc1OV+b2aO3dcNv0hShbO9f5CHa7UlxQkepH6SvCCWMX6fsd8ZDNG0notHbBAjcOvIHRT6KdsN8ZkYd50aYCrYKGk1ZzUstmsXV82Z+yqPtkAAqqiaDXwhFDwYOPJQlH9lc/oFJgQZ1yfwNgYJCE/UUWijv8o8avFiY8cAafFWi2FdJKGNt/HfWP93txBXaoiQZL+Ws+/lGfnZqska7Svi59McJQPql0yqHEpEphTkMyoqmDrIZERNm3hic7neCAJm3dc0t9aBzrZulUWcAEqcV95Xtt3bRhc1kajWwf/atrv8MpVOtrE59OT1hiUarvlVHP1GXY5fFVU091GSAuS/YpLKNCaOwGbvN49Rxutu+VAdZ6bwn/wRAqj/reOY6Oi9r7dZdvmdo1rkp+qvXD5iS2hQ/KHUOJtpzWOiVfn04N6spnED8S8oyZOKyXdBPXoPCPVFas1uD9aE1UCpOwup7uHugXUqqWw9L+BdR55KlRURus= ceph-83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:23:42.639 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-20T10:23:43.080 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:23:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.528+0000 7fc06b400700 1 -- 172.21.15.42:0/3310926109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc064108c60 msgr2=0x7fc064109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.528+0000 7fc06b400700 1 --2- 172.21.15.42:0/3310926109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc064108c60 0x7fc064109080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc058009910 tx=0x7fc058009c20 comp rx=0 tx=0).stop 2024-04-20T10:23:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.528+0000 7fc06b400700 1 -- 172.21.15.42:0/3310926109 shutdown_connections 2024-04-20T10:23:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.528+0000 7fc06b400700 1 --2- 172.21.15.42:0/3310926109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc064108c60 0x7fc064109080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.528+0000 7fc06b400700 1 -- 172.21.15.42:0/3310926109 >> 172.21.15.42:0/3310926109 conn(0x7fc064077b40 msgr2=0x7fc064079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.529+0000 7fc06b400700 1 -- 172.21.15.42:0/3310926109 shutdown_connections 2024-04-20T10:23:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.529+0000 7fc06b400700 1 -- 172.21.15.42:0/3310926109 wait complete. 2024-04-20T10:23:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.531+0000 7fc06b400700 1 Processor -- start 2024-04-20T10:23:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.531+0000 7fc06b400700 1 -- start start 2024-04-20T10:23:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.531+0000 7fc06b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06419e410 0x7fc06419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.531+0000 7fc06b400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc06410eb80 con 0x7fc06419e410 2024-04-20T10:23:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.532+0000 7fc068a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06419e410 0x7fc06419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.532+0000 7fc068a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06419e410 0x7fc06419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36170/0 (socket says 172.21.15.42:36170) 2024-04-20T10:23:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.532+0000 7fc068a00700 1 -- 172.21.15.42:0/2192178954 learned_addr learned my addr 172.21.15.42:0/2192178954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.532+0000 7fc068a00700 1 -- 172.21.15.42:0/2192178954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06419ed70 con 0x7fc06419e410 2024-04-20T10:23:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.532+0000 7fc068a00700 1 --2- 172.21.15.42:0/2192178954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06419e410 0x7fc06419e830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc058000c00 tx=0x7fc058005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.533+0000 7fc061600700 1 -- 172.21.15.42:0/2192178954 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc058026070 con 0x7fc06419e410 2024-04-20T10:23:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.533+0000 7fc061600700 1 -- 172.21.15.42:0/2192178954 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc05800bc30 con 0x7fc06419e410 2024-04-20T10:23:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.533+0000 7fc061600700 1 -- 172.21.15.42:0/2192178954 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc058003910 con 0x7fc06419e410 2024-04-20T10:23:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.533+0000 7fc06b400700 1 -- 172.21.15.42:0/2192178954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06419f000 con 0x7fc06419e410 2024-04-20T10:23:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.533+0000 7fc06b400700 1 -- 172.21.15.42:0/2192178954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0641a1ae0 con 0x7fc06419e410 2024-04-20T10:23:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.534+0000 7fc061600700 1 -- 172.21.15.42:0/2192178954 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc058029070 con 0x7fc06419e410 2024-04-20T10:23:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.534+0000 7fc06b400700 1 -- 172.21.15.42:0/2192178954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc06410f300 con 0x7fc06419e410 2024-04-20T10:23:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.535+0000 7fc061600700 1 --2- 172.21.15.42:0/2192178954 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc04c0396b0 0x7fc04c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.535+0000 7fc061600700 1 -- 172.21.15.42:0/2192178954 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc05801e070 con 0x7fc06419e410 2024-04-20T10:23:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.535+0000 7fc063e00700 1 --2- 172.21.15.42:0/2192178954 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc04c0396b0 0x7fc04c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.536+0000 7fc063e00700 1 --2- 172.21.15.42:0/2192178954 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc04c0396b0 0x7fc04c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc050006fd0 tx=0x7fc050006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.541+0000 7fc061600700 1 -- 172.21.15.42:0/2192178954 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc058013070 con 0x7fc06419e410 2024-04-20T10:23:45.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.784+0000 7fc06b400700 1 -- 172.21.15.42:0/2192178954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fc06402e1c0 con 0x7fc06419e410 2024-04-20T10:23:45.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.787+0000 7fc061600700 1 -- 172.21.15.42:0/2192178954 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fc0580300c0 con 0x7fc06419e410 2024-04-20T10:23:45.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.791+0000 7fc06b400700 1 -- 172.21.15.42:0/2192178954 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc04c0396b0 msgr2=0x7fc04c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:45.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.791+0000 7fc06b400700 1 --2- 172.21.15.42:0/2192178954 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc04c0396b0 0x7fc04c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc050006fd0 tx=0x7fc050006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:45.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.791+0000 7fc06b400700 1 -- 172.21.15.42:0/2192178954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06419e410 msgr2=0x7fc06419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:45.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.791+0000 7fc06b400700 1 --2- 172.21.15.42:0/2192178954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06419e410 0x7fc06419e830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc058000c00 tx=0x7fc058005030 comp rx=0 tx=0).stop 2024-04-20T10:23:45.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.791+0000 7fc06b400700 1 -- 172.21.15.42:0/2192178954 shutdown_connections 2024-04-20T10:23:45.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.791+0000 7fc06b400700 1 --2- 172.21.15.42:0/2192178954 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc04c0396b0 0x7fc04c03bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:45.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.791+0000 7fc06b400700 1 --2- 172.21.15.42:0/2192178954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06419e410 0x7fc06419e830 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:45.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.791+0000 7fc06b400700 1 -- 172.21.15.42:0/2192178954 >> 172.21.15.42:0/2192178954 conn(0x7fc064077b40 msgr2=0x7fc064194c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:45.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.791+0000 7fc06b400700 1 -- 172.21.15.42:0/2192178954 shutdown_connections 2024-04-20T10:23:45.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:45.791+0000 7fc06b400700 1 -- 172.21.15.42:0/2192178954 wait complete. 2024-04-20T10:23:46.754 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-20T10:23:46.755 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-20T10:23:47.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:46 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2192178954' entity='client.admin' 2024-04-20T10:23:47.190 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:23:49.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.875+0000 7fcb97400700 1 -- 172.21.15.42:0/3543872694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb90076a80 msgr2=0x7fcb90074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:49.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.875+0000 7fcb97400700 1 --2- 172.21.15.42:0/3543872694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb90076a80 0x7fcb90074ee0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009910 tx=0x7fcb78009c20 comp rx=0 tx=0).stop 2024-04-20T10:23:49.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.875+0000 7fcb97400700 1 -- 172.21.15.42:0/3543872694 shutdown_connections 2024-04-20T10:23:49.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.875+0000 7fcb97400700 1 --2- 172.21.15.42:0/3543872694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb90076a80 0x7fcb90074ee0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:49.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.875+0000 7fcb97400700 1 -- 172.21.15.42:0/3543872694 >> 172.21.15.42:0/3543872694 conn(0x7fcb900fda50 msgr2=0x7fcb900ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:49.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.875+0000 7fcb97400700 1 -- 172.21.15.42:0/3543872694 shutdown_connections 2024-04-20T10:23:49.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.875+0000 7fcb97400700 1 -- 172.21.15.42:0/3543872694 wait complete. 2024-04-20T10:23:49.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.876+0000 7fcb97400700 1 Processor -- start 2024-04-20T10:23:49.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.876+0000 7fcb97400700 1 -- start start 2024-04-20T10:23:49.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.877+0000 7fcb97400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb90076a80 0x7fcb9019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:49.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.877+0000 7fcb97400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9010ec10 con 0x7fcb90076a80 2024-04-20T10:23:49.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.877+0000 7fcb94a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb90076a80 0x7fcb9019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:49.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.877+0000 7fcb94a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb90076a80 0x7fcb9019e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34102/0 (socket says 172.21.15.42:34102) 2024-04-20T10:23:49.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.877+0000 7fcb94a00700 1 -- 172.21.15.42:0/3079428339 learned_addr learned my addr 172.21.15.42:0/3079428339 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.877+0000 7fcb94a00700 1 -- 172.21.15.42:0/3079428339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9019e6d0 con 0x7fcb90076a80 2024-04-20T10:23:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.878+0000 7fcb94a00700 1 --2- 172.21.15.42:0/3079428339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb90076a80 0x7fcb9019e190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb78005850 tx=0x7fcb78005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.878+0000 7fcb8e000700 1 -- 172.21.15.42:0/3079428339 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb78026070 con 0x7fcb90076a80 2024-04-20T10:23:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.878+0000 7fcb8e000700 1 -- 172.21.15.42:0/3079428339 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb780044a0 con 0x7fcb90076a80 2024-04-20T10:23:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.878+0000 7fcb8e000700 1 -- 172.21.15.42:0/3079428339 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb78003ab0 con 0x7fcb90076a80 2024-04-20T10:23:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.878+0000 7fcb97400700 1 -- 172.21.15.42:0/3079428339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9019e960 con 0x7fcb90076a80 2024-04-20T10:23:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.878+0000 7fcb97400700 1 -- 172.21.15.42:0/3079428339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb901a1650 con 0x7fcb90076a80 2024-04-20T10:23:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.879+0000 7fcb8e000700 1 -- 172.21.15.42:0/3079428339 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcb78029070 con 0x7fcb90076a80 2024-04-20T10:23:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.880+0000 7fcb97400700 1 -- 172.21.15.42:0/3079428339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb9010f2c0 con 0x7fcb90076a80 2024-04-20T10:23:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.880+0000 7fcb8e000700 1 --2- 172.21.15.42:0/3079428339 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fcb70039650 0x7fcb7003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.880+0000 7fcb8e000700 1 -- 172.21.15.42:0/3079428339 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcb7801e070 con 0x7fcb90076a80 2024-04-20T10:23:49.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.880+0000 7fcb87e00700 1 --2- 172.21.15.42:0/3079428339 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fcb70039650 0x7fcb7003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:49.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.881+0000 7fcb87e00700 1 --2- 172.21.15.42:0/3079428339 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fcb70039650 0x7fcb7003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcb80006fd0 tx=0x7fcb80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:49.886+0000 7fcb8e000700 1 -- 172.21.15.42:0/3079428339 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcb78013070 con 0x7fcb90076a80 2024-04-20T10:23:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi042", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:23:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:50.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:23:50.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:50.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:50.126+0000 7fcb97400700 1 -- 172.21.15.42:0/3079428339 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fcb9002e0a0 con 0x7fcb70039650 2024-04-20T10:23:50.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:50.134+0000 7fcb8e000700 1 -- 172.21.15.42:0/3079428339 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb9002e0a0 con 0x7fcb70039650 2024-04-20T10:23:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:50.137+0000 7fcb97400700 1 -- 172.21.15.42:0/3079428339 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fcb70039650 msgr2=0x7fcb7003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:50.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:50.137+0000 7fcb97400700 1 --2- 172.21.15.42:0/3079428339 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fcb70039650 0x7fcb7003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcb80006fd0 tx=0x7fcb80006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:50.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:50.137+0000 7fcb97400700 1 -- 172.21.15.42:0/3079428339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb90076a80 msgr2=0x7fcb9019e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:50.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:50.137+0000 7fcb97400700 1 --2- 172.21.15.42:0/3079428339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb90076a80 0x7fcb9019e190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb78005850 tx=0x7fcb78005030 comp rx=0 tx=0).stop 2024-04-20T10:23:50.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:50.138+0000 7fcb97400700 1 -- 172.21.15.42:0/3079428339 shutdown_connections 2024-04-20T10:23:50.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:50.138+0000 7fcb97400700 1 --2- 172.21.15.42:0/3079428339 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fcb70039650 0x7fcb7003bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:50.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:50.138+0000 7fcb97400700 1 --2- 172.21.15.42:0/3079428339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb90076a80 0x7fcb9019e190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:50.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:50.138+0000 7fcb97400700 1 -- 172.21.15.42:0/3079428339 >> 172.21.15.42:0/3079428339 conn(0x7fcb900fda50 msgr2=0x7fcb900fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:50.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:50.138+0000 7fcb97400700 1 -- 172.21.15.42:0/3079428339 shutdown_connections 2024-04-20T10:23:50.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:50.138+0000 7fcb97400700 1 -- 172.21.15.42:0/3079428339 wait complete. 2024-04-20T10:23:50.963 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi046 2024-04-20T10:23:50.963 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-20T10:23:50.963 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/etc/ceph/ceph.conf 2024-04-20T10:23:50.982 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-20T10:23:50.982 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:23:51.041 INFO:tasks.cephadm:Adding host smithi046 to orchestrator... 2024-04-20T10:23:51.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch host add smithi046 2024-04-20T10:23:51.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:51 smithi042 ceph-mon[17078]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:23:51.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:51 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:51.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:51 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:23:51.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:51 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:23:51.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:51 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:23:51.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:51 smithi042 ceph-mon[17078]: Updating smithi042:/etc/ceph/ceph.conf 2024-04-20T10:23:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:23:52.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:52 smithi042 ceph-mon[17078]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:23:52.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:52 smithi042 ceph-mon[17078]: Updating smithi042:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:23:53.311 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:53 smithi042 ceph-mon[17078]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:23:53.311 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:53.311 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:53.311 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:53.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.487+0000 7f328c200700 1 -- 172.21.15.42:0/3433203104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328410a660 msgr2=0x7f328410aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:53.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.487+0000 7f328c200700 1 --2- 172.21.15.42:0/3433203104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328410a660 0x7f328410aa40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3278009910 tx=0x7f3278009c20 comp rx=0 tx=0).stop 2024-04-20T10:23:53.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.487+0000 7f328c200700 1 -- 172.21.15.42:0/3433203104 shutdown_connections 2024-04-20T10:23:53.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.487+0000 7f328c200700 1 --2- 172.21.15.42:0/3433203104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328410a660 0x7f328410aa40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:53.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.487+0000 7f328c200700 1 -- 172.21.15.42:0/3433203104 >> 172.21.15.42:0/3433203104 conn(0x7f3284100150 msgr2=0x7f3284102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:53.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.487+0000 7f328c200700 1 -- 172.21.15.42:0/3433203104 shutdown_connections 2024-04-20T10:23:53.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.487+0000 7f328c200700 1 -- 172.21.15.42:0/3433203104 wait complete. 2024-04-20T10:23:53.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.488+0000 7f328c200700 1 Processor -- start 2024-04-20T10:23:53.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.489+0000 7f328c200700 1 -- start start 2024-04-20T10:23:53.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.489+0000 7f328c200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328410a660 0x7f328419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:53.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.489+0000 7f328c200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f328410eca0 con 0x7f328410a660 2024-04-20T10:23:53.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.489+0000 7f3289800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328410a660 0x7f328419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:53.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.489+0000 7f3289800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328410a660 0x7f328419a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34120/0 (socket says 172.21.15.42:34120) 2024-04-20T10:23:53.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.489+0000 7f3289800700 1 -- 172.21.15.42:0/432622908 learned_addr learned my addr 172.21.15.42:0/432622908 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:23:53.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.490+0000 7f3289800700 1 -- 172.21.15.42:0/432622908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328419a6e0 con 0x7f328410a660 2024-04-20T10:23:53.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.490+0000 7f3289800700 1 --2- 172.21.15.42:0/432622908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328410a660 0x7f328419a1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3278005850 tx=0x7f3278005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:53.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.490+0000 7f3282000700 1 -- 172.21.15.42:0/432622908 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3278026070 con 0x7f328410a660 2024-04-20T10:23:53.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.490+0000 7f328c200700 1 -- 172.21.15.42:0/432622908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328419a9d0 con 0x7f328410a660 2024-04-20T10:23:53.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.490+0000 7f3282000700 1 -- 172.21.15.42:0/432622908 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f327800bbd0 con 0x7f328410a660 2024-04-20T10:23:53.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.491+0000 7f3282000700 1 -- 172.21.15.42:0/432622908 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3278003970 con 0x7f328410a660 2024-04-20T10:23:53.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.491+0000 7f328c200700 1 -- 172.21.15.42:0/432622908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f328419ea70 con 0x7f328410a660 2024-04-20T10:23:53.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.492+0000 7f328c200700 1 -- 172.21.15.42:0/432622908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3284105da0 con 0x7f328410a660 2024-04-20T10:23:53.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.492+0000 7f3282000700 1 -- 172.21.15.42:0/432622908 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3278029070 con 0x7f328410a660 2024-04-20T10:23:53.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.493+0000 7f3282000700 1 --2- 172.21.15.42:0/432622908 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f326c039650 0x7f326c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:53.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.493+0000 7f3282000700 1 -- 172.21.15.42:0/432622908 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f327801e070 con 0x7f328410a660 2024-04-20T10:23:53.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.493+0000 7f3288e00700 1 --2- 172.21.15.42:0/432622908 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f326c039650 0x7f326c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:53.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.494+0000 7f3288e00700 1 --2- 172.21.15.42:0/432622908 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f326c039650 0x7f326c03bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3270006fd0 tx=0x7f3270006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:53.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.498+0000 7f3282000700 1 -- 172.21.15.42:0/432622908 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3278013070 con 0x7f328410a660 2024-04-20T10:23:53.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:53.736+0000 7f328c200700 1 -- 172.21.15.42:0/432622908 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi046", "target": ["mon-mgr", ""]}) -- 0x7f328419b500 con 0x7f326c039650 2024-04-20T10:23:54.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:54 smithi042 ceph-mon[17078]: Updating smithi042:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:23:54.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:54 smithi042 ceph-mon[17078]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:23:55.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:55 smithi042 ceph-mon[17078]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi046", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:23:56.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:56 smithi042 ceph-mon[17078]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:23:56.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:56 smithi042 ceph-mon[17078]: Deploying cephadm binary to smithi046 2024-04-20T10:23:58.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:58 smithi042 ceph-mon[17078]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:23:58.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:58.676+0000 7f3282000700 1 -- 172.21.15.42:0/432622908 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f328419b500 con 0x7f326c039650 2024-04-20T10:23:58.677 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi046' with addr '172.21.15.46' 2024-04-20T10:23:58.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:58.680+0000 7f328c200700 1 -- 172.21.15.42:0/432622908 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f326c039650 msgr2=0x7f326c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:58.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:58.680+0000 7f328c200700 1 --2- 172.21.15.42:0/432622908 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f326c039650 0x7f326c03bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3270006fd0 tx=0x7f3270006e40 comp rx=0 tx=0).stop 2024-04-20T10:23:58.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:58.680+0000 7f328c200700 1 -- 172.21.15.42:0/432622908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328410a660 msgr2=0x7f328419a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:58.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:58.680+0000 7f328c200700 1 --2- 172.21.15.42:0/432622908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328410a660 0x7f328419a1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3278005850 tx=0x7f3278005030 comp rx=0 tx=0).stop 2024-04-20T10:23:58.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:58.680+0000 7f328c200700 1 -- 172.21.15.42:0/432622908 shutdown_connections 2024-04-20T10:23:58.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:58.680+0000 7f328c200700 1 --2- 172.21.15.42:0/432622908 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f326c039650 0x7f326c03bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:58.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:58.680+0000 7f328c200700 1 --2- 172.21.15.42:0/432622908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328410a660 0x7f328419a1a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:58.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:58.680+0000 7f328c200700 1 -- 172.21.15.42:0/432622908 >> 172.21.15.42:0/432622908 conn(0x7f3284100150 msgr2=0x7f3284100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:58.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:58.680+0000 7f328c200700 1 -- 172.21.15.42:0/432622908 shutdown_connections 2024-04-20T10:23:58.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:23:58.680+0000 7f328c200700 1 -- 172.21.15.42:0/432622908 wait complete. 2024-04-20T10:23:59.410 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch host ls --format=json 2024-04-20T10:23:59.840 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:23:59.991 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:59 smithi042 ceph-mon[17078]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:23:59.992 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:59 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:23:59.992 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:59 smithi042 ceph-mon[17078]: Added host smithi046 2024-04-20T10:23:59.992 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:59 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:23:59.992 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:23:59 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:01.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:00 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.975+0000 7f3c24c00700 1 -- 172.21.15.42:0/3798832721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c20076f80 msgr2=0x7f3c20077360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.975+0000 7f3c24c00700 1 --2- 172.21.15.42:0/3798832721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c20076f80 0x7f3c20077360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009910 tx=0x7f3c10009c20 comp rx=0 tx=0).stop 2024-04-20T10:24:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.976+0000 7f3c24c00700 1 -- 172.21.15.42:0/3798832721 shutdown_connections 2024-04-20T10:24:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.976+0000 7f3c24c00700 1 --2- 172.21.15.42:0/3798832721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c20076f80 0x7f3c20077360 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.976+0000 7f3c24c00700 1 -- 172.21.15.42:0/3798832721 >> 172.21.15.42:0/3798832721 conn(0x7f3c201001a0 msgr2=0x7f3c201025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.976+0000 7f3c24c00700 1 -- 172.21.15.42:0/3798832721 shutdown_connections 2024-04-20T10:24:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.976+0000 7f3c24c00700 1 -- 172.21.15.42:0/3798832721 wait complete. 2024-04-20T10:24:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.977+0000 7f3c24c00700 1 Processor -- start 2024-04-20T10:24:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.977+0000 7f3c24c00700 1 -- start start 2024-04-20T10:24:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.977+0000 7f3c24c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c20076f80 0x7f3c2019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.977+0000 7f3c24c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20113100 con 0x7f3c20076f80 2024-04-20T10:24:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.978+0000 7f3c1de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c20076f80 0x7f3c2019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.978+0000 7f3c1de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c20076f80 0x7f3c2019e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54792/0 (socket says 172.21.15.42:54792) 2024-04-20T10:24:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.978+0000 7f3c1de00700 1 -- 172.21.15.42:0/357803511 learned_addr learned my addr 172.21.15.42:0/357803511 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:24:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.978+0000 7f3c1de00700 1 -- 172.21.15.42:0/357803511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2019eaa0 con 0x7f3c20076f80 2024-04-20T10:24:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.978+0000 7f3c1de00700 1 --2- 172.21.15.42:0/357803511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c20076f80 0x7f3c2019e560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c10006010 tx=0x7f3c10005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.979+0000 7f3c16a00700 1 -- 172.21.15.42:0/357803511 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c10026070 con 0x7f3c20076f80 2024-04-20T10:24:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.979+0000 7f3c16a00700 1 -- 172.21.15.42:0/357803511 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c1000bc30 con 0x7f3c20076f80 2024-04-20T10:24:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.979+0000 7f3c16a00700 1 -- 172.21.15.42:0/357803511 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c100039d0 con 0x7f3c20076f80 2024-04-20T10:24:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.979+0000 7f3c24c00700 1 -- 172.21.15.42:0/357803511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2019ed30 con 0x7f3c20076f80 2024-04-20T10:24:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.979+0000 7f3c24c00700 1 -- 172.21.15.42:0/357803511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2019f190 con 0x7f3c20076f80 2024-04-20T10:24:01.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.980+0000 7f3c16a00700 1 -- 172.21.15.42:0/357803511 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3c10029070 con 0x7f3c20076f80 2024-04-20T10:24:01.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.980+0000 7f3c24c00700 1 -- 172.21.15.42:0/357803511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c200786c0 con 0x7f3c20076f80 2024-04-20T10:24:01.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.981+0000 7f3c16a00700 1 --2- 172.21.15.42:0/357803511 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3c04039700 0x7f3c0403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:01.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.981+0000 7f3c16a00700 1 -- 172.21.15.42:0/357803511 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3c1001e070 con 0x7f3c20076f80 2024-04-20T10:24:01.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.981+0000 7f3c1d400700 1 --2- 172.21.15.42:0/357803511 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3c04039700 0x7f3c0403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:01.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.982+0000 7f3c1d400700 1 --2- 172.21.15.42:0/357803511 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3c04039700 0x7f3c0403bbc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c006fd0 tx=0x7f3c0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:01.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:01.987+0000 7f3c16a00700 1 -- 172.21.15.42:0/357803511 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3c10013070 con 0x7f3c20076f80 2024-04-20T10:24:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:01 smithi042 ceph-mon[17078]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:01 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:02.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:02.224+0000 7f3c24c00700 1 -- 172.21.15.42:0/357803511 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c2019f8c0 con 0x7f3c04039700 2024-04-20T10:24:02.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:02.226+0000 7f3c16a00700 1 -- 172.21.15.42:0/357803511 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f3c2019f8c0 con 0x7f3c04039700 2024-04-20T10:24:02.226 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:24:02.226 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}] 2024-04-20T10:24:02.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:02.229+0000 7f3c24c00700 1 -- 172.21.15.42:0/357803511 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3c04039700 msgr2=0x7f3c0403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:02.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:02.229+0000 7f3c24c00700 1 --2- 172.21.15.42:0/357803511 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3c04039700 0x7f3c0403bbc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c006fd0 tx=0x7f3c0c006e40 comp rx=0 tx=0).stop 2024-04-20T10:24:02.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:02.229+0000 7f3c24c00700 1 -- 172.21.15.42:0/357803511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c20076f80 msgr2=0x7f3c2019e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:02.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:02.229+0000 7f3c24c00700 1 --2- 172.21.15.42:0/357803511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c20076f80 0x7f3c2019e560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c10006010 tx=0x7f3c10005030 comp rx=0 tx=0).stop 2024-04-20T10:24:02.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:02.229+0000 7f3c24c00700 1 -- 172.21.15.42:0/357803511 shutdown_connections 2024-04-20T10:24:02.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:02.229+0000 7f3c24c00700 1 --2- 172.21.15.42:0/357803511 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3c04039700 0x7f3c0403bbc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:02.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:02.229+0000 7f3c24c00700 1 --2- 172.21.15.42:0/357803511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c20076f80 0x7f3c2019e560 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:02.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:02.229+0000 7f3c24c00700 1 -- 172.21.15.42:0/357803511 >> 172.21.15.42:0/357803511 conn(0x7f3c201001a0 msgr2=0x7f3c201019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:02.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:02.230+0000 7f3c24c00700 1 -- 172.21.15.42:0/357803511 shutdown_connections 2024-04-20T10:24:02.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:02.230+0000 7f3c24c00700 1 -- 172.21.15.42:0/357803511 wait complete. 2024-04-20T10:24:02.956 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi162 2024-04-20T10:24:02.956 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-20T10:24:02.957 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/etc/ceph/ceph.conf 2024-04-20T10:24:02.975 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-20T10:24:02.975 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:24:03.031 INFO:tasks.cephadm:Adding host smithi162 to orchestrator... 2024-04-20T10:24:03.032 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch host add smithi162 2024-04-20T10:24:03.480 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:24:03.897 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:03 smithi042 ceph-mon[17078]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:24:03.897 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:03 smithi042 ceph-mon[17078]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:05.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.400+0000 7ff00ce00700 1 -- 172.21.15.42:0/3024447165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff00810ea90 msgr2=0x7ff00810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:05.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.400+0000 7ff00ce00700 1 --2- 172.21.15.42:0/3024447165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff00810ea90 0x7ff00810ee70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feff8009910 tx=0x7feff8009c20 comp rx=0 tx=0).stop 2024-04-20T10:24:05.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.401+0000 7ff00ce00700 1 -- 172.21.15.42:0/3024447165 shutdown_connections 2024-04-20T10:24:05.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.401+0000 7ff00ce00700 1 --2- 172.21.15.42:0/3024447165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff00810ea90 0x7ff00810ee70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:05.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.401+0000 7ff00ce00700 1 -- 172.21.15.42:0/3024447165 >> 172.21.15.42:0/3024447165 conn(0x7ff0080782e0 msgr2=0x7ff00807a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:05.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.401+0000 7ff00ce00700 1 -- 172.21.15.42:0/3024447165 shutdown_connections 2024-04-20T10:24:05.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.401+0000 7ff00ce00700 1 -- 172.21.15.42:0/3024447165 wait complete. 2024-04-20T10:24:05.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.402+0000 7ff00ce00700 1 Processor -- start 2024-04-20T10:24:05.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.402+0000 7ff00ce00700 1 -- start start 2024-04-20T10:24:05.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.402+0000 7ff00ce00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff00810ea90 0x7ff00819e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:05.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.402+0000 7ff00ce00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff0081130d0 con 0x7ff00810ea90 2024-04-20T10:24:05.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.403+0000 7ff005e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff00810ea90 0x7ff00819e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:05.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.403+0000 7ff005e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff00810ea90 0x7ff00819e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54798/0 (socket says 172.21.15.42:54798) 2024-04-20T10:24:05.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.403+0000 7ff005e00700 1 -- 172.21.15.42:0/583774836 learned_addr learned my addr 172.21.15.42:0/583774836 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:24:05.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.403+0000 7ff005e00700 1 -- 172.21.15.42:0/583774836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00819ea40 con 0x7ff00810ea90 2024-04-20T10:24:05.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.403+0000 7ff005e00700 1 --2- 172.21.15.42:0/583774836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff00810ea90 0x7ff00819e500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feff8006010 tx=0x7feff8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:05.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.404+0000 7feffea00700 1 -- 172.21.15.42:0/583774836 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feff8026070 con 0x7ff00810ea90 2024-04-20T10:24:05.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.404+0000 7feffea00700 1 -- 172.21.15.42:0/583774836 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff800bbd0 con 0x7ff00810ea90 2024-04-20T10:24:05.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.404+0000 7feffea00700 1 -- 172.21.15.42:0/583774836 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feff8003910 con 0x7ff00810ea90 2024-04-20T10:24:05.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.404+0000 7ff00ce00700 1 -- 172.21.15.42:0/583774836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00819ed30 con 0x7ff00810ea90 2024-04-20T10:24:05.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.404+0000 7ff00ce00700 1 -- 172.21.15.42:0/583774836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0081a2dd0 con 0x7ff00810ea90 2024-04-20T10:24:05.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.405+0000 7ff00ce00700 1 -- 172.21.15.42:0/583774836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00810a1d0 con 0x7ff00810ea90 2024-04-20T10:24:05.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.405+0000 7feffea00700 1 -- 172.21.15.42:0/583774836 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7feff8029070 con 0x7ff00810ea90 2024-04-20T10:24:05.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.406+0000 7feffea00700 1 --2- 172.21.15.42:0/583774836 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fefec0396b0 0x7fefec03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:05.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.406+0000 7feffea00700 1 -- 172.21.15.42:0/583774836 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feff801e070 con 0x7ff00810ea90 2024-04-20T10:24:05.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.406+0000 7ff005400700 1 --2- 172.21.15.42:0/583774836 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fefec0396b0 0x7fefec03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:05.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.407+0000 7ff005400700 1 --2- 172.21.15.42:0/583774836 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fefec0396b0 0x7fefec03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feff4006fd0 tx=0x7feff4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.412+0000 7feffea00700 1 -- 172.21.15.42:0/583774836 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7feff8013070 con 0x7ff00810ea90 2024-04-20T10:24:05.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:05.652+0000 7ff00ce00700 1 -- 172.21.15.42:0/583774836 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi162", "target": ["mon-mgr", ""]}) -- 0x7ff00819f860 con 0x7fefec0396b0 2024-04-20T10:24:05.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:05 smithi042 ceph-mon[17078]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:05.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:05 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:05.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:05 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:05.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:05 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:05.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:05 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:05.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:05 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi046", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:24:05.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:05 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:05.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:05 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:24:05.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:05 smithi042 ceph-mon[17078]: Updating smithi046:/etc/ceph/ceph.conf 2024-04-20T10:24:05.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:05 smithi042 ceph-mon[17078]: Updating smithi046:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:07.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:06 smithi042 ceph-mon[17078]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:24:07.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:06 smithi042 ceph-mon[17078]: Updating smithi046:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:24:07.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:06 smithi042 ceph-mon[17078]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi162", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:24:07.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:06 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:07.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:06 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:07.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:06 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:08.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:07 smithi042 ceph-mon[17078]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:08.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:07 smithi042 ceph-mon[17078]: Deploying cephadm binary to smithi162 2024-04-20T10:24:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:09 smithi042 ceph-mon[17078]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:10.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:10.641+0000 7feffea00700 1 -- 172.21.15.42:0/583774836 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff00819f860 con 0x7fefec0396b0 2024-04-20T10:24:10.642 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi162' with addr '172.21.15.162' 2024-04-20T10:24:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:10.646+0000 7ff00ce00700 1 -- 172.21.15.42:0/583774836 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fefec0396b0 msgr2=0x7fefec03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:10.646+0000 7ff00ce00700 1 --2- 172.21.15.42:0/583774836 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fefec0396b0 0x7fefec03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feff4006fd0 tx=0x7feff4006e40 comp rx=0 tx=0).stop 2024-04-20T10:24:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:10.646+0000 7ff00ce00700 1 -- 172.21.15.42:0/583774836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff00810ea90 msgr2=0x7ff00819e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:10.646+0000 7ff00ce00700 1 --2- 172.21.15.42:0/583774836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff00810ea90 0x7ff00819e500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feff8006010 tx=0x7feff8005030 comp rx=0 tx=0).stop 2024-04-20T10:24:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:10.646+0000 7ff00ce00700 1 -- 172.21.15.42:0/583774836 shutdown_connections 2024-04-20T10:24:10.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:10.646+0000 7ff00ce00700 1 --2- 172.21.15.42:0/583774836 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fefec0396b0 0x7fefec03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:10.646+0000 7ff00ce00700 1 --2- 172.21.15.42:0/583774836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff00810ea90 0x7ff00819e500 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:10.646+0000 7ff00ce00700 1 -- 172.21.15.42:0/583774836 >> 172.21.15.42:0/583774836 conn(0x7ff0080782e0 msgr2=0x7ff008110590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:10.646+0000 7ff00ce00700 1 -- 172.21.15.42:0/583774836 shutdown_connections 2024-04-20T10:24:10.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:10.646+0000 7ff00ce00700 1 -- 172.21.15.42:0/583774836 wait complete. 2024-04-20T10:24:11.367 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch host ls --format=json 2024-04-20T10:24:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:24:11.971 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:11 smithi042 ceph-mon[17078]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:11.971 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:11.971 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:11 smithi042 ceph-mon[17078]: Added host smithi162 2024-04-20T10:24:11.971 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:24:11.971 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:12.970 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:12 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:13.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.766+0000 7f9519400700 1 -- 172.21.15.42:0/301816420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f951410d970 msgr2=0x7f951410dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:13.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.766+0000 7f9519400700 1 --2- 172.21.15.42:0/301816420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f951410d970 0x7f951410dd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9504009910 tx=0x7f9504009c20 comp rx=0 tx=0).stop 2024-04-20T10:24:13.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.767+0000 7f9519400700 1 -- 172.21.15.42:0/301816420 shutdown_connections 2024-04-20T10:24:13.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.767+0000 7f9519400700 1 --2- 172.21.15.42:0/301816420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f951410d970 0x7f951410dd50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:13.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.767+0000 7f9519400700 1 -- 172.21.15.42:0/301816420 >> 172.21.15.42:0/301816420 conn(0x7f95140781f0 msgr2=0x7f951407a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:13.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.767+0000 7f9519400700 1 -- 172.21.15.42:0/301816420 shutdown_connections 2024-04-20T10:24:13.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.767+0000 7f9519400700 1 -- 172.21.15.42:0/301816420 wait complete. 2024-04-20T10:24:13.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.768+0000 7f9519400700 1 Processor -- start 2024-04-20T10:24:13.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.768+0000 7f9519400700 1 -- start start 2024-04-20T10:24:13.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.768+0000 7f9519400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f951410d970 0x7f951419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:13.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.768+0000 7f9519400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95141130c0 con 0x7f951410d970 2024-04-20T10:24:13.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.769+0000 7f9512800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f951410d970 0x7f951419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:13.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.769+0000 7f9512800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f951410d970 0x7f951419e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46296/0 (socket says 172.21.15.42:46296) 2024-04-20T10:24:13.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.769+0000 7f9512800700 1 -- 172.21.15.42:0/3793832292 learned_addr learned my addr 172.21.15.42:0/3793832292 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:24:13.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.769+0000 7f9512800700 1 -- 172.21.15.42:0/3793832292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951419ea20 con 0x7f951410d970 2024-04-20T10:24:13.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.769+0000 7f9512800700 1 --2- 172.21.15.42:0/3793832292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f951410d970 0x7f951419e4e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9504006010 tx=0x7f9504005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:13.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.770+0000 7f950b400700 1 -- 172.21.15.42:0/3793832292 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9504026070 con 0x7f951410d970 2024-04-20T10:24:13.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.770+0000 7f950b400700 1 -- 172.21.15.42:0/3793832292 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f950400bbd0 con 0x7f951410d970 2024-04-20T10:24:13.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.770+0000 7f950b400700 1 -- 172.21.15.42:0/3793832292 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9504003a30 con 0x7f951410d970 2024-04-20T10:24:13.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.770+0000 7f9519400700 1 -- 172.21.15.42:0/3793832292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951419ed10 con 0x7f951410d970 2024-04-20T10:24:13.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.770+0000 7f9519400700 1 -- 172.21.15.42:0/3793832292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95141a2db0 con 0x7f951410d970 2024-04-20T10:24:13.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.771+0000 7f950b400700 1 -- 172.21.15.42:0/3793832292 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9504029070 con 0x7f951410d970 2024-04-20T10:24:13.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.771+0000 7f9519400700 1 -- 172.21.15.42:0/3793832292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f951410a190 con 0x7f951410d970 2024-04-20T10:24:13.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.772+0000 7f950b400700 1 --2- 172.21.15.42:0/3793832292 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f94f80396b0 0x7f94f803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:13.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.772+0000 7f950b400700 1 -- 172.21.15.42:0/3793832292 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f950401e070 con 0x7f951410d970 2024-04-20T10:24:13.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.772+0000 7f9511e00700 1 --2- 172.21.15.42:0/3793832292 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f94f80396b0 0x7f94f803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:13.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.773+0000 7f9511e00700 1 --2- 172.21.15.42:0/3793832292 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f94f80396b0 0x7f94f803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9500006fd0 tx=0x7f9500006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:13.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:13.778+0000 7f950b400700 1 -- 172.21.15.42:0/3793832292 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9504013070 con 0x7f951410d970 2024-04-20T10:24:13.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:13 smithi042 ceph-mon[17078]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:13.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:14.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:14.014+0000 7f9519400700 1 -- 172.21.15.42:0/3793832292 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f951419f840 con 0x7f94f80396b0 2024-04-20T10:24:14.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:14.016+0000 7f950b400700 1 -- 172.21.15.42:0/3793832292 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f951419f840 con 0x7f94f80396b0 2024-04-20T10:24:14.016 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:24:14.016 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.162", "hostname": "smithi162", "labels": [], "status": ""}] 2024-04-20T10:24:14.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:14.019+0000 7f9519400700 1 -- 172.21.15.42:0/3793832292 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f94f80396b0 msgr2=0x7f94f803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:14.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:14.019+0000 7f9519400700 1 --2- 172.21.15.42:0/3793832292 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f94f80396b0 0x7f94f803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9500006fd0 tx=0x7f9500006e40 comp rx=0 tx=0).stop 2024-04-20T10:24:14.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:14.019+0000 7f9519400700 1 -- 172.21.15.42:0/3793832292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f951410d970 msgr2=0x7f951419e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:14.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:14.019+0000 7f9519400700 1 --2- 172.21.15.42:0/3793832292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f951410d970 0x7f951419e4e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9504006010 tx=0x7f9504005050 comp rx=0 tx=0).stop 2024-04-20T10:24:14.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:14.019+0000 7f9519400700 1 -- 172.21.15.42:0/3793832292 shutdown_connections 2024-04-20T10:24:14.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:14.019+0000 7f9519400700 1 --2- 172.21.15.42:0/3793832292 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f94f80396b0 0x7f94f803bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:14.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:14.019+0000 7f9519400700 1 --2- 172.21.15.42:0/3793832292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f951410d970 0x7f951419e4e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:14.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:14.019+0000 7f9519400700 1 -- 172.21.15.42:0/3793832292 >> 172.21.15.42:0/3793832292 conn(0x7f95140781f0 msgr2=0x7f95141104d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:14.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:14.019+0000 7f9519400700 1 -- 172.21.15.42:0/3793832292 shutdown_connections 2024-04-20T10:24:14.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:14.019+0000 7f9519400700 1 -- 172.21.15.42:0/3793832292 wait complete. 2024-04-20T10:24:14.758 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-20T10:24:14.758 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd crush tunables default 2024-04-20T10:24:14.998 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:14 smithi042 ceph-mon[17078]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:24:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:24:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:15 smithi042 ceph-mon[17078]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:17.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.259+0000 7f108e000700 1 -- 172.21.15.42:0/2569053303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1088103600 msgr2=0x7f10881039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:17.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.259+0000 7f108e000700 1 --2- 172.21.15.42:0/2569053303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1088103600 0x7f10881039e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1078009910 tx=0x7f1078009c20 comp rx=0 tx=0).stop 2024-04-20T10:24:17.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.259+0000 7f108e000700 1 -- 172.21.15.42:0/2569053303 shutdown_connections 2024-04-20T10:24:17.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.259+0000 7f108e000700 1 --2- 172.21.15.42:0/2569053303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1088103600 0x7f10881039e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.259+0000 7f108e000700 1 -- 172.21.15.42:0/2569053303 >> 172.21.15.42:0/2569053303 conn(0x7f10880fce50 msgr2=0x7f10880ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:17.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.259+0000 7f108e000700 1 -- 172.21.15.42:0/2569053303 shutdown_connections 2024-04-20T10:24:17.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.259+0000 7f108e000700 1 -- 172.21.15.42:0/2569053303 wait complete. 2024-04-20T10:24:17.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.261+0000 7f108e000700 1 Processor -- start 2024-04-20T10:24:17.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.262+0000 7f108e000700 1 -- start start 2024-04-20T10:24:17.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.262+0000 7f108e000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1088103600 0x7f1088101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.262+0000 7f108e000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f108810caf0 con 0x7f1088103600 2024-04-20T10:24:17.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.263+0000 7f1087400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1088103600 0x7f1088101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.263+0000 7f1087400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1088103600 0x7f1088101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46308/0 (socket says 172.21.15.42:46308) 2024-04-20T10:24:17.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.263+0000 7f1087400700 1 -- 172.21.15.42:0/2562970416 learned_addr learned my addr 172.21.15.42:0/2562970416 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:24:17.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.263+0000 7f1087400700 1 -- 172.21.15.42:0/2562970416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1088102ea0 con 0x7f1088103600 2024-04-20T10:24:17.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.263+0000 7f1087400700 1 --2- 172.21.15.42:0/2562970416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1088103600 0x7f1088101260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1078006010 tx=0x7f1078005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:17.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.264+0000 7f107fe00700 1 -- 172.21.15.42:0/2562970416 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1078026070 con 0x7f1088103600 2024-04-20T10:24:17.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.264+0000 7f107fe00700 1 -- 172.21.15.42:0/2562970416 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f107800bc30 con 0x7f1088103600 2024-04-20T10:24:17.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.264+0000 7f108e000700 1 -- 172.21.15.42:0/2562970416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1088103130 con 0x7f1088103600 2024-04-20T10:24:17.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.264+0000 7f107fe00700 1 -- 172.21.15.42:0/2562970416 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10780039a0 con 0x7f1088103600 2024-04-20T10:24:17.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.264+0000 7f108e000700 1 -- 172.21.15.42:0/2562970416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1088101970 con 0x7f1088103600 2024-04-20T10:24:17.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.265+0000 7f108e000700 1 -- 172.21.15.42:0/2562970416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108806afe0 con 0x7f1088103600 2024-04-20T10:24:17.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.265+0000 7f107fe00700 1 -- 172.21.15.42:0/2562970416 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1078029070 con 0x7f1088103600 2024-04-20T10:24:17.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.266+0000 7f107fe00700 1 --2- 172.21.15.42:0/2562970416 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f106c0396b0 0x7f106c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.266+0000 7f107fe00700 1 -- 172.21.15.42:0/2562970416 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f107801e070 con 0x7f1088103600 2024-04-20T10:24:17.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.266+0000 7f1086a00700 1 --2- 172.21.15.42:0/2562970416 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f106c0396b0 0x7f106c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.267+0000 7f1086a00700 1 --2- 172.21.15.42:0/2562970416 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f106c0396b0 0x7f106c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1074006fd0 tx=0x7f1074006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:17.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.272+0000 7f107fe00700 1 -- 172.21.15.42:0/2562970416 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1078013070 con 0x7f1088103600 2024-04-20T10:24:17.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:17.513+0000 7f108e000700 1 -- 172.21.15.42:0/2562970416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1088102790 con 0x7f1088103600 2024-04-20T10:24:17.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:17 smithi042 ceph-mon[17078]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:17.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:17.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:17.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:17.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:17.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi162", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:24:17.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:17.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:24:17.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:17 smithi042 ceph-mon[17078]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-20T10:24:17.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:17 smithi042 ceph-mon[17078]: Updating smithi162:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:17.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:17 smithi042 ceph-mon[17078]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:24:18.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:18.468+0000 7f107fe00700 1 -- 172.21.15.42:0/2562970416 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f107800fe20 con 0x7f1088103600 2024-04-20T10:24:18.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:18.472+0000 7f108e000700 1 -- 172.21.15.42:0/2562970416 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f106c0396b0 msgr2=0x7f106c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:18.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:18.472+0000 7f108e000700 1 --2- 172.21.15.42:0/2562970416 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f106c0396b0 0x7f106c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1074006fd0 tx=0x7f1074006e40 comp rx=0 tx=0).stop 2024-04-20T10:24:18.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:18.472+0000 7f108e000700 1 -- 172.21.15.42:0/2562970416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1088103600 msgr2=0x7f1088101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:18.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:18.472+0000 7f108e000700 1 --2- 172.21.15.42:0/2562970416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1088103600 0x7f1088101260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1078006010 tx=0x7f1078005030 comp rx=0 tx=0).stop 2024-04-20T10:24:18.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:18.472+0000 7f108e000700 1 -- 172.21.15.42:0/2562970416 shutdown_connections 2024-04-20T10:24:18.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:18.472+0000 7f108e000700 1 --2- 172.21.15.42:0/2562970416 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f106c0396b0 0x7f106c03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:18.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:18.472+0000 7f108e000700 1 --2- 172.21.15.42:0/2562970416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1088103600 0x7f1088101260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:18.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:18.472+0000 7f108e000700 1 -- 172.21.15.42:0/2562970416 >> 172.21.15.42:0/2562970416 conn(0x7f10880fce50 msgr2=0x7f10881043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:18.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:18.473+0000 7f108e000700 1 -- 172.21.15.42:0/2562970416 shutdown_connections 2024-04-20T10:24:18.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:18.473+0000 7f108e000700 1 -- 172.21.15.42:0/2562970416 wait complete. 2024-04-20T10:24:18.473 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-04-20T10:24:18.793 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:18 smithi042 ceph-mon[17078]: Updating smithi162:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:24:18.794 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:18 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2562970416' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-20T10:24:18.794 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:18 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:18.794 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:18 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:18.794 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:18 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:19.155 INFO:tasks.cephadm:Adding mon.a on smithi042 2024-04-20T10:24:19.155 INFO:tasks.cephadm:Adding mon.b on smithi046 2024-04-20T10:24:19.156 INFO:tasks.cephadm:Adding mon.c on smithi162 2024-04-20T10:24:19.156 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch apply mon '3;smithi042:172.21.15.42=a;smithi046:172.21.15.46=b;smithi162:172.21.15.162=c' 2024-04-20T10:24:19.408 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:19.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:19 smithi042 ceph-mon[17078]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:19.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:19 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2562970416' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-20T10:24:19.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:19 smithi042 ceph-mon[17078]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T10:24:20.872 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.872+0000 7fb4e77c4700 1 -- 172.21.15.162:0/4084850811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e01084f0 msgr2=0x7fb4e01088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:20.872 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.872+0000 7fb4e77c4700 1 --2- 172.21.15.162:0/4084850811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e01084f0 0x7fb4e01088d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009910 tx=0x7fb4d0009c20 comp rx=0 tx=0).stop 2024-04-20T10:24:20.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.872+0000 7fb4e77c4700 1 -- 172.21.15.162:0/4084850811 shutdown_connections 2024-04-20T10:24:20.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.872+0000 7fb4e77c4700 1 --2- 172.21.15.162:0/4084850811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e01084f0 0x7fb4e01088d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:20.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.872+0000 7fb4e77c4700 1 -- 172.21.15.162:0/4084850811 >> 172.21.15.162:0/4084850811 conn(0x7fb4e00fdf90 msgr2=0x7fb4e01003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:20.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.872+0000 7fb4e77c4700 1 -- 172.21.15.162:0/4084850811 shutdown_connections 2024-04-20T10:24:20.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.872+0000 7fb4e77c4700 1 -- 172.21.15.162:0/4084850811 wait complete. 2024-04-20T10:24:20.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.872+0000 7fb4e77c4700 1 Processor -- start 2024-04-20T10:24:20.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.872+0000 7fb4e77c4700 1 -- start start 2024-04-20T10:24:20.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.872+0000 7fb4e77c4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e01084f0 0x7fb4e0197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:20.873 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.872+0000 7fb4e77c4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e010caa0 con 0x7fb4e01084f0 2024-04-20T10:24:20.874 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.873+0000 7fb4e5560700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e01084f0 0x7fb4e0197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:20.874 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.873+0000 7fb4e5560700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e01084f0 0x7fb4e0197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.162:35726/0 (socket says 172.21.15.162:35726) 2024-04-20T10:24:20.874 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.873+0000 7fb4e5560700 1 -- 172.21.15.162:0/1831028822 learned_addr learned my addr 172.21.15.162:0/1831028822 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-20T10:24:20.875 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.874+0000 7fb4e5560700 1 -- 172.21.15.162:0/1831028822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e01984e0 con 0x7fb4e01084f0 2024-04-20T10:24:20.875 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.874+0000 7fb4e5560700 1 --2- 172.21.15.162:0/1831028822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e01084f0 0x7fb4e0197fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005850 tx=0x7fb4d0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:20.875 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.874+0000 7fb4d67fc700 1 -- 172.21.15.162:0/1831028822 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4d0026070 con 0x7fb4e01084f0 2024-04-20T10:24:20.875 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.874+0000 7fb4e77c4700 1 -- 172.21.15.162:0/1831028822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e01987d0 con 0x7fb4e01084f0 2024-04-20T10:24:20.875 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.874+0000 7fb4e77c4700 1 -- 172.21.15.162:0/1831028822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e019c870 con 0x7fb4e01084f0 2024-04-20T10:24:20.875 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.874+0000 7fb4d67fc700 1 -- 172.21.15.162:0/1831028822 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d000bbd0 con 0x7fb4e01084f0 2024-04-20T10:24:20.875 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.874+0000 7fb4d67fc700 1 -- 172.21.15.162:0/1831028822 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4d00039b0 con 0x7fb4e01084f0 2024-04-20T10:24:20.876 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.875+0000 7fb4e77c4700 1 -- 172.21.15.162:0/1831028822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e0103ba0 con 0x7fb4e01084f0 2024-04-20T10:24:20.876 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.876+0000 7fb4d67fc700 1 -- 172.21.15.162:0/1831028822 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb4d0029070 con 0x7fb4e01084f0 2024-04-20T10:24:20.876 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.876+0000 7fb4d67fc700 1 --2- 172.21.15.162:0/1831028822 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb4cc039650 0x7fb4cc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:20.876 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.876+0000 7fb4d67fc700 1 -- 172.21.15.162:0/1831028822 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb4d001e070 con 0x7fb4e01084f0 2024-04-20T10:24:20.876 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.876+0000 7fb4e4d5f700 1 --2- 172.21.15.162:0/1831028822 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb4cc039650 0x7fb4cc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:20.877 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.877+0000 7fb4e4d5f700 1 --2- 172.21.15.162:0/1831028822 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb4cc039650 0x7fb4cc03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc006fd0 tx=0x7fb4dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:20.891 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:20.892+0000 7fb4d67fc700 1 -- 172.21.15.162:0/1831028822 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb4d0013070 con 0x7fb4e01084f0 2024-04-20T10:24:21.098 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:21.098+0000 7fb4e77c4700 1 -- 172.21.15.162:0/1831028822 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi046:172.21.15.46=b;smithi162:172.21.15.162=c", "target": ["mon-mgr", ""]}) -- 0x7fb4e0199300 con 0x7fb4cc039650 2024-04-20T10:24:21.114 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:21.114+0000 7fb4d67fc700 1 -- 172.21.15.162:0/1831028822 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb4e0199300 con 0x7fb4cc039650 2024-04-20T10:24:21.114 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled mon update... 2024-04-20T10:24:21.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:21.116+0000 7fb4e77c4700 1 -- 172.21.15.162:0/1831028822 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb4cc039650 msgr2=0x7fb4cc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:21.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:21.116+0000 7fb4e77c4700 1 --2- 172.21.15.162:0/1831028822 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb4cc039650 0x7fb4cc03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc006fd0 tx=0x7fb4dc006e40 comp rx=0 tx=0).stop 2024-04-20T10:24:21.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:21.116+0000 7fb4e77c4700 1 -- 172.21.15.162:0/1831028822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e01084f0 msgr2=0x7fb4e0197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:21.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:21.116+0000 7fb4e77c4700 1 --2- 172.21.15.162:0/1831028822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e01084f0 0x7fb4e0197fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005850 tx=0x7fb4d0005030 comp rx=0 tx=0).stop 2024-04-20T10:24:21.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:21.116+0000 7fb4e77c4700 1 -- 172.21.15.162:0/1831028822 shutdown_connections 2024-04-20T10:24:21.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:21.116+0000 7fb4e77c4700 1 --2- 172.21.15.162:0/1831028822 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb4cc039650 0x7fb4cc03bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:21.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:21.116+0000 7fb4e77c4700 1 --2- 172.21.15.162:0/1831028822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e01084f0 0x7fb4e0197fa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:21.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:21.116+0000 7fb4e77c4700 1 -- 172.21.15.162:0/1831028822 >> 172.21.15.162:0/1831028822 conn(0x7fb4e00fdf90 msgr2=0x7fb4e0109fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:21.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:21.116+0000 7fb4e77c4700 1 -- 172.21.15.162:0/1831028822 shutdown_connections 2024-04-20T10:24:21.117 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:21.116+0000 7fb4e77c4700 1 -- 172.21.15.162:0/1831028822 wait complete. 2024-04-20T10:24:21.840 DEBUG:teuthology.orchestra.run.smithi046:mon.b> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.b.service 2024-04-20T10:24:21.843 DEBUG:teuthology.orchestra.run.smithi162:mon.c> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.c.service 2024-04-20T10:24:21.845 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-20T10:24:21.845 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:24:22.599 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:22.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:22 smithi042 ceph-mon[17078]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:22.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:22 smithi042 ceph-mon[17078]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi046:172.21.15.46=b;smithi162:172.21.15.162=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:24:22.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:22 smithi042 ceph-mon[17078]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi046:172.21.15.46=b;smithi162:172.21.15.162=c;count:3 2024-04-20T10:24:22.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:22 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:22.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:22 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:24:22.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:22 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:22.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:22 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:24:22.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:22 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:22.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:22 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:24:22.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:22 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:23.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:23 smithi042 ceph-mon[17078]: Deploying daemon mon.c on smithi162 2024-04-20T10:24:24.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:24 smithi042 ceph-mon[17078]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:25.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.736+0000 7f447ac4b700 1 -- 172.21.15.162:0/1600477456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4474101600 msgr2=0x7f44741019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:25.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.736+0000 7f447ac4b700 1 --2- 172.21.15.162:0/1600477456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4474101600 0x7f44741019e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4468009910 tx=0x7f4468009c20 comp rx=0 tx=0).stop 2024-04-20T10:24:25.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.736+0000 7f447ac4b700 1 -- 172.21.15.162:0/1600477456 shutdown_connections 2024-04-20T10:24:25.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.736+0000 7f447ac4b700 1 --2- 172.21.15.162:0/1600477456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4474101600 0x7f44741019e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:25.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.736+0000 7f447ac4b700 1 -- 172.21.15.162:0/1600477456 >> 172.21.15.162:0/1600477456 conn(0x7f4474076eb0 msgr2=0x7f44740772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:25.737 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.736+0000 7f447ac4b700 1 -- 172.21.15.162:0/1600477456 shutdown_connections 2024-04-20T10:24:25.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.736+0000 7f447ac4b700 1 -- 172.21.15.162:0/1600477456 wait complete. 2024-04-20T10:24:25.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.736+0000 7f447ac4b700 1 Processor -- start 2024-04-20T10:24:25.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.736+0000 7f447ac4b700 1 -- start start 2024-04-20T10:24:25.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.737+0000 7f447ac4b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4474101600 0x7f447419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:25.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.737+0000 7f447ac4b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f447410b980 con 0x7f4474101600 2024-04-20T10:24:25.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.737+0000 7f44789e7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4474101600 0x7f447419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:25.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.737+0000 7f44789e7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4474101600 0x7f447419a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.162:35742/0 (socket says 172.21.15.162:35742) 2024-04-20T10:24:25.738 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.737+0000 7f44789e7700 1 -- 172.21.15.162:0/1239040500 learned_addr learned my addr 172.21.15.162:0/1239040500 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-20T10:24:25.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.738+0000 7f44789e7700 1 -- 172.21.15.162:0/1239040500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447419a6d0 con 0x7f4474101600 2024-04-20T10:24:25.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.738+0000 7f44789e7700 1 --2- 172.21.15.162:0/1239040500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4474101600 0x7f447419a190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4468006010 tx=0x7f4468005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:25.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.738+0000 7f4471ffb700 1 -- 172.21.15.162:0/1239040500 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4468026070 con 0x7f4474101600 2024-04-20T10:24:25.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.739+0000 7f447ac4b700 1 -- 172.21.15.162:0/1239040500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447419a9c0 con 0x7f4474101600 2024-04-20T10:24:25.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.739+0000 7f447ac4b700 1 -- 172.21.15.162:0/1239040500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447419ea60 con 0x7f4474101600 2024-04-20T10:24:25.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.739+0000 7f4471ffb700 1 -- 172.21.15.162:0/1239040500 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f446800bbd0 con 0x7f4474101600 2024-04-20T10:24:25.739 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.739+0000 7f4471ffb700 1 -- 172.21.15.162:0/1239040500 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44680038e0 con 0x7f4474101600 2024-04-20T10:24:25.740 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.739+0000 7f447ac4b700 1 -- 172.21.15.162:0/1239040500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4474102a90 con 0x7f4474101600 2024-04-20T10:24:25.740 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.740+0000 7f4471ffb700 1 -- 172.21.15.162:0/1239040500 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4468029070 con 0x7f4474101600 2024-04-20T10:24:25.740 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.740+0000 7f4471ffb700 1 --2- 172.21.15.162:0/1239040500 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4464039650 0x7f446403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:25.740 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.740+0000 7f4471ffb700 1 -- 172.21.15.162:0/1239040500 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f446801e070 con 0x7f4474101600 2024-04-20T10:24:25.741 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.741+0000 7f4473fff700 1 --2- 172.21.15.162:0/1239040500 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4464039650 0x7f446403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:25.742 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.742+0000 7f4473fff700 1 --2- 172.21.15.162:0/1239040500 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4464039650 0x7f446403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4460006fd0 tx=0x7f4460006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:25.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:25.756+0000 7f4471ffb700 1 -- 172.21.15.162:0/1239040500 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4468013070 con 0x7f4474101600 2024-04-20T10:24:26.057 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:26.057+0000 7f447ac4b700 1 -- 172.21.15.162:0/1239040500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f447419b4f0 con 0x7f4474101600 2024-04-20T10:24:26.058 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:26.058+0000 7f4471ffb700 1 -- 172.21.15.162:0/1239040500 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f446800d340 con 0x7f4474101600 2024-04-20T10:24:26.058 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:24:26.058 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":1,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","modified":"2024-04-20T10:22:16.558137Z","created":"2024-04-20T10:22:16.558137Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T10:24:26.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:26.060+0000 7f447ac4b700 1 -- 172.21.15.162:0/1239040500 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4464039650 msgr2=0x7f446403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:26.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:26.060+0000 7f447ac4b700 1 --2- 172.21.15.162:0/1239040500 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4464039650 0x7f446403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4460006fd0 tx=0x7f4460006e40 comp rx=0 tx=0).stop 2024-04-20T10:24:26.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:26.060+0000 7f447ac4b700 1 -- 172.21.15.162:0/1239040500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4474101600 msgr2=0x7f447419a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:26.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:26.060+0000 7f447ac4b700 1 --2- 172.21.15.162:0/1239040500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4474101600 0x7f447419a190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4468006010 tx=0x7f4468005030 comp rx=0 tx=0).stop 2024-04-20T10:24:26.060 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:26.060+0000 7f447ac4b700 1 -- 172.21.15.162:0/1239040500 shutdown_connections 2024-04-20T10:24:26.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:26.060+0000 7f447ac4b700 1 --2- 172.21.15.162:0/1239040500 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4464039650 0x7f446403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:26.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:26.060+0000 7f447ac4b700 1 --2- 172.21.15.162:0/1239040500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4474101600 0x7f447419a190 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:26.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:26.060+0000 7f447ac4b700 1 -- 172.21.15.162:0/1239040500 >> 172.21.15.162:0/1239040500 conn(0x7f4474076eb0 msgr2=0x7f4474069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:26.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:26.060+0000 7f447ac4b700 1 -- 172.21.15.162:0/1239040500 shutdown_connections 2024-04-20T10:24:26.061 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:26.060+0000 7f447ac4b700 1 -- 172.21.15.162:0/1239040500 wait complete. 2024-04-20T10:24:26.061 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 1 2024-04-20T10:24:27.790 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-20T10:24:27.791 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:24:28.040 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:24:31.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:24:31.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: mon.a calling monitor election 2024-04-20T10:24:31.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: mon.c calling monitor election 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: monmap epoch 2 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: fsid 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: last_changed 2024-04-20T10:24:26.102345+0000 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: created 2024-04-20T10:22:16.558137+0000 2024-04-20T10:24:31.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: min_mon_release 19 (squid) 2024-04-20T10:24:31.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: election_strategy: 1 2024-04-20T10:24:31.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-20T10:24:31.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-04-20T10:24:31.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: fsmap 2024-04-20T10:24:31.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T10:24:31.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: mgrmap e13: x(active, since 60s) 2024-04-20T10:24:31.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: overall HEALTH_OK 2024-04-20T10:24:31.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:31.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:31.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:31.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:24:31.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:32.318 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:32 smithi042 ceph-mon[17078]: Deploying daemon mon.b on smithi046 2024-04-20T10:24:32.318 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:32 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:24:32.387 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.385+0000 7f0af21f8700 1 -- 172.21.15.162:0/581669239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad40035c0 msgr2=0x7f0ad4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.385+0000 7f0af21f8700 1 --2- 172.21.15.162:0/581669239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad40035c0 0x7f0ad4005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0aec101390 tx=0x7f0adc00feb0 comp rx=0 tx=0).stop 2024-04-20T10:24:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.385+0000 7f0af21f8700 1 -- 172.21.15.162:0/581669239 shutdown_connections 2024-04-20T10:24:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.385+0000 7f0af21f8700 1 --2- 172.21.15.162:0/581669239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad40035c0 0x7f0ad4005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.385+0000 7f0af21f8700 1 -- 172.21.15.162:0/581669239 >> 172.21.15.162:0/581669239 conn(0x7f0aec0fcf90 msgr2=0x7f0aec0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.386+0000 7f0af21f8700 1 -- 172.21.15.162:0/581669239 shutdown_connections 2024-04-20T10:24:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.386+0000 7f0af21f8700 1 -- 172.21.15.162:0/581669239 wait complete. 2024-04-20T10:24:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.386+0000 7f0af21f8700 1 Processor -- start 2024-04-20T10:24:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.387+0000 7f0af21f8700 1 -- start start 2024-04-20T10:24:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.387+0000 7f0af21f8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad40035c0 0x7f0aec196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.387+0000 7f0af21f8700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aec1973b0 0x7f0aec197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:32.388 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.387+0000 7f0af21f8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec109500 con 0x7f0ad40035c0 2024-04-20T10:24:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.387+0000 7f0af21f8700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec109380 con 0x7f0aec1973b0 2024-04-20T10:24:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.387+0000 7f0aeaffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aec1973b0 0x7f0aec197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.387+0000 7f0aeaffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aec1973b0 0x7f0aec197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50968/0 (socket says 172.21.15.162:50968) 2024-04-20T10:24:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.387+0000 7f0aeaffd700 1 -- 172.21.15.162:0/3486857851 learned_addr learned my addr 172.21.15.162:0/3486857851 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-20T10:24:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.387+0000 7f0aeaffd700 1 -- 172.21.15.162:0/3486857851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aec1973b0 msgr2=0x7f0aec197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-20T10:24:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.387+0000 7f0aeaffd700 1 -- 172.21.15.162:0/3486857851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aec1973b0 msgr2=0x7f0aec197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T10:24:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.387+0000 7f0aeaffd700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aec1973b0 0x7f0aec197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:24:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.387+0000 7f0aeaffd700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aec1973b0 0x7f0aec197830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:24:32.389 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.388+0000 7f0aeb7fe700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad40035c0 0x7f0aec196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:32.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.388+0000 7f0aeb7fe700 1 -- 172.21.15.162:0/3486857851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aec1973b0 msgr2=0x7f0aec197830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:24:32.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.388+0000 7f0aeb7fe700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aec1973b0 0x7f0aec197830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:32.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.388+0000 7f0aeb7fe700 1 -- 172.21.15.162:0/3486857851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec06b9c0 con 0x7f0ad40035c0 2024-04-20T10:24:32.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.389+0000 7f0aeb7fe700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad40035c0 0x7f0aec196e70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0059d0 tx=0x7f0adc0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:32.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.389+0000 7f0ae8ff9700 1 -- 172.21.15.162:0/3486857851 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0adc004b20 con 0x7f0ad40035c0 2024-04-20T10:24:32.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.389+0000 7f0af21f8700 1 -- 172.21.15.162:0/3486857851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aec1a44e0 con 0x7f0ad40035c0 2024-04-20T10:24:32.391 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.389+0000 7f0ae8ff9700 1 -- 172.21.15.162:0/3486857851 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0adc013e40 con 0x7f0ad40035c0 2024-04-20T10:24:32.392 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.389+0000 7f0ae8ff9700 1 -- 172.21.15.162:0/3486857851 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0adc011a70 con 0x7f0ad40035c0 2024-04-20T10:24:32.392 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.389+0000 7f0af21f8700 1 -- 172.21.15.162:0/3486857851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aec1a4930 con 0x7f0ad40035c0 2024-04-20T10:24:32.392 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.390+0000 7f0af21f8700 1 -- 172.21.15.162:0/3486857851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aec102ba0 con 0x7f0ad40035c0 2024-04-20T10:24:32.392 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.390+0000 7f0ae8ff9700 1 -- 172.21.15.162:0/3486857851 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0adc023070 con 0x7f0ad40035c0 2024-04-20T10:24:32.392 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.391+0000 7f0ae8ff9700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0ad80393f0 0x7f0ad803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:32.392 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.391+0000 7f0ae8ff9700 1 -- 172.21.15.162:0/3486857851 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0adc020050 con 0x7f0ad40035c0 2024-04-20T10:24:32.392 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.391+0000 7f0aeaffd700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0ad80393f0 0x7f0ad803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:32.392 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.392+0000 7f0aeaffd700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0ad80393f0 0x7f0ad803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009180 tx=0x7f0ae0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:32.401 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.401+0000 7f0ae8ff9700 1 -- 172.21.15.162:0/3486857851 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0adc028020 con 0x7f0ad40035c0 2024-04-20T10:24:32.686 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.686+0000 7f0af21f8700 1 -- 172.21.15.162:0/3486857851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0aec02df90 con 0x7f0ad40035c0 2024-04-20T10:24:32.687 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.687+0000 7f0ae8ff9700 1 -- 172.21.15.162:0/3486857851 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f0adc01b3f0 con 0x7f0ad40035c0 2024-04-20T10:24:32.687 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:24:32.687 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":2,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","modified":"2024-04-20T10:24:26.102345Z","created":"2024-04-20T10:22:16.558137Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-20T10:24:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.689+0000 7f0af21f8700 1 -- 172.21.15.162:0/3486857851 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0ad80393f0 msgr2=0x7f0ad803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.689+0000 7f0af21f8700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0ad80393f0 0x7f0ad803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0009180 tx=0x7f0ae0008040 comp rx=0 tx=0).stop 2024-04-20T10:24:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.689+0000 7f0af21f8700 1 -- 172.21.15.162:0/3486857851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad40035c0 msgr2=0x7f0aec196e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.689+0000 7f0af21f8700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad40035c0 0x7f0aec196e70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0059d0 tx=0x7f0adc0057d0 comp rx=0 tx=0).stop 2024-04-20T10:24:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.689+0000 7f0af21f8700 1 -- 172.21.15.162:0/3486857851 shutdown_connections 2024-04-20T10:24:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.689+0000 7f0af21f8700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0ad80393f0 0x7f0ad803b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.689+0000 7f0af21f8700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad40035c0 0x7f0aec196e70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.689+0000 7f0af21f8700 1 --2- 172.21.15.162:0/3486857851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aec1973b0 0x7f0aec197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.689+0000 7f0af21f8700 1 -- 172.21.15.162:0/3486857851 >> 172.21.15.162:0/3486857851 conn(0x7f0aec0fcf90 msgr2=0x7f0aec0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.689+0000 7f0af21f8700 1 -- 172.21.15.162:0/3486857851 shutdown_connections 2024-04-20T10:24:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:32.689+0000 7f0af21f8700 1 -- 172.21.15.162:0/3486857851 wait complete. 2024-04-20T10:24:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 2 2024-04-20T10:24:33.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:33 smithi042 ceph-mon[17078]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:33.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:33 smithi042 ceph-mon[17078]: from='client.? 172.21.15.162:0/3486857851' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:24:34.242 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-20T10:24:34.242 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph mon dump -f json 2024-04-20T10:24:34.483 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:24:35.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:35 smithi046 systemd[1]: Started Ceph mon.b for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:24:40.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.363+0000 7f4b6371f700 1 -- 172.21.15.162:0/4074732793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b480057d0 msgr2=0x7f4b48007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:40.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.363+0000 7f4b6371f700 1 --2- 172.21.15.162:0/4074732793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b480057d0 0x7f4b48007c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c101370 tx=0x7f4b5800fe60 comp rx=0 tx=0).stop 2024-04-20T10:24:40.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.363+0000 7f4b6371f700 1 -- 172.21.15.162:0/4074732793 shutdown_connections 2024-04-20T10:24:40.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.363+0000 7f4b6371f700 1 --2- 172.21.15.162:0/4074732793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b480057d0 0x7f4b48007c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:40.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.363+0000 7f4b6371f700 1 -- 172.21.15.162:0/4074732793 >> 172.21.15.162:0/4074732793 conn(0x7f4b5c0fcf30 msgr2=0x7f4b5c0ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:40.366 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.364+0000 7f4b6371f700 1 -- 172.21.15.162:0/4074732793 shutdown_connections 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.364+0000 7f4b6371f700 1 -- 172.21.15.162:0/4074732793 wait complete. 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.364+0000 7f4b6371f700 1 Processor -- start 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b6371f700 1 -- start start 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b6371f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b480057d0 0x7f4b5c195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b6371f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c1964e0 0x7f4b5c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b6371f700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c19adb0 0x7f4b5c19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b6371f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c108460 con 0x7f4b5c1964e0 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b6371f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c1082e0 con 0x7f4b480057d0 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b6371f700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c1085e0 con 0x7f4b5c19adb0 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b61cbc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c19adb0 0x7f4b5c19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b61cbc700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c19adb0 0x7f4b5c19b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57140/0 (socket says 172.21.15.162:57140) 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b61cbc700 1 -- 172.21.15.162:0/1707478395 learned_addr learned my addr 172.21.15.162:0/1707478395 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-20T10:24:40.367 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b61cbc700 1 -- 172.21.15.162:0/1707478395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c19adb0 msgr2=0x7f4b5c19b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b61cbc700 1 -- 172.21.15.162:0/1707478395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c19adb0 msgr2=0x7f4b5c19b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b61cbc700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c19adb0 0x7f4b5c19b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b60cba700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c1964e0 0x7f4b5c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b61cbc700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c19adb0 0x7f4b5c19b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b614bb700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b480057d0 0x7f4b5c195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b614bb700 1 -- 172.21.15.162:0/1707478395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b480057d0 msgr2=0x7f4b5c195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b614bb700 1 -- 172.21.15.162:0/1707478395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b480057d0 msgr2=0x7f4b5c195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b614bb700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b480057d0 0x7f4b5c195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.365+0000 7f4b614bb700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b480057d0 0x7f4b5c195fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.366+0000 7f4b60cba700 1 -- 172.21.15.162:0/1707478395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c19adb0 msgr2=0x7f4b5c19b260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.366+0000 7f4b60cba700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c19adb0 0x7f4b5c19b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.366+0000 7f4b60cba700 1 -- 172.21.15.162:0/1707478395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b480057d0 msgr2=0x7f4b5c195fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:24:40.368 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.366+0000 7f4b60cba700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b480057d0 0x7f4b5c195fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:40.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.366+0000 7f4b60cba700 1 -- 172.21.15.162:0/1707478395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c1a2b20 con 0x7f4b5c1964e0 2024-04-20T10:24:40.369 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.366+0000 7f4b60cba700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c1964e0 0x7f4b5c19a870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b550 tx=0x7f4b4c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:40.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.366+0000 7f4b527fc700 1 -- 172.21.15.162:0/1707478395 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c004230 con 0x7f4b5c1964e0 2024-04-20T10:24:40.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.366+0000 7f4b527fc700 1 -- 172.21.15.162:0/1707478395 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b4c0043d0 con 0x7f4b5c1964e0 2024-04-20T10:24:40.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.366+0000 7f4b6371f700 1 -- 172.21.15.162:0/1707478395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c1a2d50 con 0x7f4b5c1964e0 2024-04-20T10:24:40.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.366+0000 7f4b527fc700 1 -- 172.21.15.162:0/1707478395 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c011c20 con 0x7f4b5c1964e0 2024-04-20T10:24:40.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.366+0000 7f4b6371f700 1 -- 172.21.15.162:0/1707478395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c1a3230 con 0x7f4b5c1964e0 2024-04-20T10:24:40.370 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.367+0000 7f4b527fc700 1 -- 172.21.15.162:0/1707478395 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4b4c01a400 con 0x7f4b5c1964e0 2024-04-20T10:24:40.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.368+0000 7f4b527fc700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4b40039ae0 0x7f4b4003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:40.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.368+0000 7f4b527fc700 1 -- 172.21.15.162:0/1707478395 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4b4c04c440 con 0x7f4b5c1964e0 2024-04-20T10:24:40.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.368+0000 7f4b6371f700 1 -- 172.21.15.162:0/1707478395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5c0640b0 con 0x7f4b5c1964e0 2024-04-20T10:24:40.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.368+0000 7f4b614bb700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4b40039ae0 0x7f4b4003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:40.371 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.369+0000 7f4b614bb700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4b40039ae0 0x7f4b4003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4b58000c00 tx=0x7f4b58005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:40.376 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.376+0000 7f4b527fc700 1 -- 172.21.15.162:0/1707478395 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4b4c01a7e0 con 0x7f4b5c1964e0 2024-04-20T10:24:40.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: mon.c calling monitor election 2024-04-20T10:24:40.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:24:40.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: mon.a calling monitor election 2024-04-20T10:24:40.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:40.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:24:40.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:40.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:40.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:40.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:40.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:40.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:40.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:40.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T10:24:40.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: monmap epoch 3 2024-04-20T10:24:40.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: fsid 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: last_changed 2024-04-20T10:24:35.214012+0000 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: created 2024-04-20T10:22:16.558137+0000 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: min_mon_release 19 (squid) 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: election_strategy: 1 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: 2: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.b 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: fsmap 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: mgrmap e13: x(active, since 69s) 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: overall HEALTH_OK 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:40.606 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:40.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:40.674 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.674+0000 7f4b6371f700 1 -- 172.21.15.162:0/1707478395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4b5c1a34f0 con 0x7f4b5c1964e0 2024-04-20T10:24:40.675 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.675+0000 7f4b527fc700 1 -- 172.21.15.162:0/1707478395 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f4b4c003c80 con 0x7f4b5c1964e0 2024-04-20T10:24:40.675 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-20T10:24:40.675 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":3,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","modified":"2024-04-20T10:24:35.214012Z","created":"2024-04-20T10:22:16.558137Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-20T10:24:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.677+0000 7f4b6371f700 1 -- 172.21.15.162:0/1707478395 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4b40039ae0 msgr2=0x7f4b4003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.677+0000 7f4b6371f700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4b40039ae0 0x7f4b4003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4b58000c00 tx=0x7f4b58005de0 comp rx=0 tx=0).stop 2024-04-20T10:24:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.677+0000 7f4b6371f700 1 -- 172.21.15.162:0/1707478395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c1964e0 msgr2=0x7f4b5c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.677+0000 7f4b6371f700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c1964e0 0x7f4b5c19a870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b550 tx=0x7f4b4c00b910 comp rx=0 tx=0).stop 2024-04-20T10:24:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.677+0000 7f4b6371f700 1 -- 172.21.15.162:0/1707478395 shutdown_connections 2024-04-20T10:24:40.677 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.677+0000 7f4b6371f700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4b40039ae0 0x7f4b4003bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.677+0000 7f4b6371f700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b480057d0 0x7f4b5c195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.677+0000 7f4b6371f700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c19adb0 0x7f4b5c19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.677+0000 7f4b6371f700 1 --2- 172.21.15.162:0/1707478395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c1964e0 0x7f4b5c19a870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.677+0000 7f4b6371f700 1 -- 172.21.15.162:0/1707478395 >> 172.21.15.162:0/1707478395 conn(0x7f4b5c0fcf30 msgr2=0x7f4b5c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.677+0000 7f4b6371f700 1 -- 172.21.15.162:0/1707478395 shutdown_connections 2024-04-20T10:24:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:40.677+0000 7f4b6371f700 1 -- 172.21.15.162:0/1707478395 wait complete. 2024-04-20T10:24:40.678 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 3 2024-04-20T10:24:41.269 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-20T10:24:41.270 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph config generate-minimal-conf 2024-04-20T10:24:41.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:41 smithi042 ceph-mon[17078]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:41.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:41.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:24:41.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:41 smithi042 ceph-mon[17078]: from='client.? 172.21.15.162:0/1707478395' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:24:41.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:41.604 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:41.685 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:24:42.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: mon.c calling monitor election 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: mon.a calling monitor election 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T10:24:42.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: monmap epoch 3 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: fsid 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: last_changed 2024-04-20T10:24:35.214012+0000 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: created 2024-04-20T10:22:16.558137+0000 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: min_mon_release 19 (squid) 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: election_strategy: 1 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: 2: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.b 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: fsmap 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: mgrmap e13: x(active, since 69s) 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: overall HEALTH_OK 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:42.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:42.556 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:42.556 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:42.556 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:42.556 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:42.556 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:24:42.556 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='client.? 172.21.15.162:0/1707478395' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T10:24:42.556 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:43.272 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: mon.b calling monitor election 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: mon.b calling monitor election 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: mon.c calling monitor election 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: mon.a calling monitor election 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: monmap epoch 3 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: fsid 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: last_changed 2024-04-20T10:24:35.214012+0000 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: created 2024-04-20T10:22:16.558137+0000 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: min_mon_release 19 (squid) 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: election_strategy: 1 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: 2: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.b 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: fsmap 2024-04-20T10:24:43.273 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T10:24:43.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: mgrmap e13: x(active, since 71s) 2024-04-20T10:24:43.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: overall HEALTH_OK 2024-04-20T10:24:43.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:43.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:43.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:43.274 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:43 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:43.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: mon.b calling monitor election 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: mon.b calling monitor election 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: mon.c calling monitor election 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: mon.a calling monitor election 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: monmap epoch 3 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: fsid 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: last_changed 2024-04-20T10:24:35.214012+0000 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: created 2024-04-20T10:22:16.558137+0000 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: min_mon_release 19 (squid) 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: election_strategy: 1 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-04-20T10:24:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: 2: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.b 2024-04-20T10:24:43.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: fsmap 2024-04-20T10:24:43.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T10:24:43.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: mgrmap e13: x(active, since 71s) 2024-04-20T10:24:43.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: overall HEALTH_OK 2024-04-20T10:24:43.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:43.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:43.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:43.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:43 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:43.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: mon.b calling monitor election 2024-04-20T10:24:43.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: mon.b calling monitor election 2024-04-20T10:24:43.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: mon.c calling monitor election 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: mon.a calling monitor election 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: monmap epoch 3 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: fsid 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: last_changed 2024-04-20T10:24:35.214012+0000 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: created 2024-04-20T10:22:16.558137+0000 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: min_mon_release 19 (squid) 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: election_strategy: 1 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: 2: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.b 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: fsmap 2024-04-20T10:24:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T10:24:43.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: mgrmap e13: x(active, since 71s) 2024-04-20T10:24:43.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: overall HEALTH_OK 2024-04-20T10:24:43.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:43.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:43.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:43.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:43 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:43.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.879+0000 7facbf400700 1 -- 172.21.15.42:0/1196875007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb8074df0 msgr2=0x7facb80751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:43.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.879+0000 7facbf400700 1 --2- 172.21.15.42:0/1196875007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb8074df0 0x7facb80751d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7facac009910 tx=0x7facac009c20 comp rx=0 tx=0).stop 2024-04-20T10:24:43.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.879+0000 7facbf400700 1 -- 172.21.15.42:0/1196875007 shutdown_connections 2024-04-20T10:24:43.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.879+0000 7facbf400700 1 --2- 172.21.15.42:0/1196875007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb8074df0 0x7facb80751d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:43.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.879+0000 7facbf400700 1 -- 172.21.15.42:0/1196875007 >> 172.21.15.42:0/1196875007 conn(0x7facb80fdf90 msgr2=0x7facb81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:43.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.879+0000 7facbf400700 1 -- 172.21.15.42:0/1196875007 shutdown_connections 2024-04-20T10:24:43.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.879+0000 7facbf400700 1 -- 172.21.15.42:0/1196875007 wait complete. 2024-04-20T10:24:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.880+0000 7facbf400700 1 Processor -- start 2024-04-20T10:24:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.880+0000 7facbf400700 1 -- start start 2024-04-20T10:24:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.881+0000 7facbf400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facb8074df0 0x7facb81a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.881+0000 7facbf400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb81a4b50 0x7facb819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.881+0000 7facbf400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb819ec20 0x7facb819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.881+0000 7facbf400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7facb8113100 con 0x7facb81a4b50 2024-04-20T10:24:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.881+0000 7facbf400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7facb8112f80 con 0x7facb819ec20 2024-04-20T10:24:43.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.881+0000 7facbf400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7facb8113280 con 0x7facb8074df0 2024-04-20T10:24:43.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.881+0000 7facbca00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facb8074df0 0x7facb81a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:43.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.881+0000 7facbca00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facb8074df0 0x7facb81a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:58510/0 (socket says 172.21.15.42:58510) 2024-04-20T10:24:43.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.881+0000 7facbca00700 1 -- 172.21.15.42:0/3606458978 learned_addr learned my addr 172.21.15.42:0/3606458978 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:24:43.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.881+0000 7facb7e00700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb81a4b50 0x7facb819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:43.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.881+0000 7facbd400700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb819ec20 0x7facb819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:43.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.882+0000 7facbca00700 1 -- 172.21.15.42:0/3606458978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb819ec20 msgr2=0x7facb819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:43.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.882+0000 7facbca00700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb819ec20 0x7facb819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:43.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.882+0000 7facbca00700 1 -- 172.21.15.42:0/3606458978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb81a4b50 msgr2=0x7facb819e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:43.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.882+0000 7facbca00700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb81a4b50 0x7facb819e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:43.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.882+0000 7facbca00700 1 -- 172.21.15.42:0/3606458978 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb81ab3c0 con 0x7facb8074df0 2024-04-20T10:24:43.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.882+0000 7facbca00700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facb8074df0 0x7facb81a4610 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7facac0092e0 tx=0x7facac004fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.882+0000 7facb5600700 1 -- 172.21.15.42:0/3606458978 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac026070 con 0x7facb8074df0 2024-04-20T10:24:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.882+0000 7facb5600700 1 -- 172.21.15.42:0/3606458978 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facac00b750 con 0x7facb8074df0 2024-04-20T10:24:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.882+0000 7facb5600700 1 -- 172.21.15.42:0/3606458978 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac003970 con 0x7facb8074df0 2024-04-20T10:24:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.882+0000 7facbf400700 1 -- 172.21.15.42:0/3606458978 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb81ab5f0 con 0x7facb8074df0 2024-04-20T10:24:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.882+0000 7facbf400700 1 -- 172.21.15.42:0/3606458978 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb81abb20 con 0x7facb8074df0 2024-04-20T10:24:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.883+0000 7facb5600700 1 -- 172.21.15.42:0/3606458978 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7facac029070 con 0x7facb8074df0 2024-04-20T10:24:43.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.884+0000 7facbf400700 1 -- 172.21.15.42:0/3606458978 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb8076530 con 0x7facb8074df0 2024-04-20T10:24:43.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.884+0000 7facb5600700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faca0039ae0 0x7faca003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:43.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.884+0000 7facb5600700 1 -- 172.21.15.42:0/3606458978 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7facac04d4b0 con 0x7facb8074df0 2024-04-20T10:24:43.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.884+0000 7facb7e00700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faca0039ae0 0x7faca003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:43.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.885+0000 7facb7e00700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faca0039ae0 0x7faca003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faca8009ad0 tx=0x7faca8009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:43.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:43.890+0000 7facb5600700 1 -- 172.21.15.42:0/3606458978 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7facac013070 con 0x7facb8074df0 2024-04-20T10:24:44.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.127+0000 7facbf400700 1 -- 172.21.15.42:0/3606458978 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7facb81abd80 con 0x7facb8074df0 2024-04-20T10:24:44.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.132+0000 7facb5600700 1 -- 172.21.15.42:0/3606458978 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7facac030020 con 0x7facb8074df0 2024-04-20T10:24:44.132 INFO:teuthology.orchestra.run.smithi042.stdout:# minimal ceph.conf for 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:24:44.132 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-04-20T10:24:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:24:44.132 INFO:teuthology.orchestra.run.smithi042.stdout: mon_host = [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-04-20T10:24:44.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.137+0000 7facbf400700 1 -- 172.21.15.42:0/3606458978 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faca0039ae0 msgr2=0x7faca003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:44.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.137+0000 7facbf400700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faca0039ae0 0x7faca003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faca8009ad0 tx=0x7faca8009210 comp rx=0 tx=0).stop 2024-04-20T10:24:44.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.137+0000 7facbf400700 1 -- 172.21.15.42:0/3606458978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facb8074df0 msgr2=0x7facb81a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:44.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.137+0000 7facbf400700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facb8074df0 0x7facb81a4610 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7facac0092e0 tx=0x7facac004fa0 comp rx=0 tx=0).stop 2024-04-20T10:24:44.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.137+0000 7facbf400700 1 -- 172.21.15.42:0/3606458978 shutdown_connections 2024-04-20T10:24:44.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.137+0000 7facbf400700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faca0039ae0 0x7faca003bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:44.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.137+0000 7facbf400700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb819ec20 0x7facb819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:44.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.137+0000 7facbf400700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facb8074df0 0x7facb81a4610 secure :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7facac0092e0 tx=0x7facac004fa0 comp rx=0 tx=0).stop 2024-04-20T10:24:44.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.137+0000 7facbf400700 1 --2- 172.21.15.42:0/3606458978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb81a4b50 0x7facb819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:44.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.137+0000 7facbf400700 1 -- 172.21.15.42:0/3606458978 >> 172.21.15.42:0/3606458978 conn(0x7facb80fdf90 msgr2=0x7facb80ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:44.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.137+0000 7facbf400700 1 -- 172.21.15.42:0/3606458978 shutdown_connections 2024-04-20T10:24:44.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:44.137+0000 7facbf400700 1 -- 172.21.15.42:0/3606458978 wait complete. 2024-04-20T10:24:44.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:44.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: Updating smithi162:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: Updating smithi046:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: Updating smithi042:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:24:44.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:44.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:44.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: Updating smithi162:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:44.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: Updating smithi046:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:44.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: Updating smithi042:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:44.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:44.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.518 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:24:44.518 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:24:44.518 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: Updating smithi162:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: Updating smithi046:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: Updating smithi042:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:24:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:24:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:44.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:24:44.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:24:44.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:44.906 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-20T10:24:44.906 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:24:44.906 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T10:24:44.977 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:24:44.977 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:24:45.094 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-20T10:24:45.094 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T10:24:45.129 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-20T10:24:45.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:24:45.198 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-20T10:24:45.199 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T10:24:45.221 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:45 smithi162 ceph-mon[21854]: Reconfiguring mon.a (unknown last config time)... 2024-04-20T10:24:45.221 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:45 smithi162 ceph-mon[21854]: Reconfiguring daemon mon.a on smithi042 2024-04-20T10:24:45.222 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:45 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3606458978' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:45.226 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-20T10:24:45.226 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:24:45.291 INFO:tasks.cephadm:Adding mgr.x on smithi042 2024-04-20T10:24:45.291 INFO:tasks.cephadm:Adding mgr.y on smithi046 2024-04-20T10:24:45.291 INFO:tasks.cephadm:Adding mgr.z on smithi162 2024-04-20T10:24:45.291 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch apply mgr '3;smithi042=x;smithi046=y;smithi162=z' 2024-04-20T10:24:45.483 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:45 smithi042 ceph-mon[17078]: Reconfiguring mon.a (unknown last config time)... 2024-04-20T10:24:45.483 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:45 smithi042 ceph-mon[17078]: Reconfiguring daemon mon.a on smithi042 2024-04-20T10:24:45.483 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:45 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3606458978' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:45.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:45 smithi046 ceph-mon[21574]: Reconfiguring mon.a (unknown last config time)... 2024-04-20T10:24:45.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:45 smithi046 ceph-mon[21574]: Reconfiguring daemon mon.a on smithi042 2024-04-20T10:24:45.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:45 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3606458978' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:45.559 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:24:46.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:46 smithi162 ceph-mon[21854]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:46.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:46 smithi046 ceph-mon[21574]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:46.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:46 smithi042 ceph-mon[17078]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:47.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.114+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2165004656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff41014f0 msgr2=0x7f4ff4101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:47.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.114+0000 7f4ffa6a9700 1 --2- 172.21.15.162:0/2165004656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff41014f0 0x7f4ff4101910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009910 tx=0x7f4fe4009c20 comp rx=0 tx=0).stop 2024-04-20T10:24:47.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.114+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2165004656 shutdown_connections 2024-04-20T10:24:47.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.114+0000 7f4ffa6a9700 1 --2- 172.21.15.162:0/2165004656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff41014f0 0x7f4ff4101910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:47.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.114+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2165004656 >> 172.21.15.162:0/2165004656 conn(0x7f4ff40fca90 msgr2=0x7f4ff40feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:47.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.114+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2165004656 shutdown_connections 2024-04-20T10:24:47.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.114+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2165004656 wait complete. 2024-04-20T10:24:47.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ffa6a9700 1 Processor -- start 2024-04-20T10:24:47.118 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ffa6a9700 1 -- start start 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ffa6a9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff41014f0 0x7f4ff4196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ffa6a9700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff41972a0 0x7f4ff419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ffa6a9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff4197720 0x7f4ff419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ffa6a9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff4107580 con 0x7f4ff4197720 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ffa6a9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff4107400 con 0x7f4ff41014f0 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ffa6a9700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff4107700 con 0x7f4ff41972a0 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ff37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff41972a0 0x7f4ff419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ff37fe700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff41972a0 0x7f4ff419c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57156/0 (socket says 172.21.15.162:57156) 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ff37fe700 1 -- 172.21.15.162:0/2446878172 learned_addr learned my addr 172.21.15.162:0/2446878172 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ff3fff700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff41014f0 0x7f4ff4196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ff37fe700 1 -- 172.21.15.162:0/2446878172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff41014f0 msgr2=0x7f4ff4196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ff8c46700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff4197720 0x7f4ff419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ff37fe700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff41014f0 0x7f4ff4196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.115+0000 7f4ff37fe700 1 -- 172.21.15.162:0/2446878172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff4197720 msgr2=0x7f4ff419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:47.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.116+0000 7f4ff37fe700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff4197720 0x7f4ff419e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.116+0000 7f4ff37fe700 1 -- 172.21.15.162:0/2446878172 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff419f170 con 0x7f4ff41972a0 2024-04-20T10:24:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.116+0000 7f4ff3fff700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff41014f0 0x7f4ff4196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:24:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.116+0000 7f4ff37fe700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff41972a0 0x7f4ff419c310 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800b780 tx=0x7f4fe800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.116+0000 7f4ff17fa700 1 -- 172.21.15.162:0/2446878172 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8004230 con 0x7f4ff41972a0 2024-04-20T10:24:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.116+0000 7f4ff17fa700 1 -- 172.21.15.162:0/2446878172 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe80043d0 con 0x7f4ff41972a0 2024-04-20T10:24:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.116+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2446878172 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff419f460 con 0x7f4ff41972a0 2024-04-20T10:24:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.116+0000 7f4ff17fa700 1 -- 172.21.15.162:0/2446878172 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8011c60 con 0x7f4ff41972a0 2024-04-20T10:24:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.116+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2446878172 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff419fa20 con 0x7f4ff41972a0 2024-04-20T10:24:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.117+0000 7f4ff17fa700 1 -- 172.21.15.162:0/2446878172 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4fe801a400 con 0x7f4ff41972a0 2024-04-20T10:24:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.117+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2446878172 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff40640b0 con 0x7f4ff41972a0 2024-04-20T10:24:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.117+0000 7f4ff17fa700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4fdc039ae0 0x7f4fdc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:47.120 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.117+0000 7f4ff17fa700 1 -- 172.21.15.162:0/2446878172 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4fe804c4c0 con 0x7f4ff41972a0 2024-04-20T10:24:47.121 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.118+0000 7f4ff3fff700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4fdc039ae0 0x7f4fdc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:47.121 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.119+0000 7f4ff3fff700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4fdc039ae0 0x7f4fdc03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009750 tx=0x7f4fe40052d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:47.123 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.122+0000 7f4ff17fa700 1 -- 172.21.15.162:0/2446878172 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4fe8020500 con 0x7f4ff41972a0 2024-04-20T10:24:47.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.331+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2446878172 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi046=y;smithi162=z", "target": ["mon-mgr", ""]}) -- 0x7f4ff4062f50 con 0x7f4fdc039ae0 2024-04-20T10:24:47.345 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.344+0000 7f4ff17fa700 1 -- 172.21.15.162:0/2446878172 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4ff4062f50 con 0x7f4fdc039ae0 2024-04-20T10:24:47.345 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled mgr update... 2024-04-20T10:24:47.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.347+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2446878172 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4fdc039ae0 msgr2=0x7f4fdc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:47.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.347+0000 7f4ffa6a9700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4fdc039ae0 0x7f4fdc03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009750 tx=0x7f4fe40052d0 comp rx=0 tx=0).stop 2024-04-20T10:24:47.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.347+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2446878172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff41972a0 msgr2=0x7f4ff419c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:47.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.347+0000 7f4ffa6a9700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff41972a0 0x7f4ff419c310 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800b780 tx=0x7f4fe800bb40 comp rx=0 tx=0).stop 2024-04-20T10:24:47.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.347+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2446878172 shutdown_connections 2024-04-20T10:24:47.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.347+0000 7f4ffa6a9700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4fdc039ae0 0x7f4fdc03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:47.347 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.347+0000 7f4ffa6a9700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ff41014f0 0x7f4ff4196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:47.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.347+0000 7f4ffa6a9700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff4197720 0x7f4ff419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:47.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.347+0000 7f4ffa6a9700 1 --2- 172.21.15.162:0/2446878172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ff41972a0 0x7f4ff419c310 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:47.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.347+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2446878172 >> 172.21.15.162:0/2446878172 conn(0x7f4ff40fca90 msgr2=0x7f4ff40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:47.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.347+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2446878172 shutdown_connections 2024-04-20T10:24:47.348 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:24:47.347+0000 7f4ffa6a9700 1 -- 172.21.15.162:0/2446878172 wait complete. 2024-04-20T10:24:47.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:47 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:47.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:47 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:47 smithi162 ceph-mon[21854]: Reconfiguring mon.b (monmap changed)... 2024-04-20T10:24:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:47 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:24:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:47 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:24:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:47 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:47 smithi162 ceph-mon[21854]: Reconfiguring daemon mon.b on smithi046 2024-04-20T10:24:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:47 smithi162 ceph-mon[21854]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:47 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:47.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:47 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:47.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:47 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:47.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:47 smithi046 ceph-mon[21574]: Reconfiguring mon.b (monmap changed)... 2024-04-20T10:24:47.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:47 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:24:47.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:47 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:24:47.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:47 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:47.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:47 smithi046 ceph-mon[21574]: Reconfiguring daemon mon.b on smithi046 2024-04-20T10:24:47.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:47 smithi046 ceph-mon[21574]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:47.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:47 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:47 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:47 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:47.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:47 smithi042 ceph-mon[17078]: Reconfiguring mon.b (monmap changed)... 2024-04-20T10:24:47.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:47 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:24:47.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:47 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:24:47.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:47 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:47.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:47 smithi042 ceph-mon[17078]: Reconfiguring daemon mon.b on smithi046 2024-04-20T10:24:47.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:47 smithi042 ceph-mon[17078]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:47.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:47 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:47.938 DEBUG:teuthology.orchestra.run.smithi046:mgr.y> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.y.service 2024-04-20T10:24:47.941 DEBUG:teuthology.orchestra.run.smithi162:mgr.z> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.z.service 2024-04-20T10:24:47.943 INFO:tasks.cephadm:Deploying OSDs... 2024-04-20T10:24:47.943 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:24:47.943 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T10:24:48.005 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T10:24:48.005 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-04-20T10:24:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T10:24:48.058 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:24:48.058 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-20T10:24:48.058 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:24:48.059 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:24:48.059 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-20 10:23:47.459462222 +0000 2024-04-20T10:24:48.059 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-20 10:14:20.152087413 +0000 2024-04-20T10:24:48.059 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-20 10:14:20.152087413 +0000 2024-04-20T10:24:48.059 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-20 10:14:20.152087413 +0000 2024-04-20T10:24:48.059 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T10:24:48.145 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-04-20T10:24:48.145 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-04-20T10:24:48.145 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000255778 s, 2.0 MB/s 2024-04-20T10:24:48.149 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T10:24:48.208 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-04-20T10:24:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T10:24:48.263 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:24:48.263 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-20T10:24:48.263 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:24:48.263 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:24:48.263 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-20 10:23:47.547461518 +0000 2024-04-20T10:24:48.264 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-20 10:14:20.771082417 +0000 2024-04-20T10:24:48.264 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-20 10:14:20.771082417 +0000 2024-04-20T10:24:48.264 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-20 10:14:20.771082417 +0000 2024-04-20T10:24:48.264 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T10:24:48.350 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-04-20T10:24:48.350 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-04-20T10:24:48.351 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000300266 s, 1.7 MB/s 2024-04-20T10:24:48.355 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T10:24:48.415 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-04-20T10:24:48.470 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T10:24:48.470 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:24:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-20T10:24:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:24:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:24:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-20 10:23:47.645460733 +0000 2024-04-20T10:24:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-20 10:14:21.411077260 +0000 2024-04-20T10:24:48.470 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-20 10:14:21.411077260 +0000 2024-04-20T10:24:48.470 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-20 10:14:21.411077260 +0000 2024-04-20T10:24:48.470 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T10:24:48.548 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-04-20T10:24:48.548 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-04-20T10:24:48.548 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000262214 s, 2.0 MB/s 2024-04-20T10:24:48.552 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T10:24:48.607 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-04-20T10:24:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T10:24:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:24:48.662 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-20T10:24:48.662 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:24:48.662 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:24:48.662 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-20 10:23:47.715460172 +0000 2024-04-20T10:24:48.662 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-20 10:14:22.019072360 +0000 2024-04-20T10:24:48.662 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-20 10:14:22.019072360 +0000 2024-04-20T10:24:48.662 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-20 10:14:22.019072360 +0000 2024-04-20T10:24:48.663 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T10:24:48.749 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-04-20T10:24:48.749 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-04-20T10:24:48.749 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000275811 s, 1.9 MB/s 2024-04-20T10:24:48.753 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T10:24:48.813 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-20T10:24:48.813 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T10:24:48.832 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T10:24:48.832 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-04-20T10:24:48.891 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T10:24:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:24:48.892 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-20T10:24:48.892 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:24:48.892 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:24:48.892 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-20 10:24:34.633541510 +0000 2024-04-20T10:24:48.892 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-20 10:14:07.579074465 +0000 2024-04-20T10:24:48.892 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-20 10:14:07.579074465 +0000 2024-04-20T10:24:48.892 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-20 10:14:07.579074465 +0000 2024-04-20T10:24:48.892 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T10:24:48.958 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-20T10:24:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-20T10:24:48.959 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.00011208 s, 4.6 MB/s 2024-04-20T10:24:48.960 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T10:24:49.020 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-04-20T10:24:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T10:24:49.077 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:24:49.077 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-20T10:24:49.077 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:24:49.078 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:24:49.078 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-20 10:24:34.634541499 +0000 2024-04-20T10:24:49.078 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-20 10:14:07.951070531 +0000 2024-04-20T10:24:49.078 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-20 10:14:07.951070531 +0000 2024-04-20T10:24:49.078 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-20 10:14:07.951070531 +0000 2024-04-20T10:24:49.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T10:24:49.141 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:48 smithi046 ceph-mon[21574]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi046=y;smithi162=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:24:49.141 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:48 smithi046 ceph-mon[21574]: Saving service mgr spec with placement smithi042=x;smithi046=y;smithi162=z;count:3 2024-04-20T10:24:49.141 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:49.141 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:49.142 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:48 smithi046 ceph-mon[21574]: Reconfiguring mon.c (monmap changed)... 2024-04-20T10:24:49.142 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:24:49.142 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:24:49.142 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:49.142 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:48 smithi046 ceph-mon[21574]: Reconfiguring daemon mon.c on smithi162 2024-04-20T10:24:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-20T10:24:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-20T10:24:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000114073 s, 4.5 MB/s 2024-04-20T10:24:49.145 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T10:24:49.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:48 smithi162 ceph-mon[21854]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi046=y;smithi162=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:24:49.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:48 smithi162 ceph-mon[21854]: Saving service mgr spec with placement smithi042=x;smithi046=y;smithi162=z;count:3 2024-04-20T10:24:49.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:49.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:49.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:48 smithi162 ceph-mon[21854]: Reconfiguring mon.c (monmap changed)... 2024-04-20T10:24:49.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:24:49.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:24:49.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:49.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:48 smithi162 ceph-mon[21854]: Reconfiguring daemon mon.c on smithi162 2024-04-20T10:24:49.203 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-04-20T10:24:49.260 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T10:24:49.261 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:24:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-20T10:24:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:24:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:24:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-20 10:24:34.634541499 +0000 2024-04-20T10:24:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-20 10:14:08.290066946 +0000 2024-04-20T10:24:49.261 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-20 10:14:08.290066946 +0000 2024-04-20T10:24:49.261 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-20 10:14:08.290066946 +0000 2024-04-20T10:24:49.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T10:24:49.325 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-20T10:24:49.325 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-20T10:24:49.325 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000118562 s, 4.3 MB/s 2024-04-20T10:24:49.326 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T10:24:49.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:48 smithi042 ceph-mon[17078]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi046=y;smithi162=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:24:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:48 smithi042 ceph-mon[17078]: Saving service mgr spec with placement smithi042=x;smithi046=y;smithi162=z;count:3 2024-04-20T10:24:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:48 smithi042 ceph-mon[17078]: Reconfiguring mon.c (monmap changed)... 2024-04-20T10:24:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T10:24:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T10:24:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:48 smithi042 ceph-mon[17078]: Reconfiguring daemon mon.c on smithi162 2024-04-20T10:24:49.384 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-04-20T10:24:49.441 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T10:24:49.442 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:24:49.442 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-20T10:24:49.442 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:24:49.442 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:24:49.442 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-20 10:24:34.634541499 +0000 2024-04-20T10:24:49.442 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-20 10:14:08.637063276 +0000 2024-04-20T10:24:49.442 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-20 10:14:08.637063276 +0000 2024-04-20T10:24:49.442 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-20 10:14:08.637063276 +0000 2024-04-20T10:24:49.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T10:24:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-20T10:24:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-20T10:24:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 9.9762e-05 s, 5.1 MB/s 2024-04-20T10:24:49.508 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T10:24:49.566 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-20T10:24:49.567 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T10:24:49.582 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T10:24:49.582 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2024-04-20T10:24:49.637 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T10:24:49.637 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:24:49.637 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-20T10:24:49.637 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:24:49.637 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:24:49.637 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-04-20 10:24:24.990349295 +0000 2024-04-20T10:24:49.637 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-20 10:14:00.228979696 +0000 2024-04-20T10:24:49.637 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-20 10:14:00.228979696 +0000 2024-04-20T10:24:49.637 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-04-20 10:14:00.228979696 +0000 2024-04-20T10:24:49.638 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T10:24:49.700 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-04-20T10:24:49.700 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-04-20T10:24:49.700 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000218775 s, 2.3 MB/s 2024-04-20T10:24:49.701 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T10:24:49.757 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2024-04-20T10:24:49.813 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T10:24:49.813 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:24:49.813 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-20T10:24:49.813 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:24:49.813 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:24:49.813 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-04-20 10:24:24.990349295 +0000 2024-04-20T10:24:49.813 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-20 10:14:00.587972407 +0000 2024-04-20T10:24:49.813 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-20 10:14:00.587972407 +0000 2024-04-20T10:24:49.813 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-04-20 10:14:00.587972407 +0000 2024-04-20T10:24:49.814 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T10:24:49.875 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-04-20T10:24:49.875 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-04-20T10:24:49.875 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000169192 s, 3.0 MB/s 2024-04-20T10:24:49.876 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T10:24:49.932 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2024-04-20T10:24:49.987 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T10:24:49.987 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:24:49.988 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-20T10:24:49.988 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:24:49.988 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:24:49.988 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-04-20 10:24:24.991349276 +0000 2024-04-20T10:24:49.988 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-20 10:14:00.919965665 +0000 2024-04-20T10:24:49.988 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-20 10:14:00.919965665 +0000 2024-04-20T10:24:49.988 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-04-20 10:14:00.919965665 +0000 2024-04-20T10:24:49.988 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T10:24:50.048 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:49 smithi162 ceph-mon[21854]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:50.048 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:50.048 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:50.048 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:24:50.048 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:50.048 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:24:50.048 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:50.048 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:24:50.049 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T10:24:50.049 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:24:50.049 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:50.050 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-04-20T10:24:50.050 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-04-20T10:24:50.051 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000183815 s, 2.8 MB/s 2024-04-20T10:24:50.052 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T10:24:50.108 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2024-04-20T10:24:50.164 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T10:24:50.164 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:24:50.164 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-20T10:24:50.164 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:24:50.164 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:24:50.164 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-04-20 10:24:24.991349276 +0000 2024-04-20T10:24:50.164 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-20 10:14:01.255958842 +0000 2024-04-20T10:24:50.164 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-20 10:14:01.255958842 +0000 2024-04-20T10:24:50.164 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-04-20 10:14:01.255958842 +0000 2024-04-20T10:24:50.165 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T10:24:50.225 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-04-20T10:24:50.226 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-04-20T10:24:50.226 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000252263 s, 2.0 MB/s 2024-04-20T10:24:50.227 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T10:24:50.282 INFO:tasks.cephadm:Deploying osd.0 on smithi042 with /dev/vg_nvme/lv_4... 2024-04-20T10:24:50.282 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T10:24:50.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:49 smithi046 ceph-mon[21574]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:50.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:50.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:50.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:24:50.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:50.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:24:50.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:50.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:24:50.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T10:24:50.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:24:50.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:50.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:49 smithi042 ceph-mon[17078]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:50.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:50.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:50.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:24:50.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:50.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:24:50.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:50.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:24:50.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T10:24:50.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:24:50.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:50.729 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:24:51.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:50 smithi162 ceph-mon[21854]: Deploying daemon mgr.z on smithi162 2024-04-20T10:24:51.190 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:50 smithi042 ceph-mon[17078]: Deploying daemon mgr.z on smithi162 2024-04-20T10:24:51.205 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:50 smithi046 ceph-mon[21574]: Deploying daemon mgr.z on smithi162 2024-04-20T10:24:51.811 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:51 smithi162 systemd[1]: Starting Ceph mgr.z for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:24:51.908 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:51 smithi042 ceph-mon[17078]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:52.129 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:51 smithi162 ceph-mon[21854]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:52.130 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:51 smithi162 podman[23725]: 2024-04-20 10:24:51.811272575 +0000 UTC m=+0.107726620 container create 5fd52e305ebf258ae5439cca6095810c726c5fd3d3af7cb414f24c1045fb1723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:24:52.130 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:51 smithi162 podman[23725]: 2024-04-20 10:24:51.721417421 +0000 UTC m=+0.017871418 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:24:52.130 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:52 smithi162 podman[23725]: 2024-04-20 10:24:52.130361664 +0000 UTC m=+0.426815645 container init 5fd52e305ebf258ae5439cca6095810c726c5fd3d3af7cb414f24c1045fb1723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-20T10:24:52.221 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:51 smithi046 ceph-mon[21574]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:52.399 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:52 smithi162 podman[23725]: 2024-04-20 10:24:52.133582829 +0000 UTC m=+0.430036803 container start 5fd52e305ebf258ae5439cca6095810c726c5fd3d3af7cb414f24c1045fb1723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-20T10:24:52.399 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:52 smithi162 ceph-mgr[23744]: -- 172.21.15.162:0/2268681527 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562a04f605a0 con 0x562a04f34c00 2024-04-20T10:24:52.399 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:52 smithi162 bash[23725]: 5fd52e305ebf258ae5439cca6095810c726c5fd3d3af7cb414f24c1045fb1723 2024-04-20T10:24:52.400 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:52 smithi162 systemd[1]: Started Ceph mgr.z for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:24:52.400 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:52.288+0000 7f108ab00200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:24:52.883 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:52.473+0000 7f108ab00200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:24:52.883 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:52.575+0000 7f108ab00200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:24:52.883 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:52.882+0000 7f108ab00200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:24:53.149 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:52.967+0000 7f108ab00200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:24:53.495 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T10:24:53.495 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T10:24:53.495 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-04-20T10:24:53.495 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-04-20T10:24:53.495 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0360266 s, 291 MB/s 2024-04-20T10:24:53.495 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-04-20T10:24:53.543 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:53.543 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:53.543 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:53.543 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:24:53.543 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T10:24:53.543 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:24:53.543 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:53.543 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:53 smithi042 ceph-mon[17078]: Deploying daemon mgr.y on smithi046 2024-04-20T10:24:53.543 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:53 smithi042 ceph-mon[17078]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:53.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:53.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:53.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:53.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:24:53.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T10:24:53.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:24:53.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:53.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:53 smithi046 ceph-mon[21574]: Deploying daemon mgr.y on smithi046 2024-04-20T10:24:53.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:53 smithi046 ceph-mon[21574]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:53.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:53.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:53.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:53.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:24:53.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T10:24:53.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:24:53.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:53.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:53 smithi162 ceph-mon[21854]: Deploying daemon mgr.y on smithi046 2024-04-20T10:24:53.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:53 smithi162 ceph-mon[21854]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:54.267 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_4 2024-04-20T10:24:54.511 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:54.172+0000 7f108ab00200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:24:54.512 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:54.340+0000 7f108ab00200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:24:54.512 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:54.511+0000 7f108ab00200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:24:54.553 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:54 smithi046 podman[22955]: 2024-04-20 10:24:54.476983203 +0000 UTC m=+0.125454357 container create 2c1fd9edaa3bb6c02292f7a8f6d8364871cf6e6c5ce2255824f0056b89360abd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-20T10:24:54.726 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:24:54.852 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:54 smithi046 podman[22955]: 2024-04-20 10:24:54.687795276 +0000 UTC m=+0.336266386 container init 2c1fd9edaa3bb6c02292f7a8f6d8364871cf6e6c5ce2255824f0056b89360abd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-20T10:24:54.852 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:54 smithi046 podman[22955]: 2024-04-20 10:24:54.690611315 +0000 UTC m=+0.339082433 container start 2c1fd9edaa3bb6c02292f7a8f6d8364871cf6e6c5ce2255824f0056b89360abd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-20T10:24:54.853 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:54 smithi046 ceph-mgr[22974]: -- 172.21.15.46:0/2600413618 <== mon.2 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5616a5b865a0 con 0x5616a5b5a800 2024-04-20T10:24:54.853 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:54 smithi046 bash[22955]: 2c1fd9edaa3bb6c02292f7a8f6d8364871cf6e6c5ce2255824f0056b89360abd 2024-04-20T10:24:54.853 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:54 smithi046 systemd[1]: Started Ceph mgr.y for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:24:55.092 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:54.796+0000 7f108ab00200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:24:55.092 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:54.889+0000 7f108ab00200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:24:55.092 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:55.092+0000 7f108ab00200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:24:55.150 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:54 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:54.852+0000 7f8e9c9aa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:24:55.150 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:55.043+0000 7f8e9c9aa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:24:55.150 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:55.150+0000 7f8e9c9aa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:24:55.399 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:55.290+0000 7f108ab00200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:24:55.803 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:55.479+0000 7f8e9c9aa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:24:55.803 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:55.569+0000 7f8e9c9aa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:24:55.964 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:55 smithi042 ceph-mon[17078]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:55.964 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:55 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:55.964 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:55 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:55.964 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:55 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:55.964 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:55 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:55.965 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:55 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:24:55.965 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:55 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:56.149 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:55.935+0000 7f108ab00200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:24:56.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:55 smithi162 ceph-mon[21854]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:56.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:55 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:56.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:55 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:56.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:55 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:56.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:55 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:56.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:55 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:24:56.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:55 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:56.213 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:55 smithi046 ceph-mon[21574]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:56.213 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:55 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:56.213 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:55 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:56.213 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:55 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:56.213 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:55 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:56.213 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:55 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:24:56.214 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:55 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:56.475 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:56.368+0000 7f108ab00200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:24:56.475 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:56.475+0000 7f108ab00200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:24:56.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.778+0000 7f252e800700 1 -- 172.21.15.42:0/3264793815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2528103cc0 msgr2=0x7f2528104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.778+0000 7f252e800700 1 --2- 172.21.15.42:0/3264793815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2528103cc0 0x7f2528104120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f25100099c0 tx=0x7f2510009cd0 comp rx=0 tx=0).stop 2024-04-20T10:24:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.778+0000 7f252e800700 1 -- 172.21.15.42:0/3264793815 shutdown_connections 2024-04-20T10:24:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.778+0000 7f252e800700 1 --2- 172.21.15.42:0/3264793815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2528104660 0x7f252810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.778+0000 7f252e800700 1 --2- 172.21.15.42:0/3264793815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2528102ac0 0x7f2528102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.778+0000 7f252e800700 1 --2- 172.21.15.42:0/3264793815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2528103cc0 0x7f2528104120 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.778+0000 7f252e800700 1 -- 172.21.15.42:0/3264793815 >> 172.21.15.42:0/3264793815 conn(0x7f25280fe020 msgr2=0x7f25281004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.778+0000 7f252e800700 1 -- 172.21.15.42:0/3264793815 shutdown_connections 2024-04-20T10:24:56.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.778+0000 7f252e800700 1 -- 172.21.15.42:0/3264793815 wait complete. 2024-04-20T10:24:56.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.779+0000 7f252e800700 1 Processor -- start 2024-04-20T10:24:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.780+0000 7f252e800700 1 -- start start 2024-04-20T10:24:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.780+0000 7f252e800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2528102ac0 0x7f252819af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.780+0000 7f252e800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2528103cc0 0x7f252819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.780+0000 7f252e800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2528104660 0x7f25281a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.780+0000 7f252e800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f252810e6f0 con 0x7f2528104660 2024-04-20T10:24:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.780+0000 7f252e800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f252810e570 con 0x7f2528103cc0 2024-04-20T10:24:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.780+0000 7f252e800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f252810e870 con 0x7f2528102ac0 2024-04-20T10:24:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.780+0000 7f2527400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2528102ac0 0x7f252819af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:56.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.780+0000 7f251fe00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2528103cc0 0x7f252819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.780+0000 7f2527400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2528102ac0 0x7f252819af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:53482/0 (socket says 172.21.15.42:53482) 2024-04-20T10:24:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.780+0000 7f2527400700 1 -- 172.21.15.42:0/4139003037 learned_addr learned my addr 172.21.15.42:0/4139003037 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:24:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.780+0000 7f2527e00700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2528104660 0x7f25281a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.781+0000 7f2527400700 1 -- 172.21.15.42:0/4139003037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2528103cc0 msgr2=0x7f252819b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.781+0000 7f2527400700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2528103cc0 0x7f252819b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:56.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.781+0000 7f2527400700 1 -- 172.21.15.42:0/4139003037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2528104660 msgr2=0x7f25281a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.781+0000 7f2527400700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2528104660 0x7f25281a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.781+0000 7f2527400700 1 -- 172.21.15.42:0/4139003037 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25281a2bb0 con 0x7f2528102ac0 2024-04-20T10:24:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.781+0000 7f2527e00700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2528104660 0x7f25281a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:24:56.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.781+0000 7f2527400700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2528102ac0 0x7f252819af50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f251800e980 tx=0x7f251800ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.781+0000 7f2524c00700 1 -- 172.21.15.42:0/4139003037 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251800cbd0 con 0x7f2528102ac0 2024-04-20T10:24:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.781+0000 7f2524c00700 1 -- 172.21.15.42:0/4139003037 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2518004510 con 0x7f2528102ac0 2024-04-20T10:24:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.782+0000 7f252e800700 1 -- 172.21.15.42:0/4139003037 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25281a2ea0 con 0x7f2528102ac0 2024-04-20T10:24:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.782+0000 7f2524c00700 1 -- 172.21.15.42:0/4139003037 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518010430 con 0x7f2528102ac0 2024-04-20T10:24:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.782+0000 7f252e800700 1 -- 172.21.15.42:0/4139003037 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25281a32f0 con 0x7f2528102ac0 2024-04-20T10:24:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.783+0000 7f2524c00700 1 -- 172.21.15.42:0/4139003037 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f25180105d0 con 0x7f2528102ac0 2024-04-20T10:24:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.783+0000 7f252e800700 1 -- 172.21.15.42:0/4139003037 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25281a3580 con 0x7f2528102ac0 2024-04-20T10:24:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.783+0000 7f2524c00700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f250c039890 0x7f250c03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.783+0000 7f2524c00700 1 -- 172.21.15.42:0/4139003037 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f251804cd80 con 0x7f2528102ac0 2024-04-20T10:24:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.784+0000 7f251fe00700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f250c039890 0x7f250c03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.784+0000 7f251fe00700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f250c039890 0x7f250c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2510000c00 tx=0x7f2510005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:56.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:56.789+0000 7f2524c00700 1 -- 172.21.15.42:0/4139003037 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2518052020 con 0x7f2528102ac0 2024-04-20T10:24:56.899 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:56.672+0000 7f108ab00200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:24:57.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:57.039+0000 7f252e800700 1 -- 172.21.15.42:0/4139003037 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f25280763a0 con 0x7f250c039890 2024-04-20T10:24:57.226 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:56 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:56.875+0000 7f8e9c9aa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:24:57.226 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:57.050+0000 7f8e9c9aa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:24:57.226 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:57.226+0000 7f8e9c9aa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:24:57.480 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:57.197+0000 7f108ab00200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:24:57.480 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:57.289+0000 7f108ab00200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:24:57.760 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:57.532+0000 7f108ab00200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:24:57.760 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:57.664+0000 7f108ab00200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:24:57.760 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:57.759+0000 7f108ab00200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:24:57.760 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:57.760 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: Reconfiguring mgr.x (unknown last config time)... 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: Reconfiguring daemon mgr.x on smithi042 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:24:57.761 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:57.803 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:57.522+0000 7f8e9c9aa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:24:57.804 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:57.616+0000 7f8e9c9aa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:24:57.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:57.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:57.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:24:57.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: Reconfiguring mgr.x (unknown last config time)... 2024-04-20T10:24:57.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:24:57.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:24:57.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:57.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: Reconfiguring daemon mgr.x on smithi042 2024-04-20T10:24:57.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:24:57.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:24:57.807 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:24:57.807 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:57.807 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:57.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:57.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:24:57.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:57.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: Reconfiguring mgr.x (unknown last config time)... 2024-04-20T10:24:57.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T10:24:57.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T10:24:57.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:57.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: Reconfiguring daemon mgr.x on smithi042 2024-04-20T10:24:57.810 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:24:57.810 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:24:57.810 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:24:57.810 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:57 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:58.149 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:57.875+0000 7f108ab00200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:24:58.303 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:57.826+0000 7f8e9c9aa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:24:58.303 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:58 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:58.024+0000 7f8e9c9aa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:24:58.649 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:58 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:58.194+0000 7f108ab00200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:24:59.053 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:58 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:58.645+0000 7f8e9c9aa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:24:59.149 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:58 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:58.795+0000 7f108ab00200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:24:59.149 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:24:59.015+0000 7f108ab00200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:24:59.386 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:59.079+0000 7f8e9c9aa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:24:59.387 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:59.179+0000 7f8e9c9aa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:24:59.387 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:59.386+0000 7f8e9c9aa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:24:59.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:24:59.953+0000 7f2524c00700 1 -- 172.21.15.42:0/4139003037 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f25180217c0 con 0x7f2528102ac0 2024-04-20T10:24:59.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:59 smithi042 ceph-mon[17078]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:24:59.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:59 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:59.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:59 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:59.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:59 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:24:59.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:59 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:24:59.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:59 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:24:59.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:59 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:24:59.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:59 smithi042 ceph-mon[17078]: Standby manager daemon z started 2024-04-20T10:24:59.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:59 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.162:0/1097811199' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T10:24:59.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:59 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.162:0/1097811199' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:24:59.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:59 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.162:0/1097811199' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T10:24:59.956 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:24:59 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.162:0/1097811199' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:25:00.273 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:24:59.928+0000 7f8e9c9aa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:25:00.273 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:25:00 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:25:00.023+0000 7f8e9c9aa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:25:00.273 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-mon[21574]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:00.273 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:00.273 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:00.273 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:00.273 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:00.274 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:00.274 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:00.274 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-mon[21574]: Standby manager daemon z started 2024-04-20T10:25:00.274 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.162:0/1097811199' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T10:25:00.274 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.162:0/1097811199' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:25:00.274 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.162:0/1097811199' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T10:25:00.274 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:24:59 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.162:0/1097811199' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:25:00.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-mon[21854]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:00.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:00.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:00.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:00.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:00.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:00.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:00.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-mon[21854]: Standby manager daemon z started 2024-04-20T10:25:00.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.162:0/1097811199' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T10:25:00.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.162:0/1097811199' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:25:00.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.162:0/1097811199' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T10:25:00.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:24:59 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.162:0/1097811199' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:25:00.553 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:25:00 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:25:00.272+0000 7f8e9c9aa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:25:00.553 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:25:00 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:25:00.409+0000 7f8e9c9aa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:25:00.553 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:25:00 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:25:00.506+0000 7f8e9c9aa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:25:00.935 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:25:00 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:25:00.624+0000 7f8e9c9aa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:25:00.935 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:25:00 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:25:00.934+0000 7f8e9c9aa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:25:01.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:00 smithi046 ceph-mon[21574]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-20T10:25:01.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:00 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T10:25:01.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:00 smithi042 ceph-mon[17078]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-20T10:25:01.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:00 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T10:25:01.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:00 smithi162 ceph-mon[21854]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-20T10:25:01.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:00 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T10:25:01.978 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:25:01.559+0000 7f8e9c9aa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:25:01.978 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:25:01.780+0000 7f8e9c9aa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:25:01.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:01.975+0000 7f2524c00700 1 -- 172.21.15.42:0/4139003037 <== mon.1 v2:172.21.15.162:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f2518016370 con 0x7f2528102ac0 2024-04-20T10:25:02.083 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:01 smithi042 ceph-mon[17078]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:02.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:01 smithi042 ceph-mon[17078]: Standby manager daemon y started 2024-04-20T10:25:02.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:01 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.46:0/816054101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T10:25:02.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:01 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.46:0/816054101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:25:02.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:01 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.46:0/816054101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T10:25:02.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:01 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.46:0/816054101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:25:02.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:01 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/514454951' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47367eda-0111-48e6-891e-e8c78dab4d43"}]: dispatch 2024-04-20T10:25:02.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:01 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47367eda-0111-48e6-891e-e8c78dab4d43"}]: dispatch 2024-04-20T10:25:02.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:01 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47367eda-0111-48e6-891e-e8c78dab4d43"}]': finished 2024-04-20T10:25:02.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:01 smithi042 ceph-mon[17078]: osdmap e5: 1 total, 0 up, 1 in 2024-04-20T10:25:02.084 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:01 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:02.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-mon[21574]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:02.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-mon[21574]: Standby manager daemon y started 2024-04-20T10:25:02.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.46:0/816054101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T10:25:02.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.46:0/816054101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:25:02.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.46:0/816054101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T10:25:02.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.46:0/816054101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:25:02.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/514454951' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47367eda-0111-48e6-891e-e8c78dab4d43"}]: dispatch 2024-04-20T10:25:02.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47367eda-0111-48e6-891e-e8c78dab4d43"}]: dispatch 2024-04-20T10:25:02.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47367eda-0111-48e6-891e-e8c78dab4d43"}]': finished 2024-04-20T10:25:02.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-mon[21574]: osdmap e5: 1 total, 0 up, 1 in 2024-04-20T10:25:02.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:01 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:02.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:01 smithi162 ceph-mon[21854]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:02.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:01 smithi162 ceph-mon[21854]: Standby manager daemon y started 2024-04-20T10:25:02.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:01 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.46:0/816054101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T10:25:02.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:01 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.46:0/816054101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:25:02.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:01 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.46:0/816054101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T10:25:02.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:01 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.46:0/816054101' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:25:02.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:01 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/514454951' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47367eda-0111-48e6-891e-e8c78dab4d43"}]: dispatch 2024-04-20T10:25:02.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:01 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47367eda-0111-48e6-891e-e8c78dab4d43"}]: dispatch 2024-04-20T10:25:02.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:01 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47367eda-0111-48e6-891e-e8c78dab4d43"}]': finished 2024-04-20T10:25:02.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:01 smithi162 ceph-mon[21854]: osdmap e5: 1 total, 0 up, 1 in 2024-04-20T10:25:02.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:01 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:03.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:02 smithi046 ceph-mon[21574]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-20T10:25:03.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:02 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T10:25:03.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:02 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3707738796' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:25:03.320 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:02 smithi162 ceph-mon[21854]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-20T10:25:03.320 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:02 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T10:25:03.320 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:02 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3707738796' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:25:03.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:02 smithi042 ceph-mon[17078]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-20T10:25:03.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:02 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T10:25:03.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:02 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3707738796' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:25:04.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:03 smithi046 ceph-mon[21574]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:04.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:03 smithi042 ceph-mon[17078]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:04.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:03 smithi162 ceph-mon[21854]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:05 smithi046 ceph-mon[21574]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:06.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:05 smithi042 ceph-mon[17078]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:06.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:05 smithi162 ceph-mon[21854]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:07 smithi042 ceph-mon[17078]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:08.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:07 smithi046 ceph-mon[21574]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:08.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:07 smithi162 ceph-mon[21854]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:10 smithi046 ceph-mon[21574]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:10 smithi042 ceph-mon[17078]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:10.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:10 smithi162 ceph-mon[21854]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:11.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T10:25:11.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T10:25:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:11.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T10:25:11.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:12.263 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:12 smithi042 ceph-mon[17078]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:12.263 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:12 smithi042 ceph-mon[17078]: Deploying daemon osd.0 on smithi042 2024-04-20T10:25:12.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:12 smithi046 ceph-mon[21574]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:12.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:12 smithi046 ceph-mon[21574]: Deploying daemon osd.0 on smithi042 2024-04-20T10:25:12.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:12 smithi162 ceph-mon[21854]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:12.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:12 smithi162 ceph-mon[21854]: Deploying daemon osd.0 on smithi042 2024-04-20T10:25:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:14 smithi046 ceph-mon[21574]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:14.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:14 smithi042 ceph-mon[17078]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:14.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:14 smithi162 ceph-mon[21854]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:16 smithi042 ceph-mon[17078]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:16.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:16 smithi046 ceph-mon[21574]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:16.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:16 smithi162 ceph-mon[21854]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:18.285 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:18 smithi042 ceph-mon[17078]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:18.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:18 smithi046 ceph-mon[21574]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:18.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:18 smithi162 ceph-mon[21854]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:20 smithi042 ceph-mon[17078]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:20 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:20 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:20.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:20 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:20 smithi046 ceph-mon[21574]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:20 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:20 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:20.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:20 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:20.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:20 smithi162 ceph-mon[21854]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:20.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:20 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:20.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:20 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:20 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:22.160 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:22 smithi042 ceph-mon[17078]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:22.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:22 smithi046 ceph-mon[21574]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:22.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:22 smithi162 ceph-mon[21854]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:23.545 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:23 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:23.545 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:23 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:23.545 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:23 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:23.545 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:23 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:23.545 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:23 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:23.545 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:23 smithi042 ceph-mon[17078]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:23.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:23 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:23.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:23 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:23.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:23 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:23.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:23 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:23.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:23 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:23.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:23 smithi046 ceph-mon[21574]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:23.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:23 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:23.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:23 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:23.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:23 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:23.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:23 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:23.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:23 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:23.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:23 smithi162 ceph-mon[21854]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:24.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.432+0000 7f2524c00700 1 -- 172.21.15.42:0/4139003037 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f25280763a0 con 0x7f250c039890 2024-04-20T10:25:24.434 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 0 on host 'smithi042' 2024-04-20T10:25:24.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.436+0000 7f252e800700 1 -- 172.21.15.42:0/4139003037 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f250c039890 msgr2=0x7f250c03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:25:24.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.436+0000 7f252e800700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f250c039890 0x7f250c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2510000c00 tx=0x7f2510005fb0 comp rx=0 tx=0).stop 2024-04-20T10:25:24.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.436+0000 7f252e800700 1 -- 172.21.15.42:0/4139003037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2528102ac0 msgr2=0x7f252819af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:25:24.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.436+0000 7f252e800700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2528102ac0 0x7f252819af50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f251800e980 tx=0x7f251800ec90 comp rx=0 tx=0).stop 2024-04-20T10:25:24.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.436+0000 7f252e800700 1 -- 172.21.15.42:0/4139003037 shutdown_connections 2024-04-20T10:25:24.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.436+0000 7f252e800700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f250c039890 0x7f250c03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:24.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.436+0000 7f252e800700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2528104660 0x7f25281a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:24.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.436+0000 7f252e800700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2528102ac0 0x7f252819af50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:24.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.436+0000 7f252e800700 1 --2- 172.21.15.42:0/4139003037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2528103cc0 0x7f252819b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:24.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.436+0000 7f252e800700 1 -- 172.21.15.42:0/4139003037 >> 172.21.15.42:0/4139003037 conn(0x7f25280fe020 msgr2=0x7f2528074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:25:24.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.436+0000 7f252e800700 1 -- 172.21.15.42:0/4139003037 shutdown_connections 2024-04-20T10:25:24.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:25:24.437+0000 7f252e800700 1 -- 172.21.15.42:0/4139003037 wait complete. 2024-04-20T10:25:24.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:24 smithi046 ceph-mon[21574]: from='osd.0 [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T10:25:24.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:24 smithi046 ceph-mon[21574]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T10:25:24.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:24 smithi042 ceph-mon[17078]: from='osd.0 [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T10:25:24.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:24 smithi042 ceph-mon[17078]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T10:25:24.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:24 smithi162 ceph-mon[21854]: from='osd.0 [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T10:25:24.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:24 smithi162 ceph-mon[21854]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T10:25:25.131 DEBUG:teuthology.orchestra.run.smithi042:osd.0> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.0.service 2024-04-20T10:25:25.136 INFO:tasks.cephadm:Deploying osd.1 on smithi046 with /dev/vg_nvme/lv_4... 2024-04-20T10:25:25.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T10:25:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:25 smithi042 ceph-mon[17078]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T10:25:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:25 smithi042 ceph-mon[17078]: from='osd.0 [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:25:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:25 smithi042 ceph-mon[17078]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T10:25:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:25 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:25 smithi042 ceph-mon[17078]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:25:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:25 smithi042 ceph-mon[17078]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:25.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:25 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:25.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:25 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:25.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:25 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:25.354 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:25:25 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0[32186]: 2024-04-20T10:25:25.316+0000 7f4958200700 -1 osd.0 0 waiting for initial osdmap 2024-04-20T10:25:25.354 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:25:25 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0[32186]: 2024-04-20T10:25:25.321+0000 7f4953000700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:25:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T10:25:25.512 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:25 smithi046 ceph-mon[21574]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T10:25:25.512 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:25 smithi046 ceph-mon[21574]: from='osd.0 [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:25:25.512 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:25 smithi046 ceph-mon[21574]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T10:25:25.512 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:25 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:25.513 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:25 smithi046 ceph-mon[21574]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:25:25.513 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:25 smithi046 ceph-mon[21574]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:25.513 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:25 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:25.513 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:25 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:25.513 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:25 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:25.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:25 smithi162 ceph-mon[21854]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T10:25:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:25 smithi162 ceph-mon[21854]: from='osd.0 [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:25:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:25 smithi162 ceph-mon[21854]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T10:25:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:25 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:25 smithi162 ceph-mon[21854]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:25:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:25 smithi162 ceph-mon[21854]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:25 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:25 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:25 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:26.566 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:26 smithi046 ceph-mon[21574]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-20T10:25:26.566 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:26 smithi046 ceph-mon[21574]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T10:25:26.567 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:26 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:26.567 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:26 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:26.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:26 smithi042 ceph-mon[17078]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-20T10:25:26.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:26 smithi042 ceph-mon[17078]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T10:25:26.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:26 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:26.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:26 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:26.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:26 smithi162 ceph-mon[21854]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-20T10:25:26.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:26 smithi162 ceph-mon[21854]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T10:25:26.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:26 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:26.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:26 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T10:25:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T10:25:27.220 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-04-20T10:25:27.220 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-04-20T10:25:27.220 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00872212 s, 1.2 GB/s 2024-04-20T10:25:27.220 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 2024-04-20T10:25:27.220 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-04-20T10:25:27.541 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:27 smithi046 ceph-mon[21574]: purged_snaps scrub starts 2024-04-20T10:25:27.541 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:27 smithi046 ceph-mon[21574]: purged_snaps scrub ok 2024-04-20T10:25:27.541 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:27 smithi046 ceph-mon[21574]: osd.0 [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493] boot 2024-04-20T10:25:27.541 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:27 smithi046 ceph-mon[21574]: osdmap e8: 1 total, 1 up, 1 in 2024-04-20T10:25:27.542 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:27 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:27.542 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:27 smithi046 ceph-mon[21574]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:27.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:27 smithi042 ceph-mon[17078]: purged_snaps scrub starts 2024-04-20T10:25:27.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:27 smithi042 ceph-mon[17078]: purged_snaps scrub ok 2024-04-20T10:25:27.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:27 smithi042 ceph-mon[17078]: osd.0 [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493] boot 2024-04-20T10:25:27.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:27 smithi042 ceph-mon[17078]: osdmap e8: 1 total, 1 up, 1 in 2024-04-20T10:25:27.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:27 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:27.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:27 smithi042 ceph-mon[17078]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:27.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:27 smithi162 ceph-mon[21854]: purged_snaps scrub starts 2024-04-20T10:25:27.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:27 smithi162 ceph-mon[21854]: purged_snaps scrub ok 2024-04-20T10:25:27.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:27 smithi162 ceph-mon[21854]: osd.0 [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493] boot 2024-04-20T10:25:27.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:27 smithi162 ceph-mon[21854]: osdmap e8: 1 total, 1 up, 1 in 2024-04-20T10:25:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:27 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:25:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:27 smithi162 ceph-mon[21854]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T10:25:27.820 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_4 2024-04-20T10:25:28.061 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T10:25:28.437 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:28 smithi046 ceph-mon[21574]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T10:25:28.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:28 smithi042 ceph-mon[17078]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T10:25:28.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:28 smithi162 ceph-mon[21854]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T10:25:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.399+0000 7f764fedf700 1 -- 172.21.15.46:0/2643421387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7648102bd0 msgr2=0x7f7648103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:25:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.399+0000 7f764fedf700 1 --2- 172.21.15.46:0/2643421387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7648102bd0 0x7f7648103030 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f76380099c0 tx=0x7f7638009cd0 comp rx=0 tx=0).stop 2024-04-20T10:25:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.400+0000 7f764fedf700 1 -- 172.21.15.46:0/2643421387 shutdown_connections 2024-04-20T10:25:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.400+0000 7f764fedf700 1 --2- 172.21.15.46:0/2643421387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648103570 0x7f7648109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.400+0000 7f764fedf700 1 --2- 172.21.15.46:0/2643421387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76481019d0 0x7f7648101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.400+0000 7f764fedf700 1 --2- 172.21.15.46:0/2643421387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7648102bd0 0x7f7648103030 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.400+0000 7f764fedf700 1 -- 172.21.15.46:0/2643421387 >> 172.21.15.46:0/2643421387 conn(0x7f76480fcf70 msgr2=0x7f76480ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:25:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.400+0000 7f764fedf700 1 -- 172.21.15.46:0/2643421387 shutdown_connections 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.400+0000 7f764fedf700 1 -- 172.21.15.46:0/2643421387 wait complete. 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.400+0000 7f764fedf700 1 Processor -- start 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.400+0000 7f764fedf700 1 -- start start 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.400+0000 7f764fedf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76481019d0 0x7f7648197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764fedf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7648102bd0 0x7f7648198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764fedf700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648103570 0x7f764819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764fedf700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f764810d680 con 0x7f7648102bd0 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764fedf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f764810d500 con 0x7f76481019d0 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764fedf700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f764810d800 con 0x7f7648103570 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764dc7b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76481019d0 0x7f7648197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764dc7b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76481019d0 0x7f7648197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45958/0 (socket says 172.21.15.46:45958) 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764dc7b700 1 -- 172.21.15.46:0/3290244232 learned_addr learned my addr 172.21.15.46:0/3290244232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764dc7b700 1 -- 172.21.15.46:0/3290244232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648103570 msgr2=0x7f764819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:25:29.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764e47c700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648103570 0x7f764819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:25:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764dc7b700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648103570 0x7f764819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764dc7b700 1 -- 172.21.15.46:0/3290244232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7648102bd0 msgr2=0x7f7648198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:25:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764dc7b700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7648102bd0 0x7f7648198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764dc7b700 1 -- 172.21.15.46:0/3290244232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764819f870 con 0x7f76481019d0 2024-04-20T10:25:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764e47c700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648103570 0x7f764819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:25:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764dc7b700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76481019d0 0x7f7648197c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f764400b500 tx=0x7f764400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:25:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f763effd700 1 -- 172.21.15.46:0/3290244232 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644015020 con 0x7f76481019d0 2024-04-20T10:25:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f763effd700 1 -- 172.21.15.46:0/3290244232 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7644011d10 con 0x7f76481019d0 2024-04-20T10:25:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764fedf700 1 -- 172.21.15.46:0/3290244232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764819fb60 con 0x7f76481019d0 2024-04-20T10:25:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f763effd700 1 -- 172.21.15.46:0/3290244232 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764400f2e0 con 0x7f76481019d0 2024-04-20T10:25:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.401+0000 7f764fedf700 1 -- 172.21.15.46:0/3290244232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481a00a0 con 0x7f76481019d0 2024-04-20T10:25:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.402+0000 7f764fedf700 1 -- 172.21.15.46:0/3290244232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7648069740 con 0x7f76481019d0 2024-04-20T10:25:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.404+0000 7f763effd700 1 -- 172.21.15.46:0/3290244232 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f7644018020 con 0x7f76481019d0 2024-04-20T10:25:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.404+0000 7f763effd700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f76340a3bb0 0x7f76340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:25:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.405+0000 7f763effd700 1 -- 172.21.15.46:0/3290244232 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f76440ce5c0 con 0x7f76481019d0 2024-04-20T10:25:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.405+0000 7f764d47a700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f76340a3bb0 0x7f76340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:25:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.406+0000 7f764d47a700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f76340a3bb0 0x7f76340a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7638009fd0 tx=0x7f7638000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:25:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.409+0000 7f763effd700 1 -- 172.21.15.46:0/3290244232 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f764401f020 con 0x7f76481019d0 2024-04-20T10:25:29.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:29 smithi046 ceph-mon[21574]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:29.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:29 smithi042 ceph-mon[17078]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:29.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:29.631+0000 7f764fedf700 1 -- 172.21.15.46:0/3290244232 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7648062e90 con 0x7f76340a3bb0 2024-04-20T10:25:29.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:29 smithi162 ceph-mon[21854]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:30.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:30 smithi046 ceph-mon[21574]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:25:30.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:25:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:25:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:25:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:30 smithi042 ceph-mon[17078]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:25:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:25:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:25:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:25:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:30 smithi162 ceph-mon[21854]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:25:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:25:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:25:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:25:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:31.520 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:31 smithi046 ceph-mon[21574]: Detected new or changed devices on smithi042 2024-04-20T10:25:31.520 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:31 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi042 to 17094M 2024-04-20T10:25:31.521 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:31 smithi046 ceph-mon[21574]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:31.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:31 smithi042 ceph-mon[17078]: Detected new or changed devices on smithi042 2024-04-20T10:25:31.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:31 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi042 to 17094M 2024-04-20T10:25:31.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:31 smithi042 ceph-mon[17078]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:31.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:31 smithi162 ceph-mon[21854]: Detected new or changed devices on smithi042 2024-04-20T10:25:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:31 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi042 to 17094M 2024-04-20T10:25:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:31 smithi162 ceph-mon[21854]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:32.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:32 smithi042 ceph-mon[17078]: from='client.? 172.21.15.46:0/164449619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4764a30-900f-4c2c-a5e4-99d346d82f17"}]: dispatch 2024-04-20T10:25:32.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:32 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4764a30-900f-4c2c-a5e4-99d346d82f17"}]: dispatch 2024-04-20T10:25:32.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:32 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4764a30-900f-4c2c-a5e4-99d346d82f17"}]': finished 2024-04-20T10:25:32.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:32 smithi042 ceph-mon[17078]: osdmap e10: 2 total, 1 up, 2 in 2024-04-20T10:25:32.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:32 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:32 smithi162 ceph-mon[21854]: from='client.? 172.21.15.46:0/164449619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4764a30-900f-4c2c-a5e4-99d346d82f17"}]: dispatch 2024-04-20T10:25:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:32 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4764a30-900f-4c2c-a5e4-99d346d82f17"}]: dispatch 2024-04-20T10:25:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:32 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4764a30-900f-4c2c-a5e4-99d346d82f17"}]': finished 2024-04-20T10:25:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:32 smithi162 ceph-mon[21854]: osdmap e10: 2 total, 1 up, 2 in 2024-04-20T10:25:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:32 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:32.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:32 smithi046 ceph-mon[21574]: from='client.? 172.21.15.46:0/164449619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4764a30-900f-4c2c-a5e4-99d346d82f17"}]: dispatch 2024-04-20T10:25:32.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:32 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4764a30-900f-4c2c-a5e4-99d346d82f17"}]: dispatch 2024-04-20T10:25:32.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:32 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4764a30-900f-4c2c-a5e4-99d346d82f17"}]': finished 2024-04-20T10:25:32.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:32 smithi046 ceph-mon[21574]: osdmap e10: 2 total, 1 up, 2 in 2024-04-20T10:25:32.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:32 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:33.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:33 smithi042 ceph-mon[17078]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:33.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:33 smithi042 ceph-mon[17078]: from='client.? 172.21.15.46:0/547978193' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:25:33.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:33 smithi162 ceph-mon[21854]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:33.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:33 smithi162 ceph-mon[21854]: from='client.? 172.21.15.46:0/547978193' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:25:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:33 smithi046 ceph-mon[21574]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:33 smithi046 ceph-mon[21574]: from='client.? 172.21.15.46:0/547978193' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:25:35.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:35 smithi046 ceph-mon[21574]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:35.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:35 smithi042 ceph-mon[17078]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:35 smithi162 ceph-mon[21854]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:37.742 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:37 smithi046 ceph-mon[21574]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:37.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:37 smithi042 ceph-mon[17078]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:37 smithi162 ceph-mon[21854]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:39.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:39 smithi046 ceph-mon[21574]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:39.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:39 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T10:25:39.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:39 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:39.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:39 smithi042 ceph-mon[17078]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:39.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:39 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T10:25:39.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:39 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:39 smithi162 ceph-mon[21854]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:39 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T10:25:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:39 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:40.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:40 smithi046 ceph-mon[21574]: Deploying daemon osd.1 on smithi046 2024-04-20T10:25:40.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:40 smithi042 ceph-mon[17078]: Deploying daemon osd.1 on smithi046 2024-04-20T10:25:40.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:40 smithi162 ceph-mon[21854]: Deploying daemon osd.1 on smithi046 2024-04-20T10:25:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:41 smithi046 ceph-mon[21574]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:41 smithi042 ceph-mon[17078]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:41 smithi162 ceph-mon[21854]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:43.664 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:43 smithi046 ceph-mon[21574]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:43.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:43 smithi042 ceph-mon[17078]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:43 smithi162 ceph-mon[21854]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:45.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:45 smithi046 ceph-mon[21574]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:45.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:45.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:45.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:45.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:45 smithi042 ceph-mon[17078]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:45.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:45.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:45.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:45 smithi162 ceph-mon[21854]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:47.565 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 1 on host 'smithi046' 2024-04-20T10:25:47.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.565+0000 7f763effd700 1 -- 172.21.15.46:0/3290244232 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7648062e90 con 0x7f76340a3bb0 2024-04-20T10:25:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.567+0000 7f764fedf700 1 -- 172.21.15.46:0/3290244232 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f76340a3bb0 msgr2=0x7f76340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:25:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.567+0000 7f764fedf700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f76340a3bb0 0x7f76340a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7638009fd0 tx=0x7f7638000bc0 comp rx=0 tx=0).stop 2024-04-20T10:25:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.567+0000 7f764fedf700 1 -- 172.21.15.46:0/3290244232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76481019d0 msgr2=0x7f7648197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:25:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.567+0000 7f764fedf700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76481019d0 0x7f7648197c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f764400b500 tx=0x7f764400b8c0 comp rx=0 tx=0).stop 2024-04-20T10:25:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.567+0000 7f764fedf700 1 -- 172.21.15.46:0/3290244232 shutdown_connections 2024-04-20T10:25:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.567+0000 7f764fedf700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f76340a3bb0 0x7f76340a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.567+0000 7f764fedf700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76481019d0 0x7f7648197c20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.567+0000 7f764fedf700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7648103570 0x7f764819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.567+0000 7f764fedf700 1 --2- 172.21.15.46:0/3290244232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7648102bd0 0x7f7648198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.567+0000 7f764fedf700 1 -- 172.21.15.46:0/3290244232 >> 172.21.15.46:0/3290244232 conn(0x7f76480fcf70 msgr2=0x7f7648105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:25:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.567+0000 7f764fedf700 1 -- 172.21.15.46:0/3290244232 shutdown_connections 2024-04-20T10:25:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:25:47.567+0000 7f764fedf700 1 -- 172.21.15.46:0/3290244232 wait complete. 2024-04-20T10:25:47.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:47 smithi046 ceph-mon[21574]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:47 smithi042 ceph-mon[17078]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:47.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:47 smithi162 ceph-mon[21854]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:48.256 DEBUG:teuthology.orchestra.run.smithi046:osd.1> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.1.service 2024-04-20T10:25:48.260 INFO:tasks.cephadm:Deploying osd.2 on smithi162 with /dev/vg_nvme/lv_4... 2024-04-20T10:25:48.260 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T10:25:48.504 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:25:48.550 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:25:48 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1[26197]: 2024-04-20T10:25:48.366+0000 7fa02ec297c0 -1 osd.1 0 log_to_monitors true 2024-04-20T10:25:48.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:48.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:48.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:48 smithi046 ceph-mon[21574]: from='osd.1 [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T10:25:48.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:48 smithi046 ceph-mon[21574]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T10:25:48.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:48.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:48.829 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:48.829 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:48.829 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:48 smithi162 ceph-mon[21854]: from='osd.1 [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T10:25:48.829 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:48 smithi162 ceph-mon[21854]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T10:25:48.829 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:48.829 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:48.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:48.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:48.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:48 smithi042 ceph-mon[17078]: from='osd.1 [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T10:25:48.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:48 smithi042 ceph-mon[17078]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T10:25:48.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:48.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:49.803 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:25:49 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1[26197]: 2024-04-20T10:25:49.570+0000 7fa02428e700 -1 osd.1 0 waiting for initial osdmap 2024-04-20T10:25:49.803 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:25:49 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1[26197]: 2024-04-20T10:25:49.574+0000 7fa020081700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:25:49.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:49 smithi046 ceph-mon[21574]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:49.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:49 smithi046 ceph-mon[21574]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T10:25:49.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:49 smithi046 ceph-mon[21574]: from='osd.1 [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:25:49.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:49 smithi046 ceph-mon[21574]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T10:25:49.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:49.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:49 smithi046 ceph-mon[21574]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:25:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:49 smithi042 ceph-mon[17078]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:49 smithi042 ceph-mon[17078]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T10:25:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:49 smithi042 ceph-mon[17078]: from='osd.1 [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:25:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:49 smithi042 ceph-mon[17078]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T10:25:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:49.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:49 smithi042 ceph-mon[17078]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:25:49.876 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:49 smithi162 ceph-mon[21854]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:49.922 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:49 smithi162 ceph-mon[21854]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T10:25:49.922 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:49 smithi162 ceph-mon[21854]: from='osd.1 [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:25:49.922 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:49 smithi162 ceph-mon[21854]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T10:25:49.922 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:49.922 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:49 smithi162 ceph-mon[21854]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:25:50.422 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T10:25:50.422 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T10:25:50.422 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-04-20T10:25:50.422 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-04-20T10:25:50.422 INFO:teuthology.orchestra.run.smithi162.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113953 s, 920 MB/s 2024-04-20T10:25:50.422 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-04-20T10:25:50.696 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:50 smithi162 ceph-mon[21854]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-20T10:25:50.697 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:50 smithi162 ceph-mon[21854]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T10:25:50.697 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:50 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:50.697 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:50 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:50.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:50 smithi042 ceph-mon[17078]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-20T10:25:50.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:50 smithi042 ceph-mon[17078]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T10:25:50.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:50 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:50.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:50 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:50.984 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_4 2024-04-20T10:25:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:50 smithi046 ceph-mon[21574]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-20T10:25:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:50 smithi046 ceph-mon[21574]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T10:25:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:50 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:50 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:51.223 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:25:51.792 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:51 smithi046 ceph-mon[21574]: purged_snaps scrub starts 2024-04-20T10:25:51.792 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:51 smithi046 ceph-mon[21574]: purged_snaps scrub ok 2024-04-20T10:25:51.792 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:51 smithi046 ceph-mon[21574]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:51.792 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:51 smithi046 ceph-mon[21574]: osd.1 [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025] boot 2024-04-20T10:25:51.792 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:51 smithi046 ceph-mon[21574]: osdmap e13: 2 total, 2 up, 2 in 2024-04-20T10:25:51.792 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:51 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:51 smithi042 ceph-mon[17078]: purged_snaps scrub starts 2024-04-20T10:25:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:51 smithi042 ceph-mon[17078]: purged_snaps scrub ok 2024-04-20T10:25:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:51 smithi042 ceph-mon[17078]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:51 smithi042 ceph-mon[17078]: osd.1 [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025] boot 2024-04-20T10:25:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:51 smithi042 ceph-mon[17078]: osdmap e13: 2 total, 2 up, 2 in 2024-04-20T10:25:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:51 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:51.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:51 smithi162 ceph-mon[21854]: purged_snaps scrub starts 2024-04-20T10:25:51.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:51 smithi162 ceph-mon[21854]: purged_snaps scrub ok 2024-04-20T10:25:51.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:51 smithi162 ceph-mon[21854]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-20T10:25:51.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:51 smithi162 ceph-mon[21854]: osd.1 [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025] boot 2024-04-20T10:25:51.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:51 smithi162 ceph-mon[21854]: osdmap e13: 2 total, 2 up, 2 in 2024-04-20T10:25:51.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:51 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:25:52.822 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3095922357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c88105f30 msgr2=0x7f8c88108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:25:52.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 --2- 172.21.15.162:0/3095922357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c88105f30 0x7f8c88108320 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009990 tx=0x7f8c74009ca0 comp rx=0 tx=0).stop 2024-04-20T10:25:52.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3095922357 shutdown_connections 2024-04-20T10:25:52.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 --2- 172.21.15.162:0/3095922357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c88103600 0x7f8c881059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:52.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 --2- 172.21.15.162:0/3095922357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c88108860 0x7f8c8810ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:52.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 --2- 172.21.15.162:0/3095922357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c88105f30 0x7f8c88108320 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:52.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3095922357 >> 172.21.15.162:0/3095922357 conn(0x7f8c880fcfb0 msgr2=0x7f8c880ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:25:52.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3095922357 shutdown_connections 2024-04-20T10:25:52.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3095922357 wait complete. 2024-04-20T10:25:52.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 Processor -- start 2024-04-20T10:25:52.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 -- start start 2024-04-20T10:25:52.823 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c88103600 0x7f8c88102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c88105f30 0x7f8c881015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c88108860 0x7f8c88101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8810e720 con 0x7f8c88105f30 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8810e5a0 con 0x7f8c88103600 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8ed8e700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8810e8a0 con 0x7f8c88108860 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.821+0000 7f8c8d32b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c88108860 0x7f8c88101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c8d32b700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c88108860 0x7f8c88101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39838/0 (socket says 172.21.15.162:39838) 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c8d32b700 1 -- 172.21.15.162:0/3308372601 learned_addr learned my addr 172.21.15.162:0/3308372601 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c8cb2a700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c88103600 0x7f8c88102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c8d32b700 1 -- 172.21.15.162:0/3308372601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c88103600 msgr2=0x7f8c88102f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c8d32b700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c88103600 0x7f8c88102f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c8d32b700 1 -- 172.21.15.162:0/3308372601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c88105f30 msgr2=0x7f8c881015b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c8d32b700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c88105f30 0x7f8c881015b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:25:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c8d32b700 1 -- 172.21.15.162:0/3308372601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c881023a0 con 0x7f8c88108860 2024-04-20T10:25:52.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c8d32b700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c88108860 0x7f8c88101af0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000b9e0 tx=0x7f8c8000bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:25:52.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c85ffb700 1 -- 172.21.15.162:0/3308372601 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80019070 con 0x7f8c88108860 2024-04-20T10:25:52.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c85ffb700 1 -- 172.21.15.162:0/3308372601 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8000ce10 con 0x7f8c88108860 2024-04-20T10:25:52.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3308372601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c881a2ec0 con 0x7f8c88108860 2024-04-20T10:25:52.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c85ffb700 1 -- 172.21.15.162:0/3308372601 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80012360 con 0x7f8c88108860 2024-04-20T10:25:52.825 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.822+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3308372601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c881a3390 con 0x7f8c88108860 2024-04-20T10:25:52.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.823+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3308372601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c8802df90 con 0x7f8c88108860 2024-04-20T10:25:52.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.824+0000 7f8c85ffb700 1 -- 172.21.15.162:0/3308372601 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f8c8001c070 con 0x7f8c88108860 2024-04-20T10:25:52.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.825+0000 7f8c85ffb700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8c780a3c30 0x7f8c780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:25:52.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.825+0000 7f8c85ffb700 1 -- 172.21.15.162:0/3308372601 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f8c800cddf0 con 0x7f8c88108860 2024-04-20T10:25:52.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.825+0000 7f8c8cb2a700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8c780a3c30 0x7f8c780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:25:52.826 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.826+0000 7f8c8cb2a700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8c780a3c30 0x7f8c780a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c005950 tx=0x7f8c7c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:25:52.829 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:52.830+0000 7f8c85ffb700 1 -- 172.21.15.162:0/3308372601 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8c8009c020 con 0x7f8c88108860 2024-04-20T10:25:52.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T10:25:52.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: Detected new or changed devices on smithi046 2024-04-20T10:25:52.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:52.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:52.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:25:52.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi046 to 17093M 2024-04-20T10:25:52.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:52.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:52.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:52.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:52.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:52.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:52.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:52.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:52 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:52.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T10:25:52.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: Detected new or changed devices on smithi046 2024-04-20T10:25:52.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:52.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:52.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:25:52.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi046 to 17093M 2024-04-20T10:25:52.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:52.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:52.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:52.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:52.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:52.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:52.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:52.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:52 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:53.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:25:53.041+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3308372601 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8c88062e90 con 0x7f8c780a3c30 2024-04-20T10:25:53.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T10:25:53.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: Detected new or changed devices on smithi046 2024-04-20T10:25:53.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:53.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:53.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:25:53.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi046 to 17093M 2024-04-20T10:25:53.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:53.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:53.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:53.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:53.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:25:53.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:53.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:25:53.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:52 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:25:53.705 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:53 smithi046 ceph-mon[21574]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:25:53.705 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:53 smithi046 ceph-mon[21574]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:25:53.705 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:25:53.705 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:25:53.705 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:53.861 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:53 smithi162 ceph-mon[21854]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:25:53.861 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:53 smithi162 ceph-mon[21854]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:25:53.861 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:25:53.861 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:25:53.861 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:54.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:53 smithi042 ceph-mon[17078]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:25:54.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:53 smithi042 ceph-mon[17078]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:25:54.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:25:54.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:25:54.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:25:55.892 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:55 smithi162 ceph-mon[21854]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:25:55.892 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:55 smithi162 ceph-mon[21854]: from='client.? 172.21.15.162:0/3284947600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f69aba92-96e7-45c8-a47a-114aafa47178"}]: dispatch 2024-04-20T10:25:55.892 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:55 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f69aba92-96e7-45c8-a47a-114aafa47178"}]: dispatch 2024-04-20T10:25:55.893 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:55 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f69aba92-96e7-45c8-a47a-114aafa47178"}]': finished 2024-04-20T10:25:55.893 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:55 smithi162 ceph-mon[21854]: osdmap e15: 3 total, 2 up, 3 in 2024-04-20T10:25:55.893 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:55 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:25:56.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:55 smithi046 ceph-mon[21574]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:25:56.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:55 smithi046 ceph-mon[21574]: from='client.? 172.21.15.162:0/3284947600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f69aba92-96e7-45c8-a47a-114aafa47178"}]: dispatch 2024-04-20T10:25:56.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:55 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f69aba92-96e7-45c8-a47a-114aafa47178"}]: dispatch 2024-04-20T10:25:56.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:55 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f69aba92-96e7-45c8-a47a-114aafa47178"}]': finished 2024-04-20T10:25:56.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:55 smithi046 ceph-mon[21574]: osdmap e15: 3 total, 2 up, 3 in 2024-04-20T10:25:56.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:55 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:25:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:55 smithi042 ceph-mon[17078]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:25:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:55 smithi042 ceph-mon[17078]: from='client.? 172.21.15.162:0/3284947600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f69aba92-96e7-45c8-a47a-114aafa47178"}]: dispatch 2024-04-20T10:25:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:55 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f69aba92-96e7-45c8-a47a-114aafa47178"}]: dispatch 2024-04-20T10:25:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:55 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f69aba92-96e7-45c8-a47a-114aafa47178"}]': finished 2024-04-20T10:25:56.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:55 smithi042 ceph-mon[17078]: osdmap e15: 3 total, 2 up, 3 in 2024-04-20T10:25:56.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:55 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:25:56.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:56 smithi162 ceph-mon[21854]: from='client.? 172.21.15.162:0/115256695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:25:57.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:56 smithi046 ceph-mon[21574]: from='client.? 172.21.15.162:0/115256695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:25:57.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:56 smithi042 ceph-mon[17078]: from='client.? 172.21.15.162:0/115256695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:25:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:57 smithi162 ceph-mon[21854]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:25:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:57 smithi046 ceph-mon[21574]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:25:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:57 smithi042 ceph-mon[17078]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:25:59.815 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:25:59 smithi162 ceph-mon[21854]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:00.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:25:59 smithi046 ceph-mon[21574]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:25:59 smithi042 ceph-mon[17078]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:01.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:01 smithi162 ceph-mon[21854]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:01.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:01 smithi042 ceph-mon[17078]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:02.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:01 smithi046 ceph-mon[21574]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:03.876 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:03 smithi162 ceph-mon[21854]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:03.876 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:03 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T10:26:03.876 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:03 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:03.876 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:03 smithi162 ceph-mon[21854]: Deploying daemon osd.2 on smithi162 2024-04-20T10:26:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:03 smithi046 ceph-mon[21574]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:03 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T10:26:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:03 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:03 smithi046 ceph-mon[21574]: Deploying daemon osd.2 on smithi162 2024-04-20T10:26:04.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:03 smithi042 ceph-mon[17078]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:04.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:03 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T10:26:04.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:03 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:04.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:03 smithi042 ceph-mon[17078]: Deploying daemon osd.2 on smithi162 2024-04-20T10:26:05.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:05 smithi162 ceph-mon[21854]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:05 smithi046 ceph-mon[21574]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:06.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:05 smithi042 ceph-mon[17078]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:07.758 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:07 smithi162 ceph-mon[21854]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:07 smithi046 ceph-mon[21574]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:07 smithi042 ceph-mon[17078]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:09.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:09 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:09.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:09 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:26:09.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:09 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:09.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:09 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:09.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:09 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:26:09.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:09 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:09.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:09 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:09.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:09 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:26:09.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:09 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:10 smithi046 ceph-mon[21574]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:10 smithi042 ceph-mon[17078]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:10.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:10 smithi162 ceph-mon[21854]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:11.731 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:11 smithi162 ceph-mon[21854]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:11.731 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:11.731 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:11.731 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:11.731 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:26:11.731 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:11 smithi046 ceph-mon[21574]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:26:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:11.827 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.827+0000 7f8c85ffb700 1 -- 172.21.15.162:0/3308372601 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8c88062e90 con 0x7f8c780a3c30 2024-04-20T10:26:11.827 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 2 on host 'smithi162' 2024-04-20T10:26:11.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.830+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3308372601 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8c780a3c30 msgr2=0x7f8c780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:26:11.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.830+0000 7f8c8ed8e700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8c780a3c30 0x7f8c780a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c005950 tx=0x7f8c7c00b410 comp rx=0 tx=0).stop 2024-04-20T10:26:11.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.830+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3308372601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c88108860 msgr2=0x7f8c88101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:26:11.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.830+0000 7f8c8ed8e700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c88108860 0x7f8c88101af0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000b9e0 tx=0x7f8c8000bcf0 comp rx=0 tx=0).stop 2024-04-20T10:26:11.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.830+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3308372601 shutdown_connections 2024-04-20T10:26:11.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.830+0000 7f8c8ed8e700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8c780a3c30 0x7f8c780a60f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:11.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.830+0000 7f8c8ed8e700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c88103600 0x7f8c88102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:11.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.830+0000 7f8c8ed8e700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c88108860 0x7f8c88101af0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:11.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.830+0000 7f8c8ed8e700 1 --2- 172.21.15.162:0/3308372601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c88105f30 0x7f8c881015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:11.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.830+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3308372601 >> 172.21.15.162:0/3308372601 conn(0x7f8c880fcfb0 msgr2=0x7f8c88103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:26:11.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.830+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3308372601 shutdown_connections 2024-04-20T10:26:11.830 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:26:11.830+0000 7f8c8ed8e700 1 -- 172.21.15.162:0/3308372601 wait complete. 2024-04-20T10:26:11.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:11 smithi042 ceph-mon[17078]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:11.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:11.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:11.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:11.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:26:11.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:12.528 DEBUG:teuthology.orchestra.run.smithi162:osd.2> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.2.service 2024-04-20T10:26:12.530 INFO:tasks.cephadm:Deploying osd.3 on smithi042 with /dev/vg_nvme/lv_3... 2024-04-20T10:26:12.531 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T10:26:13.000 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:26:13.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:12 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:13.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:12 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:13.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:12 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:26:13.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:12 smithi046 ceph-mon[21574]: from='osd.2 [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T10:26:13.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:12 smithi046 ceph-mon[21574]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T10:26:13.080 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:12 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:13.080 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:12 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:13.080 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:12 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:26:13.081 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:12 smithi162 ceph-mon[21854]: from='osd.2 [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T10:26:13.081 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:12 smithi162 ceph-mon[21854]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T10:26:13.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:12 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:13.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:12 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:13.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:12 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:26:13.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:12 smithi042 ceph-mon[17078]: from='osd.2 [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T10:26:13.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:12 smithi042 ceph-mon[17078]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T10:26:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:13 smithi046 ceph-mon[21574]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:13 smithi046 ceph-mon[21574]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T10:26:14.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:13 smithi046 ceph-mon[21574]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T10:26:14.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:13 smithi046 ceph-mon[21574]: from='osd.2 [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:26:14.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:14.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:13 smithi046 ceph-mon[21574]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:26:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:13 smithi162 ceph-mon[21854]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:13 smithi162 ceph-mon[21854]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T10:26:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:13 smithi162 ceph-mon[21854]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T10:26:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:13 smithi162 ceph-mon[21854]: from='osd.2 [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:26:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:14.069 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:13 smithi162 ceph-mon[21854]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:26:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:13 smithi042 ceph-mon[17078]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:13 smithi042 ceph-mon[17078]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T10:26:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:13 smithi042 ceph-mon[17078]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T10:26:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:13 smithi042 ceph-mon[17078]: from='osd.2 [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:26:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:13 smithi042 ceph-mon[17078]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:26:14.400 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:26:14 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2[27006]: 2024-04-20T10:26:14.227+0000 7fb9eccb2700 -1 osd.2 0 waiting for initial osdmap 2024-04-20T10:26:14.400 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:26:14 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2[27006]: 2024-04-20T10:26:14.231+0000 7fb9e4a9d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:26:14.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:14 smithi042 ceph-mon[17078]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-20T10:26:14.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:14 smithi042 ceph-mon[17078]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T10:26:14.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:14.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:14.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:14 smithi042 ceph-mon[17078]: from='osd.2 ' entity='osd.2' 2024-04-20T10:26:15.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:14 smithi046 ceph-mon[21574]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-20T10:26:15.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:14 smithi046 ceph-mon[21574]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T10:26:15.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:15.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:15.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:14 smithi046 ceph-mon[21574]: from='osd.2 ' entity='osd.2' 2024-04-20T10:26:15.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:14 smithi162 ceph-mon[21854]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-20T10:26:15.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:14 smithi162 ceph-mon[21854]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T10:26:15.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:15.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:15.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:14 smithi162 ceph-mon[21854]: from='osd.2 ' entity='osd.2' 2024-04-20T10:26:15.802 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: purged_snaps scrub starts 2024-04-20T10:26:15.802 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: purged_snaps scrub ok 2024-04-20T10:26:15.802 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:15.802 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:15.802 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: osd.2 [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810] boot 2024-04-20T10:26:15.802 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: osdmap e18: 3 total, 3 up, 3 in 2024-04-20T10:26:15.802 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:15.803 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: purged_snaps scrub starts 2024-04-20T10:26:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: purged_snaps scrub ok 2024-04-20T10:26:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: osd.2 [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810] boot 2024-04-20T10:26:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: osdmap e18: 3 total, 3 up, 3 in 2024-04-20T10:26:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:26:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:26:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: purged_snaps scrub starts 2024-04-20T10:26:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: purged_snaps scrub ok 2024-04-20T10:26:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-20T10:26:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: osd.2 [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810] boot 2024-04-20T10:26:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: osdmap e18: 3 total, 3 up, 3 in 2024-04-20T10:26:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:26:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:26:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:26:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:26:16.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:16.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:16.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:26:16.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T10:26:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T10:26:16.239 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-04-20T10:26:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-04-20T10:26:16.239 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0345657 s, 303 MB/s 2024-04-20T10:26:16.239 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-04-20T10:26:17.065 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:16 smithi042 ceph-mon[17078]: Detected new or changed devices on smithi162 2024-04-20T10:26:17.065 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:16 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi162 to 17088M 2024-04-20T10:26:17.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T10:26:17.144 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_3 2024-04-20T10:26:17.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:16 smithi162 ceph-mon[21854]: Detected new or changed devices on smithi162 2024-04-20T10:26:17.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:16 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi162 to 17088M 2024-04-20T10:26:17.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T10:26:17.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:16 smithi046 ceph-mon[21574]: Detected new or changed devices on smithi162 2024-04-20T10:26:17.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:16 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi162 to 17088M 2024-04-20T10:26:17.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T10:26:17.609 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:26:18.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:17 smithi042 ceph-mon[17078]: pgmap v91: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:18.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T10:26:18.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:17 smithi042 ceph-mon[17078]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T10:26:18.095 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T10:26:18.112 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:17 smithi162 ceph-mon[21854]: pgmap v91: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:18.112 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:17 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T10:26:18.113 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:17 smithi162 ceph-mon[21854]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T10:26:18.113 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:17 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T10:26:18.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:17 smithi046 ceph-mon[21574]: pgmap v91: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:18.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:17 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T10:26:18.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:17 smithi046 ceph-mon[21574]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T10:26:18.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:17 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T10:26:18.783 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36351]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T10:26:18.783 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36351]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T10:26:18.784 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36351]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T10:26:18.784 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36351]: pam_unix(sudo:session): session closed for user root 2024-04-20T10:26:18.784 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36354]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T10:26:18.784 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36354]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T10:26:18.784 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36354]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T10:26:18.784 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36354]: pam_unix(sudo:session): session closed for user root 2024-04-20T10:26:18.948 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30300]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T10:26:18.948 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30300]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T10:26:18.948 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30300]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T10:26:18.948 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30300]: pam_unix(sudo:session): session closed for user root 2024-04-20T10:26:18.948 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30303]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T10:26:18.948 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T10:26:18.948 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30303]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T10:26:18.948 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30303]: pam_unix(sudo:session): session closed for user root 2024-04-20T10:26:19.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36358]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T10:26:19.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36358]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T10:26:19.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36358]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T10:26:19.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36358]: pam_unix(sudo:session): session closed for user root 2024-04-20T10:26:19.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36361]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T10:26:19.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36361]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T10:26:19.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36361]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T10:26:19.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 sudo[36361]: pam_unix(sudo:session): session closed for user root 2024-04-20T10:26:19.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T10:26:19.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 ceph-mon[17078]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T10:26:19.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:19.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:19.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:19.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:18 smithi042 ceph-mon[17078]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T10:26:19.149 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30806]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T10:26:19.149 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30806]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T10:26:19.149 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30806]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T10:26:19.149 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30806]: pam_unix(sudo:session): session closed for user root 2024-04-20T10:26:19.149 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30809]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T10:26:19.150 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30809]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T10:26:19.150 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30809]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T10:26:19.150 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30809]: pam_unix(sudo:session): session closed for user root 2024-04-20T10:26:19.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30812]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T10:26:19.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30812]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T10:26:19.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30812]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T10:26:19.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30812]: pam_unix(sudo:session): session closed for user root 2024-04-20T10:26:19.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30815]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T10:26:19.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T10:26:19.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30815]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T10:26:19.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 sudo[30815]: pam_unix(sudo:session): session closed for user root 2024-04-20T10:26:19.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T10:26:19.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 ceph-mon[21854]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T10:26:19.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:19.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:19.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:19.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:18 smithi162 ceph-mon[21854]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T10:26:19.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T10:26:19.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 ceph-mon[21574]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T10:26:19.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:19.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:19.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:19.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 ceph-mon[21574]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T10:26:19.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30306]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T10:26:19.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30306]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T10:26:19.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30306]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T10:26:19.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30306]: pam_unix(sudo:session): session closed for user root 2024-04-20T10:26:19.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30309]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T10:26:19.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30309]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T10:26:19.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30309]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T10:26:19.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:18 smithi046 sudo[30309]: pam_unix(sudo:session): session closed for user root 2024-04-20T10:26:19.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.797+0000 7f2495600700 1 -- 172.21.15.42:0/3079804510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f249010ac90 msgr2=0x7f2490111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:26:19.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.797+0000 7f2495600700 1 --2- 172.21.15.42:0/3079804510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f249010ac90 0x7f2490111540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2488009fc0 tx=0x7f248800b330 comp rx=0 tx=0).stop 2024-04-20T10:26:19.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.798+0000 7f2495600700 1 -- 172.21.15.42:0/3079804510 shutdown_connections 2024-04-20T10:26:19.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.798+0000 7f2495600700 1 --2- 172.21.15.42:0/3079804510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f249010ac90 0x7f2490111540 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:19.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.798+0000 7f2495600700 1 --2- 172.21.15.42:0/3079804510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24901090f0 0x7f2490109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:19.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.798+0000 7f2495600700 1 --2- 172.21.15.42:0/3079804510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249010a2f0 0x7f249010a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:19.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.798+0000 7f2495600700 1 -- 172.21.15.42:0/3079804510 >> 172.21.15.42:0/3079804510 conn(0x7f24900780c0 msgr2=0x7f249007a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:26:19.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.798+0000 7f2495600700 1 -- 172.21.15.42:0/3079804510 shutdown_connections 2024-04-20T10:26:19.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.798+0000 7f2495600700 1 -- 172.21.15.42:0/3079804510 wait complete. 2024-04-20T10:26:19.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.799+0000 7f2495600700 1 Processor -- start 2024-04-20T10:26:19.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.799+0000 7f2495600700 1 -- start start 2024-04-20T10:26:19.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.799+0000 7f2495600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24901090f0 0x7f249019f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:26:19.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f2495600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249010a2f0 0x7f249019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:26:19.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f2495600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f249010ac90 0x7f24901a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:26:19.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f2495600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2490114cd0 con 0x7f24901090f0 2024-04-20T10:26:19.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f2495600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2490114b50 con 0x7f249010a2f0 2024-04-20T10:26:19.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f2495600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2490114e50 con 0x7f249010ac90 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f248ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24901090f0 0x7f249019f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f248ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24901090f0 0x7f249019f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49958/0 (socket says 172.21.15.42:49958) 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f248ea00700 1 -- 172.21.15.42:0/67482224 learned_addr learned my addr 172.21.15.42:0/67482224 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f248e000700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249010a2f0 0x7f249019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f248f400700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f249010ac90 0x7f24901a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f248ea00700 1 -- 172.21.15.42:0/67482224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f249010ac90 msgr2=0x7f24901a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f248ea00700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f249010ac90 0x7f24901a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f248ea00700 1 -- 172.21.15.42:0/67482224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249010a2f0 msgr2=0x7f249019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f248ea00700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249010a2f0 0x7f249019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f248ea00700 1 -- 172.21.15.42:0/67482224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24901a6e60 con 0x7f24901090f0 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.800+0000 7f248e000700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249010a2f0 0x7f249019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.801+0000 7f248ea00700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24901090f0 0x7f249019f200 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f248000ec20 tx=0x7f248000ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:26:19.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.801+0000 7f2487400700 1 -- 172.21.15.42:0/67482224 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248000cc90 con 0x7f24901090f0 2024-04-20T10:26:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.801+0000 7f2487400700 1 -- 172.21.15.42:0/67482224 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f248000ce30 con 0x7f24901090f0 2024-04-20T10:26:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.801+0000 7f2495600700 1 -- 172.21.15.42:0/67482224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24901a7150 con 0x7f24901090f0 2024-04-20T10:26:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.801+0000 7f2487400700 1 -- 172.21.15.42:0/67482224 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24800188f0 con 0x7f24901090f0 2024-04-20T10:26:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.801+0000 7f2495600700 1 -- 172.21.15.42:0/67482224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24901a7690 con 0x7f24901090f0 2024-04-20T10:26:19.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.803+0000 7f2495600700 1 -- 172.21.15.42:0/67482224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2490069740 con 0x7f24901090f0 2024-04-20T10:26:19.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.805+0000 7f2487400700 1 -- 172.21.15.42:0/67482224 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f248001f020 con 0x7f24901090f0 2024-04-20T10:26:19.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.806+0000 7f2487400700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f24740a3bb0 0x7f24740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:26:19.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.806+0000 7f2487400700 1 -- 172.21.15.42:0/67482224 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f24800cf5a0 con 0x7f24901090f0 2024-04-20T10:26:19.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.806+0000 7f248e000700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f24740a3bb0 0x7f24740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:26:19.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.807+0000 7f248e000700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f24740a3bb0 0x7f24740a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f247c011490 tx=0x7f247c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:26:19.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:19.810+0000 7f2487400700 1 -- 172.21.15.42:0/67482224 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2480016330 con 0x7f24901090f0 2024-04-20T10:26:20.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:20.054+0000 7f2495600700 1 -- 172.21.15.42:0/67482224 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2490062e90 con 0x7f24740a3bb0 2024-04-20T10:26:20.054 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:20.054 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T10:26:20.054 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:26:20.055 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:19 smithi042 ceph-mon[17078]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T10:26:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T10:26:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:26:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:26:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:26:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T10:26:20.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:26:20.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:26:20.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:26:20.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T10:26:20.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T10:26:20.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T10:26:20.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:26:20.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:26:20.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:26:20.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:19 smithi046 ceph-mon[21574]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T10:26:20.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:26:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:19 smithi162 ceph-mon[21854]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T10:26:20.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:20.975+0000 7f2487400700 1 -- 172.21.15.42:0/67482224 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f24800993a0 con 0x7f24901090f0 2024-04-20T10:26:21.221 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:20 smithi042 ceph-mon[17078]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:26:21.222 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:20 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:26:21.222 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:20 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:26:21.222 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:20 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:21.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:20 smithi046 ceph-mon[21574]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:26:21.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:20 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:26:21.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:20 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:26:21.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:20 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:21.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:20 smithi162 ceph-mon[21854]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:26:21.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:20 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:26:21.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:20 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:26:21.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:20 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:22.191 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:21 smithi042 ceph-mon[17078]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:22.191 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:21 smithi042 ceph-mon[17078]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-20T10:26:22.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:21 smithi046 ceph-mon[21574]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:22.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:21 smithi046 ceph-mon[21574]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-20T10:26:22.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:21 smithi162 ceph-mon[21854]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:22.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:21 smithi162 ceph-mon[21854]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-20T10:26:24.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:23 smithi046 ceph-mon[21574]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:23 smithi042 ceph-mon[17078]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:24.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:23 smithi162 ceph-mon[21854]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:25.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:24 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/4040247514' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "620d380c-c93c-4b80-b7c2-f6db56a31e5e"}]: dispatch 2024-04-20T10:26:25.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:24 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "620d380c-c93c-4b80-b7c2-f6db56a31e5e"}]: dispatch 2024-04-20T10:26:25.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:24 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "620d380c-c93c-4b80-b7c2-f6db56a31e5e"}]': finished 2024-04-20T10:26:25.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:24 smithi046 ceph-mon[21574]: osdmap e22: 4 total, 3 up, 4 in 2024-04-20T10:26:25.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:24 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:24 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4040247514' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "620d380c-c93c-4b80-b7c2-f6db56a31e5e"}]: dispatch 2024-04-20T10:26:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:24 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "620d380c-c93c-4b80-b7c2-f6db56a31e5e"}]: dispatch 2024-04-20T10:26:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:24 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "620d380c-c93c-4b80-b7c2-f6db56a31e5e"}]': finished 2024-04-20T10:26:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:24 smithi042 ceph-mon[17078]: osdmap e22: 4 total, 3 up, 4 in 2024-04-20T10:26:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:24 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:25.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:24 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/4040247514' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "620d380c-c93c-4b80-b7c2-f6db56a31e5e"}]: dispatch 2024-04-20T10:26:25.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:24 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "620d380c-c93c-4b80-b7c2-f6db56a31e5e"}]: dispatch 2024-04-20T10:26:25.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:24 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "620d380c-c93c-4b80-b7c2-f6db56a31e5e"}]': finished 2024-04-20T10:26:25.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:24 smithi162 ceph-mon[21854]: osdmap e22: 4 total, 3 up, 4 in 2024-04-20T10:26:25.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:24 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:26.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:25 smithi046 ceph-mon[21574]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:26.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:25 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2292821005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:26:26.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:25 smithi042 ceph-mon[17078]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:26.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:25 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2292821005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:26:26.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:25 smithi162 ceph-mon[21854]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:26.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:25 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2292821005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:26:28.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:27 smithi046 ceph-mon[21574]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:28.342 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:27 smithi042 ceph-mon[17078]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:28.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:27 smithi162 ceph-mon[21854]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:30.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:29 smithi046 ceph-mon[21574]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:30.333 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:29 smithi042 ceph-mon[17078]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:30.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:29 smithi162 ceph-mon[21854]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:32.183 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:31 smithi042 ceph-mon[17078]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:32.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:31 smithi046 ceph-mon[21574]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:32.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:31 smithi162 ceph-mon[21854]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:34.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:33 smithi046 ceph-mon[21574]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:34.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:33 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T10:26:34.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:33 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:34.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:33 smithi042 ceph-mon[17078]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:34.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:33 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T10:26:34.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:33 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:34.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:33 smithi162 ceph-mon[21854]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:34.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:33 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T10:26:34.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:33 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:35.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:35 smithi046 ceph-mon[21574]: Deploying daemon osd.3 on smithi042 2024-04-20T10:26:35.341 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:35 smithi042 ceph-mon[17078]: Deploying daemon osd.3 on smithi042 2024-04-20T10:26:35.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:35 smithi162 ceph-mon[21854]: Deploying daemon osd.3 on smithi042 2024-04-20T10:26:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:36 smithi046 ceph-mon[21574]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:36.322 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:36 smithi042 ceph-mon[17078]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:36.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:36 smithi162 ceph-mon[21854]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:38.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:38 smithi046 ceph-mon[21574]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:38.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:38 smithi042 ceph-mon[17078]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:38.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:38 smithi162 ceph-mon[21854]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:40.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:40 smithi046 ceph-mon[21574]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:40.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:40 smithi042 ceph-mon[17078]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:40.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:40 smithi162 ceph-mon[21854]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:42.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:42 smithi046 ceph-mon[21574]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:42.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:42 smithi042 ceph-mon[17078]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:42.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:42 smithi162 ceph-mon[21854]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:43.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:43 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:43.397 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:43 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:43.397 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:43 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:26:43.397 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:43 smithi042 ceph-mon[17078]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:43.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:43 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:43 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:43 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:26:43.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:43 smithi046 ceph-mon[21574]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:43.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:43 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:43.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:43 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:43.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:43 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:26:43.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:43 smithi162 ceph-mon[21854]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:45.548 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:45 smithi042 ceph-mon[17078]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:45.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:45 smithi046 ceph-mon[21574]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:45 smithi162 ceph-mon[21854]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:47.036 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:46 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:47.036 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:46 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:47.036 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:46 smithi042 ceph-mon[17078]: from='osd.3 [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T10:26:47.036 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:46 smithi042 ceph-mon[17078]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T10:26:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.159+0000 7f2487400700 1 -- 172.21.15.42:0/67482224 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2490062e90 con 0x7f24740a3bb0 2024-04-20T10:26:47.160 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 3 on host 'smithi042' 2024-04-20T10:26:47.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.162+0000 7f2495600700 1 -- 172.21.15.42:0/67482224 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f24740a3bb0 msgr2=0x7f24740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:26:47.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.162+0000 7f2495600700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f24740a3bb0 0x7f24740a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f247c011490 tx=0x7f247c009260 comp rx=0 tx=0).stop 2024-04-20T10:26:47.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.162+0000 7f2495600700 1 -- 172.21.15.42:0/67482224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24901090f0 msgr2=0x7f249019f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:26:47.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.162+0000 7f2495600700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24901090f0 0x7f249019f200 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f248000ec20 tx=0x7f248000ef30 comp rx=0 tx=0).stop 2024-04-20T10:26:47.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.163+0000 7f2495600700 1 -- 172.21.15.42:0/67482224 shutdown_connections 2024-04-20T10:26:47.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.163+0000 7f2495600700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f24740a3bb0 0x7f24740a6070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:47.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.163+0000 7f2495600700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f249010ac90 0x7f24901a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:47.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.163+0000 7f2495600700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24901090f0 0x7f249019f200 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:47.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.163+0000 7f2495600700 1 --2- 172.21.15.42:0/67482224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f249010a2f0 0x7f249019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:47.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.163+0000 7f2495600700 1 -- 172.21.15.42:0/67482224 >> 172.21.15.42:0/67482224 conn(0x7f24900780c0 msgr2=0x7f249010d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:26:47.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.163+0000 7f2495600700 1 -- 172.21.15.42:0/67482224 shutdown_connections 2024-04-20T10:26:47.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:26:47.163+0000 7f2495600700 1 -- 172.21.15.42:0/67482224 wait complete. 2024-04-20T10:26:47.175 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:46 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:47.175 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:46 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:47.175 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:46 smithi162 ceph-mon[21854]: from='osd.3 [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T10:26:47.175 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:46 smithi162 ceph-mon[21854]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T10:26:47.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:46 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:47.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:46 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:47.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:46 smithi046 ceph-mon[21574]: from='osd.3 [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T10:26:47.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:46 smithi046 ceph-mon[21574]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T10:26:48.021 DEBUG:teuthology.orchestra.run.smithi042:osd.3> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.3.service 2024-04-20T10:26:48.026 INFO:tasks.cephadm:Deploying osd.4 on smithi046 with /dev/vg_nvme/lv_3... 2024-04-20T10:26:48.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T10:26:48.257 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:47 smithi042 ceph-mon[17078]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:48.257 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:47 smithi042 ceph-mon[17078]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T10:26:48.257 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:47 smithi042 ceph-mon[17078]: from='osd.3 [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:26:48.258 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:47 smithi042 ceph-mon[17078]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T10:26:48.258 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:47 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:48.258 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:47 smithi042 ceph-mon[17078]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:26:48.258 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:47 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:48.258 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:47 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:48.280 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T10:26:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:47 smithi046 ceph-mon[21574]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:47 smithi046 ceph-mon[21574]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T10:26:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:47 smithi046 ceph-mon[21574]: from='osd.3 [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:26:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:47 smithi046 ceph-mon[21574]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T10:26:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:47 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:47 smithi046 ceph-mon[21574]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:26:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:47 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:47 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:48.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:47 smithi162 ceph-mon[21854]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:48.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:47 smithi162 ceph-mon[21854]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T10:26:48.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:47 smithi162 ceph-mon[21854]: from='osd.3 [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:26:48.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:47 smithi162 ceph-mon[21854]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T10:26:48.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:47 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:48.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:47 smithi162 ceph-mon[21854]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:26:48.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:47 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:48.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:47 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:49.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:48 smithi046 ceph-mon[21574]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-20T10:26:49.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:48 smithi046 ceph-mon[21574]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T10:26:49.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:49.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:49.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:48 smithi046 ceph-mon[21574]: osd.3 [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467] boot 2024-04-20T10:26:49.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:48 smithi046 ceph-mon[21574]: osdmap e25: 4 total, 4 up, 4 in 2024-04-20T10:26:49.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:48 smithi042 ceph-mon[17078]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-20T10:26:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:48 smithi042 ceph-mon[17078]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T10:26:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:48 smithi042 ceph-mon[17078]: osd.3 [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467] boot 2024-04-20T10:26:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:48 smithi042 ceph-mon[17078]: osdmap e25: 4 total, 4 up, 4 in 2024-04-20T10:26:49.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:49.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:48 smithi162 ceph-mon[21854]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-20T10:26:49.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:48 smithi162 ceph-mon[21854]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T10:26:49.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:49.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:49.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:48 smithi162 ceph-mon[21854]: osd.3 [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467] boot 2024-04-20T10:26:49.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:48 smithi162 ceph-mon[21854]: osdmap e25: 4 total, 4 up, 4 in 2024-04-20T10:26:49.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:26:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T10:26:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T10:26:50.205 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-04-20T10:26:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-04-20T10:26:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00878937 s, 1.2 GB/s 2024-04-20T10:26:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-04-20T10:26:50.205 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:49 smithi046 ceph-mon[21574]: purged_snaps scrub starts 2024-04-20T10:26:50.205 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:49 smithi046 ceph-mon[21574]: purged_snaps scrub ok 2024-04-20T10:26:50.205 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:49 smithi046 ceph-mon[21574]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:50.205 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:49 smithi046 ceph-mon[21574]: osdmap e26: 4 total, 4 up, 4 in 2024-04-20T10:26:50.214 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:49 smithi042 ceph-mon[17078]: purged_snaps scrub starts 2024-04-20T10:26:50.214 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:49 smithi042 ceph-mon[17078]: purged_snaps scrub ok 2024-04-20T10:26:50.214 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:49 smithi042 ceph-mon[17078]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:50.215 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:49 smithi042 ceph-mon[17078]: osdmap e26: 4 total, 4 up, 4 in 2024-04-20T10:26:50.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:49 smithi162 ceph-mon[21854]: purged_snaps scrub starts 2024-04-20T10:26:50.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:49 smithi162 ceph-mon[21854]: purged_snaps scrub ok 2024-04-20T10:26:50.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:49 smithi162 ceph-mon[21854]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-20T10:26:50.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:49 smithi162 ceph-mon[21854]: osdmap e26: 4 total, 4 up, 4 in 2024-04-20T10:26:50.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_3 2024-04-20T10:26:51.230 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T10:26:51.960 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:51 smithi046 ceph-mon[21574]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:26:51.960 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:51 smithi046 ceph-mon[21574]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T10:26:51.980 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:51 smithi042 ceph-mon[17078]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:26:51.981 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:51 smithi042 ceph-mon[17078]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T10:26:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:51 smithi162 ceph-mon[21854]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:26:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:51 smithi162 ceph-mon[21854]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T10:26:52.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 -- 172.21.15.46:0/792073974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7888102bd0 msgr2=0x7f7888103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:26:52.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 --2- 172.21.15.46:0/792073974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7888102bd0 0x7f7888103030 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f78740099c0 tx=0x7f7874009cd0 comp rx=0 tx=0).stop 2024-04-20T10:26:52.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 -- 172.21.15.46:0/792073974 shutdown_connections 2024-04-20T10:26:52.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 --2- 172.21.15.46:0/792073974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7888103570 0x7f7888109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:52.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 --2- 172.21.15.46:0/792073974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78881019d0 0x7f7888101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 --2- 172.21.15.46:0/792073974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7888102bd0 0x7f7888103030 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 -- 172.21.15.46:0/792073974 >> 172.21.15.46:0/792073974 conn(0x7f78880fcf70 msgr2=0x7f78880ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 -- 172.21.15.46:0/792073974 shutdown_connections 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 -- 172.21.15.46:0/792073974 wait complete. 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 Processor -- start 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 -- start start 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78881019d0 0x7f7888105960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7888102bd0 0x7f7888103fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7888103570 0x7f78881044f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f788810d630 con 0x7f7888102bd0 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f788810d4b0 con 0x7f78881019d0 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.717+0000 7f788ee69700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f788810d7b0 con 0x7f7888103570 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f788cc05700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78881019d0 0x7f7888105960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f788cc05700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78881019d0 0x7f7888105960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46142/0 (socket says 172.21.15.46:46142) 2024-04-20T10:26:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f788cc05700 1 -- 172.21.15.46:0/3898455338 learned_addr learned my addr 172.21.15.46:0/3898455338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f788d406700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7888103570 0x7f78881044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f7887fff700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7888102bd0 0x7f7888103fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f788cc05700 1 -- 172.21.15.46:0/3898455338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7888103570 msgr2=0x7f78881044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f788cc05700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7888103570 0x7f78881044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f788cc05700 1 -- 172.21.15.46:0/3898455338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7888102bd0 msgr2=0x7f7888103fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f788cc05700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7888102bd0 0x7f7888103fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f788cc05700 1 -- 172.21.15.46:0/3898455338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7888104da0 con 0x7f78881019d0 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f788cc05700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78881019d0 0x7f7888105960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f787c00d980 tx=0x7f787c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f7885ffb700 1 -- 172.21.15.46:0/3898455338 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c017070 con 0x7f78881019d0 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f7885ffb700 1 -- 172.21.15.46:0/3898455338 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f787c009e10 con 0x7f78881019d0 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f788ee69700 1 -- 172.21.15.46:0/3898455338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78881a2ea0 con 0x7f78881019d0 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.718+0000 7f7885ffb700 1 -- 172.21.15.46:0/3898455338 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c00f620 con 0x7f78881019d0 2024-04-20T10:26:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.719+0000 7f788ee69700 1 -- 172.21.15.46:0/3898455338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78881a3320 con 0x7f78881019d0 2024-04-20T10:26:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.720+0000 7f788ee69700 1 -- 172.21.15.46:0/3898455338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7888069770 con 0x7f78881019d0 2024-04-20T10:26:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.721+0000 7f7885ffb700 1 -- 172.21.15.46:0/3898455338 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f787c009930 con 0x7f78881019d0 2024-04-20T10:26:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.721+0000 7f7885ffb700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f78780a3c40 0x7f78780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:26:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.721+0000 7f7885ffb700 1 -- 172.21.15.46:0/3898455338 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f787c0ced40 con 0x7f78881019d0 2024-04-20T10:26:52.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.722+0000 7f7887fff700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f78780a3c40 0x7f78780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:26:52.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.723+0000 7f7887fff700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f78780a3c40 0x7f78780a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7874000c00 tx=0x7f7874005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:26:52.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.726+0000 7f7885ffb700 1 -- 172.21.15.46:0/3898455338 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7888069770 con 0x7f78881019d0 2024-04-20T10:26:52.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:52 smithi046 ceph-mon[21574]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T10:26:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:26:52.948+0000 7f788ee69700 1 -- 172.21.15.46:0/3898455338 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f78880ff120 con 0x7f78780a3c40 2024-04-20T10:26:53.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:52 smithi042 ceph-mon[17078]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T10:26:53.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:52 smithi162 ceph-mon[21854]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T10:26:53.908 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: pgmap v119: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T10:26:53.908 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T10:26:53.908 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: Detected new or changed devices on smithi042 2024-04-20T10:26:53.908 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:53.908 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi042 to 8547M 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:26:53.909 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:53.910 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:26:53.910 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:54.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: pgmap v119: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T10:26:54.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T10:26:54.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: Detected new or changed devices on smithi042 2024-04-20T10:26:54.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:54.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:54.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:26:54.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi042 to 8547M 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:26:54.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:54.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: pgmap v119: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T10:26:54.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T10:26:54.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: Detected new or changed devices on smithi042 2024-04-20T10:26:54.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:54.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:54.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:26:54.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:26:54.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi042 to 8547M 2024-04-20T10:26:54.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:54.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:54.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:26:54.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:54.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:26:54.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:26:54.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:26:54.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:54.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:26:54.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:26:54.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:26:54.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:26:56.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:55 smithi046 ceph-mon[21574]: pgmap v121: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-20T10:26:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:55 smithi042 ceph-mon[17078]: pgmap v121: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-20T10:26:56.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:55 smithi162 ceph-mon[21854]: pgmap v121: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-20T10:26:57.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:56 smithi046 ceph-mon[21574]: from='client.? 172.21.15.46:0/2358860827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d67ed7c7-d9b2-4e22-ad08-633fa00b77f8"}]: dispatch 2024-04-20T10:26:57.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:56 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d67ed7c7-d9b2-4e22-ad08-633fa00b77f8"}]: dispatch 2024-04-20T10:26:57.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:56 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d67ed7c7-d9b2-4e22-ad08-633fa00b77f8"}]': finished 2024-04-20T10:26:57.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:56 smithi046 ceph-mon[21574]: osdmap e30: 5 total, 4 up, 5 in 2024-04-20T10:26:57.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:56 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:26:57.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:56 smithi046 ceph-mon[21574]: from='client.? 172.21.15.46:0/36449573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:26:57.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:56 smithi042 ceph-mon[17078]: from='client.? 172.21.15.46:0/2358860827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d67ed7c7-d9b2-4e22-ad08-633fa00b77f8"}]: dispatch 2024-04-20T10:26:57.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:56 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d67ed7c7-d9b2-4e22-ad08-633fa00b77f8"}]: dispatch 2024-04-20T10:26:57.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:56 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d67ed7c7-d9b2-4e22-ad08-633fa00b77f8"}]': finished 2024-04-20T10:26:57.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:56 smithi042 ceph-mon[17078]: osdmap e30: 5 total, 4 up, 5 in 2024-04-20T10:26:57.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:56 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:26:57.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:56 smithi042 ceph-mon[17078]: from='client.? 172.21.15.46:0/36449573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:26:57.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:56 smithi162 ceph-mon[21854]: from='client.? 172.21.15.46:0/2358860827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d67ed7c7-d9b2-4e22-ad08-633fa00b77f8"}]: dispatch 2024-04-20T10:26:57.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:56 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d67ed7c7-d9b2-4e22-ad08-633fa00b77f8"}]: dispatch 2024-04-20T10:26:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:56 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d67ed7c7-d9b2-4e22-ad08-633fa00b77f8"}]': finished 2024-04-20T10:26:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:56 smithi162 ceph-mon[21854]: osdmap e30: 5 total, 4 up, 5 in 2024-04-20T10:26:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:56 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:26:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:56 smithi162 ceph-mon[21854]: from='client.? 172.21.15.46:0/36449573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:26:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:57 smithi162 ceph-mon[21854]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-20T10:26:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:57 smithi046 ceph-mon[21574]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-20T10:26:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:57 smithi042 ceph-mon[17078]: pgmap v123: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-20T10:27:00.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:26:59 smithi046 ceph-mon[21574]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:26:59 smithi042 ceph-mon[17078]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:00.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:26:59 smithi162 ceph-mon[21854]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:02.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:01 smithi046 ceph-mon[21574]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:01 smithi042 ceph-mon[17078]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:01 smithi162 ceph-mon[21854]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:03.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:03 smithi042 ceph-mon[17078]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:03.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:03 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T10:27:03.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:03 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:04.103 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:03 smithi046 ceph-mon[21574]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:04.103 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:03 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T10:27:04.103 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:03 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:03 smithi162 ceph-mon[21854]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:03 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T10:27:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:03 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:05.065 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:04 smithi046 ceph-mon[21574]: Deploying daemon osd.4 on smithi046 2024-04-20T10:27:05.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:04 smithi042 ceph-mon[17078]: Deploying daemon osd.4 on smithi046 2024-04-20T10:27:05.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:04 smithi162 ceph-mon[21854]: Deploying daemon osd.4 on smithi046 2024-04-20T10:27:06.080 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:05 smithi046 ceph-mon[21574]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:06.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:05 smithi042 ceph-mon[17078]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:05 smithi162 ceph-mon[21854]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:08.080 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:07 smithi046 ceph-mon[21574]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:07 smithi042 ceph-mon[17078]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:07 smithi162 ceph-mon[21854]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:09.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:09 smithi162 ceph-mon[21854]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:09.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:09 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:09.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:09 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:27:09.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:09 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:09 smithi046 ceph-mon[21574]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:09 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:10.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:09 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:27:10.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:09 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:09 smithi042 ceph-mon[17078]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:09 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:09 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:27:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:09 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:11 smithi046 ceph-mon[21574]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:27:11.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:11 smithi042 ceph-mon[17078]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:27:12.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:11 smithi162 ceph-mon[21854]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:12.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:12.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:27:12.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.529+0000 7f7885ffb700 1 -- 172.21.15.46:0/3898455338 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f78880ff120 con 0x7f78780a3c40 2024-04-20T10:27:12.531 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 4 on host 'smithi046' 2024-04-20T10:27:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.531+0000 7f788ee69700 1 -- 172.21.15.46:0/3898455338 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f78780a3c40 msgr2=0x7f78780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.532+0000 7f788ee69700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f78780a3c40 0x7f78780a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7874000c00 tx=0x7f7874005fd0 comp rx=0 tx=0).stop 2024-04-20T10:27:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.532+0000 7f788ee69700 1 -- 172.21.15.46:0/3898455338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78881019d0 msgr2=0x7f7888105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.532+0000 7f788ee69700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78881019d0 0x7f7888105960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f787c00d980 tx=0x7f787c00dc90 comp rx=0 tx=0).stop 2024-04-20T10:27:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.532+0000 7f788ee69700 1 -- 172.21.15.46:0/3898455338 shutdown_connections 2024-04-20T10:27:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.532+0000 7f788ee69700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f78780a3c40 0x7f78780a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.532+0000 7f788ee69700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78881019d0 0x7f7888105960 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.532+0000 7f788ee69700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7888103570 0x7f78881044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.532+0000 7f788ee69700 1 --2- 172.21.15.46:0/3898455338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7888102bd0 0x7f7888103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.532+0000 7f788ee69700 1 -- 172.21.15.46:0/3898455338 >> 172.21.15.46:0/3898455338 conn(0x7f78880fcf70 msgr2=0x7f78880fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.532+0000 7f788ee69700 1 -- 172.21.15.46:0/3898455338 shutdown_connections 2024-04-20T10:27:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:27:12.532+0000 7f788ee69700 1 -- 172.21.15.46:0/3898455338 wait complete. 2024-04-20T10:27:13.146 DEBUG:teuthology.orchestra.run.smithi046:osd.4> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.4.service 2024-04-20T10:27:13.148 INFO:tasks.cephadm:Deploying osd.5 on smithi162 with /dev/vg_nvme/lv_3... 2024-04-20T10:27:13.149 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T10:27:13.398 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:27:13.764 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:13 smithi162 ceph-mon[21854]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:13.764 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:13.764 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:27:13.764 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:13.764 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:13 smithi162 ceph-mon[21854]: from='osd.4 [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T10:27:13.764 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:13 smithi162 ceph-mon[21854]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T10:27:13.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:13 smithi046 ceph-mon[21574]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:13.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:13.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:27:13.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:13.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:13 smithi046 ceph-mon[21574]: from='osd.4 [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T10:27:13.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:13 smithi046 ceph-mon[21574]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T10:27:13.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:13 smithi042 ceph-mon[17078]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:13.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:13.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:27:13.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:13.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:13 smithi042 ceph-mon[17078]: from='osd.4 [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T10:27:13.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:13 smithi042 ceph-mon[17078]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T10:27:14.780 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:14 smithi162 ceph-mon[21854]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T10:27:14.780 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:14 smithi162 ceph-mon[21854]: from='osd.4 [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:27:14.780 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:14 smithi162 ceph-mon[21854]: osdmap e31: 5 total, 4 up, 5 in 2024-04-20T10:27:14.780 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:27:14.780 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:14 smithi162 ceph-mon[21854]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:27:14.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:14 smithi046 ceph-mon[21574]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T10:27:14.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:14 smithi046 ceph-mon[21574]: from='osd.4 [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:27:14.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:14 smithi046 ceph-mon[21574]: osdmap e31: 5 total, 4 up, 5 in 2024-04-20T10:27:14.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:27:14.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:14 smithi046 ceph-mon[21574]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:27:14.804 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:27:14 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4[33146]: 2024-04-20T10:27:14.505+0000 7f151c169700 -1 osd.4 0 waiting for initial osdmap 2024-04-20T10:27:14.804 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:27:14 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4[33146]: 2024-04-20T10:27:14.511+0000 7f1517f5c700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:27:14.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:14 smithi042 ceph-mon[17078]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T10:27:14.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:14 smithi042 ceph-mon[17078]: from='osd.4 [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:27:14.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:14 smithi042 ceph-mon[17078]: osdmap e31: 5 total, 4 up, 5 in 2024-04-20T10:27:14.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:27:14.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:14 smithi042 ceph-mon[17078]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:27:15.374 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T10:27:15.375 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T10:27:15.375 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-04-20T10:27:15.375 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-04-20T10:27:15.375 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112235 s, 934 MB/s 2024-04-20T10:27:15.375 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-04-20T10:27:15.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:15 smithi046 ceph-mon[21574]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:15.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:15 smithi046 ceph-mon[21574]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-20T10:27:15.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:15 smithi046 ceph-mon[21574]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T10:27:15.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:27:15.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:27:15.708 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:15 smithi162 ceph-mon[21854]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:15.708 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:15 smithi162 ceph-mon[21854]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-20T10:27:15.708 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:15 smithi162 ceph-mon[21854]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T10:27:15.708 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:27:15.708 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:27:15.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:15 smithi042 ceph-mon[17078]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:15.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:15 smithi042 ceph-mon[17078]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-20T10:27:15.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:15 smithi042 ceph-mon[17078]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T10:27:15.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:27:15.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:27:15.987 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_3 2024-04-20T10:27:16.229 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:27:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: purged_snaps scrub starts 2024-04-20T10:27:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: purged_snaps scrub ok 2024-04-20T10:27:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: osd.4 [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758] boot 2024-04-20T10:27:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: osdmap e33: 5 total, 5 up, 5 in 2024-04-20T10:27:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:27:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: Detected new or changed devices on smithi046 2024-04-20T10:27:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:27:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:27:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi046 to 8546M 2024-04-20T10:27:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:16.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:16.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:27:16.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:16.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: purged_snaps scrub starts 2024-04-20T10:27:16.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: purged_snaps scrub ok 2024-04-20T10:27:16.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: osd.4 [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758] boot 2024-04-20T10:27:16.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: osdmap e33: 5 total, 5 up, 5 in 2024-04-20T10:27:16.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:27:16.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: Detected new or changed devices on smithi046 2024-04-20T10:27:16.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:16.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:16.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:27:16.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:27:16.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi046 to 8546M 2024-04-20T10:27:16.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:16.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:16.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:27:16.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:16.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: purged_snaps scrub starts 2024-04-20T10:27:16.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: purged_snaps scrub ok 2024-04-20T10:27:16.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: osd.4 [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758] boot 2024-04-20T10:27:16.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: osdmap e33: 5 total, 5 up, 5 in 2024-04-20T10:27:16.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:27:16.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: Detected new or changed devices on smithi046 2024-04-20T10:27:16.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:16.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:16.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:27:16.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:27:16.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi046 to 8546M 2024-04-20T10:27:16.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:16.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:16.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:27:16.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:17.593 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:17 smithi162 ceph-mon[21854]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:17.593 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:17 smithi162 ceph-mon[21854]: osdmap e34: 5 total, 5 up, 5 in 2024-04-20T10:27:17.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.796+0000 7f907908c700 1 -- 172.21.15.162:0/614895874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90741019c0 msgr2=0x7f9074101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:17.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.796+0000 7f907908c700 1 --2- 172.21.15.162:0/614895874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90741019c0 0x7f9074101de0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9064009990 tx=0x7f9064009ca0 comp rx=0 tx=0).stop 2024-04-20T10:27:17.798 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.796+0000 7f907908c700 1 -- 172.21.15.162:0/614895874 shutdown_connections 2024-04-20T10:27:17.798 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.796+0000 7f907908c700 1 --2- 172.21.15.162:0/614895874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9074103560 0x7f9074109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:17.798 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.796+0000 7f907908c700 1 --2- 172.21.15.162:0/614895874 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90741019c0 0x7f9074101de0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:17.798 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.796+0000 7f907908c700 1 --2- 172.21.15.162:0/614895874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9074102bc0 0x7f9074103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:17.798 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.796+0000 7f907908c700 1 -- 172.21.15.162:0/614895874 >> 172.21.15.162:0/614895874 conn(0x7f90740fcf80 msgr2=0x7f90740ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:17.798 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.796+0000 7f907908c700 1 -- 172.21.15.162:0/614895874 shutdown_connections 2024-04-20T10:27:17.798 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.796+0000 7f907908c700 1 -- 172.21.15.162:0/614895874 wait complete. 2024-04-20T10:27:17.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.797+0000 7f907908c700 1 Processor -- start 2024-04-20T10:27:17.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.797+0000 7f907908c700 1 -- start start 2024-04-20T10:27:17.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.797+0000 7f907908c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90741019c0 0x7f9074198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:17.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.797+0000 7f907908c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9074102bc0 0x7f9074199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:17.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.797+0000 7f907908c700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9074103560 0x7f90741a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.797+0000 7f907908c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f907410d600 con 0x7f90741019c0 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.797+0000 7f907908c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f907410d480 con 0x7f9074102bc0 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.797+0000 7f907908c700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f907410d780 con 0x7f9074103560 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f907359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9074103560 0x7f90741a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f907359e700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9074103560 0x7f90741a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:36382/0 (socket says 172.21.15.162:36382) 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f907359e700 1 -- 172.21.15.162:0/1349435718 learned_addr learned my addr 172.21.15.162:0/1349435718 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f907259c700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9074102bc0 0x7f9074199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f907359e700 1 -- 172.21.15.162:0/1349435718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9074102bc0 msgr2=0x7f9074199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f907359e700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9074102bc0 0x7f9074199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f907359e700 1 -- 172.21.15.162:0/1349435718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90741019c0 msgr2=0x7f9074198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f907359e700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90741019c0 0x7f9074198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f907359e700 1 -- 172.21.15.162:0/1349435718 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90741a0990 con 0x7f9074103560 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f907359e700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9074103560 0x7f90741a02f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f906c00b9a0 tx=0x7f906c00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:17.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f905bfff700 1 -- 172.21.15.162:0/1349435718 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c019070 con 0x7f9074103560 2024-04-20T10:27:17.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f907908c700 1 -- 172.21.15.162:0/1349435718 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90741a0c80 con 0x7f9074103560 2024-04-20T10:27:17.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f905bfff700 1 -- 172.21.15.162:0/1349435718 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c014440 con 0x7f9074103560 2024-04-20T10:27:17.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f905bfff700 1 -- 172.21.15.162:0/1349435718 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c0075d0 con 0x7f9074103560 2024-04-20T10:27:17.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.798+0000 7f907908c700 1 -- 172.21.15.162:0/1349435718 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90741a11c0 con 0x7f9074103560 2024-04-20T10:27:17.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.799+0000 7f907908c700 1 -- 172.21.15.162:0/1349435718 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9074069740 con 0x7f9074103560 2024-04-20T10:27:17.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.800+0000 7f905bfff700 1 -- 172.21.15.162:0/1349435718 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f906c01c070 con 0x7f9074103560 2024-04-20T10:27:17.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.801+0000 7f905bfff700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f905c0a3c40 0x7f905c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:17.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.801+0000 7f905bfff700 1 -- 172.21.15.162:0/1349435718 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f906c0cec70 con 0x7f9074103560 2024-04-20T10:27:17.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.802+0000 7f9072d9d700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f905c0a3c40 0x7f905c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:17.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:17 smithi046 ceph-mon[21574]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:17.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:17 smithi046 ceph-mon[21574]: osdmap e34: 5 total, 5 up, 5 in 2024-04-20T10:27:17.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.802+0000 7f9072d9d700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f905c0a3c40 0x7f905c0a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f90640039c0 tx=0x7f9064005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:17.806 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:17.805+0000 7f905bfff700 1 -- 172.21.15.162:0/1349435718 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f906c09c750 con 0x7f9074103560 2024-04-20T10:27:17.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:17 smithi042 ceph-mon[17078]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-20T10:27:17.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:17 smithi042 ceph-mon[17078]: osdmap e34: 5 total, 5 up, 5 in 2024-04-20T10:27:18.017 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:18.016+0000 7f907908c700 1 -- 172.21.15.162:0/1349435718 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f907406b8b0 con 0x7f905c0a3c40 2024-04-20T10:27:18.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:18 smithi046 ceph-mon[21574]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:27:18.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:18 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:27:18.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:18 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:27:18.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:18 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:18.809 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:18 smithi162 ceph-mon[21854]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:27:18.809 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:18 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:27:18.809 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:18 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:27:18.809 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:18 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:18.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:18 smithi042 ceph-mon[17078]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:27:18.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:18 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:27:18.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:18 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:27:18.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:18 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:19.635 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:19 smithi162 ceph-mon[21854]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:19.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:19 smithi046 ceph-mon[21574]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:19.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:19 smithi042 ceph-mon[17078]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:21.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:21 smithi042 ceph-mon[17078]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:21.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:21 smithi042 ceph-mon[17078]: from='client.? 172.21.15.162:0/2869479070' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe973de7-c11c-402d-89ce-138f33c0a706"}]: dispatch 2024-04-20T10:27:21.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:21 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe973de7-c11c-402d-89ce-138f33c0a706"}]: dispatch 2024-04-20T10:27:21.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:21 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe973de7-c11c-402d-89ce-138f33c0a706"}]': finished 2024-04-20T10:27:21.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:21 smithi042 ceph-mon[17078]: osdmap e35: 6 total, 5 up, 6 in 2024-04-20T10:27:21.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:21.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:21 smithi042 ceph-mon[17078]: from='client.? 172.21.15.162:0/3950158949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:27:21.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:21 smithi162 ceph-mon[21854]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:21.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:21 smithi162 ceph-mon[21854]: from='client.? 172.21.15.162:0/2869479070' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe973de7-c11c-402d-89ce-138f33c0a706"}]: dispatch 2024-04-20T10:27:21.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:21 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe973de7-c11c-402d-89ce-138f33c0a706"}]: dispatch 2024-04-20T10:27:21.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:21 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe973de7-c11c-402d-89ce-138f33c0a706"}]': finished 2024-04-20T10:27:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:21 smithi162 ceph-mon[21854]: osdmap e35: 6 total, 5 up, 6 in 2024-04-20T10:27:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:21 smithi162 ceph-mon[21854]: from='client.? 172.21.15.162:0/3950158949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:27:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:21 smithi046 ceph-mon[21574]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:21 smithi046 ceph-mon[21574]: from='client.? 172.21.15.162:0/2869479070' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe973de7-c11c-402d-89ce-138f33c0a706"}]: dispatch 2024-04-20T10:27:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:21 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe973de7-c11c-402d-89ce-138f33c0a706"}]: dispatch 2024-04-20T10:27:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:21 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe973de7-c11c-402d-89ce-138f33c0a706"}]': finished 2024-04-20T10:27:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:21 smithi046 ceph-mon[21574]: osdmap e35: 6 total, 5 up, 6 in 2024-04-20T10:27:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:21 smithi046 ceph-mon[21574]: from='client.? 172.21.15.162:0/3950158949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:27:23.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:23 smithi042 ceph-mon[17078]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:23.855 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:23 smithi046 ceph-mon[21574]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:23.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:23 smithi162 ceph-mon[21854]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:25.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:25 smithi042 ceph-mon[17078]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:25.854 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:25 smithi162 ceph-mon[21854]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:26.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:25 smithi046 ceph-mon[21574]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:27.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:27 smithi042 ceph-mon[17078]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:27.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:27 smithi162 ceph-mon[21854]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:28.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:27 smithi046 ceph-mon[21574]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:28.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:28 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T10:27:28.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:28 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:28.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:28 smithi042 ceph-mon[17078]: Deploying daemon osd.5 on smithi162 2024-04-20T10:27:28.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:28 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T10:27:28.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:28 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:28.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:28 smithi162 ceph-mon[21854]: Deploying daemon osd.5 on smithi162 2024-04-20T10:27:29.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:28 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T10:27:29.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:28 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:29.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:28 smithi046 ceph-mon[21574]: Deploying daemon osd.5 on smithi162 2024-04-20T10:27:29.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:29 smithi162 ceph-mon[21854]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:29.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:29 smithi042 ceph-mon[17078]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:30.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:29 smithi046 ceph-mon[21574]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:31.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:31 smithi042 ceph-mon[17078]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:31.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:31 smithi162 ceph-mon[21854]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:32.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:31 smithi046 ceph-mon[21574]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:33.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:33 smithi162 ceph-mon[21854]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:33.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:33 smithi042 ceph-mon[17078]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:34.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:33 smithi046 ceph-mon[21574]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:34.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:34.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:34.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:27:35.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:35.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:35.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:27:35.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:35.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:35.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:27:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:35 smithi162 ceph-mon[21854]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:35 smithi046 ceph-mon[21574]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:36.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:35 smithi042 ceph-mon[17078]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:37.455 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.455+0000 7f905bfff700 1 -- 172.21.15.162:0/1349435718 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f907406b8b0 con 0x7f905c0a3c40 2024-04-20T10:27:37.456 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 5 on host 'smithi162' 2024-04-20T10:27:37.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.458+0000 7f907908c700 1 -- 172.21.15.162:0/1349435718 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f905c0a3c40 msgr2=0x7f905c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:37.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.458+0000 7f907908c700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f905c0a3c40 0x7f905c0a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f90640039c0 tx=0x7f9064005fb0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.458+0000 7f907908c700 1 -- 172.21.15.162:0/1349435718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9074103560 msgr2=0x7f90741a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:37.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.458+0000 7f907908c700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9074103560 0x7f90741a02f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f906c00b9a0 tx=0x7f906c00bd60 comp rx=0 tx=0).stop 2024-04-20T10:27:37.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.458+0000 7f907908c700 1 -- 172.21.15.162:0/1349435718 shutdown_connections 2024-04-20T10:27:37.458 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.458+0000 7f907908c700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f905c0a3c40 0x7f905c0a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.458+0000 7f907908c700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9074103560 0x7f90741a02f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.458+0000 7f907908c700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90741019c0 0x7f9074198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.458+0000 7f907908c700 1 --2- 172.21.15.162:0/1349435718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9074102bc0 0x7f9074199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.458+0000 7f907908c700 1 -- 172.21.15.162:0/1349435718 >> 172.21.15.162:0/1349435718 conn(0x7f90740fcf80 msgr2=0x7f907406b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:37.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.458+0000 7f907908c700 1 -- 172.21.15.162:0/1349435718 shutdown_connections 2024-04-20T10:27:37.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:27:37.458+0000 7f907908c700 1 -- 172.21.15.162:0/1349435718 wait complete. 2024-04-20T10:27:37.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:37 smithi162 ceph-mon[21854]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:37.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:37.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:27:37.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:27:37.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:37 smithi046 ceph-mon[21574]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:37.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:37.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:27:37.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:27:37.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:37 smithi042 ceph-mon[17078]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:37.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:37.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:27:37.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:37.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:27:38.113 DEBUG:teuthology.orchestra.run.smithi162:osd.5> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.5.service 2024-04-20T10:27:38.116 INFO:tasks.cephadm:Deploying osd.6 on smithi042 with /dev/vg_nvme/lv_2... 2024-04-20T10:27:38.116 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T10:27:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:27:38.745 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:38 smithi042 ceph-mon[17078]: from='osd.5 [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T10:27:38.745 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:38 smithi042 ceph-mon[17078]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T10:27:38.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:38 smithi046 ceph-mon[21574]: from='osd.5 [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T10:27:38.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:38 smithi046 ceph-mon[21574]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T10:27:38.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:38 smithi162 ceph-mon[21854]: from='osd.5 [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T10:27:38.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:38 smithi162 ceph-mon[21854]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T10:27:39.803 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:39 smithi042 ceph-mon[17078]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:39.803 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:39 smithi042 ceph-mon[17078]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T10:27:39.803 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:39 smithi042 ceph-mon[17078]: osdmap e36: 6 total, 5 up, 6 in 2024-04-20T10:27:39.803 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:39 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:39.803 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:39 smithi042 ceph-mon[17078]: from='osd.5 [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:27:39.803 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:39 smithi042 ceph-mon[17078]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:27:39.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:39 smithi046 ceph-mon[21574]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:39.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:39 smithi046 ceph-mon[21574]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T10:27:39.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:39 smithi046 ceph-mon[21574]: osdmap e36: 6 total, 5 up, 6 in 2024-04-20T10:27:39.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:39 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:39.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:39 smithi046 ceph-mon[21574]: from='osd.5 [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:27:39.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:39 smithi046 ceph-mon[21574]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:27:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:39 smithi162 ceph-mon[21854]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:39.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:39 smithi162 ceph-mon[21854]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T10:27:39.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:39 smithi162 ceph-mon[21854]: osdmap e36: 6 total, 5 up, 6 in 2024-04-20T10:27:39.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:39 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:39.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:39 smithi162 ceph-mon[21854]: from='osd.5 [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:27:39.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:39 smithi162 ceph-mon[21854]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:27:40.400 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:27:39 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:27:39.943+0000 7f7da24e2700 -1 osd.5 0 waiting for initial osdmap 2024-04-20T10:27:40.400 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:27:39 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:27:39.952+0000 7f7d9a2cd700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:27:40.709 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:40 smithi042 ceph-mon[17078]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-20T10:27:40.710 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:40 smithi042 ceph-mon[17078]: osdmap e37: 6 total, 5 up, 6 in 2024-04-20T10:27:40.710 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:40.710 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:40 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:40.710 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:40 smithi042 ceph-mon[17078]: from='osd.5 ' entity='osd.5' 2024-04-20T10:27:40.792 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:40 smithi162 ceph-mon[21854]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-20T10:27:40.792 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:40 smithi162 ceph-mon[21854]: osdmap e37: 6 total, 5 up, 6 in 2024-04-20T10:27:40.792 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:40 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:40.792 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:40 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:40.792 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:40 smithi162 ceph-mon[21854]: from='osd.5 ' entity='osd.5' 2024-04-20T10:27:40.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:40 smithi046 ceph-mon[21574]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-20T10:27:40.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:40 smithi046 ceph-mon[21574]: osdmap e37: 6 total, 5 up, 6 in 2024-04-20T10:27:40.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:40 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:40.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:40 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:40.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:40 smithi046 ceph-mon[21574]: from='osd.5 ' entity='osd.5' 2024-04-20T10:27:41.770 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T10:27:41.770 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T10:27:41.770 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-04-20T10:27:41.770 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-04-20T10:27:41.770 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0346134 s, 303 MB/s 2024-04-20T10:27:41.771 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 2024-04-20T10:27:41.771 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-04-20T10:27:41.771 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: purged_snaps scrub starts 2024-04-20T10:27:41.771 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: purged_snaps scrub ok 2024-04-20T10:27:41.771 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:41.771 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:41.771 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: osd.5 [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096] boot 2024-04-20T10:27:41.771 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: osdmap e38: 6 total, 6 up, 6 in 2024-04-20T10:27:41.771 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:41.771 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:41.771 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:41.771 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:27:41.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:27:41.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:41.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:41.772 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:27:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: purged_snaps scrub starts 2024-04-20T10:27:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: purged_snaps scrub ok 2024-04-20T10:27:41.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:41.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:41.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: osd.5 [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096] boot 2024-04-20T10:27:41.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: osdmap e38: 6 total, 6 up, 6 in 2024-04-20T10:27:41.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:41.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:41.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:41.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:27:41.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:27:41.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:41.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:41.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:27:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: purged_snaps scrub starts 2024-04-20T10:27:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: purged_snaps scrub ok 2024-04-20T10:27:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-20T10:27:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: osd.5 [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096] boot 2024-04-20T10:27:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: osdmap e38: 6 total, 6 up, 6 in 2024-04-20T10:27:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:27:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:27:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:27:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:41.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:27:42.639 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_2 2024-04-20T10:27:42.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:42 smithi042 ceph-mon[17078]: Detected new or changed devices on smithi162 2024-04-20T10:27:42.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:42 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi162 to 8544M 2024-04-20T10:27:42.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:42 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:42.775 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:42 smithi042 ceph-mon[17078]: osdmap e39: 6 total, 6 up, 6 in 2024-04-20T10:27:42.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:42 smithi046 ceph-mon[21574]: Detected new or changed devices on smithi162 2024-04-20T10:27:42.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:42 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi162 to 8544M 2024-04-20T10:27:42.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:42.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:42 smithi046 ceph-mon[21574]: osdmap e39: 6 total, 6 up, 6 in 2024-04-20T10:27:42.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:42 smithi162 ceph-mon[21854]: Detected new or changed devices on smithi162 2024-04-20T10:27:42.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:42 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi162 to 8544M 2024-04-20T10:27:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:42 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:27:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:42 smithi162 ceph-mon[21854]: osdmap e39: 6 total, 6 up, 6 in 2024-04-20T10:27:43.104 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:27:43.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:43 smithi046 ceph-mon[21574]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:27:43.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:43 smithi046 ceph-mon[21574]: osdmap e40: 6 total, 6 up, 6 in 2024-04-20T10:27:43.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:43 smithi042 ceph-mon[17078]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:27:43.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:43 smithi042 ceph-mon[17078]: osdmap e40: 6 total, 6 up, 6 in 2024-04-20T10:27:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:43 smithi162 ceph-mon[21854]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:27:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:43 smithi162 ceph-mon[21854]: osdmap e40: 6 total, 6 up, 6 in 2024-04-20T10:27:44.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:44 smithi042 ceph-mon[17078]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T10:27:44.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:44 smithi162 ceph-mon[21854]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T10:27:45.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:44 smithi046 ceph-mon[21574]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T10:27:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.525+0000 7f15cee00700 1 -- 172.21.15.42:0/4150046370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15c806a350 msgr2=0x7f15c81015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.525+0000 7f15cee00700 1 --2- 172.21.15.42:0/4150046370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15c806a350 0x7f15c81015d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009990 tx=0x7f15b4009ca0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.526+0000 7f15cee00700 1 -- 172.21.15.42:0/4150046370 shutdown_connections 2024-04-20T10:27:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.526+0000 7f15cee00700 1 --2- 172.21.15.42:0/4150046370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15c8069ea0 0x7f15c8101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.526+0000 7f15cee00700 1 --2- 172.21.15.42:0/4150046370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15c8102250 0x7f15c81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.526+0000 7f15cee00700 1 --2- 172.21.15.42:0/4150046370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15c806a350 0x7f15c81015d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.526+0000 7f15cee00700 1 -- 172.21.15.42:0/4150046370 >> 172.21.15.42:0/4150046370 conn(0x7f15c80fc5f0 msgr2=0x7f15c80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.526+0000 7f15cee00700 1 -- 172.21.15.42:0/4150046370 shutdown_connections 2024-04-20T10:27:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.526+0000 7f15cee00700 1 -- 172.21.15.42:0/4150046370 wait complete. 2024-04-20T10:27:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.527+0000 7f15cee00700 1 Processor -- start 2024-04-20T10:27:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.527+0000 7f15cee00700 1 -- start start 2024-04-20T10:27:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.528+0000 7f15cee00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15c8069ea0 0x7f15c81060f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.528+0000 7f15cee00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15c806a350 0x7f15c8104740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.528+0000 7f15cee00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15c8102250 0x7f15c8104c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.528+0000 7f15cee00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f15c810d4a0 con 0x7f15c8069ea0 2024-04-20T10:27:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.528+0000 7f15cee00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15c810d320 con 0x7f15c8102250 2024-04-20T10:27:45.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.528+0000 7f15cee00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f15c810d620 con 0x7f15c806a350 2024-04-20T10:27:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.528+0000 7f15cda00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15c8069ea0 0x7f15c81060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.528+0000 7f15ce400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15c8102250 0x7f15c8104c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.528+0000 7f15ce400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15c8102250 0x7f15c8104c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47692/0 (socket says 172.21.15.42:47692) 2024-04-20T10:27:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.528+0000 7f15ce400700 1 -- 172.21.15.42:0/2708754518 learned_addr learned my addr 172.21.15.42:0/2708754518 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:27:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.528+0000 7f15cd000700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15c806a350 0x7f15c8104740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.529+0000 7f15ce400700 1 -- 172.21.15.42:0/2708754518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15c806a350 msgr2=0x7f15c8104740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.529+0000 7f15ce400700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15c806a350 0x7f15c8104740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.529+0000 7f15ce400700 1 -- 172.21.15.42:0/2708754518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15c8069ea0 msgr2=0x7f15c81060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.529+0000 7f15ce400700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15c8069ea0 0x7f15c81060f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.529+0000 7f15ce400700 1 -- 172.21.15.42:0/2708754518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c8105530 con 0x7f15c8102250 2024-04-20T10:27:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.529+0000 7f15ce400700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15c8102250 0x7f15c8104c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f15c000e940 tx=0x7f15c000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.529+0000 7f15c6000700 1 -- 172.21.15.42:0/2708754518 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0004a30 con 0x7f15c8102250 2024-04-20T10:27:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.529+0000 7f15cee00700 1 -- 172.21.15.42:0/2708754518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c81a2d30 con 0x7f15c8102250 2024-04-20T10:27:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.529+0000 7f15c6000700 1 -- 172.21.15.42:0/2708754518 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c0005500 con 0x7f15c8102250 2024-04-20T10:27:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.530+0000 7f15c6000700 1 -- 172.21.15.42:0/2708754518 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c000ce20 con 0x7f15c8102250 2024-04-20T10:27:45.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.530+0000 7f15cee00700 1 -- 172.21.15.42:0/2708754518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c81a31e0 con 0x7f15c8102250 2024-04-20T10:27:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.531+0000 7f15cee00700 1 -- 172.21.15.42:0/2708754518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c8069740 con 0x7f15c8102250 2024-04-20T10:27:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.533+0000 7f15c6000700 1 -- 172.21.15.42:0/2708754518 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f15c0005020 con 0x7f15c8102250 2024-04-20T10:27:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.534+0000 7f15c6000700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f15b00a3c40 0x7f15b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.534+0000 7f15c6000700 1 -- 172.21.15.42:0/2708754518 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f15c00cf7a0 con 0x7f15c8102250 2024-04-20T10:27:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.534+0000 7f15cda00700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f15b00a3c40 0x7f15b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.535+0000 7f15cda00700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f15b00a3c40 0x7f15b00a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f15c81020b0 tx=0x7f15bc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.539+0000 7f15c6000700 1 -- 172.21.15.42:0/2708754518 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f15c009cfa0 con 0x7f15c8102250 2024-04-20T10:27:45.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:45 smithi042 ceph-mon[17078]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:27:45.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:45 smithi042 ceph-mon[17078]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T10:27:45.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:27:45.784+0000 7f15cee00700 1 -- 172.21.15.42:0/2708754518 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f15c8062e90 con 0x7f15b00a3c40 2024-04-20T10:27:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:45 smithi162 ceph-mon[21854]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:27:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:45 smithi162 ceph-mon[21854]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T10:27:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:45 smithi046 ceph-mon[21574]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:27:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:45 smithi046 ceph-mon[21574]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T10:27:46.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:46 smithi162 ceph-mon[21854]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:27:46.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:46 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:27:46.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:46 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:27:46.816 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:46 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:46.829 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:46 smithi042 ceph-mon[17078]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:27:46.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:46 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:27:46.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:46 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:27:46.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:46 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:47.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:46 smithi046 ceph-mon[21574]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:27:47.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:46 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:27:47.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:46 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:27:47.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:46 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:27:47.841 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:47 smithi042 ceph-mon[17078]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:27:47.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:47 smithi162 ceph-mon[21854]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:27:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:47 smithi046 ceph-mon[21574]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:27:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:49 smithi042 ceph-mon[17078]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T10:27:49.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:49 smithi162 ceph-mon[21854]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T10:27:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:49 smithi046 ceph-mon[21574]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T10:27:50.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:50 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/333265419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67420dab-1d28-4626-9f42-6774ce3b19af"}]: dispatch 2024-04-20T10:27:50.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:50 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67420dab-1d28-4626-9f42-6774ce3b19af"}]: dispatch 2024-04-20T10:27:50.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:50 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67420dab-1d28-4626-9f42-6774ce3b19af"}]': finished 2024-04-20T10:27:50.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:50 smithi042 ceph-mon[17078]: osdmap e43: 7 total, 6 up, 7 in 2024-04-20T10:27:50.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:50 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:27:50.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:50 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/333265419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67420dab-1d28-4626-9f42-6774ce3b19af"}]: dispatch 2024-04-20T10:27:50.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:50 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67420dab-1d28-4626-9f42-6774ce3b19af"}]: dispatch 2024-04-20T10:27:50.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:50 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67420dab-1d28-4626-9f42-6774ce3b19af"}]': finished 2024-04-20T10:27:50.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:50 smithi162 ceph-mon[21854]: osdmap e43: 7 total, 6 up, 7 in 2024-04-20T10:27:50.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:50 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:27:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:50 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/333265419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67420dab-1d28-4626-9f42-6774ce3b19af"}]: dispatch 2024-04-20T10:27:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:50 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67420dab-1d28-4626-9f42-6774ce3b19af"}]: dispatch 2024-04-20T10:27:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:50 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67420dab-1d28-4626-9f42-6774ce3b19af"}]': finished 2024-04-20T10:27:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:50 smithi046 ceph-mon[21574]: osdmap e43: 7 total, 6 up, 7 in 2024-04-20T10:27:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:50 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:27:52.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:52 smithi046 ceph-mon[21574]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-20T10:27:52.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:52 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2391485070' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:27:52.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:52 smithi042 ceph-mon[17078]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-20T10:27:52.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:52 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2391485070' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:27:52.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:52 smithi162 ceph-mon[21854]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-20T10:27:52.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:52 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2391485070' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:27:54.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:54 smithi042 ceph-mon[17078]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T10:27:54.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:54 smithi162 ceph-mon[21854]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T10:27:54.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:54 smithi046 ceph-mon[21574]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T10:27:56.324 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:56 smithi042 ceph-mon[17078]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T10:27:56.365 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:56 smithi046 ceph-mon[21574]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T10:27:56.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:56 smithi162 ceph-mon[21854]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T10:27:58.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:27:58 smithi042 ceph-mon[17078]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T10:27:58.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:27:58 smithi162 ceph-mon[21854]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T10:27:58.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:27:58 smithi046 ceph-mon[21574]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T10:28:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:00 smithi042 ceph-mon[17078]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:00 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-20T10:28:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:00 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:00.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:00 smithi162 ceph-mon[21854]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:00.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:00 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-20T10:28:00.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:00 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:00.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:00 smithi046 ceph-mon[21574]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:00.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:00 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-20T10:28:00.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:00 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:01.292 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:01 smithi042 ceph-mon[17078]: Deploying daemon osd.6 on smithi042 2024-04-20T10:28:01.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:01 smithi162 ceph-mon[21854]: Deploying daemon osd.6 on smithi042 2024-04-20T10:28:01.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:01 smithi046 ceph-mon[21574]: Deploying daemon osd.6 on smithi042 2024-04-20T10:28:02.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:02 smithi042 ceph-mon[17078]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:02.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:02 smithi162 ceph-mon[21854]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:02.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:02 smithi046 ceph-mon[21574]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:04.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:04 smithi042 ceph-mon[17078]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:04.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:04 smithi162 ceph-mon[21854]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:04.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:04 smithi046 ceph-mon[21574]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:06.286 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:06 smithi042 ceph-mon[17078]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:06.305 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:06 smithi046 ceph-mon[21574]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:06.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:06 smithi162 ceph-mon[21854]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:08.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:08 smithi042 ceph-mon[17078]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:08.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:08 smithi162 ceph-mon[21854]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:08.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:08 smithi046 ceph-mon[21574]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:10 smithi042 ceph-mon[17078]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:28:10.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:10 smithi162 ceph-mon[21854]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:28:10.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:10 smithi046 ceph-mon[21574]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:10.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:28:12.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:12 smithi042 ceph-mon[17078]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:12.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:12 smithi162 ceph-mon[21854]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:12.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:12 smithi046 ceph-mon[21574]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:13.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:13 smithi042 ceph-mon[17078]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:13.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:13.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:13.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:13 smithi042 ceph-mon[17078]: from='osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T10:28:13.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:13 smithi162 ceph-mon[21854]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:13.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:13.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:13.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:13 smithi162 ceph-mon[21854]: from='osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T10:28:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:13 smithi046 ceph-mon[21574]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:13 smithi046 ceph-mon[21574]: from='osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T10:28:14.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.454+0000 7f15c6000700 1 -- 172.21.15.42:0/2708754518 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f15c8062e90 con 0x7f15b00a3c40 2024-04-20T10:28:14.455 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 6 on host 'smithi042' 2024-04-20T10:28:14.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.458+0000 7f15cee00700 1 -- 172.21.15.42:0/2708754518 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f15b00a3c40 msgr2=0x7f15b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.458+0000 7f15cee00700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f15b00a3c40 0x7f15b00a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f15c81020b0 tx=0x7f15bc00b040 comp rx=0 tx=0).stop 2024-04-20T10:28:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.458+0000 7f15cee00700 1 -- 172.21.15.42:0/2708754518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15c8102250 msgr2=0x7f15c8104c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.458+0000 7f15cee00700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15c8102250 0x7f15c8104c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f15c000e940 tx=0x7f15c000ec50 comp rx=0 tx=0).stop 2024-04-20T10:28:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.459+0000 7f15cee00700 1 -- 172.21.15.42:0/2708754518 shutdown_connections 2024-04-20T10:28:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.459+0000 7f15cee00700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f15b00a3c40 0x7f15b00a6100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.459+0000 7f15cee00700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15c8102250 0x7f15c8104c80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.459+0000 7f15cee00700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15c8069ea0 0x7f15c81060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.459+0000 7f15cee00700 1 --2- 172.21.15.42:0/2708754518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15c806a350 0x7f15c8104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.459+0000 7f15cee00700 1 -- 172.21.15.42:0/2708754518 >> 172.21.15.42:0/2708754518 conn(0x7f15c80fc5f0 msgr2=0x7f15c8103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.459+0000 7f15cee00700 1 -- 172.21.15.42:0/2708754518 shutdown_connections 2024-04-20T10:28:14.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:28:14.459+0000 7f15cee00700 1 -- 172.21.15.42:0/2708754518 wait complete. 2024-04-20T10:28:14.893 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:14.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:28:14.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:14.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:14 smithi042 ceph-mon[17078]: from='osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-20T10:28:14.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:14 smithi042 ceph-mon[17078]: osdmap e44: 7 total, 6 up, 7 in 2024-04-20T10:28:14.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:14 smithi042 ceph-mon[17078]: from='osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:28:14.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:28:14.895 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:14.895 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:14.895 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:28:14.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:14.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:28:14.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:14.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:14 smithi162 ceph-mon[21854]: from='osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-20T10:28:14.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:14 smithi162 ceph-mon[21854]: osdmap e44: 7 total, 6 up, 7 in 2024-04-20T10:28:14.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:14 smithi162 ceph-mon[21854]: from='osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:28:14.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:28:14.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:14.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:14.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:28:15.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:15.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:28:15.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:15.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:14 smithi046 ceph-mon[21574]: from='osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-20T10:28:15.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:14 smithi046 ceph-mon[21574]: osdmap e44: 7 total, 6 up, 7 in 2024-04-20T10:28:15.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:14 smithi046 ceph-mon[21574]: from='osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:28:15.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:28:15.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:15.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:15.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:28:15.556 DEBUG:teuthology.orchestra.run.smithi042:osd.6> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.6.service 2024-04-20T10:28:15.562 INFO:tasks.cephadm:Deploying osd.7 on smithi046 with /dev/vg_nvme/lv_2... 2024-04-20T10:28:15.562 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T10:28:15.817 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T10:28:16.191 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:16 smithi046 ceph-mon[21574]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:16.192 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:16 smithi046 ceph-mon[21574]: from='osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-20T10:28:16.192 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:16 smithi046 ceph-mon[21574]: osdmap e45: 7 total, 6 up, 7 in 2024-04-20T10:28:16.192 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:28:16.192 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:28:16.201 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:16 smithi042 ceph-mon[17078]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:16.201 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:16 smithi042 ceph-mon[17078]: from='osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-20T10:28:16.201 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:16 smithi042 ceph-mon[17078]: osdmap e45: 7 total, 6 up, 7 in 2024-04-20T10:28:16.201 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:28:16.202 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:28:16.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:16 smithi162 ceph-mon[21854]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T10:28:16.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:16 smithi162 ceph-mon[21854]: from='osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-20T10:28:16.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:16 smithi162 ceph-mon[21854]: osdmap e45: 7 total, 6 up, 7 in 2024-04-20T10:28:16.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:28:16.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:28:17.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:17 smithi042 ceph-mon[17078]: purged_snaps scrub starts 2024-04-20T10:28:17.096 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:17 smithi042 ceph-mon[17078]: purged_snaps scrub ok 2024-04-20T10:28:17.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:17 smithi046 ceph-mon[21574]: purged_snaps scrub starts 2024-04-20T10:28:17.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:17 smithi046 ceph-mon[21574]: purged_snaps scrub ok 2024-04-20T10:28:17.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:17 smithi046 ceph-mon[21574]: osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714] boot 2024-04-20T10:28:17.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:17 smithi046 ceph-mon[21574]: osdmap e46: 7 total, 7 up, 7 in 2024-04-20T10:28:17.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:17 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:28:17.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:17 smithi042 ceph-mon[17078]: osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714] boot 2024-04-20T10:28:17.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:17 smithi042 ceph-mon[17078]: osdmap e46: 7 total, 7 up, 7 in 2024-04-20T10:28:17.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:28:17.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:17 smithi162 ceph-mon[21854]: purged_snaps scrub starts 2024-04-20T10:28:17.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:17 smithi162 ceph-mon[21854]: purged_snaps scrub ok 2024-04-20T10:28:17.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:17 smithi162 ceph-mon[21854]: osd.6 [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714] boot 2024-04-20T10:28:17.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:17 smithi162 ceph-mon[21854]: osdmap e46: 7 total, 7 up, 7 in 2024-04-20T10:28:17.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:17 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:28:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T10:28:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T10:28:17.933 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-04-20T10:28:17.933 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-04-20T10:28:17.934 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0088782 s, 1.2 GB/s 2024-04-20T10:28:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-04-20T10:28:18.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:18 smithi042 ceph-mon[17078]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:18.432 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:18 smithi042 ceph-mon[17078]: osdmap e47: 7 total, 7 up, 7 in 2024-04-20T10:28:18.433 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:18 smithi162 ceph-mon[21854]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:18.433 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:18 smithi162 ceph-mon[21854]: osdmap e47: 7 total, 7 up, 7 in 2024-04-20T10:28:18.541 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:18 smithi046 ceph-mon[21574]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:18.542 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:18 smithi046 ceph-mon[21574]: osdmap e47: 7 total, 7 up, 7 in 2024-04-20T10:28:18.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_2 2024-04-20T10:28:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T10:28:20.281 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:20 smithi046 ceph-mon[21574]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:20.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:20 smithi042 ceph-mon[17078]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:20.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:20 smithi162 ceph-mon[21854]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.508+0000 7fa6b031d700 1 -- 172.21.15.46:0/2580926089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6a8107640 msgr2=0x7fa6a8109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.508+0000 7fa6b031d700 1 --2- 172.21.15.46:0/2580926089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6a8107640 0x7fa6a8109b00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000b0a0 tx=0x7fa6a000b3b0 comp rx=0 tx=0).stop 2024-04-20T10:28:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.508+0000 7fa6b031d700 1 -- 172.21.15.46:0/2580926089 shutdown_connections 2024-04-20T10:28:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.508+0000 7fa6b031d700 1 --2- 172.21.15.46:0/2580926089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6a8107640 0x7fa6a8109b00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.508+0000 7fa6b031d700 1 --2- 172.21.15.46:0/2580926089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a81021c0 0x7fa6a81045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:20.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.508+0000 7fa6b031d700 1 --2- 172.21.15.46:0/2580926089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6a8104b80 0x7fa6a8106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.508+0000 7fa6b031d700 1 -- 172.21.15.46:0/2580926089 >> 172.21.15.46:0/2580926089 conn(0x7fa6a80fc2a0 msgr2=0x7fa6a80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.508+0000 7fa6b031d700 1 -- 172.21.15.46:0/2580926089 shutdown_connections 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.508+0000 7fa6b031d700 1 -- 172.21.15.46:0/2580926089 wait complete. 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.508+0000 7fa6b031d700 1 Processor -- start 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6b031d700 1 -- start start 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6b031d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6a81021c0 0x7fa6a8104110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6b031d700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6a8104b80 0x7fa6a8102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6b031d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a8107640 0x7fa6a8102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6b031d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a810d600 con 0x7fa6a8107640 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6b031d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a810d480 con 0x7fa6a81021c0 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6b031d700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a810d780 con 0x7fa6a8104b80 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6ae0b9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6a81021c0 0x7fa6a8104110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:20.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6ae0b9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6a81021c0 0x7fa6a8104110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55936/0 (socket says 172.21.15.46:55936) 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6ae0b9700 1 -- 172.21.15.46:0/1088301918 learned_addr learned my addr 172.21.15.46:0/1088301918 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6ad8b8700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6a8104b80 0x7fa6a8102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6ae0b9700 1 -- 172.21.15.46:0/1088301918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6a8104b80 msgr2=0x7fa6a8102760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6ae0b9700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6a8104b80 0x7fa6a8102760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6ae0b9700 1 -- 172.21.15.46:0/1088301918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a8107640 msgr2=0x7fa6a8102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6ae0b9700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a8107640 0x7fa6a8102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6ae0b9700 1 -- 172.21.15.46:0/1088301918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a8103550 con 0x7fa6a81021c0 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.509+0000 7fa6ae0b9700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6a81021c0 0x7fa6a8104110 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400b530 tx=0x7fa6a400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.510+0000 7fa69f7fe700 1 -- 172.21.15.46:0/1088301918 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4015020 con 0x7fa6a81021c0 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.510+0000 7fa69f7fe700 1 -- 172.21.15.46:0/1088301918 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4011d10 con 0x7fa6a81021c0 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.510+0000 7fa6b031d700 1 -- 172.21.15.46:0/1088301918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a81a2e00 con 0x7fa6a81021c0 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.510+0000 7fa69f7fe700 1 -- 172.21.15.46:0/1088301918 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a400f2e0 con 0x7fa6a81021c0 2024-04-20T10:28:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.510+0000 7fa6b031d700 1 -- 172.21.15.46:0/1088301918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a81a3250 con 0x7fa6a81021c0 2024-04-20T10:28:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.511+0000 7fa6b031d700 1 -- 172.21.15.46:0/1088301918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a8069740 con 0x7fa6a81021c0 2024-04-20T10:28:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.512+0000 7fa69f7fe700 1 -- 172.21.15.46:0/1088301918 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa6a4018020 con 0x7fa6a81021c0 2024-04-20T10:28:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.512+0000 7fa69f7fe700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa6940a3d40 0x7fa6940a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.513+0000 7fa69f7fe700 1 -- 172.21.15.46:0/1088301918 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7fa6a40ced90 con 0x7fa6a81021c0 2024-04-20T10:28:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.513+0000 7fa6ad8b8700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa6940a3d40 0x7fa6940a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.514+0000 7fa6ad8b8700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa6940a3d40 0x7fa6940a6200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa698005950 tx=0x7fa69800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:20.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.518+0000 7fa69f7fe700 1 -- 172.21.15.46:0/1088301918 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa6a401f030 con 0x7fa6a81021c0 2024-04-20T10:28:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:20.736+0000 7fa6b031d700 1 -- 172.21.15.46:0/1088301918 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa6a80fd690 con 0x7fa6940a3d40 2024-04-20T10:28:21.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: Detected new or changed devices on smithi042 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi042 to 5698M 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:28:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:28:21.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:28:21.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:21 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: Detected new or changed devices on smithi042 2024-04-20T10:28:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi042 to 5698M 2024-04-20T10:28:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:28:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:28:22.055 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:28:22.055 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:28:22.055 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:21 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: Detected new or changed devices on smithi042 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi042 to 5698M 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:28:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:28:22.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:28:22.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:21 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:23.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:23 smithi162 ceph-mon[21854]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:23.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:23 smithi162 ceph-mon[21854]: from='client.? 172.21.15.46:0/1022237151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13021f92-8ad1-4ff6-b0c7-52d1a2d174a5"}]: dispatch 2024-04-20T10:28:23.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:23 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13021f92-8ad1-4ff6-b0c7-52d1a2d174a5"}]: dispatch 2024-04-20T10:28:23.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:23 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13021f92-8ad1-4ff6-b0c7-52d1a2d174a5"}]': finished 2024-04-20T10:28:23.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:23 smithi162 ceph-mon[21854]: osdmap e48: 8 total, 7 up, 8 in 2024-04-20T10:28:23.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:23 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:28:23.906 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:23 smithi046 ceph-mon[21574]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:23.906 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:23 smithi046 ceph-mon[21574]: from='client.? 172.21.15.46:0/1022237151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13021f92-8ad1-4ff6-b0c7-52d1a2d174a5"}]: dispatch 2024-04-20T10:28:23.907 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:23 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13021f92-8ad1-4ff6-b0c7-52d1a2d174a5"}]: dispatch 2024-04-20T10:28:23.907 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:23 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13021f92-8ad1-4ff6-b0c7-52d1a2d174a5"}]': finished 2024-04-20T10:28:23.907 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:23 smithi046 ceph-mon[21574]: osdmap e48: 8 total, 7 up, 8 in 2024-04-20T10:28:23.907 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:23 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:28:24.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:23 smithi042 ceph-mon[17078]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:24.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:23 smithi042 ceph-mon[17078]: from='client.? 172.21.15.46:0/1022237151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13021f92-8ad1-4ff6-b0c7-52d1a2d174a5"}]: dispatch 2024-04-20T10:28:24.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:23 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13021f92-8ad1-4ff6-b0c7-52d1a2d174a5"}]: dispatch 2024-04-20T10:28:24.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:23 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13021f92-8ad1-4ff6-b0c7-52d1a2d174a5"}]': finished 2024-04-20T10:28:24.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:23 smithi042 ceph-mon[17078]: osdmap e48: 8 total, 7 up, 8 in 2024-04-20T10:28:24.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:23 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:28:25.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:24 smithi046 ceph-mon[21574]: from='client.? 172.21.15.46:0/4193069118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:28:25.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:24 smithi042 ceph-mon[17078]: from='client.? 172.21.15.46:0/4193069118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:28:25.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:24 smithi162 ceph-mon[21854]: from='client.? 172.21.15.46:0/4193069118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:28:26.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:25 smithi046 ceph-mon[21574]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:26.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:25 smithi042 ceph-mon[17078]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:26.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:25 smithi162 ceph-mon[21854]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:27.884 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:27 smithi046 ceph-mon[21574]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:28.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:27 smithi042 ceph-mon[17078]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:28.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:27 smithi162 ceph-mon[21854]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:29.932 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:29 smithi046 ceph-mon[21574]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:30.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:29 smithi042 ceph-mon[17078]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:30.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:29 smithi162 ceph-mon[21854]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:31.929 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:31 smithi046 ceph-mon[21574]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:31.929 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-20T10:28:31.929 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:31.929 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:31 smithi046 ceph-mon[21574]: Deploying daemon osd.7 on smithi046 2024-04-20T10:28:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:31 smithi042 ceph-mon[17078]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-20T10:28:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:31 smithi042 ceph-mon[17078]: Deploying daemon osd.7 on smithi046 2024-04-20T10:28:32.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:31 smithi162 ceph-mon[21854]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:32.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-20T10:28:32.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:32.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:31 smithi162 ceph-mon[21854]: Deploying daemon osd.7 on smithi046 2024-04-20T10:28:34.038 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:33 smithi046 ceph-mon[21574]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:34.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:33 smithi042 ceph-mon[17078]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:34.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:33 smithi162 ceph-mon[21854]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:35.990 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:35 smithi046 ceph-mon[21574]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:36.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:35 smithi042 ceph-mon[17078]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:36.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:35 smithi162 ceph-mon[21854]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:37.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:37 smithi042 ceph-mon[17078]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:37.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:37.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:37.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:28:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:37 smithi162 ceph-mon[21854]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:28:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:37 smithi046 ceph-mon[21574]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:28:39.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:39 smithi042 ceph-mon[17078]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:39 smithi162 ceph-mon[21854]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:39.902 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:39 smithi046 ceph-mon[21574]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.315+0000 7fa69f7fe700 1 -- 172.21.15.46:0/1088301918 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa6a80fd690 con 0x7fa6940a3d40 2024-04-20T10:28:40.317 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 7 on host 'smithi046' 2024-04-20T10:28:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.318+0000 7fa6b031d700 1 -- 172.21.15.46:0/1088301918 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa6940a3d40 msgr2=0x7fa6940a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.318+0000 7fa6b031d700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa6940a3d40 0x7fa6940a6200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa698005950 tx=0x7fa69800b410 comp rx=0 tx=0).stop 2024-04-20T10:28:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.318+0000 7fa6b031d700 1 -- 172.21.15.46:0/1088301918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6a81021c0 msgr2=0x7fa6a8104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.318+0000 7fa6b031d700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6a81021c0 0x7fa6a8104110 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400b530 tx=0x7fa6a400b840 comp rx=0 tx=0).stop 2024-04-20T10:28:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.318+0000 7fa6b031d700 1 -- 172.21.15.46:0/1088301918 shutdown_connections 2024-04-20T10:28:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.318+0000 7fa6b031d700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa6940a3d40 0x7fa6940a6200 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.318+0000 7fa6b031d700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6a81021c0 0x7fa6a8104110 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.318+0000 7fa6b031d700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6a8107640 0x7fa6a8102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.318+0000 7fa6b031d700 1 --2- 172.21.15.46:0/1088301918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6a8104b80 0x7fa6a8102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.318+0000 7fa6b031d700 1 -- 172.21.15.46:0/1088301918 >> 172.21.15.46:0/1088301918 conn(0x7fa6a80fc2a0 msgr2=0x7fa6a8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.318+0000 7fa6b031d700 1 -- 172.21.15.46:0/1088301918 shutdown_connections 2024-04-20T10:28:40.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:28:40.318+0000 7fa6b031d700 1 -- 172.21.15.46:0/1088301918 wait complete. 2024-04-20T10:28:41.003 DEBUG:teuthology.orchestra.run.smithi046:osd.7> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.7.service 2024-04-20T10:28:41.005 INFO:tasks.cephadm:Deploying osd.8 on smithi162 with /dev/vg_nvme/lv_2... 2024-04-20T10:28:41.005 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T10:28:41.252 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:28:41.374 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:41.374 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:41.374 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:41 smithi162 ceph-mon[21854]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:41.374 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:41 smithi162 ceph-mon[21854]: from='osd.7 [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T10:28:41.375 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:41 smithi162 ceph-mon[21854]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T10:28:41.375 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:41.375 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:41.507 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:41.507 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:41.507 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:41 smithi046 ceph-mon[21574]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:41.507 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:41 smithi046 ceph-mon[21574]: from='osd.7 [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T10:28:41.507 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:41 smithi046 ceph-mon[21574]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T10:28:41.507 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:41.507 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:41.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:41.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:41.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:41 smithi042 ceph-mon[17078]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:41.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:41 smithi042 ceph-mon[17078]: from='osd.7 [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T10:28:41.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:41 smithi042 ceph-mon[17078]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T10:28:41.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:41.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:42.485 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:42 smithi046 ceph-mon[21574]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-20T10:28:42.485 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:42 smithi046 ceph-mon[21574]: from='osd.7 [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:28:42.485 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:42 smithi046 ceph-mon[21574]: osdmap e49: 8 total, 7 up, 8 in 2024-04-20T10:28:42.485 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:28:42.485 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:42 smithi046 ceph-mon[21574]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:28:42.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:42 smithi042 ceph-mon[17078]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-20T10:28:42.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:42 smithi042 ceph-mon[17078]: from='osd.7 [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:28:42.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:42 smithi042 ceph-mon[17078]: osdmap e49: 8 total, 7 up, 8 in 2024-04-20T10:28:42.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:42 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:28:42.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:42 smithi042 ceph-mon[17078]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:28:42.636 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:42 smithi162 ceph-mon[21854]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-20T10:28:42.636 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:42 smithi162 ceph-mon[21854]: from='osd.7 [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:28:42.636 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:42 smithi162 ceph-mon[21854]: osdmap e49: 8 total, 7 up, 8 in 2024-04-20T10:28:42.636 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:42 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:28:42.637 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:42 smithi162 ceph-mon[21854]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:28:43.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:28:42 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:28:42.748+0000 7fca92c41700 -1 osd.7 0 waiting for initial osdmap 2024-04-20T10:28:43.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:28:42 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:28:42.758+0000 7fca8aa2c700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:28:43.218 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T10:28:43.218 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T10:28:43.218 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-04-20T10:28:43.218 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-04-20T10:28:43.218 INFO:teuthology.orchestra.run.smithi162.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111228 s, 943 MB/s 2024-04-20T10:28:43.219 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-04-20T10:28:43.522 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:43 smithi162 ceph-mon[21854]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-20T10:28:43.522 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:43 smithi162 ceph-mon[21854]: osdmap e50: 8 total, 7 up, 8 in 2024-04-20T10:28:43.522 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:43 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:28:43.522 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:43 smithi162 ceph-mon[21854]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:43.522 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:43 smithi162 ceph-mon[21854]: from='osd.7 ' entity='osd.7' 2024-04-20T10:28:43.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:43 smithi046 ceph-mon[21574]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-20T10:28:43.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:43 smithi046 ceph-mon[21574]: osdmap e50: 8 total, 7 up, 8 in 2024-04-20T10:28:43.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:43 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:28:43.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:43 smithi046 ceph-mon[21574]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:43.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:43 smithi046 ceph-mon[21574]: from='osd.7 ' entity='osd.7' 2024-04-20T10:28:43.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:43 smithi042 ceph-mon[17078]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-20T10:28:43.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:43 smithi042 ceph-mon[17078]: osdmap e50: 8 total, 7 up, 8 in 2024-04-20T10:28:43.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:43 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:28:43.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:43 smithi042 ceph-mon[17078]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T10:28:43.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:43 smithi042 ceph-mon[17078]: from='osd.7 ' entity='osd.7' 2024-04-20T10:28:43.860 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_2 2024-04-20T10:28:44.102 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:28:44.475 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:44 smithi162 ceph-mon[21854]: purged_snaps scrub starts 2024-04-20T10:28:44.475 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:44 smithi162 ceph-mon[21854]: purged_snaps scrub ok 2024-04-20T10:28:44.475 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:44 smithi162 ceph-mon[21854]: osd.7 [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643] boot 2024-04-20T10:28:44.475 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:44 smithi162 ceph-mon[21854]: osdmap e51: 8 total, 8 up, 8 in 2024-04-20T10:28:44.475 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:28:44.475 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:44 smithi162 ceph-mon[21854]: osdmap e52: 8 total, 8 up, 8 in 2024-04-20T10:28:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:44 smithi046 ceph-mon[21574]: purged_snaps scrub starts 2024-04-20T10:28:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:44 smithi046 ceph-mon[21574]: purged_snaps scrub ok 2024-04-20T10:28:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:44 smithi046 ceph-mon[21574]: osd.7 [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643] boot 2024-04-20T10:28:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:44 smithi046 ceph-mon[21574]: osdmap e51: 8 total, 8 up, 8 in 2024-04-20T10:28:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:28:44.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:44 smithi046 ceph-mon[21574]: osdmap e52: 8 total, 8 up, 8 in 2024-04-20T10:28:44.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:44 smithi042 ceph-mon[17078]: purged_snaps scrub starts 2024-04-20T10:28:44.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:44 smithi042 ceph-mon[17078]: purged_snaps scrub ok 2024-04-20T10:28:44.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:44 smithi042 ceph-mon[17078]: osd.7 [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643] boot 2024-04-20T10:28:44.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:44 smithi042 ceph-mon[17078]: osdmap e51: 8 total, 8 up, 8 in 2024-04-20T10:28:44.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:28:44.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:44 smithi042 ceph-mon[17078]: osdmap e52: 8 total, 8 up, 8 in 2024-04-20T10:28:45.545 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: Detected new or changed devices on smithi046 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi046 to 5697M 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:28:45.546 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:45.547 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:28:45.547 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.547 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:45 smithi162 ceph-mon[21854]: osdmap e53: 8 total, 8 up, 8 in 2024-04-20T10:28:45.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: Detected new or changed devices on smithi046 2024-04-20T10:28:45.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:45.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:45.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:45.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi046 to 5697M 2024-04-20T10:28:45.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:45.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:28:45.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:45.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:28:45.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:45.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:28:45.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:45 smithi042 ceph-mon[17078]: osdmap e53: 8 total, 8 up, 8 in 2024-04-20T10:28:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.781+0000 7f189b1a2700 1 -- 172.21.15.162:0/3942075399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1894105d90 msgr2=0x7f1894108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.781+0000 7f189b1a2700 1 --2- 172.21.15.162:0/3942075399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1894105d90 0x7f1894108180 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1884009990 tx=0x7f1884009ca0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.781+0000 7f189b1a2700 1 -- 172.21.15.162:0/3942075399 shutdown_connections 2024-04-20T10:28:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.781+0000 7f189b1a2700 1 --2- 172.21.15.162:0/3942075399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18941086c0 0x7f189410ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.781+0000 7f189b1a2700 1 --2- 172.21.15.162:0/3942075399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1894103460 0x7f1894105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.781+0000 7f189b1a2700 1 --2- 172.21.15.162:0/3942075399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1894105d90 0x7f1894108180 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.781+0000 7f189b1a2700 1 -- 172.21.15.162:0/3942075399 >> 172.21.15.162:0/3942075399 conn(0x7f18940fce70 msgr2=0x7f18940ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.781+0000 7f189b1a2700 1 -- 172.21.15.162:0/3942075399 shutdown_connections 2024-04-20T10:28:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.781+0000 7f189b1a2700 1 -- 172.21.15.162:0/3942075399 wait complete. 2024-04-20T10:28:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189b1a2700 1 Processor -- start 2024-04-20T10:28:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189b1a2700 1 -- start start 2024-04-20T10:28:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189b1a2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894103460 0x7f1894102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189b1a2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1894105d90 0x7f1894101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189b1a2700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18941086c0 0x7f1894101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189b1a2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f189410e620 con 0x7f1894105d90 2024-04-20T10:28:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189b1a2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f189410e4a0 con 0x7f1894103460 2024-04-20T10:28:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189b1a2700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f189410e7a0 con 0x7f18941086c0 2024-04-20T10:28:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189a9a1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18941086c0 0x7f1894101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189a9a1700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18941086c0 0x7f1894101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56306/0 (socket says 172.21.15.162:56306) 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189a9a1700 1 -- 172.21.15.162:0/955366581 learned_addr learned my addr 172.21.15.162:0/955366581 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189a1a0700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894103460 0x7f1894102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189a9a1700 1 -- 172.21.15.162:0/955366581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894103460 msgr2=0x7f1894102d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189a9a1700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894103460 0x7f1894102d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189a9a1700 1 -- 172.21.15.162:0/955366581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1894105d90 msgr2=0x7f1894101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189a9a1700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1894105d90 0x7f1894101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189a9a1700 1 -- 172.21.15.162:0/955366581 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1894102220 con 0x7f18941086c0 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189a9a1700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18941086c0 0x7f1894101950 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f188c00ef90 tx=0x7f188c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f188b7fe700 1 -- 172.21.15.162:0/955366581 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c0092e0 con 0x7f18941086c0 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f188b7fe700 1 -- 172.21.15.162:0/955366581 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f188c004bb0 con 0x7f18941086c0 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.782+0000 7f189b1a2700 1 -- 172.21.15.162:0/955366581 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18941a2d90 con 0x7f18941086c0 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.783+0000 7f188b7fe700 1 -- 172.21.15.162:0/955366581 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c004280 con 0x7f18941086c0 2024-04-20T10:28:45.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.783+0000 7f189b1a2700 1 -- 172.21.15.162:0/955366581 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18941a3190 con 0x7f18941086c0 2024-04-20T10:28:45.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.784+0000 7f189b1a2700 1 -- 172.21.15.162:0/955366581 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1894069740 con 0x7f18941086c0 2024-04-20T10:28:45.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.785+0000 7f188b7fe700 1 -- 172.21.15.162:0/955366581 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f188c01b030 con 0x7f18941086c0 2024-04-20T10:28:45.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.785+0000 7f188b7fe700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f18800a3c40 0x7f18800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.785+0000 7f188b7fe700 1 -- 172.21.15.162:0/955366581 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f188c0cf4c0 con 0x7f18941086c0 2024-04-20T10:28:45.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.786+0000 7f189a1a0700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f18800a3c40 0x7f18800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.787+0000 7f189a1a0700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f18800a3c40 0x7f18800a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1890009a10 tx=0x7f1890009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:45.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.790+0000 7f188b7fe700 1 -- 172.21.15.162:0/955366581 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f188c022020 con 0x7f18941086c0 2024-04-20T10:28:45.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: Detected new or changed devices on smithi046 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi046 to 5697M 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:28:45.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:45.807 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:28:45.807 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:28:45.807 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:45 smithi046 ceph-mon[21574]: osdmap e53: 8 total, 8 up, 8 in 2024-04-20T10:28:45.995 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:28:45.995+0000 7f189b1a2700 1 -- 172.21.15.162:0/955366581 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1894062e90 con 0x7f18800a3c40 2024-04-20T10:28:46.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:46 smithi042 ceph-mon[17078]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:28:46.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:46 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:28:46.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:46 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:28:46.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:46 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:46.643 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:46 smithi162 ceph-mon[21854]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:28:46.643 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:46 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:28:46.643 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:46 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:28:46.643 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:46 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:46.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:46 smithi046 ceph-mon[21574]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:28:46.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:46 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:28:46.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:46 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:28:46.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:46 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:47.648 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:47 smithi162 ceph-mon[21854]: osdmap e54: 8 total, 8 up, 8 in 2024-04-20T10:28:47.648 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:47 smithi162 ceph-mon[21854]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:47.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:47 smithi046 ceph-mon[21574]: osdmap e54: 8 total, 8 up, 8 in 2024-04-20T10:28:47.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:47 smithi046 ceph-mon[21574]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:47 smithi042 ceph-mon[17078]: osdmap e54: 8 total, 8 up, 8 in 2024-04-20T10:28:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:47 smithi042 ceph-mon[17078]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:48.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:48 smithi162 ceph-mon[21854]: osdmap e55: 8 total, 8 up, 8 in 2024-04-20T10:28:48.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:48 smithi046 ceph-mon[21574]: osdmap e55: 8 total, 8 up, 8 in 2024-04-20T10:28:48.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:48 smithi042 ceph-mon[17078]: osdmap e55: 8 total, 8 up, 8 in 2024-04-20T10:28:49.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:49 smithi162 ceph-mon[21854]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:49 smithi162 ceph-mon[21854]: from='client.? 172.21.15.162:0/1815688154' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8185b6db-3e13-40ed-950c-b330c983331e"}]: dispatch 2024-04-20T10:28:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:49 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8185b6db-3e13-40ed-950c-b330c983331e"}]: dispatch 2024-04-20T10:28:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:49 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8185b6db-3e13-40ed-950c-b330c983331e"}]': finished 2024-04-20T10:28:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:49 smithi162 ceph-mon[21854]: osdmap e56: 9 total, 8 up, 9 in 2024-04-20T10:28:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:28:49.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:49 smithi046 ceph-mon[21574]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:49.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:49 smithi046 ceph-mon[21574]: from='client.? 172.21.15.162:0/1815688154' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8185b6db-3e13-40ed-950c-b330c983331e"}]: dispatch 2024-04-20T10:28:49.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:49 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8185b6db-3e13-40ed-950c-b330c983331e"}]: dispatch 2024-04-20T10:28:49.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:49 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8185b6db-3e13-40ed-950c-b330c983331e"}]': finished 2024-04-20T10:28:49.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:49 smithi046 ceph-mon[21574]: osdmap e56: 9 total, 8 up, 9 in 2024-04-20T10:28:49.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:28:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:49 smithi042 ceph-mon[17078]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:49 smithi042 ceph-mon[17078]: from='client.? 172.21.15.162:0/1815688154' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8185b6db-3e13-40ed-950c-b330c983331e"}]: dispatch 2024-04-20T10:28:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:49 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8185b6db-3e13-40ed-950c-b330c983331e"}]: dispatch 2024-04-20T10:28:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:49 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8185b6db-3e13-40ed-950c-b330c983331e"}]': finished 2024-04-20T10:28:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:49 smithi042 ceph-mon[17078]: osdmap e56: 9 total, 8 up, 9 in 2024-04-20T10:28:49.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:28:50.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:50 smithi162 ceph-mon[21854]: from='client.? 172.21.15.162:0/4168744041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:28:50.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:50 smithi046 ceph-mon[21574]: from='client.? 172.21.15.162:0/4168744041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:28:50.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:50 smithi042 ceph-mon[17078]: from='client.? 172.21.15.162:0/4168744041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:28:51.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:51 smithi162 ceph-mon[21854]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:51.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:51 smithi046 ceph-mon[21574]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:51 smithi042 ceph-mon[17078]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:53.790 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:53 smithi162 ceph-mon[21854]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:53.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:53 smithi046 ceph-mon[21574]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:53.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:53 smithi042 ceph-mon[17078]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:55.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:55 smithi046 ceph-mon[21574]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:55.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:55 smithi042 ceph-mon[17078]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:55.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:55 smithi162 ceph-mon[21854]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:56.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:56 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-20T10:28:56.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:56 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:56.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:56 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-20T10:28:56.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:56 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:56.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:56 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-20T10:28:56.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:56 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:28:57.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:57 smithi046 ceph-mon[21574]: Deploying daemon osd.8 on smithi162 2024-04-20T10:28:57.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:57 smithi046 ceph-mon[21574]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:57.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:57 smithi042 ceph-mon[17078]: Deploying daemon osd.8 on smithi162 2024-04-20T10:28:57.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:57 smithi042 ceph-mon[17078]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:57 smithi162 ceph-mon[21854]: Deploying daemon osd.8 on smithi162 2024-04-20T10:28:57.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:57 smithi162 ceph-mon[21854]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:59.791 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:28:59 smithi162 ceph-mon[21854]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:59.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:28:59 smithi046 ceph-mon[21574]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:28:59.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:28:59 smithi042 ceph-mon[17078]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:01.772 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:01 smithi162 ceph-mon[21854]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:01.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:01 smithi046 ceph-mon[21574]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:01.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:01 smithi042 ceph-mon[17078]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:02.763 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:02 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:29:02.763 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:02 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:02.763 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:02 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:02.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:02 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:29:02.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:02 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:02.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:02 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:02.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:02 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:29:02.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:02 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:02.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:02 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:04.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:04 smithi042 ceph-mon[17078]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:04.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:04 smithi162 ceph-mon[21854]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:04.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:04 smithi046 ceph-mon[21574]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:06.038 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.038+0000 7f188b7fe700 1 -- 172.21.15.162:0/955366581 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1894062e90 con 0x7f18800a3c40 2024-04-20T10:29:06.039 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 8 on host 'smithi162' 2024-04-20T10:29:06.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.040+0000 7f189b1a2700 1 -- 172.21.15.162:0/955366581 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f18800a3c40 msgr2=0x7f18800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:06.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.040+0000 7f189b1a2700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f18800a3c40 0x7f18800a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1890009a10 tx=0x7f1890009190 comp rx=0 tx=0).stop 2024-04-20T10:29:06.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.040+0000 7f189b1a2700 1 -- 172.21.15.162:0/955366581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18941086c0 msgr2=0x7f1894101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:06.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.040+0000 7f189b1a2700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18941086c0 0x7f1894101950 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f188c00ef90 tx=0x7f188c00c3a0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.040+0000 7f189b1a2700 1 -- 172.21.15.162:0/955366581 shutdown_connections 2024-04-20T10:29:06.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.040+0000 7f189b1a2700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f18800a3c40 0x7f18800a6100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.040+0000 7f189b1a2700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1894103460 0x7f1894102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.040+0000 7f189b1a2700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18941086c0 0x7f1894101950 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.040+0000 7f189b1a2700 1 --2- 172.21.15.162:0/955366581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1894105d90 0x7f1894101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.040+0000 7f189b1a2700 1 -- 172.21.15.162:0/955366581 >> 172.21.15.162:0/955366581 conn(0x7f18940fce70 msgr2=0x7f1894103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:06.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.040+0000 7f189b1a2700 1 -- 172.21.15.162:0/955366581 shutdown_connections 2024-04-20T10:29:06.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:29:06.040+0000 7f189b1a2700 1 -- 172.21.15.162:0/955366581 wait complete. 2024-04-20T10:29:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:06 smithi046 ceph-mon[21574]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:06 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:06 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:06 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:06.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:06 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:29:06.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:06 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:06 smithi046 ceph-mon[21574]: from='osd.8 [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T10:29:06.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:06 smithi046 ceph-mon[21574]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T10:29:06.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:06 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:06 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:06 smithi042 ceph-mon[17078]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:06.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:06 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:06 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:06 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:06.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:06 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:29:06.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:06 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:06 smithi042 ceph-mon[17078]: from='osd.8 [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T10:29:06.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:06 smithi042 ceph-mon[17078]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T10:29:06.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:06 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:06 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:06 smithi162 ceph-mon[21854]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:06.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:06 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:06 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:06 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:06.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:06 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:29:06.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:06 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:06 smithi162 ceph-mon[21854]: from='osd.8 [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T10:29:06.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:06 smithi162 ceph-mon[21854]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T10:29:06.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:06 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:06 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:06.709 DEBUG:teuthology.orchestra.run.smithi162:osd.8> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.8.service 2024-04-20T10:29:06.712 INFO:tasks.cephadm:Deploying osd.9 on smithi042 with /dev/vg_nvme/lv_1... 2024-04-20T10:29:06.712 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-20T10:29:07.166 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:29:07.289 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:07 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:29:07.289 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:07 smithi162 ceph-mon[21854]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-20T10:29:07.290 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:07 smithi162 ceph-mon[21854]: from='osd.8 [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:29:07.290 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:07 smithi162 ceph-mon[21854]: osdmap e57: 9 total, 8 up, 9 in 2024-04-20T10:29:07.290 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:07 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:29:07.290 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:07 smithi162 ceph-mon[21854]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:29:07.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:07 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:29:07.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:07 smithi046 ceph-mon[21574]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-20T10:29:07.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:07 smithi046 ceph-mon[21574]: from='osd.8 [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:29:07.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:07 smithi046 ceph-mon[21574]: osdmap e57: 9 total, 8 up, 9 in 2024-04-20T10:29:07.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:07 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:29:07.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:07 smithi046 ceph-mon[21574]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:29:07.320 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:07 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:29:07.320 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:07 smithi042 ceph-mon[17078]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-20T10:29:07.320 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:07 smithi042 ceph-mon[17078]: from='osd.8 [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:29:07.320 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:07 smithi042 ceph-mon[17078]: osdmap e57: 9 total, 8 up, 9 in 2024-04-20T10:29:07.320 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:07 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:29:07.320 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:07 smithi042 ceph-mon[17078]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:29:07.618 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:29:07 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:29:07.521+0000 7f5d6203c700 -1 osd.8 0 waiting for initial osdmap 2024-04-20T10:29:07.618 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:29:07 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:29:07.532+0000 7f5d59626700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:29:08.241 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:08 smithi162 ceph-mon[21854]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:08.241 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:08 smithi162 ceph-mon[21854]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-20T10:29:08.242 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:08 smithi162 ceph-mon[21854]: osdmap e58: 9 total, 8 up, 9 in 2024-04-20T10:29:08.242 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:08 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:29:08.242 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:08 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:29:08.242 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:08 smithi162 ceph-mon[21854]: from='osd.8 ' entity='osd.8' 2024-04-20T10:29:08.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:08 smithi042 ceph-mon[17078]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:08.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:08 smithi042 ceph-mon[17078]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-20T10:29:08.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:08 smithi042 ceph-mon[17078]: osdmap e58: 9 total, 8 up, 9 in 2024-04-20T10:29:08.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:08 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:29:08.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:08 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:29:08.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:08 smithi042 ceph-mon[17078]: from='osd.8 ' entity='osd.8' 2024-04-20T10:29:08.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:08 smithi046 ceph-mon[21574]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-20T10:29:08.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:08 smithi046 ceph-mon[21574]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-20T10:29:08.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:08 smithi046 ceph-mon[21574]: osdmap e58: 9 total, 8 up, 9 in 2024-04-20T10:29:08.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:08 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:29:08.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:08 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:29:08.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:08 smithi046 ceph-mon[21574]: from='osd.8 ' entity='osd.8' 2024-04-20T10:29:09.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:09 smithi162 ceph-mon[21854]: purged_snaps scrub starts 2024-04-20T10:29:09.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:09 smithi162 ceph-mon[21854]: purged_snaps scrub ok 2024-04-20T10:29:09.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:09 smithi162 ceph-mon[21854]: osd.8 [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377] boot 2024-04-20T10:29:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:09 smithi162 ceph-mon[21854]: osdmap e59: 9 total, 9 up, 9 in 2024-04-20T10:29:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:09 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:29:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:09 smithi162 ceph-mon[21854]: osdmap e60: 9 total, 9 up, 9 in 2024-04-20T10:29:09.464 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:09 smithi042 ceph-mon[17078]: purged_snaps scrub starts 2024-04-20T10:29:09.464 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:09 smithi042 ceph-mon[17078]: purged_snaps scrub ok 2024-04-20T10:29:09.464 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:09 smithi042 ceph-mon[17078]: osd.8 [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377] boot 2024-04-20T10:29:09.464 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:09 smithi042 ceph-mon[17078]: osdmap e59: 9 total, 9 up, 9 in 2024-04-20T10:29:09.464 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:09 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:29:09.464 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:09 smithi042 ceph-mon[17078]: osdmap e60: 9 total, 9 up, 9 in 2024-04-20T10:29:09.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:09 smithi046 ceph-mon[21574]: purged_snaps scrub starts 2024-04-20T10:29:09.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:09 smithi046 ceph-mon[21574]: purged_snaps scrub ok 2024-04-20T10:29:09.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:09 smithi046 ceph-mon[21574]: osd.8 [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377] boot 2024-04-20T10:29:09.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:09 smithi046 ceph-mon[21574]: osdmap e59: 9 total, 9 up, 9 in 2024-04-20T10:29:09.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:09 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:29:09.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:09 smithi046 ceph-mon[21574]: osdmap e60: 9 total, 9 up, 9 in 2024-04-20T10:29:10.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:10 smithi162 ceph-mon[21854]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:10 smithi162 ceph-mon[21854]: osdmap e61: 9 total, 9 up, 9 in 2024-04-20T10:29:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:29:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-20T10:29:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-20T10:29:10.418 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-04-20T10:29:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-04-20T10:29:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0387718 s, 270 MB/s 2024-04-20T10:29:10.418 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-04-20T10:29:10.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:10 smithi042 ceph-mon[17078]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:10.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:10 smithi042 ceph-mon[17078]: osdmap e61: 9 total, 9 up, 9 in 2024-04-20T10:29:10.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:10.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:10.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:10.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:10.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:10.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:10.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:10.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:29:10.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:10 smithi046 ceph-mon[21574]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:10 smithi046 ceph-mon[21574]: osdmap e61: 9 total, 9 up, 9 in 2024-04-20T10:29:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:29:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:11 smithi042 ceph-mon[17078]: Detected new or changed devices on smithi162 2024-04-20T10:29:11.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:11 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi162 to 5696M 2024-04-20T10:29:11.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:11 smithi042 ceph-mon[17078]: osdmap e62: 9 total, 9 up, 9 in 2024-04-20T10:29:11.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:11 smithi162 ceph-mon[21854]: Detected new or changed devices on smithi162 2024-04-20T10:29:11.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:11 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi162 to 5696M 2024-04-20T10:29:11.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:11 smithi162 ceph-mon[21854]: osdmap e62: 9 total, 9 up, 9 in 2024-04-20T10:29:11.504 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_1 2024-04-20T10:29:11.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:11 smithi046 ceph-mon[21574]: Detected new or changed devices on smithi162 2024-04-20T10:29:11.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:11 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi162 to 5696M 2024-04-20T10:29:11.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:11 smithi046 ceph-mon[21574]: osdmap e62: 9 total, 9 up, 9 in 2024-04-20T10:29:11.948 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:29:12.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:12 smithi042 ceph-mon[17078]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:12.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:12 smithi042 ceph-mon[17078]: osdmap e63: 9 total, 9 up, 9 in 2024-04-20T10:29:12.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:12 smithi162 ceph-mon[21854]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:12.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:12 smithi162 ceph-mon[21854]: osdmap e63: 9 total, 9 up, 9 in 2024-04-20T10:29:12.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:12 smithi046 ceph-mon[21574]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:12.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:12 smithi046 ceph-mon[21574]: osdmap e63: 9 total, 9 up, 9 in 2024-04-20T10:29:14.206 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:14 smithi042 ceph-mon[17078]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:14.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.384+0000 7f6595000700 1 -- 172.21.15.42:0/2117968480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6590105ef0 msgr2=0x7f6590106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:14.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.384+0000 7f6595000700 1 --2- 172.21.15.42:0/2117968480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6590105ef0 0x7f6590106350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f657c0099c0 tx=0x7f657c009cd0 comp rx=0 tx=0).stop 2024-04-20T10:29:14.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.384+0000 7f6595000700 1 -- 172.21.15.42:0/2117968480 shutdown_connections 2024-04-20T10:29:14.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.384+0000 7f6595000700 1 --2- 172.21.15.42:0/2117968480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6590106890 0x7f659010d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.384+0000 7f6595000700 1 --2- 172.21.15.42:0/2117968480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6590104cf0 0x7f6590105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.384+0000 7f6595000700 1 --2- 172.21.15.42:0/2117968480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6590105ef0 0x7f6590106350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.384+0000 7f6595000700 1 -- 172.21.15.42:0/2117968480 >> 172.21.15.42:0/2117968480 conn(0x7f65901002b0 msgr2=0x7f65901026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.384+0000 7f6595000700 1 -- 172.21.15.42:0/2117968480 shutdown_connections 2024-04-20T10:29:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.384+0000 7f6595000700 1 -- 172.21.15.42:0/2117968480 wait complete. 2024-04-20T10:29:14.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.386+0000 7f6595000700 1 Processor -- start 2024-04-20T10:29:14.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.386+0000 7f6595000700 1 -- start start 2024-04-20T10:29:14.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.386+0000 7f6595000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6590104cf0 0x7f659019aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:14.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.386+0000 7f6595000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6590105ef0 0x7f659019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:14.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.386+0000 7f6595000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6590106890 0x7f65901a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:14.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.386+0000 7f6595000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6590110930 con 0x7f6590106890 2024-04-20T10:29:14.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.386+0000 7f6595000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65901107b0 con 0x7f6590104cf0 2024-04-20T10:29:14.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.386+0000 7f6595000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6590110ab0 con 0x7f6590105ef0 2024-04-20T10:29:14.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.386+0000 7f658de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6590104cf0 0x7f659019aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:14.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.387+0000 7f658d400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6590105ef0 0x7f659019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:14.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.387+0000 7f658de00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6590104cf0 0x7f659019aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59474/0 (socket says 172.21.15.42:59474) 2024-04-20T10:29:14.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.387+0000 7f658de00700 1 -- 172.21.15.42:0/614782736 learned_addr learned my addr 172.21.15.42:0/614782736 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:29:14.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.387+0000 7f658e800700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6590106890 0x7f65901a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:14.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.387+0000 7f658d400700 1 -- 172.21.15.42:0/614782736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6590104cf0 msgr2=0x7f659019aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:14.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.387+0000 7f658d400700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6590104cf0 0x7f659019aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:14.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.387+0000 7f658d400700 1 -- 172.21.15.42:0/614782736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6590106890 msgr2=0x7f65901a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:14.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.387+0000 7f658d400700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6590106890 0x7f65901a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:14.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.387+0000 7f658d400700 1 -- 172.21.15.42:0/614782736 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65901a2b20 con 0x7f6590105ef0 2024-04-20T10:29:14.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.387+0000 7f658e800700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6590106890 0x7f65901a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:29:14.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.387+0000 7f658d400700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6590105ef0 0x7f659019b400 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f657c003790 tx=0x7f657c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:14.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.388+0000 7f6586a00700 1 -- 172.21.15.42:0/614782736 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c027070 con 0x7f6590105ef0 2024-04-20T10:29:14.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.388+0000 7f6586a00700 1 -- 172.21.15.42:0/614782736 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f657c022470 con 0x7f6590105ef0 2024-04-20T10:29:14.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.388+0000 7f6586a00700 1 -- 172.21.15.42:0/614782736 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c009e60 con 0x7f6590105ef0 2024-04-20T10:29:14.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.388+0000 7f6595000700 1 -- 172.21.15.42:0/614782736 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65901a2db0 con 0x7f6590105ef0 2024-04-20T10:29:14.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.388+0000 7f6595000700 1 -- 172.21.15.42:0/614782736 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65901a31b0 con 0x7f6590105ef0 2024-04-20T10:29:14.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.389+0000 7f6595000700 1 -- 172.21.15.42:0/614782736 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659002df90 con 0x7f6590105ef0 2024-04-20T10:29:14.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.391+0000 7f6586a00700 1 -- 172.21.15.42:0/614782736 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f657c02a070 con 0x7f6590105ef0 2024-04-20T10:29:14.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.392+0000 7f6586a00700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f65740a3cc0 0x7f65740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:14.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.392+0000 7f6586a00700 1 -- 172.21.15.42:0/614782736 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f657c0d0820 con 0x7f6590105ef0 2024-04-20T10:29:14.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.393+0000 7f658de00700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f65740a3cc0 0x7f65740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:14.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.393+0000 7f658de00700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f65740a3cc0 0x7f65740a6180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6580005950 tx=0x7f6580009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:14.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.395+0000 7f6586a00700 1 -- 172.21.15.42:0/614782736 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f657c013070 con 0x7f6590105ef0 2024-04-20T10:29:14.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:14 smithi162 ceph-mon[21854]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:14.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:14 smithi046 ceph-mon[21574]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:14.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:14.638+0000 7f6595000700 1 -- 172.21.15.42:0/614782736 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6590062e90 con 0x7f65740a3cc0 2024-04-20T10:29:15.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:29:15.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:29:15.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:15.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:29:15.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:29:15.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:15.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:29:15.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:29:15.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:16.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:16 smithi042 ceph-mon[17078]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-20T10:29:16.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:16 smithi042 ceph-mon[17078]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:29:16.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:16 smithi162 ceph-mon[21854]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-20T10:29:16.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:16 smithi162 ceph-mon[21854]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:29:16.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:16 smithi046 ceph-mon[21574]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-20T10:29:16.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:16 smithi046 ceph-mon[21574]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:29:18.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:18 smithi162 ceph-mon[21854]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-20T10:29:18.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:18 smithi046 ceph-mon[21574]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-20T10:29:18.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:18 smithi042 ceph-mon[17078]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-20T10:29:19.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:19 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2813397941' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cd34bd8-f661-4903-88cb-7c44fd178f1d"}]: dispatch 2024-04-20T10:29:19.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:19 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2813397941' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cd34bd8-f661-4903-88cb-7c44fd178f1d"}]': finished 2024-04-20T10:29:19.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:19 smithi046 ceph-mon[21574]: osdmap e64: 10 total, 9 up, 10 in 2024-04-20T10:29:19.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:19.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:19 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2813397941' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cd34bd8-f661-4903-88cb-7c44fd178f1d"}]: dispatch 2024-04-20T10:29:19.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:19 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2813397941' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cd34bd8-f661-4903-88cb-7c44fd178f1d"}]': finished 2024-04-20T10:29:19.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:19 smithi042 ceph-mon[17078]: osdmap e64: 10 total, 9 up, 10 in 2024-04-20T10:29:19.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:19.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:19 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2813397941' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cd34bd8-f661-4903-88cb-7c44fd178f1d"}]: dispatch 2024-04-20T10:29:19.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:19 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2813397941' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cd34bd8-f661-4903-88cb-7c44fd178f1d"}]': finished 2024-04-20T10:29:19.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:19 smithi162 ceph-mon[21854]: osdmap e64: 10 total, 9 up, 10 in 2024-04-20T10:29:19.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:20.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:20 smithi046 ceph-mon[21574]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T10:29:20.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:20 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/221235937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:29:20.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:20 smithi042 ceph-mon[17078]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T10:29:20.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:20 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/221235937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:29:20.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:20 smithi162 ceph-mon[21854]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T10:29:20.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:20 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/221235937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:29:22.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:22 smithi046 ceph-mon[21574]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-20T10:29:22.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:22 smithi042 ceph-mon[17078]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-20T10:29:22.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:22 smithi162 ceph-mon[21854]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-20T10:29:24.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:24 smithi046 ceph-mon[21574]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T10:29:24.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:24 smithi042 ceph-mon[17078]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T10:29:24.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:24 smithi162 ceph-mon[21854]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T10:29:26.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:26 smithi042 ceph-mon[17078]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:26.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:26 smithi046 ceph-mon[21574]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:26.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:26 smithi162 ceph-mon[21854]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:28.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:28 smithi042 ceph-mon[17078]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:28.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:28 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-20T10:29:28.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:28 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:28.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:28 smithi162 ceph-mon[21854]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:28.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:28 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-20T10:29:28.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:28 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:28.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:28 smithi046 ceph-mon[21574]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:28.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:28 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-20T10:29:28.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:28 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:29.293 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:29 smithi042 ceph-mon[17078]: Deploying daemon osd.9 on smithi042 2024-04-20T10:29:29.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:29 smithi046 ceph-mon[21574]: Deploying daemon osd.9 on smithi042 2024-04-20T10:29:29.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:29 smithi162 ceph-mon[21854]: Deploying daemon osd.9 on smithi042 2024-04-20T10:29:30.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:30 smithi046 ceph-mon[21574]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:30.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:30 smithi042 ceph-mon[17078]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:30.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:30 smithi162 ceph-mon[21854]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:32.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:32 smithi042 ceph-mon[17078]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:32.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:32 smithi046 ceph-mon[21574]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:32 smithi162 ceph-mon[21854]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:34.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:34 smithi046 ceph-mon[21574]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:34.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:34 smithi042 ceph-mon[17078]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:34.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:34 smithi162 ceph-mon[21854]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:36.287 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:36 smithi042 ceph-mon[17078]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:36.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:36 smithi046 ceph-mon[21574]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:36.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:36 smithi162 ceph-mon[21854]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:37.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:37 smithi046 ceph-mon[21574]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:37.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:37.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:37.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:29:37.841 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:37 smithi042 ceph-mon[17078]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:37.841 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:37.841 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:37.841 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:29:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:37 smithi162 ceph-mon[21854]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:29:39.648 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:39 smithi042 ceph-mon[17078]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:39.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:39 smithi046 ceph-mon[21574]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:39 smithi162 ceph-mon[21854]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:41.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.695+0000 7f6586a00700 1 -- 172.21.15.42:0/614782736 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6590062e90 con 0x7f65740a3cc0 2024-04-20T10:29:41.696 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 9 on host 'smithi042' 2024-04-20T10:29:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.698+0000 7f6595000700 1 -- 172.21.15.42:0/614782736 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f65740a3cc0 msgr2=0x7f65740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.698+0000 7f6595000700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f65740a3cc0 0x7f65740a6180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6580005950 tx=0x7f6580009260 comp rx=0 tx=0).stop 2024-04-20T10:29:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.698+0000 7f6595000700 1 -- 172.21.15.42:0/614782736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6590105ef0 msgr2=0x7f659019b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.698+0000 7f6595000700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6590105ef0 0x7f659019b400 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f657c003790 tx=0x7f657c009580 comp rx=0 tx=0).stop 2024-04-20T10:29:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.699+0000 7f6595000700 1 -- 172.21.15.42:0/614782736 shutdown_connections 2024-04-20T10:29:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.699+0000 7f6595000700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f65740a3cc0 0x7f65740a6180 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.699+0000 7f6595000700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6590104cf0 0x7f659019aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.699+0000 7f6595000700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6590106890 0x7f65901a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.699+0000 7f6595000700 1 --2- 172.21.15.42:0/614782736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6590105ef0 0x7f659019b400 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.699+0000 7f6595000700 1 -- 172.21.15.42:0/614782736 >> 172.21.15.42:0/614782736 conn(0x7f65901002b0 msgr2=0x7f6590109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.699+0000 7f6595000700 1 -- 172.21.15.42:0/614782736 shutdown_connections 2024-04-20T10:29:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:29:41.699+0000 7f6595000700 1 -- 172.21.15.42:0/614782736 wait complete. 2024-04-20T10:29:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:41 smithi046 ceph-mon[21574]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:41 smithi046 ceph-mon[21574]: from='osd.9 [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T10:29:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:41 smithi046 ceph-mon[21574]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T10:29:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:41 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:41.852 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:41 smithi042 ceph-mon[17078]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:41.852 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:41 smithi042 ceph-mon[17078]: from='osd.9 [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T10:29:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:41 smithi042 ceph-mon[17078]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T10:29:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:41 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:41 smithi162 ceph-mon[21854]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:41 smithi162 ceph-mon[21854]: from='osd.9 [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T10:29:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:41 smithi162 ceph-mon[21854]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T10:29:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:41 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:42.651 DEBUG:teuthology.orchestra.run.smithi042:osd.9> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.9.service 2024-04-20T10:29:42.655 INFO:tasks.cephadm:Deploying osd.10 on smithi046 with /dev/vg_nvme/lv_1... 2024-04-20T10:29:42.656 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-20T10:29:42.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:42 smithi046 ceph-mon[21574]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-20T10:29:42.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:42 smithi046 ceph-mon[21574]: from='osd.9 [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:29:42.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:42 smithi046 ceph-mon[21574]: osdmap e65: 10 total, 9 up, 10 in 2024-04-20T10:29:42.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:42.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:42 smithi046 ceph-mon[21574]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:29:42.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:42.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:42 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:42.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:42 smithi046 ceph-mon[21574]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-20T10:29:42.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:42 smithi042 ceph-mon[17078]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-20T10:29:42.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:42 smithi042 ceph-mon[17078]: from='osd.9 [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:29:42.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:42 smithi042 ceph-mon[17078]: osdmap e65: 10 total, 9 up, 10 in 2024-04-20T10:29:42.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:42 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:42.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:42 smithi042 ceph-mon[17078]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:29:42.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:42 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:42.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:42 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:42.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:42 smithi042 ceph-mon[17078]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-20T10:29:42.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:42 smithi162 ceph-mon[21854]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-20T10:29:42.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:42 smithi162 ceph-mon[21854]: from='osd.9 [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:29:42.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:42 smithi162 ceph-mon[21854]: osdmap e65: 10 total, 9 up, 10 in 2024-04-20T10:29:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:42 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:42 smithi162 ceph-mon[21854]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-20T10:29:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:42 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:42 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:42 smithi162 ceph-mon[21854]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-20T10:29:42.912 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T10:29:43.550 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:29:43 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:29:43.251+0000 7fb6e4800700 -1 osd.9 0 waiting for initial osdmap 2024-04-20T10:29:43.550 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:29:43 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:29:43.284+0000 7fb6da600700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:29:43.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:43 smithi046 ceph-mon[21574]: purged_snaps scrub starts 2024-04-20T10:29:43.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:43 smithi046 ceph-mon[21574]: purged_snaps scrub ok 2024-04-20T10:29:43.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:43 smithi046 ceph-mon[21574]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:43.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:43 smithi046 ceph-mon[21574]: osdmap e66: 10 total, 9 up, 10 in 2024-04-20T10:29:43.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:43 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:43.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:43 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:43.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:43 smithi046 ceph-mon[21574]: from='osd.9 ' entity='osd.9' 2024-04-20T10:29:43.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:43 smithi042 ceph-mon[17078]: purged_snaps scrub starts 2024-04-20T10:29:43.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:43 smithi042 ceph-mon[17078]: purged_snaps scrub ok 2024-04-20T10:29:43.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:43 smithi042 ceph-mon[17078]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:43.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:43 smithi042 ceph-mon[17078]: osdmap e66: 10 total, 9 up, 10 in 2024-04-20T10:29:43.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:43 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:43.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:43 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:43.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:43 smithi042 ceph-mon[17078]: from='osd.9 ' entity='osd.9' 2024-04-20T10:29:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:43 smithi162 ceph-mon[21854]: purged_snaps scrub starts 2024-04-20T10:29:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:43 smithi162 ceph-mon[21854]: purged_snaps scrub ok 2024-04-20T10:29:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:43 smithi162 ceph-mon[21854]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:43 smithi162 ceph-mon[21854]: osdmap e66: 10 total, 9 up, 10 in 2024-04-20T10:29:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:43 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:43 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:43 smithi162 ceph-mon[21854]: from='osd.9 ' entity='osd.9' 2024-04-20T10:29:44.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:44.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:44 smithi042 ceph-mon[17078]: osd.9 [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076] boot 2024-04-20T10:29:44.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:44 smithi042 ceph-mon[17078]: osdmap e67: 10 total, 10 up, 10 in 2024-04-20T10:29:44.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:44 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:44.683 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:44.683 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:44 smithi046 ceph-mon[21574]: osd.9 [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076] boot 2024-04-20T10:29:44.683 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:44 smithi046 ceph-mon[21574]: osdmap e67: 10 total, 10 up, 10 in 2024-04-20T10:29:44.683 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:44 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:44.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:44.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:44 smithi162 ceph-mon[21854]: osd.9 [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076] boot 2024-04-20T10:29:44.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:44 smithi162 ceph-mon[21854]: osdmap e67: 10 total, 10 up, 10 in 2024-04-20T10:29:44.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:44 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:29:45.140 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-20T10:29:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-20T10:29:45.141 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-04-20T10:29:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-04-20T10:29:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00896747 s, 1.2 GB/s 2024-04-20T10:29:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-04-20T10:29:45.787 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:45 smithi046 ceph-mon[21574]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:45.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_1 2024-04-20T10:29:45.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:45 smithi042 ceph-mon[17078]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:45 smithi162 ceph-mon[21854]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-20T10:29:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T10:29:46.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:46 smithi046 ceph-mon[21574]: osdmap e68: 10 total, 10 up, 10 in 2024-04-20T10:29:46.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:46 smithi042 ceph-mon[17078]: osdmap e68: 10 total, 10 up, 10 in 2024-04-20T10:29:46.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:46 smithi162 ceph-mon[21854]: osdmap e68: 10 total, 10 up, 10 in 2024-04-20T10:29:47.751 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:47 smithi046 ceph-mon[21574]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:47 smithi042 ceph-mon[17078]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:47.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:47 smithi162 ceph-mon[21854]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:47.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.968+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1975827810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc934103cd0 msgr2=0x7fc934104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:47.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.968+0000 7fc93a4b7700 1 --2- 172.21.15.46:0/1975827810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc934103cd0 0x7fc934104130 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc9280099c0 tx=0x7fc928009cd0 comp rx=0 tx=0).stop 2024-04-20T10:29:47.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.968+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1975827810 shutdown_connections 2024-04-20T10:29:47.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.968+0000 7fc93a4b7700 1 --2- 172.21.15.46:0/1975827810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc934104670 0x7fc93410af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:47.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.968+0000 7fc93a4b7700 1 --2- 172.21.15.46:0/1975827810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc934102ad0 0x7fc934102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:47.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.968+0000 7fc93a4b7700 1 --2- 172.21.15.46:0/1975827810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc934103cd0 0x7fc934104130 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:47.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.968+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1975827810 >> 172.21.15.46:0/1975827810 conn(0x7fc9340fe070 msgr2=0x7fc9341004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:47.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.968+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1975827810 shutdown_connections 2024-04-20T10:29:47.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.968+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1975827810 wait complete. 2024-04-20T10:29:47.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.969+0000 7fc93a4b7700 1 Processor -- start 2024-04-20T10:29:47.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.969+0000 7fc93a4b7700 1 -- start start 2024-04-20T10:29:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.969+0000 7fc93a4b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc934102ad0 0x7fc934198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.969+0000 7fc93a4b7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc934103cd0 0x7fc934199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.969+0000 7fc93a4b7700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc934104670 0x7fc9341a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.969+0000 7fc93a4b7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc93410e730 con 0x7fc934103cd0 2024-04-20T10:29:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.969+0000 7fc93a4b7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc93410e5b0 con 0x7fc934102ad0 2024-04-20T10:29:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.969+0000 7fc93a4b7700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc93410e8b0 con 0x7fc934104670 2024-04-20T10:29:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.969+0000 7fc933fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc934102ad0 0x7fc934198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:47.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.969+0000 7fc933fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc934102ad0 0x7fc934198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33694/0 (socket says 172.21.15.46:33694) 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.969+0000 7fc933fff700 1 -- 172.21.15.46:0/1525016044 learned_addr learned my addr 172.21.15.46:0/1525016044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.969+0000 7fc938a54700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc934104670 0x7fc9341a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.970+0000 7fc933fff700 1 -- 172.21.15.46:0/1525016044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc934104670 msgr2=0x7fc9341a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.970+0000 7fc933fff700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc934104670 0x7fc9341a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.970+0000 7fc933fff700 1 -- 172.21.15.46:0/1525016044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc934103cd0 msgr2=0x7fc934199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.970+0000 7fc933fff700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc934103cd0 0x7fc934199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.970+0000 7fc933fff700 1 -- 172.21.15.46:0/1525016044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc934074c80 con 0x7fc934102ad0 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.970+0000 7fc938a54700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc934104670 0x7fc9341a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.970+0000 7fc933fff700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc934102ad0 0x7fc934198d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc92400e950 tx=0x7fc92400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.970+0000 7fc9317fa700 1 -- 172.21.15.46:0/1525016044 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92400cbd0 con 0x7fc934102ad0 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.970+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1525016044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc934074f70 con 0x7fc934102ad0 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.970+0000 7fc9317fa700 1 -- 172.21.15.46:0/1525016044 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc924004510 con 0x7fc934102ad0 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.970+0000 7fc9317fa700 1 -- 172.21.15.46:0/1525016044 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924010430 con 0x7fc934102ad0 2024-04-20T10:29:47.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.970+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1525016044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc934075530 con 0x7fc934102ad0 2024-04-20T10:29:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.971+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1525016044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc934069740 con 0x7fc934102ad0 2024-04-20T10:29:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.972+0000 7fc9317fa700 1 -- 172.21.15.46:0/1525016044 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc924003f30 con 0x7fc934102ad0 2024-04-20T10:29:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.973+0000 7fc9317fa700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc91c0a3c40 0x7fc91c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.973+0000 7fc9317fa700 1 -- 172.21.15.46:0/1525016044 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7fc9240d0150 con 0x7fc934102ad0 2024-04-20T10:29:47.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.973+0000 7fc9337fe700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc91c0a3c40 0x7fc91c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.974+0000 7fc9337fe700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc91c0a3c40 0x7fc91c0a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc928003790 tx=0x7fc928000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:47.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:47.976+0000 7fc9317fa700 1 -- 172.21.15.46:0/1525016044 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc92409d290 con 0x7fc934102ad0 2024-04-20T10:29:48.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:29:48.195+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1525016044 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc934109a60 con 0x7fc91c0a3c40 2024-04-20T10:29:48.892 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: Detected new or changed devices on smithi042 2024-04-20T10:29:48.892 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:48.892 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:48.892 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:48.892 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:48.892 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:48.892 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:48.892 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi042 to 4273M 2024-04-20T10:29:48.892 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:48.892 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:48.892 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:29:48.893 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:48.893 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:29:48.893 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:48.893 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:29:48.893 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:48.893 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:29:48.893 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:29:48.893 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:29:48.893 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:48 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:48.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: Detected new or changed devices on smithi042 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi042 to 4273M 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:29:48.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:29:48.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:29:48.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:48 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:49.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: Detected new or changed devices on smithi042 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi042 to 4273M 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:49.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:29:49.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:29:49.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:29:49.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:29:49.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:29:49.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:48 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:50.292 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:50 smithi046 ceph-mon[21574]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:50.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:50 smithi042 ceph-mon[17078]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:50.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:50 smithi162 ceph-mon[21854]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:52.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:52 smithi046 ceph-mon[21574]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:52.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:52 smithi046 ceph-mon[21574]: from='client.? 172.21.15.46:0/4107577649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d4fba56-28b6-449d-9b0c-fc927cbe6aa7"}]: dispatch 2024-04-20T10:29:52.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:52 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d4fba56-28b6-449d-9b0c-fc927cbe6aa7"}]: dispatch 2024-04-20T10:29:52.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:52 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d4fba56-28b6-449d-9b0c-fc927cbe6aa7"}]': finished 2024-04-20T10:29:52.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:52 smithi046 ceph-mon[21574]: osdmap e69: 11 total, 10 up, 11 in 2024-04-20T10:29:52.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:52 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:29:52.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:52 smithi046 ceph-mon[21574]: from='client.? 172.21.15.46:0/759963030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:29:52.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:52 smithi042 ceph-mon[17078]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:52.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:52 smithi042 ceph-mon[17078]: from='client.? 172.21.15.46:0/4107577649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d4fba56-28b6-449d-9b0c-fc927cbe6aa7"}]: dispatch 2024-04-20T10:29:52.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:52 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d4fba56-28b6-449d-9b0c-fc927cbe6aa7"}]: dispatch 2024-04-20T10:29:52.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:52 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d4fba56-28b6-449d-9b0c-fc927cbe6aa7"}]': finished 2024-04-20T10:29:52.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:52 smithi042 ceph-mon[17078]: osdmap e69: 11 total, 10 up, 11 in 2024-04-20T10:29:52.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:52 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:29:52.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:52 smithi042 ceph-mon[17078]: from='client.? 172.21.15.46:0/759963030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:29:52.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:52 smithi162 ceph-mon[21854]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:52.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:52 smithi162 ceph-mon[21854]: from='client.? 172.21.15.46:0/4107577649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d4fba56-28b6-449d-9b0c-fc927cbe6aa7"}]: dispatch 2024-04-20T10:29:52.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:52 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d4fba56-28b6-449d-9b0c-fc927cbe6aa7"}]: dispatch 2024-04-20T10:29:52.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:52 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d4fba56-28b6-449d-9b0c-fc927cbe6aa7"}]': finished 2024-04-20T10:29:52.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:52 smithi162 ceph-mon[21854]: osdmap e69: 11 total, 10 up, 11 in 2024-04-20T10:29:52.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:52 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:29:52.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:52 smithi162 ceph-mon[21854]: from='client.? 172.21.15.46:0/759963030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:29:54.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:54 smithi046 ceph-mon[21574]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:54.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:54 smithi042 ceph-mon[17078]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:54.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:54 smithi162 ceph-mon[21854]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:56.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:56 smithi046 ceph-mon[21574]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:56.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:56 smithi042 ceph-mon[17078]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:56.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:56 smithi162 ceph-mon[21854]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:58.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:58 smithi046 ceph-mon[21574]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:58.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:58 smithi042 ceph-mon[17078]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:58.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:58 smithi162 ceph-mon[21854]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:29:59.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:59 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-20T10:29:59.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:29:59 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:59.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:59 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-20T10:29:59.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:29:59 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:29:59.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:59 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-20T10:29:59.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:29:59 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:00.183 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:00 smithi046 ceph-mon[21574]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:00.183 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:00 smithi046 ceph-mon[21574]: Deploying daemon osd.10 on smithi046 2024-04-20T10:30:00.183 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:00 smithi046 ceph-mon[21574]: overall HEALTH_OK 2024-04-20T10:30:00.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:00 smithi042 ceph-mon[17078]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:00.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:00 smithi042 ceph-mon[17078]: Deploying daemon osd.10 on smithi046 2024-04-20T10:30:00.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:00 smithi042 ceph-mon[17078]: overall HEALTH_OK 2024-04-20T10:30:00.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:00 smithi162 ceph-mon[21854]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:00.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:00 smithi162 ceph-mon[21854]: Deploying daemon osd.10 on smithi046 2024-04-20T10:30:00.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:00 smithi162 ceph-mon[21854]: overall HEALTH_OK 2024-04-20T10:30:02.105 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:02 smithi046 ceph-mon[21574]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:02.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:02 smithi042 ceph-mon[17078]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:02.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:02 smithi162 ceph-mon[21854]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:04.274 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:04 smithi046 ceph-mon[21574]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:04.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:04 smithi042 ceph-mon[17078]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:04.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:04 smithi162 ceph-mon[21854]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:05 smithi162 ceph-mon[21854]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:05 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:05 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:06.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:05 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:05 smithi046 ceph-mon[21574]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:05 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:05 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:05 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:06.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:05 smithi042 ceph-mon[17078]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:06.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:05 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:06.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:05 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:06.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:05 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:07 smithi162 ceph-mon[21854]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:08.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:07 smithi046 ceph-mon[21574]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:08.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:07 smithi042 ceph-mon[17078]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.938+0000 7fc9317fa700 1 -- 172.21.15.46:0/1525016044 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc934109a60 con 0x7fc91c0a3c40 2024-04-20T10:30:08.938 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 10 on host 'smithi046' 2024-04-20T10:30:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.940+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1525016044 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc91c0a3c40 msgr2=0x7fc91c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.940+0000 7fc93a4b7700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc91c0a3c40 0x7fc91c0a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc928003790 tx=0x7fc928000bc0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.940+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1525016044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc934102ad0 msgr2=0x7fc934198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.940+0000 7fc93a4b7700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc934102ad0 0x7fc934198d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc92400e950 tx=0x7fc92400ed10 comp rx=0 tx=0).stop 2024-04-20T10:30:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.940+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1525016044 shutdown_connections 2024-04-20T10:30:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.940+0000 7fc93a4b7700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc91c0a3c40 0x7fc91c0a6100 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.940+0000 7fc93a4b7700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc934102ad0 0x7fc934198d00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.940+0000 7fc93a4b7700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc934104670 0x7fc9341a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.940+0000 7fc93a4b7700 1 --2- 172.21.15.46:0/1525016044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc934103cd0 0x7fc934199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.940+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1525016044 >> 172.21.15.46:0/1525016044 conn(0x7fc9340fe070 msgr2=0x7fc934106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.940+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1525016044 shutdown_connections 2024-04-20T10:30:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-20T10:30:08.941+0000 7fc93a4b7700 1 -- 172.21.15.46:0/1525016044 wait complete. 2024-04-20T10:30:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:08 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:08 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:08 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:08 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:08 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:08 smithi046 ceph-mon[21574]: from='osd.10 [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T10:30:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:08 smithi046 ceph-mon[21574]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T10:30:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:08 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:08 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:08 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:08 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:08 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:09.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:08 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:09.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:08 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:08 smithi042 ceph-mon[17078]: from='osd.10 [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T10:30:09.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:08 smithi042 ceph-mon[17078]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T10:30:09.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:08 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:08 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:08 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:08 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:08 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:08 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:08 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:08 smithi162 ceph-mon[21854]: from='osd.10 [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T10:30:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:08 smithi162 ceph-mon[21854]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T10:30:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:08 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:08 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:09.603 DEBUG:teuthology.orchestra.run.smithi046:osd.10> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.10.service 2024-04-20T10:30:09.605 INFO:tasks.cephadm:Deploying osd.11 on smithi162 with /dev/vg_nvme/lv_1... 2024-04-20T10:30:09.606 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-20T10:30:09.851 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:30:10.205 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:09 smithi162 ceph-mon[21854]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:10.205 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:09 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:10.205 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:09 smithi162 ceph-mon[21854]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-20T10:30:10.205 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:09 smithi162 ceph-mon[21854]: from='osd.10 [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:30:10.205 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:09 smithi162 ceph-mon[21854]: osdmap e70: 11 total, 10 up, 11 in 2024-04-20T10:30:10.205 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:09 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:30:10.205 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:09 smithi162 ceph-mon[21854]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:30:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:09 smithi046 ceph-mon[21574]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:09 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:09 smithi046 ceph-mon[21574]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-20T10:30:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:09 smithi046 ceph-mon[21574]: from='osd.10 [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:30:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:09 smithi046 ceph-mon[21574]: osdmap e70: 11 total, 10 up, 11 in 2024-04-20T10:30:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:09 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:30:10.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:09 smithi046 ceph-mon[21574]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:30:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:09 smithi042 ceph-mon[17078]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:09 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:09 smithi042 ceph-mon[17078]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-20T10:30:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:09 smithi042 ceph-mon[17078]: from='osd.10 [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:30:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:09 smithi042 ceph-mon[17078]: osdmap e70: 11 total, 10 up, 11 in 2024-04-20T10:30:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:09 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:30:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:09 smithi042 ceph-mon[17078]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-04-20T10:30:10.803 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:30:10 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:30:10.461+0000 7faf1c3d0700 -1 osd.10 0 waiting for initial osdmap 2024-04-20T10:30:10.803 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:30:10 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:30:10.476+0000 7faf141bb700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:30:11.273 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:10 smithi162 ceph-mon[21854]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-20T10:30:11.273 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:10 smithi162 ceph-mon[21854]: osdmap e71: 11 total, 10 up, 11 in 2024-04-20T10:30:11.273 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:30:11.273 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:10 smithi162 ceph-mon[21854]: from='osd.10 ' entity='osd.10' 2024-04-20T10:30:11.292 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:10 smithi046 ceph-mon[21574]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-20T10:30:11.292 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:10 smithi046 ceph-mon[21574]: osdmap e71: 11 total, 10 up, 11 in 2024-04-20T10:30:11.292 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:30:11.292 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:10 smithi046 ceph-mon[21574]: from='osd.10 ' entity='osd.10' 2024-04-20T10:30:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:10 smithi042 ceph-mon[17078]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-04-20T10:30:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:10 smithi042 ceph-mon[17078]: osdmap e71: 11 total, 10 up, 11 in 2024-04-20T10:30:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:30:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:10 smithi042 ceph-mon[17078]: from='osd.10 ' entity='osd.10' 2024-04-20T10:30:11.964 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-20T10:30:11.964 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-20T10:30:11.964 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-04-20T10:30:11.964 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-04-20T10:30:11.964 INFO:teuthology.orchestra.run.smithi162.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112228 s, 934 MB/s 2024-04-20T10:30:11.965 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 2024-04-20T10:30:11.965 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-04-20T10:30:12.262 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:11 smithi162 ceph-mon[21854]: purged_snaps scrub starts 2024-04-20T10:30:12.263 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:11 smithi162 ceph-mon[21854]: purged_snaps scrub ok 2024-04-20T10:30:12.263 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:11 smithi162 ceph-mon[21854]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:12.263 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:11 smithi162 ceph-mon[21854]: osd.10 [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617] boot 2024-04-20T10:30:12.263 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:11 smithi162 ceph-mon[21854]: osdmap e72: 11 total, 11 up, 11 in 2024-04-20T10:30:12.263 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:30:12.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:11 smithi046 ceph-mon[21574]: purged_snaps scrub starts 2024-04-20T10:30:12.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:11 smithi046 ceph-mon[21574]: purged_snaps scrub ok 2024-04-20T10:30:12.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:11 smithi046 ceph-mon[21574]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:12.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:11 smithi046 ceph-mon[21574]: osd.10 [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617] boot 2024-04-20T10:30:12.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:11 smithi046 ceph-mon[21574]: osdmap e72: 11 total, 11 up, 11 in 2024-04-20T10:30:12.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:30:12.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:11 smithi042 ceph-mon[17078]: purged_snaps scrub starts 2024-04-20T10:30:12.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:11 smithi042 ceph-mon[17078]: purged_snaps scrub ok 2024-04-20T10:30:12.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:11 smithi042 ceph-mon[17078]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-20T10:30:12.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:11 smithi042 ceph-mon[17078]: osd.10 [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617] boot 2024-04-20T10:30:12.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:11 smithi042 ceph-mon[17078]: osdmap e72: 11 total, 11 up, 11 in 2024-04-20T10:30:12.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:30:12.631 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_1 2024-04-20T10:30:12.876 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:30:13.255 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:12 smithi162 ceph-mon[21854]: osdmap e73: 11 total, 11 up, 11 in 2024-04-20T10:30:13.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:12 smithi046 ceph-mon[21574]: osdmap e73: 11 total, 11 up, 11 in 2024-04-20T10:30:13.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:12 smithi042 ceph-mon[17078]: osdmap e73: 11 total, 11 up, 11 in 2024-04-20T10:30:14.287 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:14.287 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: osdmap e74: 11 total, 11 up, 11 in 2024-04-20T10:30:14.287 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: Detected new or changed devices on smithi046 2024-04-20T10:30:14.287 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:14.288 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:14.288 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:14.288 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:14.288 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:14.288 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:14.288 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi046 to 4273M 2024-04-20T10:30:14.288 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:14.288 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:14.288 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:14.288 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:14.288 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:13 smithi162 ceph-mon[21854]: osdmap e75: 11 total, 11 up, 11 in 2024-04-20T10:30:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: osdmap e74: 11 total, 11 up, 11 in 2024-04-20T10:30:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: Detected new or changed devices on smithi046 2024-04-20T10:30:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi046 to 4273M 2024-04-20T10:30:14.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:14.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:14.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:14.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:14.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:13 smithi046 ceph-mon[21574]: osdmap e75: 11 total, 11 up, 11 in 2024-04-20T10:30:14.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:14.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: osdmap e74: 11 total, 11 up, 11 in 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: Detected new or changed devices on smithi046 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi046 to 4273M 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:13 smithi042 ceph-mon[17078]: osdmap e75: 11 total, 11 up, 11 in 2024-04-20T10:30:14.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.524+0000 7f8df7ada700 1 -- 172.21.15.162:0/1633710380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df01035a0 msgr2=0x7f8df0105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:14.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.524+0000 7f8df7ada700 1 --2- 172.21.15.162:0/1633710380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df01035a0 0x7f8df0105990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009990 tx=0x7f8dec009ca0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.524+0000 7f8df7ada700 1 -- 172.21.15.162:0/1633710380 shutdown_connections 2024-04-20T10:30:14.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.524+0000 7f8df7ada700 1 --2- 172.21.15.162:0/1633710380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df0108800 0x7f8df010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.524+0000 7f8df7ada700 1 --2- 172.21.15.162:0/1633710380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df01035a0 0x7f8df0105990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.524+0000 7f8df7ada700 1 --2- 172.21.15.162:0/1633710380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df0105ed0 0x7f8df01082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.524+0000 7f8df7ada700 1 -- 172.21.15.162:0/1633710380 >> 172.21.15.162:0/1633710380 conn(0x7f8df00fcf50 msgr2=0x7f8df00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.524+0000 7f8df7ada700 1 -- 172.21.15.162:0/1633710380 shutdown_connections 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.524+0000 7f8df7ada700 1 -- 172.21.15.162:0/1633710380 wait complete. 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df7ada700 1 Processor -- start 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df7ada700 1 -- start start 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df7ada700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df01035a0 0x7f8df0198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df7ada700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df0105ed0 0x7f8df01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df7ada700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df0108800 0x7f8df01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df7ada700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8df010e720 con 0x7f8df0105ed0 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df7ada700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8df010e5a0 con 0x7f8df01035a0 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df7ada700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8df010e8a0 con 0x7f8df0108800 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df6077700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df0108800 0x7f8df01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:14.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df6077700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df0108800 0x7f8df01a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:40530/0 (socket says 172.21.15.162:40530) 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df6077700 1 -- 172.21.15.162:0/3836381194 learned_addr learned my addr 172.21.15.162:0/3836381194 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df5876700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df01035a0 0x7f8df0198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df6077700 1 -- 172.21.15.162:0/3836381194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df01035a0 msgr2=0x7f8df0198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df6077700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df01035a0 0x7f8df0198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df6077700 1 -- 172.21.15.162:0/3836381194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df0105ed0 msgr2=0x7f8df01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df6077700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df0105ed0 0x7f8df01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df6077700 1 -- 172.21.15.162:0/3836381194 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df01a09c0 con 0x7f8df0108800 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8df6077700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df0108800 0x7f8df01a0320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8de8007ee0 tx=0x7f8de800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.525+0000 7f8de6ffd700 1 -- 172.21.15.162:0/3836381194 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de8004db0 con 0x7f8df0108800 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.526+0000 7f8df7ada700 1 -- 172.21.15.162:0/3836381194 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df01a0cb0 con 0x7f8df0108800 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.526+0000 7f8de6ffd700 1 -- 172.21.15.162:0/3836381194 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8de800dbd0 con 0x7f8df0108800 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.526+0000 7f8de6ffd700 1 -- 172.21.15.162:0/3836381194 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de8015660 con 0x7f8df0108800 2024-04-20T10:30:14.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.526+0000 7f8df7ada700 1 -- 172.21.15.162:0/3836381194 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df01a1260 con 0x7f8df0108800 2024-04-20T10:30:14.529 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.526+0000 7f8df7ada700 1 -- 172.21.15.162:0/3836381194 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df002df90 con 0x7f8df0108800 2024-04-20T10:30:14.529 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.529+0000 7f8de6ffd700 1 -- 172.21.15.162:0/3836381194 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8de8005000 con 0x7f8df0108800 2024-04-20T10:30:14.529 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.529+0000 7f8de6ffd700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8ddc0a3c40 0x7f8ddc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:14.529 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.529+0000 7f8de6ffd700 1 -- 172.21.15.162:0/3836381194 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7f8de80d0080 con 0x7f8df0108800 2024-04-20T10:30:14.530 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.530+0000 7f8df5876700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8ddc0a3c40 0x7f8ddc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:14.531 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.531+0000 7f8df5876700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8ddc0a3c40 0x7f8ddc0a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0039c0 tx=0x7f8dec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:14.532 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.533+0000 7f8de6ffd700 1 -- 172.21.15.162:0/3836381194 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8de8010070 con 0x7f8df0108800 2024-04-20T10:30:14.741 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:14.741+0000 7f8df7ada700 1 -- 172.21.15.162:0/3836381194 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8df0062e90 con 0x7f8ddc0a3c40 2024-04-20T10:30:15.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:15 smithi162 ceph-mon[21854]: pgmap v267: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-04-20T10:30:15.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:15 smithi162 ceph-mon[21854]: osdmap e76: 11 total, 11 up, 11 in 2024-04-20T10:30:15.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:15 smithi162 ceph-mon[21854]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:30:15.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:30:15.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:30:15.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:15 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:15.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:15 smithi162 ceph-mon[21854]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-20T10:30:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:15 smithi046 ceph-mon[21574]: pgmap v267: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-04-20T10:30:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:15 smithi046 ceph-mon[21574]: osdmap e76: 11 total, 11 up, 11 in 2024-04-20T10:30:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:15 smithi046 ceph-mon[21574]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:30:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:30:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:30:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:15 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:15 smithi046 ceph-mon[21574]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-20T10:30:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:15 smithi042 ceph-mon[17078]: pgmap v267: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-04-20T10:30:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:15 smithi042 ceph-mon[17078]: osdmap e76: 11 total, 11 up, 11 in 2024-04-20T10:30:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:15 smithi042 ceph-mon[17078]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:30:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T10:30:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T10:30:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:15 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:15 smithi042 ceph-mon[17078]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-20T10:30:18.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:17 smithi046 ceph-mon[21574]: pgmap v269: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-04-20T10:30:18.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:17 smithi046 ceph-mon[21574]: from='client.? 172.21.15.162:0/2452798391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b31e91e-bfbb-4672-b2c3-55fc49d9aae8"}]: dispatch 2024-04-20T10:30:18.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:17 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b31e91e-bfbb-4672-b2c3-55fc49d9aae8"}]: dispatch 2024-04-20T10:30:18.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:17 smithi046 ceph-mon[21574]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b31e91e-bfbb-4672-b2c3-55fc49d9aae8"}]': finished 2024-04-20T10:30:18.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:17 smithi046 ceph-mon[21574]: osdmap e77: 12 total, 11 up, 12 in 2024-04-20T10:30:18.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:17 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:18.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:17 smithi042 ceph-mon[17078]: pgmap v269: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-04-20T10:30:18.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:17 smithi042 ceph-mon[17078]: from='client.? 172.21.15.162:0/2452798391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b31e91e-bfbb-4672-b2c3-55fc49d9aae8"}]: dispatch 2024-04-20T10:30:18.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:17 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b31e91e-bfbb-4672-b2c3-55fc49d9aae8"}]: dispatch 2024-04-20T10:30:18.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:17 smithi042 ceph-mon[17078]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b31e91e-bfbb-4672-b2c3-55fc49d9aae8"}]': finished 2024-04-20T10:30:18.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:17 smithi042 ceph-mon[17078]: osdmap e77: 12 total, 11 up, 12 in 2024-04-20T10:30:18.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:18.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:17 smithi162 ceph-mon[21854]: pgmap v269: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-04-20T10:30:18.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:17 smithi162 ceph-mon[21854]: from='client.? 172.21.15.162:0/2452798391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b31e91e-bfbb-4672-b2c3-55fc49d9aae8"}]: dispatch 2024-04-20T10:30:18.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:17 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b31e91e-bfbb-4672-b2c3-55fc49d9aae8"}]: dispatch 2024-04-20T10:30:18.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:17 smithi162 ceph-mon[21854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b31e91e-bfbb-4672-b2c3-55fc49d9aae8"}]': finished 2024-04-20T10:30:18.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:17 smithi162 ceph-mon[21854]: osdmap e77: 12 total, 11 up, 12 in 2024-04-20T10:30:18.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:17 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:19.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:18 smithi046 ceph-mon[21574]: from='client.? 172.21.15.162:0/507015046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:30:19.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:18 smithi042 ceph-mon[17078]: from='client.? 172.21.15.162:0/507015046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:30:19.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:18 smithi162 ceph-mon[21854]: from='client.? 172.21.15.162:0/507015046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T10:30:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:19 smithi046 ceph-mon[21574]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-20T10:30:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:19 smithi046 ceph-mon[21574]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-20T10:30:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:19 smithi046 ceph-mon[21574]: Cluster is now healthy 2024-04-20T10:30:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:19 smithi042 ceph-mon[17078]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-20T10:30:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:19 smithi042 ceph-mon[17078]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-20T10:30:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:19 smithi042 ceph-mon[17078]: Cluster is now healthy 2024-04-20T10:30:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:19 smithi162 ceph-mon[21854]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-20T10:30:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:19 smithi162 ceph-mon[21854]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-20T10:30:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:19 smithi162 ceph-mon[21854]: Cluster is now healthy 2024-04-20T10:30:21.872 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:21 smithi162 ceph-mon[21854]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-20T10:30:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:21 smithi046 ceph-mon[21574]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-20T10:30:22.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:21 smithi042 ceph-mon[17078]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-20T10:30:23.890 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:23 smithi162 ceph-mon[21854]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T10:30:24.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:23 smithi046 ceph-mon[21574]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T10:30:24.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:23 smithi042 ceph-mon[17078]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T10:30:25.908 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:25 smithi162 ceph-mon[21854]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-20T10:30:25.908 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:25 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-20T10:30:25.908 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:25 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:25.908 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:25 smithi162 ceph-mon[21854]: Deploying daemon osd.11 on smithi162 2024-04-20T10:30:26.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:25 smithi046 ceph-mon[21574]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-20T10:30:26.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:25 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-20T10:30:26.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:25 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:26.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:25 smithi046 ceph-mon[21574]: Deploying daemon osd.11 on smithi162 2024-04-20T10:30:26.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:25 smithi042 ceph-mon[17078]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-20T10:30:26.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:25 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-20T10:30:26.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:25 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:26.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:25 smithi042 ceph-mon[17078]: Deploying daemon osd.11 on smithi162 2024-04-20T10:30:28.023 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:27 smithi162 ceph-mon[21854]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T10:30:28.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:27 smithi046 ceph-mon[21574]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T10:30:28.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:27 smithi042 ceph-mon[17078]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T10:30:29.891 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:29 smithi162 ceph-mon[21854]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-20T10:30:30.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:29 smithi046 ceph-mon[21574]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-20T10:30:30.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:29 smithi042 ceph-mon[17078]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-20T10:30:32.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:31 smithi046 ceph-mon[21574]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:32.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:32.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:32.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:31 smithi042 ceph-mon[17078]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:32.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:31 smithi162 ceph-mon[21854]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:32.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:32.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:32.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:33 smithi162 ceph-mon[21854]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:34.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:33 smithi046 ceph-mon[21574]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:34.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:33 smithi042 ceph-mon[17078]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:34.732 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 11 on host 'smithi162' 2024-04-20T10:30:34.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.730+0000 7f8de6ffd700 1 -- 172.21.15.162:0/3836381194 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8df0062e90 con 0x7f8ddc0a3c40 2024-04-20T10:30:34.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.733+0000 7f8df7ada700 1 -- 172.21.15.162:0/3836381194 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8ddc0a3c40 msgr2=0x7f8ddc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:34.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.733+0000 7f8df7ada700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8ddc0a3c40 0x7f8ddc0a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0039c0 tx=0x7f8dec005fb0 comp rx=0 tx=0).stop 2024-04-20T10:30:34.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.733+0000 7f8df7ada700 1 -- 172.21.15.162:0/3836381194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df0108800 msgr2=0x7f8df01a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:34.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.733+0000 7f8df7ada700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df0108800 0x7f8df01a0320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8de8007ee0 tx=0x7f8de800d3b0 comp rx=0 tx=0).stop 2024-04-20T10:30:34.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.733+0000 7f8df7ada700 1 -- 172.21.15.162:0/3836381194 shutdown_connections 2024-04-20T10:30:34.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.733+0000 7f8df7ada700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8ddc0a3c40 0x7f8ddc0a6100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:34.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.733+0000 7f8df7ada700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df01035a0 0x7f8df0198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:34.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.733+0000 7f8df7ada700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8df0108800 0x7f8df01a0320 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:34.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.733+0000 7f8df7ada700 1 --2- 172.21.15.162:0/3836381194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df0105ed0 0x7f8df01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:34.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.733+0000 7f8df7ada700 1 -- 172.21.15.162:0/3836381194 >> 172.21.15.162:0/3836381194 conn(0x7f8df00fcf50 msgr2=0x7f8df0104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:34.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.733+0000 7f8df7ada700 1 -- 172.21.15.162:0/3836381194 shutdown_connections 2024-04-20T10:30:34.735 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:34.733+0000 7f8df7ada700 1 -- 172.21.15.162:0/3836381194 wait complete. 2024-04-20T10:30:35.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:35.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:35.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:34 smithi046 ceph-mon[21574]: from='osd.11 [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T10:30:35.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:34 smithi046 ceph-mon[21574]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T10:30:35.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:35.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:34 smithi042 ceph-mon[17078]: from='osd.11 [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T10:30:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:34 smithi042 ceph-mon[17078]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T10:30:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:35.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:35.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:35.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:34 smithi162 ceph-mon[21854]: from='osd.11 [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T10:30:35.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:34 smithi162 ceph-mon[21854]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T10:30:35.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:35.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:35.466 DEBUG:teuthology.orchestra.run.smithi162:osd.11> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.11.service 2024-04-20T10:30:35.469 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-20T10:30:35.470 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd stat -f json 2024-04-20T10:30:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:30:35.963 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:35 smithi162 ceph-mon[21854]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:35.963 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:35 smithi162 ceph-mon[21854]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-20T10:30:35.964 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:35 smithi162 ceph-mon[21854]: from='osd.11 [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:30:35.964 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:35 smithi162 ceph-mon[21854]: osdmap e78: 12 total, 11 up, 12 in 2024-04-20T10:30:35.964 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:35 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:35.964 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:35 smithi162 ceph-mon[21854]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:30:35.964 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:30:35 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:30:35.942+0000 7f9bc4abd700 -1 osd.11 0 waiting for initial osdmap 2024-04-20T10:30:35.964 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:30:35 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:30:35.962+0000 7f9bc08b0700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:30:36.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:35 smithi042 ceph-mon[17078]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:36.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:35 smithi042 ceph-mon[17078]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-20T10:30:36.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:35 smithi042 ceph-mon[17078]: from='osd.11 [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:30:36.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:35 smithi042 ceph-mon[17078]: osdmap e78: 12 total, 11 up, 12 in 2024-04-20T10:30:36.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:35 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:36.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:35 smithi042 ceph-mon[17078]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:30:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:35 smithi046 ceph-mon[21574]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:35 smithi046 ceph-mon[21574]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-20T10:30:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:35 smithi046 ceph-mon[21574]: from='osd.11 [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:30:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:35 smithi046 ceph-mon[21574]: osdmap e78: 12 total, 11 up, 12 in 2024-04-20T10:30:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:35 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:35 smithi046 ceph-mon[21574]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-04-20T10:30:37.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:36 smithi042 ceph-mon[17078]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-20T10:30:37.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:36 smithi042 ceph-mon[17078]: osdmap e79: 12 total, 11 up, 12 in 2024-04-20T10:30:37.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:36 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:37.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:36 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:37.224 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:36 smithi162 ceph-mon[21854]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-20T10:30:37.224 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:36 smithi162 ceph-mon[21854]: osdmap e79: 12 total, 11 up, 12 in 2024-04-20T10:30:37.224 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:36 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:37.224 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:36 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:37.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:36 smithi046 ceph-mon[21574]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-04-20T10:30:37.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:36 smithi046 ceph-mon[21574]: osdmap e79: 12 total, 11 up, 12 in 2024-04-20T10:30:37.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:36 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:37.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:36 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:38.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.144+0000 7f5459a00700 1 -- 172.21.15.42:0/3671156405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f545410a8b0 msgr2=0x7f545410cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:38.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.144+0000 7f5459a00700 1 --2- 172.21.15.42:0/3671156405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f545410a8b0 0x7f545410cd40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f544c00b0a0 tx=0x7f544c00b3b0 comp rx=0 tx=0).stop 2024-04-20T10:30:38.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.144+0000 7f5459a00700 1 -- 172.21.15.42:0/3671156405 shutdown_connections 2024-04-20T10:30:38.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.144+0000 7f5459a00700 1 --2- 172.21.15.42:0/3671156405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454076a20 0x7f5454074e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:38.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.144+0000 7f5459a00700 1 --2- 172.21.15.42:0/3671156405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f545410a8b0 0x7f545410cd40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:38.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.144+0000 7f5459a00700 1 --2- 172.21.15.42:0/3671156405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5454075450 0x7f54540758d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:38.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.144+0000 7f5459a00700 1 -- 172.21.15.42:0/3671156405 >> 172.21.15.42:0/3671156405 conn(0x7f54540fdf90 msgr2=0x7f54541003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:38.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.145+0000 7f5459a00700 1 -- 172.21.15.42:0/3671156405 shutdown_connections 2024-04-20T10:30:38.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.145+0000 7f5459a00700 1 -- 172.21.15.42:0/3671156405 wait complete. 2024-04-20T10:30:38.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.146+0000 7f5459a00700 1 Processor -- start 2024-04-20T10:30:38.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.146+0000 7f5459a00700 1 -- start start 2024-04-20T10:30:38.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.146+0000 7f5459a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5454075450 0x7f545419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:38.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.146+0000 7f5459a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454076a20 0x7f545419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:38.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.146+0000 7f5459a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f545410a8b0 0x7f54541a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:38.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.146+0000 7f5459a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5454110790 con 0x7f5454075450 2024-04-20T10:30:38.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.146+0000 7f5459a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5454110610 con 0x7f5454076a20 2024-04-20T10:30:38.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.146+0000 7f5459a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5454110910 con 0x7f545410a8b0 2024-04-20T10:30:38.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.147+0000 7f5453e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5454075450 0x7f545419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:38.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.147+0000 7f5453e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5454075450 0x7f545419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60470/0 (socket says 172.21.15.42:60470) 2024-04-20T10:30:38.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.147+0000 7f5453e00700 1 -- 172.21.15.42:0/2589787117 learned_addr learned my addr 172.21.15.42:0/2589787117 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:30:38.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.147+0000 7f5459000700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f545410a8b0 0x7f54541a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:38.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.147+0000 7f5453400700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454076a20 0x7f545419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:38.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.147+0000 7f5459000700 1 -- 172.21.15.42:0/2589787117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454076a20 msgr2=0x7f545419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:38.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.147+0000 7f5459000700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454076a20 0x7f545419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:38.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.147+0000 7f5459000700 1 -- 172.21.15.42:0/2589787117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5454075450 msgr2=0x7f545419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:38.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.147+0000 7f5459000700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5454075450 0x7f545419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:38.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.147+0000 7f5459000700 1 -- 172.21.15.42:0/2589787117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54541a2960 con 0x7f545410a8b0 2024-04-20T10:30:38.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.147+0000 7f5453e00700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5454075450 0x7f545419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:38.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.148+0000 7f5459000700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f545410a8b0 0x7f54541a2290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f544c009bc0 tx=0x7f544c004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:38.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.148+0000 7f5450c00700 1 -- 172.21.15.42:0/2589787117 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c021070 con 0x7f545410a8b0 2024-04-20T10:30:38.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.148+0000 7f5450c00700 1 -- 172.21.15.42:0/2589787117 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f544c007b80 con 0x7f545410a8b0 2024-04-20T10:30:38.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.148+0000 7f5450c00700 1 -- 172.21.15.42:0/2589787117 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c01d450 con 0x7f545410a8b0 2024-04-20T10:30:38.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.148+0000 7f5459a00700 1 -- 172.21.15.42:0/2589787117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54541a2bf0 con 0x7f545410a8b0 2024-04-20T10:30:38.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.148+0000 7f5459a00700 1 -- 172.21.15.42:0/2589787117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54541a3050 con 0x7f545410a8b0 2024-04-20T10:30:38.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:37 smithi162 ceph-mon[21854]: purged_snaps scrub starts 2024-04-20T10:30:38.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:37 smithi162 ceph-mon[21854]: purged_snaps scrub ok 2024-04-20T10:30:38.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:37 smithi162 ceph-mon[21854]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:38.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:38.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:37 smithi162 ceph-mon[21854]: osd.11 [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339] boot 2024-04-20T10:30:38.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:37 smithi162 ceph-mon[21854]: osdmap e80: 12 total, 12 up, 12 in 2024-04-20T10:30:38.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:38.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.149+0000 7f5459a00700 1 -- 172.21.15.42:0/2589787117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5454069740 con 0x7f545410a8b0 2024-04-20T10:30:38.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.151+0000 7f5450c00700 1 -- 172.21.15.42:0/2589787117 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f544c004000 con 0x7f545410a8b0 2024-04-20T10:30:38.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.152+0000 7f5450c00700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f54380a3cc0 0x7f54380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:38.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.153+0000 7f5450c00700 1 -- 172.21.15.42:0/2589787117 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f544c0d0f30 con 0x7f545410a8b0 2024-04-20T10:30:38.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.153+0000 7f5453e00700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f54380a3cc0 0x7f54380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:38.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.153+0000 7f5453e00700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f54380a3cc0 0x7f54380a6180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f545419c1f0 tx=0x7f5444009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:38.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.156+0000 7f5450c00700 1 -- 172.21.15.42:0/2589787117 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f544c09dda0 con 0x7f545410a8b0 2024-04-20T10:30:38.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:37 smithi046 ceph-mon[21574]: purged_snaps scrub starts 2024-04-20T10:30:38.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:37 smithi046 ceph-mon[21574]: purged_snaps scrub ok 2024-04-20T10:30:38.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:37 smithi046 ceph-mon[21574]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:38.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:38.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:37 smithi046 ceph-mon[21574]: osd.11 [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339] boot 2024-04-20T10:30:38.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:37 smithi046 ceph-mon[21574]: osdmap e80: 12 total, 12 up, 12 in 2024-04-20T10:30:38.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:38.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:37 smithi042 ceph-mon[17078]: purged_snaps scrub starts 2024-04-20T10:30:38.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:37 smithi042 ceph-mon[17078]: purged_snaps scrub ok 2024-04-20T10:30:38.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:37 smithi042 ceph-mon[17078]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-20T10:30:38.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:38.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:37 smithi042 ceph-mon[17078]: osd.11 [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339] boot 2024-04-20T10:30:38.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:37 smithi042 ceph-mon[17078]: osdmap e80: 12 total, 12 up, 12 in 2024-04-20T10:30:38.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:30:38.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.394+0000 7f5459a00700 1 -- 172.21.15.42:0/2589787117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f5454063660 con 0x7f545410a8b0 2024-04-20T10:30:38.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.394+0000 7f5450c00700 1 -- 172.21.15.42:0/2589787117 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f544c09d6a0 con 0x7f545410a8b0 2024-04-20T10:30:38.394 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:30:38.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.397+0000 7f5459a00700 1 -- 172.21.15.42:0/2589787117 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f54380a3cc0 msgr2=0x7f54380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.397+0000 7f5459a00700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f54380a3cc0 0x7f54380a6180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f545419c1f0 tx=0x7f5444009260 comp rx=0 tx=0).stop 2024-04-20T10:30:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.397+0000 7f5459a00700 1 -- 172.21.15.42:0/2589787117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f545410a8b0 msgr2=0x7f54541a2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.397+0000 7f5459a00700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f545410a8b0 0x7f54541a2290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f544c009bc0 tx=0x7f544c004d00 comp rx=0 tx=0).stop 2024-04-20T10:30:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.398+0000 7f5459a00700 1 -- 172.21.15.42:0/2589787117 shutdown_connections 2024-04-20T10:30:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.398+0000 7f5459a00700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f54380a3cc0 0x7f54380a6180 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.398+0000 7f5459a00700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f545410a8b0 0x7f54541a2290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.398+0000 7f5459a00700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5454075450 0x7f545419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.398+0000 7f5459a00700 1 --2- 172.21.15.42:0/2589787117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454076a20 0x7f545419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.398+0000 7f5459a00700 1 -- 172.21.15.42:0/2589787117 >> 172.21.15.42:0/2589787117 conn(0x7f54540fdf90 msgr2=0x7f5454106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.398+0000 7f5459a00700 1 -- 172.21.15.42:0/2589787117 shutdown_connections 2024-04-20T10:30:38.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:38.398+0000 7f5459a00700 1 -- 172.21.15.42:0/2589787117 wait complete. 2024-04-20T10:30:39.090 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:38 smithi042 ceph-mon[17078]: osdmap e81: 12 total, 12 up, 12 in 2024-04-20T10:30:39.090 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:38 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2589787117' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T10:30:39.090 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:38 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:39.090 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:38 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:39.090 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:38 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:39.090 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:38 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:39.090 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:38 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:39.090 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:38 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:39.090 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:38 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:39.091 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:38 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:39.091 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:38 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:39.091 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:38 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:39.144 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713609036,"num_in_osds":12,"osd_in_since":1713609017,"num_remapped_pgs":0} 2024-04-20T10:30:39.145 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd dump --format=json 2024-04-20T10:30:39.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:38 smithi046 ceph-mon[21574]: osdmap e81: 12 total, 12 up, 12 in 2024-04-20T10:30:39.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:38 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2589787117' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T10:30:39.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:38 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:39.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:38 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:39.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:38 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:39.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:38 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:39.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:38 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:39.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:38 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:39.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:38 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:39.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:38 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:39.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:38 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:39.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:38 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:39.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:38 smithi162 ceph-mon[21854]: osdmap e81: 12 total, 12 up, 12 in 2024-04-20T10:30:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:38 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2589787117' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T10:30:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:38 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:38 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:38 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:38 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:38 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:38 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:30:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:38 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:38 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:38 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:38 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:39.581 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:30:40.052 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:39 smithi042 ceph-mon[17078]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:40.052 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:39 smithi042 ceph-mon[17078]: Detected new or changed devices on smithi162 2024-04-20T10:30:40.052 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:39 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi162 to 4272M 2024-04-20T10:30:40.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:39 smithi046 ceph-mon[21574]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:40.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:39 smithi046 ceph-mon[21574]: Detected new or changed devices on smithi162 2024-04-20T10:30:40.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:39 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi162 to 4272M 2024-04-20T10:30:40.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:39 smithi162 ceph-mon[21854]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:40.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:39 smithi162 ceph-mon[21854]: Detected new or changed devices on smithi162 2024-04-20T10:30:40.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:39 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi162 to 4272M 2024-04-20T10:30:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.777+0000 7f07ab800700 1 -- 172.21.15.42:0/2392316834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a4102ae0 msgr2=0x7f07a4102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.777+0000 7f07ab800700 1 --2- 172.21.15.42:0/2392316834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a4102ae0 0x7f07a4102f60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0790009990 tx=0x7f0790009ca0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.777+0000 7f07ab800700 1 -- 172.21.15.42:0/2392316834 shutdown_connections 2024-04-20T10:30:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.777+0000 7f07ab800700 1 --2- 172.21.15.42:0/2392316834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a4108ab0 0x7f07a4108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.777+0000 7f07ab800700 1 --2- 172.21.15.42:0/2392316834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a41034a0 0x7f07a410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.777+0000 7f07ab800700 1 --2- 172.21.15.42:0/2392316834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a4102ae0 0x7f07a4102f60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.777+0000 7f07ab800700 1 -- 172.21.15.42:0/2392316834 >> 172.21.15.42:0/2392316834 conn(0x7f07a4079c40 msgr2=0x7f07a407a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:41.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.777+0000 7f07ab800700 1 -- 172.21.15.42:0/2392316834 shutdown_connections 2024-04-20T10:30:41.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.777+0000 7f07ab800700 1 -- 172.21.15.42:0/2392316834 wait complete. 2024-04-20T10:30:41.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.779+0000 7f07ab800700 1 Processor -- start 2024-04-20T10:30:41.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.779+0000 7f07ab800700 1 -- start start 2024-04-20T10:30:41.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.780+0000 7f07ab800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a4102ae0 0x7f07a4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:41.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.780+0000 7f07ab800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a41034a0 0x7f07a4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:41.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.780+0000 7f07ab800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a4108ab0 0x7f07a419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:41.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.780+0000 7f07ab800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4112e90 con 0x7f07a4102ae0 2024-04-20T10:30:41.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.780+0000 7f07ab800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4112d10 con 0x7f07a4108ab0 2024-04-20T10:30:41.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.780+0000 7f07ab800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4113010 con 0x7f07a41034a0 2024-04-20T10:30:41.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.780+0000 7f07a8e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a4102ae0 0x7f07a4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:41.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.780+0000 7f07a3e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a41034a0 0x7f07a4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:41.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.781+0000 7f07a3e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a41034a0 0x7f07a4199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:32998/0 (socket says 172.21.15.42:32998) 2024-04-20T10:30:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.781+0000 7f07a3e00700 1 -- 172.21.15.42:0/3465050534 learned_addr learned my addr 172.21.15.42:0/3465050534 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:30:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.781+0000 7f07a9800700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a4108ab0 0x7f07a419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.781+0000 7f07a3e00700 1 -- 172.21.15.42:0/3465050534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a4108ab0 msgr2=0x7f07a419dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.781+0000 7f07a3e00700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a4108ab0 0x7f07a419dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.781+0000 7f07a3e00700 1 -- 172.21.15.42:0/3465050534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a4102ae0 msgr2=0x7f07a4199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.781+0000 7f07a3e00700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a4102ae0 0x7f07a4199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.781+0000 7f07a3e00700 1 -- 172.21.15.42:0/3465050534 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a419e310 con 0x7f07a41034a0 2024-04-20T10:30:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.781+0000 7f07a8e00700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a4102ae0 0x7f07a4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.781+0000 7f07a3e00700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a41034a0 0x7f07a4199840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0790005910 tx=0x7f079000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.782+0000 7f07a1600700 1 -- 172.21.15.42:0/3465050534 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790027070 con 0x7f07a41034a0 2024-04-20T10:30:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.782+0000 7f07a1600700 1 -- 172.21.15.42:0/3465050534 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0790003d70 con 0x7f07a41034a0 2024-04-20T10:30:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.782+0000 7f07ab800700 1 -- 172.21.15.42:0/3465050534 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a419e5a0 con 0x7f07a41034a0 2024-04-20T10:30:41.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.782+0000 7f07a1600700 1 -- 172.21.15.42:0/3465050534 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079000f460 con 0x7f07a41034a0 2024-04-20T10:30:41.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.782+0000 7f07ab800700 1 -- 172.21.15.42:0/3465050534 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a41a5e80 con 0x7f07a41034a0 2024-04-20T10:30:41.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.783+0000 7f07ab800700 1 -- 172.21.15.42:0/3465050534 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a4069740 con 0x7f07a41034a0 2024-04-20T10:30:41.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.785+0000 7f07a1600700 1 -- 172.21.15.42:0/3465050534 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f079002a070 con 0x7f07a41034a0 2024-04-20T10:30:41.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.787+0000 7f07a1600700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f078c0a3cc0 0x7f078c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:41.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.787+0000 7f07a1600700 1 -- 172.21.15.42:0/3465050534 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f07900d0e50 con 0x7f07a41034a0 2024-04-20T10:30:41.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.787+0000 7f07a8e00700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f078c0a3cc0 0x7f078c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:41.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.788+0000 7f07a8e00700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f078c0a3cc0 0x7f078c0a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0798009fd0 tx=0x7f0798009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:41.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:41.790+0000 7f07a1600700 1 -- 172.21.15.42:0/3465050534 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0790013070 con 0x7f07a41034a0 2024-04-20T10:30:42.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.021+0000 7f07ab800700 1 -- 172.21.15.42:0/3465050534 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07a402df90 con 0x7f07a41034a0 2024-04-20T10:30:42.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.023+0000 7f07a1600700 1 -- 172.21.15.42:0/3465050534 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18434 (secure 0 0 0) 0x7f0790099030 con 0x7f07a41034a0 2024-04-20T10:30:42.023 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:30:42.023 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:30:37.958499+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:30:42.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.026+0000 7f07ab800700 1 -- 172.21.15.42:0/3465050534 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f078c0a3cc0 msgr2=0x7f078c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:42.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.026+0000 7f07ab800700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f078c0a3cc0 0x7f078c0a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0798009fd0 tx=0x7f0798009260 comp rx=0 tx=0).stop 2024-04-20T10:30:42.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.026+0000 7f07ab800700 1 -- 172.21.15.42:0/3465050534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a41034a0 msgr2=0x7f07a4199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:42.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.026+0000 7f07ab800700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a41034a0 0x7f07a4199840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0790005910 tx=0x7f079000bab0 comp rx=0 tx=0).stop 2024-04-20T10:30:42.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.026+0000 7f07ab800700 1 -- 172.21.15.42:0/3465050534 shutdown_connections 2024-04-20T10:30:42.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.026+0000 7f07ab800700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f078c0a3cc0 0x7f078c0a6180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:42.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.026+0000 7f07ab800700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07a4108ab0 0x7f07a419dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:42.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.026+0000 7f07ab800700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07a4102ae0 0x7f07a4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:42.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.026+0000 7f07ab800700 1 --2- 172.21.15.42:0/3465050534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07a41034a0 0x7f07a4199840 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:42.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.026+0000 7f07ab800700 1 -- 172.21.15.42:0/3465050534 >> 172.21.15.42:0/3465050534 conn(0x7f07a4079c40 msgr2=0x7f07a4100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:42.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.026+0000 7f07ab800700 1 -- 172.21.15.42:0/3465050534 shutdown_connections 2024-04-20T10:30:42.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:42.026+0000 7f07ab800700 1 -- 172.21.15.42:0/3465050534 wait complete. 2024-04-20T10:30:42.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:41 smithi046 ceph-mon[21574]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:42.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:41 smithi042 ceph-mon[17078]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:42.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:41 smithi162 ceph-mon[21854]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:42.788 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:26:16.537072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:30:42.789 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-20T10:30:43.243 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:30:43.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:42 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3465050534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:30:43.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:42 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3465050534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:30:43.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:42 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3465050534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:30:44.263 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:43 smithi042 ceph-mon[17078]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:44.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:43 smithi046 ceph-mon[21574]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:44.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:43 smithi162 ceph-mon[21854]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:45.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.470+0000 7faecf200700 1 -- 172.21.15.42:0/2745874714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed0102970 msgr2=0x7faed0102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:45.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.470+0000 7faecf200700 1 --2- 172.21.15.42:0/2745874714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed0102970 0x7faed0102df0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009990 tx=0x7faeb8009ca0 comp rx=0 tx=0).stop 2024-04-20T10:30:45.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.471+0000 7faecf200700 1 -- 172.21.15.42:0/2745874714 shutdown_connections 2024-04-20T10:30:45.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.471+0000 7faecf200700 1 --2- 172.21.15.42:0/2745874714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faed0108970 0x7faed0108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:45.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.471+0000 7faecf200700 1 --2- 172.21.15.42:0/2745874714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faed0103330 0x7faed010f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:45.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.471+0000 7faecf200700 1 --2- 172.21.15.42:0/2745874714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed0102970 0x7faed0102df0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:45.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.471+0000 7faecf200700 1 -- 172.21.15.42:0/2745874714 >> 172.21.15.42:0/2745874714 conn(0x7faed00fe460 msgr2=0x7faed0100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:45.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.471+0000 7faecf200700 1 -- 172.21.15.42:0/2745874714 shutdown_connections 2024-04-20T10:30:45.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.471+0000 7faecf200700 1 -- 172.21.15.42:0/2745874714 wait complete. 2024-04-20T10:30:45.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.472+0000 7faecf200700 1 Processor -- start 2024-04-20T10:30:45.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.472+0000 7faecf200700 1 -- start start 2024-04-20T10:30:45.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.472+0000 7faecf200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed0102970 0x7faed01991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:45.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faecf200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faed0103330 0x7faed0199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:45.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faecf200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faed0108970 0x7faed019da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:45.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faecf200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faed0112d50 con 0x7faed0103330 2024-04-20T10:30:45.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faecf200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faed0112bd0 con 0x7faed0108970 2024-04-20T10:30:45.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faecf200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7faed0112ed0 con 0x7faed0102970 2024-04-20T10:30:45.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faec5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faed0103330 0x7faed0199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:45.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faecde00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed0102970 0x7faed01991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:45.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faecde00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed0102970 0x7faed01991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:33020/0 (socket says 172.21.15.42:33020) 2024-04-20T10:30:45.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faecde00700 1 -- 172.21.15.42:0/1433458592 learned_addr learned my addr 172.21.15.42:0/1433458592 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:30:45.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faece800700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faed0108970 0x7faed019da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:45.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faecde00700 1 -- 172.21.15.42:0/1433458592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faed0108970 msgr2=0x7faed019da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:45.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faecde00700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faed0108970 0x7faed019da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:45.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.473+0000 7faecde00700 1 -- 172.21.15.42:0/1433458592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faed0103330 msgr2=0x7faed0199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:45.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.474+0000 7faecde00700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faed0103330 0x7faed0199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:45.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.474+0000 7faecde00700 1 -- 172.21.15.42:0/1433458592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed019e180 con 0x7faed0102970 2024-04-20T10:30:45.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.474+0000 7faec5400700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faed0103330 0x7faed0199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:45.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.474+0000 7faecde00700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed0102970 0x7faed01991c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faec000ef30 tx=0x7faec000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.474+0000 7faec7400700 1 -- 172.21.15.42:0/1433458592 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec00092e0 con 0x7faed0102970 2024-04-20T10:30:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.474+0000 7faec7400700 1 -- 172.21.15.42:0/1433458592 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec0004bb0 con 0x7faed0102970 2024-04-20T10:30:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.474+0000 7faecf200700 1 -- 172.21.15.42:0/1433458592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed019e410 con 0x7faed0102970 2024-04-20T10:30:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.474+0000 7faec7400700 1 -- 172.21.15.42:0/1433458592 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0004280 con 0x7faed0102970 2024-04-20T10:30:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.474+0000 7faecf200700 1 -- 172.21.15.42:0/1433458592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed0074c80 con 0x7faed0102970 2024-04-20T10:30:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.475+0000 7faecf200700 1 -- 172.21.15.42:0/1433458592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed01040b0 con 0x7faed0102970 2024-04-20T10:30:45.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.477+0000 7faec7400700 1 -- 172.21.15.42:0/1433458592 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faec001b030 con 0x7faed0102970 2024-04-20T10:30:45.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.479+0000 7faec7400700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faeb40a3c40 0x7faeb40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:45.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.479+0000 7faec7400700 1 -- 172.21.15.42:0/1433458592 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7faec00cfa50 con 0x7faed0102970 2024-04-20T10:30:45.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.479+0000 7faec5400700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faeb40a3c40 0x7faeb40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:45.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.480+0000 7faec5400700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faeb40a3c40 0x7faeb40a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faed0100a50 tx=0x7faeb800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:45.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.484+0000 7faec7400700 1 -- 172.21.15.42:0/1433458592 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7faec0022020 con 0x7faed0102970 2024-04-20T10:30:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.713+0000 7faecf200700 1 -- 172.21.15.42:0/1433458592 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faed019a4d0 con 0x7faed0102970 2024-04-20T10:30:45.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.714+0000 7faec7400700 1 -- 172.21.15.42:0/1433458592 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7faec0010040 con 0x7faed0102970 2024-04-20T10:30:45.714 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-20T10:30:45.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.717+0000 7faecf200700 1 -- 172.21.15.42:0/1433458592 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faeb40a3c40 msgr2=0x7faeb40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:45.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.717+0000 7faecf200700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faeb40a3c40 0x7faeb40a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faed0100a50 tx=0x7faeb800b260 comp rx=0 tx=0).stop 2024-04-20T10:30:45.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.717+0000 7faecf200700 1 -- 172.21.15.42:0/1433458592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed0102970 msgr2=0x7faed01991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:45.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.717+0000 7faecf200700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed0102970 0x7faed01991c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faec000ef30 tx=0x7faec000c3a0 comp rx=0 tx=0).stop 2024-04-20T10:30:45.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.717+0000 7faecf200700 1 -- 172.21.15.42:0/1433458592 shutdown_connections 2024-04-20T10:30:45.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.717+0000 7faecf200700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faeb40a3c40 0x7faeb40a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:45.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.717+0000 7faecf200700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faed0108970 0x7faed019da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:45.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.717+0000 7faecf200700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed0102970 0x7faed01991c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:45.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.717+0000 7faecf200700 1 --2- 172.21.15.42:0/1433458592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faed0103330 0x7faed0199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:45.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.717+0000 7faecf200700 1 -- 172.21.15.42:0/1433458592 >> 172.21.15.42:0/1433458592 conn(0x7faed00fe460 msgr2=0x7faed00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:45.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.717+0000 7faecf200700 1 -- 172.21.15.42:0/1433458592 shutdown_connections 2024-04-20T10:30:45.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:45.717+0000 7faecf200700 1 -- 172.21.15.42:0/1433458592 wait complete. 2024-04-20T10:30:46.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:46 smithi046 ceph-mon[21574]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:46.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:46 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1433458592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:30:46.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:46 smithi042 ceph-mon[17078]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:46.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:46 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1433458592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:30:46.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:46 smithi162 ceph-mon[21854]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:46.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:46 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1433458592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:30:46.557 INFO:tasks.cephadm:Adding mds.a on smithi042 2024-04-20T10:30:46.557 INFO:tasks.cephadm:Adding mds.d on smithi042 2024-04-20T10:30:46.557 INFO:tasks.cephadm:Adding mds.g on smithi042 2024-04-20T10:30:46.557 INFO:tasks.cephadm:Adding mds.j on smithi042 2024-04-20T10:30:46.557 INFO:tasks.cephadm:Adding mds.b on smithi046 2024-04-20T10:30:46.557 INFO:tasks.cephadm:Adding mds.e on smithi046 2024-04-20T10:30:46.557 INFO:tasks.cephadm:Adding mds.h on smithi046 2024-04-20T10:30:46.557 INFO:tasks.cephadm:Adding mds.k on smithi046 2024-04-20T10:30:46.557 INFO:tasks.cephadm:Adding mds.c on smithi162 2024-04-20T10:30:46.557 INFO:tasks.cephadm:Adding mds.f on smithi162 2024-04-20T10:30:46.557 INFO:tasks.cephadm:Adding mds.i on smithi162 2024-04-20T10:30:46.558 INFO:tasks.cephadm:Adding mds.l on smithi162 2024-04-20T10:30:46.558 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph orch apply mds all '12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi046=b;smithi046=e;smithi046=h;smithi046=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l' 2024-04-20T10:30:46.819 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:30:48.072 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:48 smithi162 ceph-mon[21854]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:48 smithi046 ceph-mon[21574]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:48.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:48 smithi042 ceph-mon[17078]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:48.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.493+0000 7fb2b2159700 1 -- 172.21.15.162:0/781234351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ac1035a0 msgr2=0x7fb2ac105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:48.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.493+0000 7fb2b2159700 1 --2- 172.21.15.162:0/781234351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ac1035a0 0x7fb2ac105990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009990 tx=0x7fb29c009ca0 comp rx=0 tx=0).stop 2024-04-20T10:30:48.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.493+0000 7fb2b2159700 1 -- 172.21.15.162:0/781234351 shutdown_connections 2024-04-20T10:30:48.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.493+0000 7fb2b2159700 1 --2- 172.21.15.162:0/781234351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2ac108800 0x7fb2ac10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:48.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.493+0000 7fb2b2159700 1 --2- 172.21.15.162:0/781234351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ac1035a0 0x7fb2ac105990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:48.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.493+0000 7fb2b2159700 1 --2- 172.21.15.162:0/781234351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac105ed0 0x7fb2ac1082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:48.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.493+0000 7fb2b2159700 1 -- 172.21.15.162:0/781234351 >> 172.21.15.162:0/781234351 conn(0x7fb2ac0fcf70 msgr2=0x7fb2ac0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:48.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.493+0000 7fb2b2159700 1 -- 172.21.15.162:0/781234351 shutdown_connections 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.493+0000 7fb2b2159700 1 -- 172.21.15.162:0/781234351 wait complete. 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.493+0000 7fb2b2159700 1 Processor -- start 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.493+0000 7fb2b2159700 1 -- start start 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2b2159700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2ac1035a0 0x7fb2ac198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2b2159700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ac105ed0 0x7fb2ac199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2b2159700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac108800 0x7fb2ac1a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2b2159700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac10e780 con 0x7fb2ac108800 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2b2159700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac10e600 con 0x7fb2ac1035a0 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2b2159700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac10e900 con 0x7fb2ac105ed0 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2aaffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ac105ed0 0x7fb2ac199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2aaffd700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ac105ed0 0x7fb2ac199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47998/0 (socket says 172.21.15.162:47998) 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2aaffd700 1 -- 172.21.15.162:0/2642192406 learned_addr learned my addr 172.21.15.162:0/2642192406 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-20T10:30:48.499 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2ab7fe700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2ac1035a0 0x7fb2ac198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2aaffd700 1 -- 172.21.15.162:0/2642192406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2ac1035a0 msgr2=0x7fb2ac198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2aaffd700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2ac1035a0 0x7fb2ac198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2aaffd700 1 -- 172.21.15.162:0/2642192406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac108800 msgr2=0x7fb2ac1a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2aaffd700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac108800 0x7fb2ac1a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2aaffd700 1 -- 172.21.15.162:0/2642192406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ac1a0a50 con 0x7fb2ac105ed0 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2aaffd700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ac105ed0 0x7fb2ac199330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000b560 tx=0x7fb2a000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2a8ff9700 1 -- 172.21.15.162:0/2642192406 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0015020 con 0x7fb2ac105ed0 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2a8ff9700 1 -- 172.21.15.162:0/2642192406 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2a0011d10 con 0x7fb2ac105ed0 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2b2159700 1 -- 172.21.15.162:0/2642192406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ac1a0d40 con 0x7fb2ac105ed0 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2a8ff9700 1 -- 172.21.15.162:0/2642192406 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0010e50 con 0x7fb2ac105ed0 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.494+0000 7fb2b2159700 1 -- 172.21.15.162:0/2642192406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ac1a12f0 con 0x7fb2ac105ed0 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.495+0000 7fb2b2159700 1 -- 172.21.15.162:0/2642192406 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2ac02df90 con 0x7fb2ac105ed0 2024-04-20T10:30:48.500 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.496+0000 7fb2a8ff9700 1 -- 172.21.15.162:0/2642192406 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb2a0018020 con 0x7fb2ac105ed0 2024-04-20T10:30:48.501 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.497+0000 7fb2a8ff9700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb2940a3e60 0x7fb2940a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:48.501 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.497+0000 7fb2a8ff9700 1 -- 172.21.15.162:0/2642192406 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb2a00cf640 con 0x7fb2ac105ed0 2024-04-20T10:30:48.501 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.497+0000 7fb2ab7fe700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb2940a3e60 0x7fb2940a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:48.501 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.498+0000 7fb2ab7fe700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb2940a3e60 0x7fb2940a6320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00b5c0 tx=0x7fb29c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:48.501 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.502+0000 7fb2a8ff9700 1 -- 172.21.15.162:0/2642192406 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb2a001f020 con 0x7fb2ac105ed0 2024-04-20T10:30:48.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.713+0000 7fb2b2159700 1 -- 172.21.15.162:0/2642192406 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi046=b;smithi046=e;smithi046=h;smithi046=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l", "target": ["mon-mgr", ""]}) -- 0x7fb2ac062f90 con 0x7fb2940a3e60 2024-04-20T10:30:48.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.731+0000 7fb2a8ff9700 1 -- 172.21.15.162:0/2642192406 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fb2ac062f90 con 0x7fb2940a3e60 2024-04-20T10:30:48.731 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled mds.all update... 2024-04-20T10:30:48.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.733+0000 7fb2b2159700 1 -- 172.21.15.162:0/2642192406 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb2940a3e60 msgr2=0x7fb2940a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:48.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.733+0000 7fb2b2159700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb2940a3e60 0x7fb2940a6320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00b5c0 tx=0x7fb29c005fd0 comp rx=0 tx=0).stop 2024-04-20T10:30:48.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.733+0000 7fb2b2159700 1 -- 172.21.15.162:0/2642192406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ac105ed0 msgr2=0x7fb2ac199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:48.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.733+0000 7fb2b2159700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ac105ed0 0x7fb2ac199330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000b560 tx=0x7fb2a000b870 comp rx=0 tx=0).stop 2024-04-20T10:30:48.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.733+0000 7fb2b2159700 1 -- 172.21.15.162:0/2642192406 shutdown_connections 2024-04-20T10:30:48.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.733+0000 7fb2b2159700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb2940a3e60 0x7fb2940a6320 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:48.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.733+0000 7fb2b2159700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2ac1035a0 0x7fb2ac198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:48.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.733+0000 7fb2b2159700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac108800 0x7fb2ac1a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:48.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.733+0000 7fb2b2159700 1 --2- 172.21.15.162:0/2642192406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2ac105ed0 0x7fb2ac199330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:48.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.733+0000 7fb2b2159700 1 -- 172.21.15.162:0/2642192406 >> 172.21.15.162:0/2642192406 conn(0x7fb2ac0fcf70 msgr2=0x7fb2ac103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:48.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.733+0000 7fb2b2159700 1 -- 172.21.15.162:0/2642192406 shutdown_connections 2024-04-20T10:30:48.734 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-20T10:30:48.733+0000 7fb2b2159700 1 -- 172.21.15.162:0/2642192406 wait complete. 2024-04-20T10:30:49.456 DEBUG:teuthology.orchestra.run.smithi042:mds.a> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.a.service 2024-04-20T10:30:49.461 DEBUG:teuthology.orchestra.run.smithi042:mds.d> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.d.service 2024-04-20T10:30:49.506 DEBUG:teuthology.orchestra.run.smithi042:mds.g> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.g.service 2024-04-20T10:30:49.552 DEBUG:teuthology.orchestra.run.smithi042:mds.j> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.j.service 2024-04-20T10:30:49.597 DEBUG:teuthology.orchestra.run.smithi046:mds.b> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.b.service 2024-04-20T10:30:49.599 DEBUG:teuthology.orchestra.run.smithi046:mds.e> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.e.service 2024-04-20T10:30:49.643 DEBUG:teuthology.orchestra.run.smithi046:mds.h> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.h.service 2024-04-20T10:30:49.685 DEBUG:teuthology.orchestra.run.smithi046:mds.k> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.k.service 2024-04-20T10:30:49.728 DEBUG:teuthology.orchestra.run.smithi162:mds.c> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.c.service 2024-04-20T10:30:49.731 DEBUG:teuthology.orchestra.run.smithi162:mds.f> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.f.service 2024-04-20T10:30:49.773 DEBUG:teuthology.orchestra.run.smithi162:mds.i> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.i.service 2024-04-20T10:30:49.817 DEBUG:teuthology.orchestra.run.smithi162:mds.l> sudo journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.l.service 2024-04-20T10:30:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:49 smithi042 ceph-mon[17078]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:49 smithi042 ceph-mon[17078]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi046=b;smithi046=e;smithi046=h;smithi046=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:30:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:49 smithi042 ceph-mon[17078]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi046=b;smithi046=e;smithi046=h;smithi046=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l;count:12 2024-04-20T10:30:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:49.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:49 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:49.860 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-20T10:30:49.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:30:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:49 smithi046 ceph-mon[21574]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:49 smithi046 ceph-mon[21574]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi046=b;smithi046=e;smithi046=h;smithi046=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:30:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:49 smithi046 ceph-mon[21574]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi046=b;smithi046=e;smithi046=h;smithi046=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l;count:12 2024-04-20T10:30:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:49 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:50.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.096+0000 7f89a0a00640 1 -- 172.21.15.42:0/3974478484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f899c109f90 msgr2=0x7f899c110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:50.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.096+0000 7f89a0a00640 1 --2- 172.21.15.42:0/3974478484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f899c109f90 0x7f899c110820 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8994009ca0 tx=0x7f899402c0f0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.096+0000 7f89a0a00640 1 -- 172.21.15.42:0/3974478484 shutdown_connections 2024-04-20T10:30:50.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.096+0000 7f89a0a00640 1 --2- 172.21.15.42:0/3974478484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f899c109f90 0x7f899c110820 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.096+0000 7f89a0a00640 1 --2- 172.21.15.42:0/3974478484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c1095d0 0x7f899c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.096+0000 7f89a0a00640 1 --2- 172.21.15.42:0/3974478484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f899c1083e0 0x7f899c108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.096+0000 7f89a0a00640 1 -- 172.21.15.42:0/3974478484 >> 172.21.15.42:0/3974478484 conn(0x7f899c0773a0 msgr2=0x7f899c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:50.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.096+0000 7f89a0a00640 1 -- 172.21.15.42:0/3974478484 shutdown_connections 2024-04-20T10:30:50.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.096+0000 7f89a0a00640 1 -- 172.21.15.42:0/3974478484 wait complete. 2024-04-20T10:30:50.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.097+0000 7f89a0a00640 1 Processor -- start 2024-04-20T10:30:50.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f89a0a00640 1 -- start start 2024-04-20T10:30:50.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f89a0a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f899c1083e0 0x7f899c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:50.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f89a0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f899c1095d0 0x7f899c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:50.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f89a0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c109f90 0x7f899c1a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:50.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f89a0a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f899c114070 con 0x7f899c1095d0 2024-04-20T10:30:50.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f89a0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f899c113ef0 con 0x7f899c109f90 2024-04-20T10:30:50.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f89a0a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f899c1141f0 con 0x7f899c1083e0 2024-04-20T10:30:50.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f8999e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f899c1095d0 0x7f899c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:50.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f899a800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f899c1083e0 0x7f899c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:50.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f899a800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f899c1083e0 0x7f899c19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52676/0 (socket says 172.21.15.42:52676) 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f899b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c109f90 0x7f899c1a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f8999e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f899c1095d0 0x7f899c19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33742/0 (socket says 172.21.15.42:33742) 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.098+0000 7f899a800640 1 -- 172.21.15.42:0/3329696179 learned_addr learned my addr 172.21.15.42:0/3329696179 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.099+0000 7f899a800640 1 -- 172.21.15.42:0/3329696179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c109f90 msgr2=0x7f899c1a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.099+0000 7f899a800640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c109f90 0x7f899c1a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.099+0000 7f899a800640 1 -- 172.21.15.42:0/3329696179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f899c1095d0 msgr2=0x7f899c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.099+0000 7f899a800640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f899c1095d0 0x7f899c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.099+0000 7f899a800640 1 -- 172.21.15.42:0/3329696179 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899c1a6290 con 0x7f899c1083e0 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.099+0000 7f899b200640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c109f90 0x7f899c1a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.099+0000 7f899a800640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f899c1083e0 0x7f899c19e670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e9c0 tx=0x7f898c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.099+0000 7f8993400640 1 -- 172.21.15.42:0/3329696179 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c00cd80 con 0x7f899c1083e0 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.099+0000 7f8993400640 1 -- 172.21.15.42:0/3329696179 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f898c0035f0 con 0x7f899c1083e0 2024-04-20T10:30:50.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.099+0000 7f89a0a00640 1 -- 172.21.15.42:0/3329696179 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899c1a6580 con 0x7f899c1083e0 2024-04-20T10:30:50.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.099+0000 7f8993400640 1 -- 172.21.15.42:0/3329696179 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c010690 con 0x7f899c1083e0 2024-04-20T10:30:50.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.100+0000 7f89a0a00640 1 -- 172.21.15.42:0/3329696179 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899c1a6a80 con 0x7f899c1083e0 2024-04-20T10:30:50.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.101+0000 7f89a0a00640 1 -- 172.21.15.42:0/3329696179 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899c10f700 con 0x7f899c1083e0 2024-04-20T10:30:50.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.102+0000 7f8993400640 1 -- 172.21.15.42:0/3329696179 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f898c0037a0 con 0x7f899c1083e0 2024-04-20T10:30:50.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.104+0000 7f8993400640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f89800a3bb0 0x7f89800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:50.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.104+0000 7f8993400640 1 -- 172.21.15.42:0/3329696179 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f898c0d0360 con 0x7f899c1083e0 2024-04-20T10:30:50.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.104+0000 7f8999e00640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f89800a3bb0 0x7f89800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:50.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.105+0000 7f8999e00640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f89800a3bb0 0x7f89800a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f899c19fb80 tx=0x7f898800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:50.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.108+0000 7f8993400640 1 -- 172.21.15.42:0/3329696179 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f898c09d140 con 0x7f899c1083e0 2024-04-20T10:30:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:49 smithi162 ceph-mon[21854]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:49 smithi162 ceph-mon[21854]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi046=b;smithi046=e;smithi046=h;smithi046=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:30:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:49 smithi162 ceph-mon[21854]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi046=b;smithi046=e;smithi046=h;smithi046=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l;count:12 2024-04-20T10:30:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:49 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:30:50.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.319+0000 7f89a0a00640 1 -- 172.21.15.42:0/3329696179 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f899c062b60 con 0x7f899c1083e0 2024-04-20T10:30:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.320+0000 7f8993400640 1 -- 172.21.15.42:0/3329696179 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18434 (secure 0 0 0) 0x7f898c09ca40 con 0x7f899c1083e0 2024-04-20T10:30:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:30:50.320 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:30:37.958499+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:30:50.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.323+0000 7f89a0a00640 1 -- 172.21.15.42:0/3329696179 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f89800a3bb0 msgr2=0x7f89800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:50.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.323+0000 7f89a0a00640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f89800a3bb0 0x7f89800a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f899c19fb80 tx=0x7f898800b410 comp rx=0 tx=0).stop 2024-04-20T10:30:50.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.323+0000 7f89a0a00640 1 -- 172.21.15.42:0/3329696179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f899c1083e0 msgr2=0x7f899c19e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:50.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.323+0000 7f89a0a00640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f899c1083e0 0x7f899c19e670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e9c0 tx=0x7f898c00ee80 comp rx=0 tx=0).stop 2024-04-20T10:30:50.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.323+0000 7f89a0a00640 1 -- 172.21.15.42:0/3329696179 shutdown_connections 2024-04-20T10:30:50.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.323+0000 7f89a0a00640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899c109f90 0x7f899c1a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.323+0000 7f89a0a00640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f899c1095d0 0x7f899c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.323+0000 7f89a0a00640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f89800a3bb0 0x7f89800a6070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.323+0000 7f89a0a00640 1 --2- 172.21.15.42:0/3329696179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f899c1083e0 0x7f899c19e670 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.323+0000 7f89a0a00640 1 -- 172.21.15.42:0/3329696179 >> 172.21.15.42:0/3329696179 conn(0x7f899c0773a0 msgr2=0x7f899c10c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.324+0000 7f89a0a00640 1 -- 172.21.15.42:0/3329696179 shutdown_connections 2024-04-20T10:30:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.324+0000 7f89a0a00640 1 -- 172.21.15.42:0/3329696179 wait complete. 2024-04-20T10:30:50.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:26:16.537072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:30:50.343 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:30:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.556+0000 7f1e3d400640 1 -- 172.21.15.42:0/1213599966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e380749c0 msgr2=0x7f1e38074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.556+0000 7f1e3d400640 1 --2- 172.21.15.42:0/1213599966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e380749c0 0x7f1e38074e40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009930 tx=0x7f1e2402bc20 comp rx=0 tx=0).stop 2024-04-20T10:30:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.556+0000 7f1e3d400640 1 -- 172.21.15.42:0/1213599966 shutdown_connections 2024-04-20T10:30:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.556+0000 7f1e3d400640 1 --2- 172.21.15.42:0/1213599966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e3810e370 0x7f1e38110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.556+0000 7f1e3d400640 1 --2- 172.21.15.42:0/1213599966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e380749c0 0x7f1e38074e40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.556+0000 7f1e3d400640 1 --2- 172.21.15.42:0/1213599966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e38075fa0 0x7f1e380743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.556+0000 7f1e3d400640 1 -- 172.21.15.42:0/1213599966 >> 172.21.15.42:0/1213599966 conn(0x7f1e380fd310 msgr2=0x7f1e380ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:50.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.556+0000 7f1e3d400640 1 -- 172.21.15.42:0/1213599966 shutdown_connections 2024-04-20T10:30:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.556+0000 7f1e3d400640 1 -- 172.21.15.42:0/1213599966 wait complete. 2024-04-20T10:30:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.558+0000 7f1e3d400640 1 Processor -- start 2024-04-20T10:30:50.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.558+0000 7f1e3d400640 1 -- start start 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.558+0000 7f1e3d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e380749c0 0x7f1e3819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.558+0000 7f1e3d400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e38075fa0 0x7f1e3819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.558+0000 7f1e3d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e3810e370 0x7f1e381a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.558+0000 7f1e3d400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e38114080 con 0x7f1e3810e370 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.558+0000 7f1e3d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e38113f00 con 0x7f1e380749c0 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e3d400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1e38114200 con 0x7f1e38075fa0 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e380749c0 0x7f1e3819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e37400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e38075fa0 0x7f1e3819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e3ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e3810e370 0x7f1e381a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e380749c0 0x7f1e3819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42446/0 (socket says 172.21.15.42:42446) 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e37400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e38075fa0 0x7f1e3819ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52684/0 (socket says 172.21.15.42:52684) 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e37e00640 1 -- 172.21.15.42:0/77468572 learned_addr learned my addr 172.21.15.42:0/77468572 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e37400640 1 -- 172.21.15.42:0/77468572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e380749c0 msgr2=0x7f1e3819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e37400640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e380749c0 0x7f1e3819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e37400640 1 -- 172.21.15.42:0/77468572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e3810e370 msgr2=0x7f1e381a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e37400640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e3810e370 0x7f1e381a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e37400640 1 -- 172.21.15.42:0/77468572 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e381a62d0 con 0x7f1e38075fa0 2024-04-20T10:30:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e3ca00640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e3810e370 0x7f1e381a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.559+0000 7f1e37e00640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e380749c0 0x7f1e3819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.560+0000 7f1e37400640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e38075fa0 0x7f1e3819ebf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1e2402c160 tx=0x7f1e2402c630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.560+0000 7f1e34c00640 1 -- 172.21.15.42:0/77468572 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24043020 con 0x7f1e38075fa0 2024-04-20T10:30:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.560+0000 7f1e34c00640 1 -- 172.21.15.42:0/77468572 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e240054e0 con 0x7f1e38075fa0 2024-04-20T10:30:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.560+0000 7f1e3d400640 1 -- 172.21.15.42:0/77468572 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e381a6560 con 0x7f1e38075fa0 2024-04-20T10:30:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.560+0000 7f1e34c00640 1 -- 172.21.15.42:0/77468572 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24003b90 con 0x7f1e38075fa0 2024-04-20T10:30:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.560+0000 7f1e3d400640 1 -- 172.21.15.42:0/77468572 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e381a6a80 con 0x7f1e38075fa0 2024-04-20T10:30:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.561+0000 7f1e3d400640 1 -- 172.21.15.42:0/77468572 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e38104540 con 0x7f1e38075fa0 2024-04-20T10:30:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.563+0000 7f1e34c00640 1 -- 172.21.15.42:0/77468572 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1e24046020 con 0x7f1e38075fa0 2024-04-20T10:30:50.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.565+0000 7f1e34c00640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e1c0a3bb0 0x7f1e1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:50.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.565+0000 7f1e34c00640 1 -- 172.21.15.42:0/77468572 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1e240f1f30 con 0x7f1e38075fa0 2024-04-20T10:30:50.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.565+0000 7f1e37e00640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e1c0a3bb0 0x7f1e1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:50.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.566+0000 7f1e37e00640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e1c0a3bb0 0x7f1e1c0a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e28011550 tx=0x7f1e28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:50.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.568+0000 7f1e34c00640 1 -- 172.21.15.42:0/77468572 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1e2404d020 con 0x7f1e38075fa0 2024-04-20T10:30:50.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.777+0000 7f1e3d400640 1 -- 172.21.15.42:0/77468572 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e38062b60 con 0x7f1e38075fa0 2024-04-20T10:30:50.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.778+0000 7f1e34c00640 1 -- 172.21.15.42:0/77468572 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f1e240beda0 con 0x7f1e38075fa0 2024-04-20T10:30:50.777 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-20T10:30:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.781+0000 7f1e3d400640 1 -- 172.21.15.42:0/77468572 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e1c0a3bb0 msgr2=0x7f1e1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.781+0000 7f1e3d400640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e1c0a3bb0 0x7f1e1c0a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e28011550 tx=0x7f1e28005f90 comp rx=0 tx=0).stop 2024-04-20T10:30:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.781+0000 7f1e3d400640 1 -- 172.21.15.42:0/77468572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e38075fa0 msgr2=0x7f1e3819ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.781+0000 7f1e3d400640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e38075fa0 0x7f1e3819ebf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1e2402c160 tx=0x7f1e2402c630 comp rx=0 tx=0).stop 2024-04-20T10:30:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.781+0000 7f1e3d400640 1 -- 172.21.15.42:0/77468572 shutdown_connections 2024-04-20T10:30:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.781+0000 7f1e3d400640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e3810e370 0x7f1e381a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.781+0000 7f1e3d400640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e1c0a3bb0 0x7f1e1c0a6070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.781+0000 7f1e3d400640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e38075fa0 0x7f1e3819ebf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.781+0000 7f1e3d400640 1 --2- 172.21.15.42:0/77468572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e380749c0 0x7f1e3819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.781+0000 7f1e3d400640 1 -- 172.21.15.42:0/77468572 >> 172.21.15.42:0/77468572 conn(0x7f1e380fd310 msgr2=0x7f1e38101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:50.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.781+0000 7f1e3d400640 1 -- 172.21.15.42:0/77468572 shutdown_connections 2024-04-20T10:30:50.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:50.781+0000 7f1e3d400640 1 -- 172.21.15.42:0/77468572 wait complete. 2024-04-20T10:30:50.801 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:50.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:50 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:50.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:50 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:50.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:50 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:50.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:50 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:30:50.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:50 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:30:50.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:50 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:50.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:50 smithi162 ceph-mon[21854]: Deploying daemon mds.l on smithi162 2024-04-20T10:30:50.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:50 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3329696179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:30:51.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.011+0000 7f99bcc00640 1 -- 172.21.15.42:0/2484784728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99b8105bb0 msgr2=0x7f99b810c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:51.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.011+0000 7f99bcc00640 1 --2- 172.21.15.42:0/2484784728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99b8105bb0 0x7f99b810c440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009be0 tx=0x7f99b002bef0 comp rx=0 tx=0).stop 2024-04-20T10:30:51.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.011+0000 7f99bcc00640 1 -- 172.21.15.42:0/2484784728 shutdown_connections 2024-04-20T10:30:51.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.011+0000 7f99bcc00640 1 --2- 172.21.15.42:0/2484784728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99b8105bb0 0x7f99b810c440 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:51.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.011+0000 7f99bcc00640 1 --2- 172.21.15.42:0/2484784728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b81051f0 0x7f99b8105670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:51.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.011+0000 7f99bcc00640 1 --2- 172.21.15.42:0/2484784728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b8104000 0x7f99b8104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:51.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.011+0000 7f99bcc00640 1 -- 172.21.15.42:0/2484784728 >> 172.21.15.42:0/2484784728 conn(0x7f99b80ff570 msgr2=0x7f99b81019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:51.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.011+0000 7f99bcc00640 1 -- 172.21.15.42:0/2484784728 shutdown_connections 2024-04-20T10:30:51.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.011+0000 7f99bcc00640 1 -- 172.21.15.42:0/2484784728 wait complete. 2024-04-20T10:30:51.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.012+0000 7f99bcc00640 1 Processor -- start 2024-04-20T10:30:51.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.012+0000 7f99bcc00640 1 -- start start 2024-04-20T10:30:51.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.013+0000 7f99bcc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b8104000 0x7f99b819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:51.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.013+0000 7f99bcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b81051f0 0x7f99b819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:51.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.013+0000 7f99bcc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99b8105bb0 0x7f99b81a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:51.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.013+0000 7f99bcc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99b810fca0 con 0x7f99b8105bb0 2024-04-20T10:30:51.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.013+0000 7f99bcc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99b810fb20 con 0x7f99b81051f0 2024-04-20T10:30:51.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.013+0000 7f99bcc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f99b810fe20 con 0x7f99b8104000 2024-04-20T10:30:51.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.013+0000 7f99b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99b8105bb0 0x7f99b81a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:51.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.013+0000 7f99b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99b8105bb0 0x7f99b81a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33764/0 (socket says 172.21.15.42:33764) 2024-04-20T10:30:51.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.013+0000 7f99b7e00640 1 -- 172.21.15.42:0/1543873317 learned_addr learned my addr 172.21.15.42:0/1543873317 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:30:51.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.013+0000 7f99b7400640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b8104000 0x7f99b819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.013+0000 7f99aea00640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b81051f0 0x7f99b819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.014+0000 7f99b7400640 1 -- 172.21.15.42:0/1543873317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b81051f0 msgr2=0x7f99b819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.014+0000 7f99b7400640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b81051f0 0x7f99b819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.014+0000 7f99b7400640 1 -- 172.21.15.42:0/1543873317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99b8105bb0 msgr2=0x7f99b81a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.014+0000 7f99b7400640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99b8105bb0 0x7f99b81a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.014+0000 7f99b7400640 1 -- 172.21.15.42:0/1543873317 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b81a1ea0 con 0x7f99b8104000 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.014+0000 7f99b7e00640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99b8105bb0 0x7f99b81a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.014+0000 7f99b7400640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b8104000 0x7f99b819a280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99a800e9c0 tx=0x7f99a800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.014+0000 7f99b4c00640 1 -- 172.21.15.42:0/1543873317 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a800ccc0 con 0x7f99b8104000 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.014+0000 7f99b4c00640 1 -- 172.21.15.42:0/1543873317 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a80035f0 con 0x7f99b8104000 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.014+0000 7f99b4c00640 1 -- 172.21.15.42:0/1543873317 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a800ce60 con 0x7f99b8104000 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.014+0000 7f99bcc00640 1 -- 172.21.15.42:0/1543873317 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b81a2190 con 0x7f99b8104000 2024-04-20T10:30:51.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.014+0000 7f99bcc00640 1 -- 172.21.15.42:0/1543873317 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b81a26e0 con 0x7f99b8104000 2024-04-20T10:30:51.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.016+0000 7f99bcc00640 1 -- 172.21.15.42:0/1543873317 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b810b320 con 0x7f99b8104000 2024-04-20T10:30:51.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.017+0000 7f99b4c00640 1 -- 172.21.15.42:0/1543873317 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f99a80037a0 con 0x7f99b8104000 2024-04-20T10:30:51.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.019+0000 7f99b4c00640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f999c0a3bb0 0x7f999c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:51.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.019+0000 7f99b4c00640 1 -- 172.21.15.42:0/1543873317 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f99a80d0320 con 0x7f99b8104000 2024-04-20T10:30:51.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.019+0000 7f99aea00640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f999c0a3bb0 0x7f999c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:51.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.019+0000 7f99aea00640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f999c0a3bb0 0x7f999c0a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99b819b790 tx=0x7f99a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:51.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.022+0000 7f99b4c00640 1 -- 172.21.15.42:0/1543873317 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f99a809d100 con 0x7f99b8104000 2024-04-20T10:30:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:50 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:50 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:50 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:50 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:30:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:50 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:30:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:50 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:50 smithi046 ceph-mon[21574]: Deploying daemon mds.l on smithi162 2024-04-20T10:30:51.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:50 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3329696179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:30:51.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:50 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:51.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:50 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:30:51.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:50 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:51.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:50 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:30:51.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:50 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:30:51.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:50 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:51.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:50 smithi042 ceph-mon[17078]: Deploying daemon mds.l on smithi162 2024-04-20T10:30:51.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:50 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3329696179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:30:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.271+0000 7f99bcc00640 1 -- 172.21.15.42:0/1543873317 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99b8062b60 con 0x7f99b8104000 2024-04-20T10:30:51.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.272+0000 7f99b4c00640 1 -- 172.21.15.42:0/1543873317 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f99a809ca00 con 0x7f99b8104000 2024-04-20T10:30:51.271 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:30:51.272 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-20T10:30:51.272 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 1 2024-04-20T10:30:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.275+0000 7f99bcc00640 1 -- 172.21.15.42:0/1543873317 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f999c0a3bb0 msgr2=0x7f999c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.275+0000 7f99bcc00640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f999c0a3bb0 0x7f999c0a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99b819b790 tx=0x7f99a0009260 comp rx=0 tx=0).stop 2024-04-20T10:30:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.275+0000 7f99bcc00640 1 -- 172.21.15.42:0/1543873317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b8104000 msgr2=0x7f99b819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.275+0000 7f99bcc00640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b8104000 0x7f99b819a280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99a800e9c0 tx=0x7f99a800edc0 comp rx=0 tx=0).stop 2024-04-20T10:30:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.275+0000 7f99bcc00640 1 -- 172.21.15.42:0/1543873317 shutdown_connections 2024-04-20T10:30:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.275+0000 7f99bcc00640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99b8105bb0 0x7f99b81a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.275+0000 7f99bcc00640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b81051f0 0x7f99b819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.275+0000 7f99bcc00640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f999c0a3bb0 0x7f999c0a6070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.275+0000 7f99bcc00640 1 --2- 172.21.15.42:0/1543873317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b8104000 0x7f99b819a280 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.275+0000 7f99bcc00640 1 -- 172.21.15.42:0/1543873317 >> 172.21.15.42:0/1543873317 conn(0x7f99b80ff570 msgr2=0x7f99b81084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.275+0000 7f99bcc00640 1 -- 172.21.15.42:0/1543873317 shutdown_connections 2024-04-20T10:30:51.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:51.275+0000 7f99bcc00640 1 -- 172.21.15.42:0/1543873317 wait complete. 2024-04-20T10:30:51.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:51 smithi162 ceph-mon[21854]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:51.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:51 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/77468572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:30:51.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:51 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1543873317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:30:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:51 smithi046 ceph-mon[21574]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:51 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/77468572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:30:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:51 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1543873317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:30:52.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:51 smithi042 ceph-mon[17078]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:52.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:51 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/77468572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:30:52.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:51 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1543873317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:30:52.316 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:30:52 smithi162 systemd[1]: Starting Ceph mds.l for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:30:52.570 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:30:52 smithi162 podman[51613]: 2024-04-20 10:30:52.219294529 +0000 UTC m=+0.017844388 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:30:52.570 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:30:52 smithi162 podman[51613]: 2024-04-20 10:30:52.332028629 +0000 UTC m=+0.130578539 container create d0514c17601a4f8bf2a936e6fb4cf6c1f59904d0012db13abdd0e69114d738ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-l, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-20T10:30:52.570 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:30:52 smithi162 podman[51613]: 2024-04-20 10:30:52.543147575 +0000 UTC m=+0.341697435 container init d0514c17601a4f8bf2a936e6fb4cf6c1f59904d0012db13abdd0e69114d738ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-l, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:30:52.570 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:30:52 smithi162 podman[51613]: 2024-04-20 10:30:52.545786326 +0000 UTC m=+0.344336186 container start d0514c17601a4f8bf2a936e6fb4cf6c1f59904d0012db13abdd0e69114d738ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:30:52.900 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:30:52 smithi162 ceph-mds[51631]: -- 172.21.15.162:0/1254594736 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c02c385a40 con 0x55c02c358800 2024-04-20T10:30:52.900 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:30:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-l[51627]: starting mds.l at 2024-04-20T10:30:52.900 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:30:52 smithi162 bash[51613]: d0514c17601a4f8bf2a936e6fb4cf6c1f59904d0012db13abdd0e69114d738ec 2024-04-20T10:30:52.900 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:30:52 smithi162 systemd[1]: Started Ceph mds.l for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:30:53.295 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:53.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.519+0000 7fda11000640 1 -- 172.21.15.42:0/2338708777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1095d0 msgr2=0x7fda0c109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.519+0000 7fda11000640 1 --2- 172.21.15.42:0/2338708777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1095d0 0x7fda0c109a50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009900 tx=0x7fd9f802bc30 comp rx=0 tx=0).stop 2024-04-20T10:30:53.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.519+0000 7fda11000640 1 -- 172.21.15.42:0/2338708777 shutdown_connections 2024-04-20T10:30:53.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.519+0000 7fda11000640 1 --2- 172.21.15.42:0/2338708777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda0c109f90 0x7fda0c110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.519+0000 7fda11000640 1 --2- 172.21.15.42:0/2338708777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1095d0 0x7fda0c109a50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.519+0000 7fda11000640 1 --2- 172.21.15.42:0/2338708777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda0c1083e0 0x7fda0c108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.519+0000 7fda11000640 1 -- 172.21.15.42:0/2338708777 >> 172.21.15.42:0/2338708777 conn(0x7fda0c0773a0 msgr2=0x7fda0c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:53.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.519+0000 7fda11000640 1 -- 172.21.15.42:0/2338708777 shutdown_connections 2024-04-20T10:30:53.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.519+0000 7fda11000640 1 -- 172.21.15.42:0/2338708777 wait complete. 2024-04-20T10:30:53.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.520+0000 7fda11000640 1 Processor -- start 2024-04-20T10:30:53.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda11000640 1 -- start start 2024-04-20T10:30:53.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda11000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda0c1083e0 0x7fda0c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda11000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1095d0 0x7fda0c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda11000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda0c109f90 0x7fda0c1a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda11000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c114070 con 0x7fda0c1095d0 2024-04-20T10:30:53.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda11000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c113ef0 con 0x7fda0c1083e0 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda11000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1141f0 con 0x7fda0c109f90 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda0aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1095d0 0x7fda0c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda0b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda0c1083e0 0x7fda0c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda0aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1095d0 0x7fda0c19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33778/0 (socket says 172.21.15.42:33778) 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda0aa00640 1 -- 172.21.15.42:0/927294578 learned_addr learned my addr 172.21.15.42:0/927294578 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda0b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda0c1083e0 0x7fda0c19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42486/0 (socket says 172.21.15.42:42486) 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.521+0000 7fda0be00640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda0c109f90 0x7fda0c1a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.522+0000 7fda0b400640 1 -- 172.21.15.42:0/927294578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda0c109f90 msgr2=0x7fda0c1a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.522+0000 7fda0b400640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda0c109f90 0x7fda0c1a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.522+0000 7fda0b400640 1 -- 172.21.15.42:0/927294578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1095d0 msgr2=0x7fda0c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.522+0000 7fda0b400640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1095d0 0x7fda0c19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.522+0000 7fda0b400640 1 -- 172.21.15.42:0/927294578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c1a6340 con 0x7fda0c1083e0 2024-04-20T10:30:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.522+0000 7fda0aa00640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1095d0 0x7fda0c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:53.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.522+0000 7fda0b400640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda0c1083e0 0x7fda0c19e720 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e9c0 tx=0x7fd9fc00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.522+0000 7fda03e00640 1 -- 172.21.15.42:0/927294578 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00ccc0 con 0x7fda0c1083e0 2024-04-20T10:30:53.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.522+0000 7fda03e00640 1 -- 172.21.15.42:0/927294578 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9fc0035f0 con 0x7fda0c1083e0 2024-04-20T10:30:53.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.522+0000 7fda11000640 1 -- 172.21.15.42:0/927294578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c1a6630 con 0x7fda0c1083e0 2024-04-20T10:30:53.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.522+0000 7fda03e00640 1 -- 172.21.15.42:0/927294578 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00ce60 con 0x7fda0c1083e0 2024-04-20T10:30:53.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.522+0000 7fda11000640 1 -- 172.21.15.42:0/927294578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c1a6b00 con 0x7fda0c1083e0 2024-04-20T10:30:53.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.523+0000 7fda11000640 1 -- 172.21.15.42:0/927294578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0c10f780 con 0x7fda0c1083e0 2024-04-20T10:30:53.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.526+0000 7fda03e00640 1 -- 172.21.15.42:0/927294578 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd9fc0037a0 con 0x7fda0c1083e0 2024-04-20T10:30:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.527+0000 7fda03e00640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd9f00a3bb0 0x7fd9f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.527+0000 7fda03e00640 1 -- 172.21.15.42:0/927294578 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd9fc0d02b0 con 0x7fda0c1083e0 2024-04-20T10:30:53.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.527+0000 7fda0aa00640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd9f00a3bb0 0x7fd9f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.528+0000 7fda0aa00640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd9f00a3bb0 0x7fd9f00a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8000c00 tx=0x7fd9f802c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.529+0000 7fda03e00640 1 -- 172.21.15.42:0/927294578 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd9fc09d090 con 0x7fda0c1083e0 2024-04-20T10:30:53.755 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:53 smithi046 ceph-mon[21574]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:53.755 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:53 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.162:6832/4094679184,v1:172.21.15.162:6833/4094679184] up:boot 2024-04-20T10:30:53.755 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:53 smithi046 ceph-mon[21574]: fsmap 1 up:standby 2024-04-20T10:30:53.755 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T10:30:53.755 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:53.756 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:53.756 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:53.756 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:30:53.756 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:30:53.756 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:53 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:53.756 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:53 smithi046 ceph-mon[21574]: Deploying daemon mds.k on smithi046 2024-04-20T10:30:53.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.779+0000 7fda11000640 1 -- 172.21.15.42:0/927294578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda0c02d5d0 con 0x7fda0c1083e0 2024-04-20T10:30:53.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.780+0000 7fda03e00640 1 -- 172.21.15.42:0/927294578 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fd9fc09c990 con 0x7fda0c1083e0 2024-04-20T10:30:53.779 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:30:53.780 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T10:30:53.780 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 2 2024-04-20T10:30:53.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.782+0000 7fda11000640 1 -- 172.21.15.42:0/927294578 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd9f00a3bb0 msgr2=0x7fd9f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.782+0000 7fda11000640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd9f00a3bb0 0x7fd9f00a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8000c00 tx=0x7fd9f802c880 comp rx=0 tx=0).stop 2024-04-20T10:30:53.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.782+0000 7fda11000640 1 -- 172.21.15.42:0/927294578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda0c1083e0 msgr2=0x7fda0c19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.782+0000 7fda11000640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda0c1083e0 0x7fda0c19e720 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e9c0 tx=0x7fd9fc00edc0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.782+0000 7fda11000640 1 -- 172.21.15.42:0/927294578 shutdown_connections 2024-04-20T10:30:53.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.782+0000 7fda11000640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd9f00a3bb0 0x7fd9f00a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.782+0000 7fda11000640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda0c109f90 0x7fda0c1a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.783+0000 7fda11000640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda0c1095d0 0x7fda0c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.783+0000 7fda11000640 1 --2- 172.21.15.42:0/927294578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda0c1083e0 0x7fda0c19e720 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.783+0000 7fda11000640 1 -- 172.21.15.42:0/927294578 >> 172.21.15.42:0/927294578 conn(0x7fda0c0773a0 msgr2=0x7fda0c10c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:53.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.783+0000 7fda11000640 1 -- 172.21.15.42:0/927294578 shutdown_connections 2024-04-20T10:30:53.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:53.783+0000 7fda11000640 1 -- 172.21.15.42:0/927294578 wait complete. 2024-04-20T10:30:53.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:53 smithi042 ceph-mon[17078]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:53.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:53 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.162:6832/4094679184,v1:172.21.15.162:6833/4094679184] up:boot 2024-04-20T10:30:53.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:53 smithi042 ceph-mon[17078]: fsmap 1 up:standby 2024-04-20T10:30:53.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T10:30:53.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:53.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:53.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:53.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:30:53.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:30:53.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:53 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:53.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:53 smithi042 ceph-mon[17078]: Deploying daemon mds.k on smithi046 2024-04-20T10:30:53.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:53 smithi162 ceph-mon[21854]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:53.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:53 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.162:6832/4094679184,v1:172.21.15.162:6833/4094679184] up:boot 2024-04-20T10:30:53.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:53 smithi162 ceph-mon[21854]: fsmap 1 up:standby 2024-04-20T10:30:53.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T10:30:53.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:53.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:53.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:53.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:30:53.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:30:53.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:53 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:53.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:53 smithi162 ceph-mon[21854]: Deploying daemon mds.k on smithi046 2024-04-20T10:30:54.777 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:54 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/927294578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:30:54.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:54 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/927294578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:30:54.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:54 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/927294578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:30:55.782 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:55 smithi046 ceph-mon[21574]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:55.782 INFO:journalctl@ceph.mds.k.smithi046.stdout:Apr 20 10:30:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-k[51242]: starting mds.k at 2024-04-20T10:30:55.783 INFO:journalctl@ceph.mds.k.smithi046.stdout:Apr 20 10:30:55 smithi046 bash[51227]: e50c2e20b7c2bcb4d91d9d93db1dd8357b0ec9915fc2a59824386827da4962bd 2024-04-20T10:30:55.802 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:55.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:55 smithi042 ceph-mon[17078]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:55.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:55 smithi162 ceph-mon[21854]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:56.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.035+0000 7f8034600640 1 -- 172.21.15.42:0/3946985094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1053b0 msgr2=0x7f802c105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:56.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.035+0000 7f8034600640 1 --2- 172.21.15.42:0/3946985094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1053b0 0x7f802c105830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8018009900 tx=0x7f801802bc30 comp rx=0 tx=0).stop 2024-04-20T10:30:56.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.035+0000 7f8034600640 1 -- 172.21.15.42:0/3946985094 shutdown_connections 2024-04-20T10:30:56.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.036+0000 7f8034600640 1 --2- 172.21.15.42:0/3946985094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f802c105d70 0x7f802c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.036+0000 7f8034600640 1 --2- 172.21.15.42:0/3946985094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1053b0 0x7f802c105830 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.036+0000 7f8034600640 1 --2- 172.21.15.42:0/3946985094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f802c1041c0 0x7f802c1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.036+0000 7f8034600640 1 -- 172.21.15.42:0/3946985094 >> 172.21.15.42:0/3946985094 conn(0x7f802c0ff750 msgr2=0x7f802c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:56.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.036+0000 7f8034600640 1 -- 172.21.15.42:0/3946985094 shutdown_connections 2024-04-20T10:30:56.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.036+0000 7f8034600640 1 -- 172.21.15.42:0/3946985094 wait complete. 2024-04-20T10:30:56.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.037+0000 7f8034600640 1 Processor -- start 2024-04-20T10:30:56.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.037+0000 7f8034600640 1 -- start start 2024-04-20T10:30:56.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.037+0000 7f8034600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1041c0 0x7f802c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:56.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.037+0000 7f8034600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f802c1053b0 0x7f802c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:56.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.037+0000 7f8034600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f802c105d70 0x7f802c1a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:56.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.037+0000 7f8034600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f802c10fea0 con 0x7f802c105d70 2024-04-20T10:30:56.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.037+0000 7f8034600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f802c10fd20 con 0x7f802c1041c0 2024-04-20T10:30:56.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.037+0000 7f8034600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f802c110020 con 0x7f802c1053b0 2024-04-20T10:30:56.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.037+0000 7f8032600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f802c105d70 0x7f802c1a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:56.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.038+0000 7f8031c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1041c0 0x7f802c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:56.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.038+0000 7f8031200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f802c1053b0 0x7f802c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:56.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.038+0000 7f8031c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1041c0 0x7f802c19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42504/0 (socket says 172.21.15.42:42504) 2024-04-20T10:30:56.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.038+0000 7f8031c00640 1 -- 172.21.15.42:0/3400754351 learned_addr learned my addr 172.21.15.42:0/3400754351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:30:56.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.038+0000 7f8031c00640 1 -- 172.21.15.42:0/3400754351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f802c1053b0 msgr2=0x7f802c19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:56.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.038+0000 7f8031c00640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f802c1053b0 0x7f802c19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.038+0000 7f8031c00640 1 -- 172.21.15.42:0/3400754351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f802c105d70 msgr2=0x7f802c1a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:56.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.038+0000 7f8031c00640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f802c105d70 0x7f802c1a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.038+0000 7f8031c00640 1 -- 172.21.15.42:0/3400754351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c1a2130 con 0x7f802c1041c0 2024-04-20T10:30:56.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.038+0000 7f8031200640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f802c1053b0 0x7f802c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:56.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.038+0000 7f8032600640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f802c105d70 0x7f802c1a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:56.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.038+0000 7f8031c00640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1041c0 0x7f802c19a510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f802000e9c0 tx=0x7f802000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:56.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.039+0000 7f802a000640 1 -- 172.21.15.42:0/3400754351 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802000cd80 con 0x7f802c1041c0 2024-04-20T10:30:56.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.039+0000 7f802a000640 1 -- 172.21.15.42:0/3400754351 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80200035f0 con 0x7f802c1041c0 2024-04-20T10:30:56.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.039+0000 7f802a000640 1 -- 172.21.15.42:0/3400754351 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020010690 con 0x7f802c1041c0 2024-04-20T10:30:56.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.039+0000 7f8034600640 1 -- 172.21.15.42:0/3400754351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802c1a2420 con 0x7f802c1041c0 2024-04-20T10:30:56.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.039+0000 7f8034600640 1 -- 172.21.15.42:0/3400754351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802c1a28f0 con 0x7f802c1041c0 2024-04-20T10:30:56.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.040+0000 7f8034600640 1 -- 172.21.15.42:0/3400754351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802c10b480 con 0x7f802c1041c0 2024-04-20T10:30:56.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.042+0000 7f802a000640 1 -- 172.21.15.42:0/3400754351 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f80200037a0 con 0x7f802c1041c0 2024-04-20T10:30:56.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.043+0000 7f802a000640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f80140a3bb0 0x7f80140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:56.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.043+0000 7f802a000640 1 -- 172.21.15.42:0/3400754351 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f80200d0400 con 0x7f802c1041c0 2024-04-20T10:30:56.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.043+0000 7f8031200640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f80140a3bb0 0x7f80140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:56.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.044+0000 7f8031200640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f80140a3bb0 0x7f80140a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8018000c00 tx=0x7f801802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:56.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.047+0000 7f802a000640 1 -- 172.21.15.42:0/3400754351 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f802009d1e0 con 0x7f802c1041c0 2024-04-20T10:30:56.053 INFO:journalctl@ceph.mds.k.smithi046.stdout:Apr 20 10:30:55 smithi046 systemd[1]: Started Ceph mds.k for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:30:56.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.299+0000 7f8034600640 1 -- 172.21.15.42:0/3400754351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f802c02d5d0 con 0x7f802c1041c0 2024-04-20T10:30:56.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.300+0000 7f802a000640 1 -- 172.21.15.42:0/3400754351 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f802009cae0 con 0x7f802c1041c0 2024-04-20T10:30:56.300 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:30:56.300 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T10:30:56.300 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 2 2024-04-20T10:30:56.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.304+0000 7f8034600640 1 -- 172.21.15.42:0/3400754351 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f80140a3bb0 msgr2=0x7f80140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:56.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.304+0000 7f8034600640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f80140a3bb0 0x7f80140a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8018000c00 tx=0x7f801802df90 comp rx=0 tx=0).stop 2024-04-20T10:30:56.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.304+0000 7f8034600640 1 -- 172.21.15.42:0/3400754351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1041c0 msgr2=0x7f802c19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:56.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.304+0000 7f8034600640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1041c0 0x7f802c19a510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f802000e9c0 tx=0x7f802000ee80 comp rx=0 tx=0).stop 2024-04-20T10:30:56.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.304+0000 7f8034600640 1 -- 172.21.15.42:0/3400754351 shutdown_connections 2024-04-20T10:30:56.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.304+0000 7f8034600640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f802c105d70 0x7f802c1a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.304+0000 7f8034600640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f80140a3bb0 0x7f80140a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.304+0000 7f8034600640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f802c1053b0 0x7f802c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.304+0000 7f8034600640 1 --2- 172.21.15.42:0/3400754351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1041c0 0x7f802c19a510 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.304+0000 7f8034600640 1 -- 172.21.15.42:0/3400754351 >> 172.21.15.42:0/3400754351 conn(0x7f802c0ff750 msgr2=0x7f802c108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:56.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.304+0000 7f8034600640 1 -- 172.21.15.42:0/3400754351 shutdown_connections 2024-04-20T10:30:56.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:56.304+0000 7f8034600640 1 -- 172.21.15.42:0/3400754351 wait complete. 2024-04-20T10:30:57.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:56 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:57.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:56 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:57.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:56 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:57.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:56 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:30:57.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:56 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:30:57.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:56 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:57.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:56 smithi042 ceph-mon[17078]: Deploying daemon mds.j on smithi042 2024-04-20T10:30:57.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:56 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3400754351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:30:57.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:56 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.46:6832/1854750771,v1:172.21.15.46:6833/1854750771] up:boot 2024-04-20T10:30:57.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:56 smithi042 ceph-mon[17078]: fsmap 2 up:standby 2024-04-20T10:30:57.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:56 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T10:30:57.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:56 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:56 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:56 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:56 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:30:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:56 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:30:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:56 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:56 smithi162 ceph-mon[21854]: Deploying daemon mds.j on smithi042 2024-04-20T10:30:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:56 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3400754351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:30:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:56 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.46:6832/1854750771,v1:172.21.15.46:6833/1854750771] up:boot 2024-04-20T10:30:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:56 smithi162 ceph-mon[21854]: fsmap 2 up:standby 2024-04-20T10:30:57.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:56 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T10:30:57.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:56 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:57.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:56 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:57.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:56 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:30:57.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:56 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:30:57.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:56 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:30:57.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:56 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:30:57.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:56 smithi046 ceph-mon[21574]: Deploying daemon mds.j on smithi042 2024-04-20T10:30:57.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:56 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3400754351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:30:57.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:56 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.46:6832/1854750771,v1:172.21.15.46:6833/1854750771] up:boot 2024-04-20T10:30:57.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:56 smithi046 ceph-mon[21574]: fsmap 2 up:standby 2024-04-20T10:30:57.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:56 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T10:30:58.085 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:57 smithi042 ceph-mon[17078]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:58.117 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:57 smithi046 ceph-mon[21574]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:58.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:57 smithi162 ceph-mon[21854]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:30:58.324 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.686+0000 7f92eb800640 1 -- 172.21.15.42:0/323755626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e4101fd0 msgr2=0x7f92e41023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.686+0000 7f92eb800640 1 --2- 172.21.15.42:0/323755626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e4101fd0 0x7f92e41023f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009900 tx=0x7f92d802bc30 comp rx=0 tx=0).stop 2024-04-20T10:30:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.686+0000 7f92eb800640 1 -- 172.21.15.42:0/323755626 shutdown_connections 2024-04-20T10:30:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.686+0000 7f92eb800640 1 --2- 172.21.15.42:0/323755626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e4103b80 0x7f92e410a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.686+0000 7f92eb800640 1 --2- 172.21.15.42:0/323755626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e41031c0 0x7f92e4103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.686+0000 7f92eb800640 1 --2- 172.21.15.42:0/323755626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e4101fd0 0x7f92e41023f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.686+0000 7f92eb800640 1 -- 172.21.15.42:0/323755626 >> 172.21.15.42:0/323755626 conn(0x7f92e40fd540 msgr2=0x7f92e40ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:58.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.689+0000 7f92eb800640 1 -- 172.21.15.42:0/323755626 shutdown_connections 2024-04-20T10:30:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.689+0000 7f92eb800640 1 -- 172.21.15.42:0/323755626 wait complete. 2024-04-20T10:30:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.690+0000 7f92eb800640 1 Processor -- start 2024-04-20T10:30:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.691+0000 7f92eb800640 1 -- start start 2024-04-20T10:30:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.691+0000 7f92eb800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e4101fd0 0x7f92e4112ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.691+0000 7f92eb800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e41031c0 0x7f92e4113000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.692+0000 7f92eb800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e4103b80 0x7f92e4111100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.692+0000 7f92eb800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f92e410ba70 con 0x7f92e41031c0 2024-04-20T10:30:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.692+0000 7f92eb800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92e410b8f0 con 0x7f92e4101fd0 2024-04-20T10:30:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.692+0000 7f92eb800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f92e410bbf0 con 0x7f92e4103b80 2024-04-20T10:30:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.692+0000 7f92e9800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e4103b80 0x7f92e4111100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.692+0000 7f92e9800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e4103b80 0x7f92e4111100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52762/0 (socket says 172.21.15.42:52762) 2024-04-20T10:30:58.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.692+0000 7f92e9800640 1 -- 172.21.15.42:0/3259257765 learned_addr learned my addr 172.21.15.42:0/3259257765 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:30:58.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.692+0000 7f92e3e00640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e41031c0 0x7f92e4113000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:58.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.693+0000 7f92e8e00640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e4101fd0 0x7f92e4112ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:58.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.693+0000 7f92e8e00640 1 -- 172.21.15.42:0/3259257765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e4103b80 msgr2=0x7f92e4111100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:58.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.693+0000 7f92e8e00640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e4103b80 0x7f92e4111100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.693+0000 7f92e8e00640 1 -- 172.21.15.42:0/3259257765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e41031c0 msgr2=0x7f92e4113000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:58.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.693+0000 7f92e8e00640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e41031c0 0x7f92e4113000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.693+0000 7f92e8e00640 1 -- 172.21.15.42:0/3259257765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e4111830 con 0x7f92e4101fd0 2024-04-20T10:30:58.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.694+0000 7f92e8e00640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e4101fd0 0x7f92e4112ac0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009230 tx=0x7f92d802c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:58.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.694+0000 7f92e1600640 1 -- 172.21.15.42:0/3259257765 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8043020 con 0x7f92e4101fd0 2024-04-20T10:30:58.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.694+0000 7f92e1600640 1 -- 172.21.15.42:0/3259257765 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d80054e0 con 0x7f92e4101fd0 2024-04-20T10:30:58.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.694+0000 7f92e1600640 1 -- 172.21.15.42:0/3259257765 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8003c60 con 0x7f92e4101fd0 2024-04-20T10:30:58.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.695+0000 7f92eb800640 1 -- 172.21.15.42:0/3259257765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e4111ac0 con 0x7f92e4101fd0 2024-04-20T10:30:58.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.695+0000 7f92eb800640 1 -- 172.21.15.42:0/3259257765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e4111ed0 con 0x7f92e4101fd0 2024-04-20T10:30:58.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.698+0000 7f92eb800640 1 -- 172.21.15.42:0/3259257765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e4109290 con 0x7f92e4101fd0 2024-04-20T10:30:58.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.699+0000 7f92e1600640 1 -- 172.21.15.42:0/3259257765 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f92d8046020 con 0x7f92e4101fd0 2024-04-20T10:30:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.702+0000 7f92e1600640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f92cc0a3c20 0x7f92cc0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:58.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.702+0000 7f92e3e00640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f92cc0a3c20 0x7f92cc0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:58.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.702+0000 7f92e1600640 1 -- 172.21.15.42:0/3259257765 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f92d80f2080 con 0x7f92e4101fd0 2024-04-20T10:30:58.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.703+0000 7f92e3e00640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f92cc0a3c20 0x7f92cc0a60e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f92e4112830 tx=0x7f92d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:58.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.705+0000 7f92e1600640 1 -- 172.21.15.42:0/3259257765 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f92d804d020 con 0x7f92e4101fd0 2024-04-20T10:30:58.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.960+0000 7f92eb800640 1 -- 172.21.15.42:0/3259257765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92e4062b60 con 0x7f92e4101fd0 2024-04-20T10:30:58.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.961+0000 7f92e1600640 1 -- 172.21.15.42:0/3259257765 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f92d80bee60 con 0x7f92e4101fd0 2024-04-20T10:30:58.961 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:30:58.961 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T10:30:58.961 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 3 2024-04-20T10:30:58.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.964+0000 7f92eb800640 1 -- 172.21.15.42:0/3259257765 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f92cc0a3c20 msgr2=0x7f92cc0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:58.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.964+0000 7f92eb800640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f92cc0a3c20 0x7f92cc0a60e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f92e4112830 tx=0x7f92d0005f90 comp rx=0 tx=0).stop 2024-04-20T10:30:58.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.964+0000 7f92eb800640 1 -- 172.21.15.42:0/3259257765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e4101fd0 msgr2=0x7f92e4112ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:58.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.964+0000 7f92eb800640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e4101fd0 0x7f92e4112ac0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009230 tx=0x7f92d802c6c0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.964+0000 7f92eb800640 1 -- 172.21.15.42:0/3259257765 shutdown_connections 2024-04-20T10:30:58.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.964+0000 7f92eb800640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f92cc0a3c20 0x7f92cc0a60e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.964+0000 7f92eb800640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92e4103b80 0x7f92e4111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.964+0000 7f92eb800640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e41031c0 0x7f92e4113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.964+0000 7f92eb800640 1 --2- 172.21.15.42:0/3259257765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e4101fd0 0x7f92e4112ac0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.964+0000 7f92eb800640 1 -- 172.21.15.42:0/3259257765 >> 172.21.15.42:0/3259257765 conn(0x7f92e40fd540 msgr2=0x7f92e4106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:58.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.964+0000 7f92eb800640 1 -- 172.21.15.42:0/3259257765 shutdown_connections 2024-04-20T10:30:58.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:30:58.964+0000 7f92eb800640 1 -- 172.21.15.42:0/3259257765 wait complete. 2024-04-20T10:30:59.604 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:30:59 smithi042 podman[57066]: 2024-04-20 10:30:59.329954597 +0000 UTC m=+0.175430951 container create f99cf4e9e713dc9c99495999a07f419ea8788f31e9a995e94ea9309b627f8ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:30:59.859 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:30:59 smithi042 podman[57066]: 2024-04-20 10:30:59.608615197 +0000 UTC m=+0.454091558 container init f99cf4e9e713dc9c99495999a07f419ea8788f31e9a995e94ea9309b627f8ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-20T10:30:59.859 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:30:59 smithi042 podman[57066]: 2024-04-20 10:30:59.620030423 +0000 UTC m=+0.465506800 container start f99cf4e9e713dc9c99495999a07f419ea8788f31e9a995e94ea9309b627f8ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-j, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-20T10:30:59.859 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:30:59 smithi042 ceph-mds[57085]: -- 172.21.15.42:0/1672893430 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556b323e5a40 con 0x556b323b9000 2024-04-20T10:30:59.859 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:30:59 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-j[57081]: starting mds.j at 2024-04-20T10:30:59.860 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:30:59 smithi042 bash[57066]: f99cf4e9e713dc9c99495999a07f419ea8788f31e9a995e94ea9309b627f8ea9 2024-04-20T10:31:00.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:59 smithi046 ceph-mon[21574]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:00.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:30:59 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3259257765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:00.353 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:30:59 smithi042 systemd[1]: Started Ceph mds.j for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:31:00.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:59 smithi042 ceph-mon[17078]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:00.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:30:59 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3259257765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:00.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:59 smithi162 ceph-mon[21854]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:00.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:30:59 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3259257765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:00.983 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:01.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.173+0000 7f7cf9800640 1 -- 172.21.15.42:0/206807019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cf4075f40 msgr2=0x7f7cf4074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.173+0000 7f7cf9800640 1 --2- 172.21.15.42:0/206807019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cf4075f40 0x7f7cf4074390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009ca0 tx=0x7f7ce402c180 comp rx=0 tx=0).stop 2024-04-20T10:31:01.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.173+0000 7f7cf9800640 1 -- 172.21.15.42:0/206807019 shutdown_connections 2024-04-20T10:31:01.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.173+0000 7f7cf9800640 1 --2- 172.21.15.42:0/206807019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cf4109c60 0x7f7cf410c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.173+0000 7f7cf9800640 1 --2- 172.21.15.42:0/206807019 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cf4074960 0x7f7cf4074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.173+0000 7f7cf9800640 1 --2- 172.21.15.42:0/206807019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cf4075f40 0x7f7cf4074390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.173+0000 7f7cf9800640 1 -- 172.21.15.42:0/206807019 >> 172.21.15.42:0/206807019 conn(0x7f7cf40fd340 msgr2=0x7f7cf40ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:01.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.174+0000 7f7cf9800640 1 -- 172.21.15.42:0/206807019 shutdown_connections 2024-04-20T10:31:01.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.174+0000 7f7cf9800640 1 -- 172.21.15.42:0/206807019 wait complete. 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.174+0000 7f7cf9800640 1 Processor -- start 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.175+0000 7f7cf9800640 1 -- start start 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.175+0000 7f7cf9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cf4074960 0x7f7cf419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.175+0000 7f7cf9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cf4075f40 0x7f7cf419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.175+0000 7f7cf9800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cf4109c60 0x7f7cf41a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.175+0000 7f7cf9800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf410fc60 con 0x7f7cf4074960 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.175+0000 7f7cf9800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf410fae0 con 0x7f7cf4075f40 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.175+0000 7f7cf9800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf410fde0 con 0x7f7cf4109c60 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.175+0000 7f7ceb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cf4075f40 0x7f7cf419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.175+0000 7f7ceb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cf4075f40 0x7f7cf419a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52954/0 (socket says 172.21.15.42:52954) 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.175+0000 7f7ceb400640 1 -- 172.21.15.42:0/1433737698 learned_addr learned my addr 172.21.15.42:0/1433737698 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.175+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cf4074960 0x7f7cf419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.175+0000 7f7cf8e00640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cf4109c60 0x7f7cf41a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.176+0000 7f7ceb400640 1 -- 172.21.15.42:0/1433737698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cf4109c60 msgr2=0x7f7cf41a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.176+0000 7f7ceb400640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cf4109c60 0x7f7cf41a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.176+0000 7f7ceb400640 1 -- 172.21.15.42:0/1433737698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cf4074960 msgr2=0x7f7cf419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.176+0000 7f7ceb400640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cf4074960 0x7f7cf419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.176+0000 7f7ceb400640 1 -- 172.21.15.42:0/1433737698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf41a1ec0 con 0x7f7cf4075f40 2024-04-20T10:31:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.176+0000 7f7ceb400640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cf4075f40 0x7f7cf419a830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00e9f0 tx=0x7f7cdc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.176+0000 7f7cf1600640 1 -- 172.21.15.42:0/1433737698 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc00cd80 con 0x7f7cf4075f40 2024-04-20T10:31:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.176+0000 7f7cf1600640 1 -- 172.21.15.42:0/1433737698 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cdc010910 con 0x7f7cf4075f40 2024-04-20T10:31:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.176+0000 7f7cf1600640 1 -- 172.21.15.42:0/1433737698 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc018990 con 0x7f7cf4075f40 2024-04-20T10:31:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.176+0000 7f7cf9800640 1 -- 172.21.15.42:0/1433737698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf41a21b0 con 0x7f7cf4075f40 2024-04-20T10:31:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.176+0000 7f7cf9800640 1 -- 172.21.15.42:0/1433737698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf41a2700 con 0x7f7cf4075f40 2024-04-20T10:31:01.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.178+0000 7f7cf9800640 1 -- 172.21.15.42:0/1433737698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cf4108a20 con 0x7f7cf4075f40 2024-04-20T10:31:01.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.179+0000 7f7cf1600640 1 -- 172.21.15.42:0/1433737698 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7cdc010ac0 con 0x7f7cf4075f40 2024-04-20T10:31:01.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.181+0000 7f7cf1600640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f7cd80a3bb0 0x7f7cd80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.181+0000 7f7cf1600640 1 -- 172.21.15.42:0/1433737698 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7cdc0d0320 con 0x7f7cf4075f40 2024-04-20T10:31:01.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.181+0000 7f7cf3e00640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f7cd80a3bb0 0x7f7cd80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.182+0000 7f7cf3e00640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f7cd80a3bb0 0x7f7cd80a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4006010 tx=0x7f7ce4032670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:01.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.185+0000 7f7cf1600640 1 -- 172.21.15.42:0/1433737698 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7cdc09cab0 con 0x7f7cf4075f40 2024-04-20T10:31:01.203 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:00 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.42:6834/2947757102,v1:172.21.15.42:6835/2947757102] up:boot 2024-04-20T10:31:01.203 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:00 smithi042 ceph-mon[17078]: fsmap 3 up:standby 2024-04-20T10:31:01.203 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:00 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T10:31:01.203 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:00 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:01.204 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:00 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:01.204 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:00 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:01.204 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:00 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:01.204 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:00 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:01.204 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:00 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:01.204 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:00 smithi042 ceph-mon[17078]: Deploying daemon mds.d on smithi042 2024-04-20T10:31:01.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:00 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.42:6834/2947757102,v1:172.21.15.42:6835/2947757102] up:boot 2024-04-20T10:31:01.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:00 smithi046 ceph-mon[21574]: fsmap 3 up:standby 2024-04-20T10:31:01.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:00 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T10:31:01.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:00 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:01.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:00 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:01.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:00 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:01.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:00 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:01.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:00 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:01.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:00 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:01.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:00 smithi046 ceph-mon[21574]: Deploying daemon mds.d on smithi042 2024-04-20T10:31:01.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:00 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.42:6834/2947757102,v1:172.21.15.42:6835/2947757102] up:boot 2024-04-20T10:31:01.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:00 smithi162 ceph-mon[21854]: fsmap 3 up:standby 2024-04-20T10:31:01.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:00 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T10:31:01.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:00 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:01.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:00 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:01.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:00 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:01.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:00 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:01.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:00 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:01.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:00 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:01.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:00 smithi162 ceph-mon[21854]: Deploying daemon mds.d on smithi042 2024-04-20T10:31:01.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.441+0000 7f7cf9800640 1 -- 172.21.15.42:0/1433737698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cf4062b60 con 0x7f7cf4075f40 2024-04-20T10:31:01.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.442+0000 7f7cf1600640 1 -- 172.21.15.42:0/1433737698 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f7cdc0a0db0 con 0x7f7cf4075f40 2024-04-20T10:31:01.443 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:01.443 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-20T10:31:01.443 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 4 2024-04-20T10:31:01.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.445+0000 7f7cf9800640 1 -- 172.21.15.42:0/1433737698 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f7cd80a3bb0 msgr2=0x7f7cd80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.445+0000 7f7cf9800640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f7cd80a3bb0 0x7f7cd80a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4006010 tx=0x7f7ce4032670 comp rx=0 tx=0).stop 2024-04-20T10:31:01.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.445+0000 7f7cf9800640 1 -- 172.21.15.42:0/1433737698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cf4075f40 msgr2=0x7f7cf419a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.445+0000 7f7cf9800640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cf4075f40 0x7f7cf419a830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00e9f0 tx=0x7f7cdc00eeb0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.445+0000 7f7cf9800640 1 -- 172.21.15.42:0/1433737698 shutdown_connections 2024-04-20T10:31:01.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.445+0000 7f7cf9800640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f7cd80a3bb0 0x7f7cd80a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.445+0000 7f7cf9800640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cf4109c60 0x7f7cf41a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.445+0000 7f7cf9800640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cf4075f40 0x7f7cf419a830 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.445+0000 7f7cf9800640 1 --2- 172.21.15.42:0/1433737698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7cf4074960 0x7f7cf419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.445+0000 7f7cf9800640 1 -- 172.21.15.42:0/1433737698 >> 172.21.15.42:0/1433737698 conn(0x7f7cf40fd340 msgr2=0x7f7cf4105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:01.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.445+0000 7f7cf9800640 1 -- 172.21.15.42:0/1433737698 shutdown_connections 2024-04-20T10:31:01.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:01.445+0000 7f7cf9800640 1 -- 172.21.15.42:0/1433737698 wait complete. 2024-04-20T10:31:02.327 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:02 smithi042 ceph-mon[17078]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:02.327 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:02 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1433737698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:02.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:02 smithi162 ceph-mon[21854]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:02.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:02 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1433737698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:02.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:02 smithi046 ceph-mon[21574]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:02.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:02 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1433737698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:03.467 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:03.531 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:31:03 smithi042 systemd[1]: Starting Ceph mds.d for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:31:03.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.667+0000 7f50e3000640 1 -- 172.21.15.42:0/1155528771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50dc1039b0 msgr2=0x7f50dc10a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.667+0000 7f50e3000640 1 --2- 172.21.15.42:0/1155528771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50dc1039b0 0x7f50dc10a240 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009be0 tx=0x7f50d402bef0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.667+0000 7f50e3000640 1 -- 172.21.15.42:0/1155528771 shutdown_connections 2024-04-20T10:31:03.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.667+0000 7f50e3000640 1 --2- 172.21.15.42:0/1155528771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50dc1039b0 0x7f50dc10a240 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.667+0000 7f50e3000640 1 --2- 172.21.15.42:0/1155528771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50dc102ff0 0x7f50dc103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.667+0000 7f50e3000640 1 --2- 172.21.15.42:0/1155528771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50dc101e00 0x7f50dc102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.667+0000 7f50e3000640 1 -- 172.21.15.42:0/1155528771 >> 172.21.15.42:0/1155528771 conn(0x7f50dc0fd370 msgr2=0x7f50dc0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:03.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.667+0000 7f50e3000640 1 -- 172.21.15.42:0/1155528771 shutdown_connections 2024-04-20T10:31:03.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.667+0000 7f50e3000640 1 -- 172.21.15.42:0/1155528771 wait complete. 2024-04-20T10:31:03.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.668+0000 7f50e3000640 1 Processor -- start 2024-04-20T10:31:03.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.668+0000 7f50e3000640 1 -- start start 2024-04-20T10:31:03.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.668+0000 7f50e3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50dc101e00 0x7f50dc19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e3000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50dc102ff0 0x7f50dc19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e3000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50dc1039b0 0x7f50dc1a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e3000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc10da60 con 0x7f50dc102ff0 2024-04-20T10:31:03.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc10d8e0 con 0x7f50dc101e00 2024-04-20T10:31:03.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e3000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc10dbe0 con 0x7f50dc1039b0 2024-04-20T10:31:03.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50dc102ff0 0x7f50dc19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50dc101e00 0x7f50dc19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50dc101e00 0x7f50dc19a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52962/0 (socket says 172.21.15.42:52962) 2024-04-20T10:31:03.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e1c00640 1 -- 172.21.15.42:0/2159138530 learned_addr learned my addr 172.21.15.42:0/2159138530 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:03.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e2600640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50dc1039b0 0x7f50dc1a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e1200640 1 -- 172.21.15.42:0/2159138530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50dc1039b0 msgr2=0x7f50dc1a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e1200640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50dc1039b0 0x7f50dc1a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e1200640 1 -- 172.21.15.42:0/2159138530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50dc101e00 msgr2=0x7f50dc19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e1200640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50dc101e00 0x7f50dc19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e1200640 1 -- 172.21.15.42:0/2159138530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50dc1a1eb0 con 0x7f50dc102ff0 2024-04-20T10:31:03.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.669+0000 7f50e1c00640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50dc101e00 0x7f50dc19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:03.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.670+0000 7f50e1200640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50dc102ff0 0x7f50dc19a820 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f50c8005fd0 tx=0x7f50c800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:03.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.670+0000 7f50da000640 1 -- 172.21.15.42:0/2159138530 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c800cd00 con 0x7f50dc102ff0 2024-04-20T10:31:03.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.670+0000 7f50da000640 1 -- 172.21.15.42:0/2159138530 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50c80035f0 con 0x7f50dc102ff0 2024-04-20T10:31:03.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.670+0000 7f50da000640 1 -- 172.21.15.42:0/2159138530 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8010670 con 0x7f50dc102ff0 2024-04-20T10:31:03.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.670+0000 7f50e3000640 1 -- 172.21.15.42:0/2159138530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50dc1a21a0 con 0x7f50dc102ff0 2024-04-20T10:31:03.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.670+0000 7f50e3000640 1 -- 172.21.15.42:0/2159138530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50dc1a26a0 con 0x7f50dc102ff0 2024-04-20T10:31:03.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.671+0000 7f50e3000640 1 -- 172.21.15.42:0/2159138530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50dc1a2960 con 0x7f50dc102ff0 2024-04-20T10:31:03.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.673+0000 7f50da000640 1 -- 172.21.15.42:0/2159138530 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f50c80037a0 con 0x7f50dc102ff0 2024-04-20T10:31:03.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.675+0000 7f50da000640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f50c40a3c30 0x7f50c40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.675+0000 7f50da000640 1 -- 172.21.15.42:0/2159138530 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f50c80d0060 con 0x7f50dc102ff0 2024-04-20T10:31:03.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.676+0000 7f50e1c00640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f50c40a3c30 0x7f50c40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.676+0000 7f50e1c00640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f50c40a3c30 0x7f50c40a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f50d00058c0 tx=0x7f50d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:03.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.680+0000 7f50da000640 1 -- 172.21.15.42:0/2159138530 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f50c8009510 con 0x7f50dc102ff0 2024-04-20T10:31:03.818 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:31:03 smithi042 podman[57412]: 2024-04-20 10:31:03.446875932 +0000 UTC m=+0.035470142 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:31:03.818 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:31:03 smithi042 podman[57412]: 2024-04-20 10:31:03.563711084 +0000 UTC m=+0.152305336 container create 3f1f0f598855bc25fb42b1d47db5379bd5648d6df37827b319a3e2695e9af862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:31:03.818 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:31:03 smithi042 podman[57412]: 2024-04-20 10:31:03.818452108 +0000 UTC m=+0.407046363 container init 3f1f0f598855bc25fb42b1d47db5379bd5648d6df37827b319a3e2695e9af862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-20T10:31:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.932+0000 7f50e3000640 1 -- 172.21.15.42:0/2159138530 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50dc1a2bd0 con 0x7f50dc102ff0 2024-04-20T10:31:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.933+0000 7f50da000640 1 -- 172.21.15.42:0/2159138530 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f50c809cee0 con 0x7f50dc102ff0 2024-04-20T10:31:03.933 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:03.933 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-20T10:31:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 4 2024-04-20T10:31:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.935+0000 7f50e3000640 1 -- 172.21.15.42:0/2159138530 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f50c40a3c30 msgr2=0x7f50c40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.935+0000 7f50e3000640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f50c40a3c30 0x7f50c40a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f50d00058c0 tx=0x7f50d0005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.935+0000 7f50e3000640 1 -- 172.21.15.42:0/2159138530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50dc102ff0 msgr2=0x7f50dc19a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.935+0000 7f50e3000640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50dc102ff0 0x7f50dc19a820 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f50c8005fd0 tx=0x7f50c800ee50 comp rx=0 tx=0).stop 2024-04-20T10:31:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.935+0000 7f50e3000640 1 -- 172.21.15.42:0/2159138530 shutdown_connections 2024-04-20T10:31:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.935+0000 7f50e3000640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f50c40a3c30 0x7f50c40a60f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.935+0000 7f50e3000640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50dc1039b0 0x7f50dc1a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.935+0000 7f50e3000640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50dc102ff0 0x7f50dc19a820 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.935+0000 7f50e3000640 1 --2- 172.21.15.42:0/2159138530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50dc101e00 0x7f50dc19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.935+0000 7f50e3000640 1 -- 172.21.15.42:0/2159138530 >> 172.21.15.42:0/2159138530 conn(0x7f50dc0fd370 msgr2=0x7f50dc074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.935+0000 7f50e3000640 1 -- 172.21.15.42:0/2159138530 shutdown_connections 2024-04-20T10:31:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:03.935+0000 7f50e3000640 1 -- 172.21.15.42:0/2159138530 wait complete. 2024-04-20T10:31:04.103 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:31:03 smithi042 podman[57412]: 2024-04-20 10:31:03.830272555 +0000 UTC m=+0.418866775 container start 3f1f0f598855bc25fb42b1d47db5379bd5648d6df37827b319a3e2695e9af862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-04-20T10:31:04.103 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:31:03 smithi042 ceph-mds[57469]: -- 172.21.15.42:0/4202878727 <== mon.2 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56517c50fa40 con 0x56517c4e3000 2024-04-20T10:31:04.103 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:31:03 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-d[57465]: starting mds.d at 2024-04-20T10:31:04.103 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:31:03 smithi042 bash[57412]: 3f1f0f598855bc25fb42b1d47db5379bd5648d6df37827b319a3e2695e9af862 2024-04-20T10:31:04.104 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:31:03 smithi042 systemd[1]: Started Ceph mds.d for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:31:04.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:04 smithi162 ceph-mon[21854]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:04.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:04 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2159138530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:04.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:04 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:04.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:04 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:04.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:04 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:04.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:04 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:04.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:04 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:04.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:04 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:04.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:04 smithi046 ceph-mon[21574]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:04.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:04 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2159138530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:04.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:04 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:04.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:04 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:04.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:04 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:04.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:04 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:04.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:04 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:04.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:04 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:04.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:04 smithi042 ceph-mon[17078]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:04.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:04 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2159138530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:04.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:04 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:04.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:04 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:04.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:04 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:04.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:04 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:04.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:04 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:04.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:04 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:05.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:05 smithi046 ceph-mon[21574]: Deploying daemon mds.e on smithi046 2024-04-20T10:31:05.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:05 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.42:6836/2898732601,v1:172.21.15.42:6837/2898732601] up:boot 2024-04-20T10:31:05.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:05 smithi046 ceph-mon[21574]: fsmap 4 up:standby 2024-04-20T10:31:05.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:05 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T10:31:05.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:05 smithi042 ceph-mon[17078]: Deploying daemon mds.e on smithi046 2024-04-20T10:31:05.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:05 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.42:6836/2898732601,v1:172.21.15.42:6837/2898732601] up:boot 2024-04-20T10:31:05.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:05 smithi042 ceph-mon[17078]: fsmap 4 up:standby 2024-04-20T10:31:05.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:05 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T10:31:05.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:05 smithi162 ceph-mon[21854]: Deploying daemon mds.e on smithi046 2024-04-20T10:31:05.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:05 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.42:6836/2898732601,v1:172.21.15.42:6837/2898732601] up:boot 2024-04-20T10:31:05.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:05 smithi162 ceph-mon[21854]: fsmap 4 up:standby 2024-04-20T10:31:05.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:05 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T10:31:05.956 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:06.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.172+0000 7f945e800640 1 -- 172.21.15.42:0/3773386895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9458107c90 msgr2=0x7f945810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.172+0000 7f945e800640 1 --2- 172.21.15.42:0/3773386895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9458107c90 0x7f945810a0e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9450009ca0 tx=0x7f945002c100 comp rx=0 tx=0).stop 2024-04-20T10:31:06.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.172+0000 7f945e800640 1 -- 172.21.15.42:0/3773386895 shutdown_connections 2024-04-20T10:31:06.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.172+0000 7f945e800640 1 --2- 172.21.15.42:0/3773386895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9458107c90 0x7f945810a0e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.172+0000 7f945e800640 1 --2- 172.21.15.42:0/3773386895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9458105360 0x7f9458107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.172+0000 7f945e800640 1 --2- 172.21.15.42:0/3773386895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458102a30 0x7f9458104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.172+0000 7f945e800640 1 -- 172.21.15.42:0/3773386895 >> 172.21.15.42:0/3773386895 conn(0x7f94580fc440 msgr2=0x7f94580fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:06.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.172+0000 7f945e800640 1 -- 172.21.15.42:0/3773386895 shutdown_connections 2024-04-20T10:31:06.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.172+0000 7f945e800640 1 -- 172.21.15.42:0/3773386895 wait complete. 2024-04-20T10:31:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.173+0000 7f945e800640 1 Processor -- start 2024-04-20T10:31:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.173+0000 7f945e800640 1 -- start start 2024-04-20T10:31:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f945e800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9458102a30 0x7f945819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f945e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458105360 0x7f945819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f945e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9458107c90 0x7f94581a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f945e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f945810dc40 con 0x7f9458105360 2024-04-20T10:31:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f945e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f945810dac0 con 0x7f9458107c90 2024-04-20T10:31:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f945e800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f945810ddc0 con 0x7f9458102a30 2024-04-20T10:31:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f9457400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9458102a30 0x7f945819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f9457e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9458107c90 0x7f94581a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f9456a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458105360 0x7f945819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f9457400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9458102a30 0x7f945819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:56194/0 (socket says 172.21.15.42:56194) 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f9457e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9458107c90 0x7f94581a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53002/0 (socket says 172.21.15.42:53002) 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f9457400640 1 -- 172.21.15.42:0/2687216026 learned_addr learned my addr 172.21.15.42:0/2687216026 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f9457e00640 1 -- 172.21.15.42:0/2687216026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9458102a30 msgr2=0x7f945819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f9457e00640 1 --2- 172.21.15.42:0/2687216026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9458102a30 0x7f945819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f9457e00640 1 -- 172.21.15.42:0/2687216026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458105360 msgr2=0x7f945819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f9457e00640 1 --2- 172.21.15.42:0/2687216026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458105360 0x7f945819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.174+0000 7f9457e00640 1 -- 172.21.15.42:0/2687216026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94581a20b0 con 0x7f9458107c90 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.175+0000 7f9457e00640 1 --2- 172.21.15.42:0/2687216026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9458107c90 0x7f94581a1a40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9450037040 tx=0x7f9450035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.175+0000 7f944fe00640 1 -- 172.21.15.42:0/2687216026 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450039040 con 0x7f9458107c90 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.175+0000 7f944fe00640 1 -- 172.21.15.42:0/2687216026 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9450044070 con 0x7f9458107c90 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.175+0000 7f944fe00640 1 -- 172.21.15.42:0/2687216026 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450004350 con 0x7f9458107c90 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.175+0000 7f945e800640 1 -- 172.21.15.42:0/2687216026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94581a2340 con 0x7f9458107c90 2024-04-20T10:31:06.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.175+0000 7f945e800640 1 -- 172.21.15.42:0/2687216026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94581a27b0 con 0x7f9458107c90 2024-04-20T10:31:06.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.176+0000 7f945e800640 1 -- 172.21.15.42:0/2687216026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9458109780 con 0x7f9458107c90 2024-04-20T10:31:06.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.178+0000 7f944fe00640 1 -- 172.21.15.42:0/2687216026 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f94500044f0 con 0x7f9458107c90 2024-04-20T10:31:06.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.180+0000 7f944fe00640 1 --2- 172.21.15.42:0/2687216026 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f943c0a3c30 0x7f943c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.180+0000 7f944fe00640 1 -- 172.21.15.42:0/2687216026 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f94500f24d0 con 0x7f9458107c90 2024-04-20T10:31:06.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.180+0000 7f9457400640 1 --2- 172.21.15.42:0/2687216026 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f943c0a3c30 0x7f943c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.181+0000 7f9457400640 1 --2- 172.21.15.42:0/2687216026 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f943c0a3c30 0x7f943c0a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f94480095f0 tx=0x7f9448005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.184+0000 7f944fe00640 1 -- 172.21.15.42:0/2687216026 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f94500bf2b0 con 0x7f9458107c90 2024-04-20T10:31:06.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:06 smithi042 ceph-mon[17078]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:06.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:06 smithi162 ceph-mon[21854]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.433+0000 7f945e800640 1 -- 172.21.15.42:0/2687216026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f945802d5d0 con 0x7f9458107c90 2024-04-20T10:31:06.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.434+0000 7f944fe00640 1 -- 172.21.15.42:0/2687216026 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f94500098b0 con 0x7f9458107c90 2024-04-20T10:31:06.435 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:06.435 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-20T10:31:06.435 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 5 2024-04-20T10:31:06.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.436+0000 7f945e800640 1 -- 172.21.15.42:0/2687216026 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f943c0a3c30 msgr2=0x7f943c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.436+0000 7f945e800640 1 --2- 172.21.15.42:0/2687216026 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f943c0a3c30 0x7f943c0a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f94480095f0 tx=0x7f9448005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.436+0000 7f945e800640 1 -- 172.21.15.42:0/2687216026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9458107c90 msgr2=0x7f94581a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.436+0000 7f945e800640 1 --2- 172.21.15.42:0/2687216026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9458107c90 0x7f94581a1a40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9450037040 tx=0x7f9450035d10 comp rx=0 tx=0).stop 2024-04-20T10:31:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.437+0000 7f945e800640 1 -- 172.21.15.42:0/2687216026 shutdown_connections 2024-04-20T10:31:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.437+0000 7f945e800640 1 --2- 172.21.15.42:0/2687216026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9458107c90 0x7f94581a1a40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.437+0000 7f945e800640 1 --2- 172.21.15.42:0/2687216026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9458105360 0x7f945819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.437+0000 7f945e800640 1 --2- 172.21.15.42:0/2687216026 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f943c0a3c30 0x7f943c0a60f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.437+0000 7f945e800640 1 --2- 172.21.15.42:0/2687216026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9458102a30 0x7f945819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.437+0000 7f945e800640 1 -- 172.21.15.42:0/2687216026 >> 172.21.15.42:0/2687216026 conn(0x7f94580fc440 msgr2=0x7f9458102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.437+0000 7f945e800640 1 -- 172.21.15.42:0/2687216026 shutdown_connections 2024-04-20T10:31:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:06.437+0000 7f945e800640 1 -- 172.21.15.42:0/2687216026 wait complete. 2024-04-20T10:31:06.483 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:06 smithi046 ceph-mon[21574]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:06.483 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:31:06 smithi046 systemd[1]: Starting Ceph mds.e for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:31:06.753 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:31:06 smithi046 podman[51551]: 2024-04-20 10:31:06.386895573 +0000 UTC m=+0.018642269 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:31:06.753 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:31:06 smithi046 podman[51551]: 2024-04-20 10:31:06.49302157 +0000 UTC m=+0.124768266 container create e15407ae12275cc1561cd340a3afd7c0d007e967e92b7938799c25b8bb4d2295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T10:31:06.753 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:31:06 smithi046 podman[51551]: 2024-04-20 10:31:06.754068713 +0000 UTC m=+0.385815408 container init e15407ae12275cc1561cd340a3afd7c0d007e967e92b7938799c25b8bb4d2295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:31:07.053 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:31:06 smithi046 podman[51551]: 2024-04-20 10:31:06.756787436 +0000 UTC m=+0.388534140 container start e15407ae12275cc1561cd340a3afd7c0d007e967e92b7938799c25b8bb4d2295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-e, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-20T10:31:07.053 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:31:06 smithi046 ceph-mds[51570]: -- 172.21.15.46:0/3852735712 <== mon.2 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b9e80b3a40 con 0x55b9e8086800 2024-04-20T10:31:07.053 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:31:06 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-e[51566]: starting mds.e at 2024-04-20T10:31:07.053 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:31:06 smithi046 bash[51551]: e15407ae12275cc1561cd340a3afd7c0d007e967e92b7938799c25b8bb4d2295 2024-04-20T10:31:07.053 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:31:06 smithi046 systemd[1]: Started Ceph mds.e for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:31:07.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:07 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2687216026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:07.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:07 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:07.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:07 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:07.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:07 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:07 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:07 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:07 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:07.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:07 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2687216026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:07.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:07 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:07.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:07 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:07.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:07 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:07.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:07 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:07.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:07 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:07.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:07 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:07.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:07 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2687216026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:07.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:07 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:07.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:07 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:07.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:07 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:07.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:07 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:07.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:07 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:07.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:07 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:08 smithi162 ceph-mon[21854]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:08 smithi162 ceph-mon[21854]: Deploying daemon mds.f on smithi162 2024-04-20T10:31:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:08 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.46:6834/4194525549,v1:172.21.15.46:6835/4194525549] up:boot 2024-04-20T10:31:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:08 smithi162 ceph-mon[21854]: fsmap 5 up:standby 2024-04-20T10:31:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:08 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T10:31:08.456 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:08.537 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:08 smithi042 ceph-mon[17078]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:08.538 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:08 smithi042 ceph-mon[17078]: Deploying daemon mds.f on smithi162 2024-04-20T10:31:08.538 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:08 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.46:6834/4194525549,v1:172.21.15.46:6835/4194525549] up:boot 2024-04-20T10:31:08.538 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:08 smithi042 ceph-mon[17078]: fsmap 5 up:standby 2024-04-20T10:31:08.538 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:08 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T10:31:08.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:08 smithi046 ceph-mon[21574]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:08.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:08 smithi046 ceph-mon[21574]: Deploying daemon mds.f on smithi162 2024-04-20T10:31:08.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:08 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.46:6834/4194525549,v1:172.21.15.46:6835/4194525549] up:boot 2024-04-20T10:31:08.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:08 smithi046 ceph-mon[21574]: fsmap 5 up:standby 2024-04-20T10:31:08.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:08 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T10:31:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.681+0000 7f6f6fe00640 1 -- 172.21.15.42:0/622034521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f701000b0 msgr2=0x7f6f701004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.681+0000 7f6f6fe00640 1 --2- 172.21.15.42:0/622034521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f701000b0 0x7f6f701004d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009900 tx=0x7f6f6002bc40 comp rx=0 tx=0).stop 2024-04-20T10:31:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.681+0000 7f6f6fe00640 1 -- 172.21.15.42:0/622034521 shutdown_connections 2024-04-20T10:31:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.681+0000 7f6f6fe00640 1 --2- 172.21.15.42:0/622034521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f701017d0 0x7f6f70106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.681+0000 7f6f6fe00640 1 --2- 172.21.15.42:0/622034521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f70100e10 0x7f6f70101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.681+0000 7f6f6fe00640 1 --2- 172.21.15.42:0/622034521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f701000b0 0x7f6f701004d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.681+0000 7f6f6fe00640 1 -- 172.21.15.42:0/622034521 >> 172.21.15.42:0/622034521 conn(0x7f6f700fb9e0 msgr2=0x7f6f700fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.681+0000 7f6f6fe00640 1 -- 172.21.15.42:0/622034521 shutdown_connections 2024-04-20T10:31:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.681+0000 7f6f6fe00640 1 -- 172.21.15.42:0/622034521 wait complete. 2024-04-20T10:31:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.682+0000 7f6f6fe00640 1 Processor -- start 2024-04-20T10:31:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.683+0000 7f6f6fe00640 1 -- start start 2024-04-20T10:31:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.683+0000 7f6f6fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f701000b0 0x7f6f7019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.683+0000 7f6f6fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f70100e10 0x7f6f7019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.683+0000 7f6f6fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f701017d0 0x7f6f701a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.683+0000 7f6f6fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70114130 con 0x7f6f70100e10 2024-04-20T10:31:08.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.683+0000 7f6f6fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70113fb0 con 0x7f6f701017d0 2024-04-20T10:31:08.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.683+0000 7f6f6fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6f701142b0 con 0x7f6f701000b0 2024-04-20T10:31:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.683+0000 7f6f6e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f70100e10 0x7f6f7019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.683+0000 7f6f6e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f70100e10 0x7f6f7019ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57642/0 (socket says 172.21.15.42:57642) 2024-04-20T10:31:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.683+0000 7f6f6ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f701000b0 0x7f6f7019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.683+0000 7f6f6e000640 1 -- 172.21.15.42:0/4057037204 learned_addr learned my addr 172.21.15.42:0/4057037204 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.683+0000 7f6f6f400640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f701017d0 0x7f6f701a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.684+0000 7f6f6ea00640 1 -- 172.21.15.42:0/4057037204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f701017d0 msgr2=0x7f6f701a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.684+0000 7f6f6ea00640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f701017d0 0x7f6f701a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.684+0000 7f6f6ea00640 1 -- 172.21.15.42:0/4057037204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f70100e10 msgr2=0x7f6f7019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.684+0000 7f6f6ea00640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f70100e10 0x7f6f7019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.684+0000 7f6f6ea00640 1 -- 172.21.15.42:0/4057037204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f701a6360 con 0x7f6f701000b0 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.684+0000 7f6f6f400640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f701017d0 0x7f6f701a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.684+0000 7f6f6e000640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f70100e10 0x7f6f7019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.684+0000 7f6f6ea00640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f701000b0 0x7f6f7019e740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f60036040 tx=0x7f6f6002c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.685+0000 7f6f67400640 1 -- 172.21.15.42:0/4057037204 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60041070 con 0x7f6f701000b0 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.685+0000 7f6f6fe00640 1 -- 172.21.15.42:0/4057037204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f701a65f0 con 0x7f6f701000b0 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.685+0000 7f6f67400640 1 -- 172.21.15.42:0/4057037204 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f60004450 con 0x7f6f701000b0 2024-04-20T10:31:08.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.685+0000 7f6f67400640 1 -- 172.21.15.42:0/4057037204 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60003e60 con 0x7f6f701000b0 2024-04-20T10:31:08.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.685+0000 7f6f6fe00640 1 -- 172.21.15.42:0/4057037204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f701a6ab0 con 0x7f6f701000b0 2024-04-20T10:31:08.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.686+0000 7f6f6fe00640 1 -- 172.21.15.42:0/4057037204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f701a6d10 con 0x7f6f701000b0 2024-04-20T10:31:08.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.688+0000 7f6f67400640 1 -- 172.21.15.42:0/4057037204 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6f60044070 con 0x7f6f701000b0 2024-04-20T10:31:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.689+0000 7f6f67400640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6f540a3bb0 0x7f6f540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.689+0000 7f6f67400640 1 -- 172.21.15.42:0/4057037204 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6f600f1d60 con 0x7f6f701000b0 2024-04-20T10:31:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.689+0000 7f6f6e000640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6f540a3bb0 0x7f6f540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.690+0000 7f6f6e000640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6f540a3bb0 0x7f6f540a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f7019fc50 tx=0x7f6f5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:08.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.691+0000 7f6f67400640 1 -- 172.21.15.42:0/4057037204 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6f600beb40 con 0x7f6f701000b0 2024-04-20T10:31:08.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.939+0000 7f6f6fe00640 1 -- 172.21.15.42:0/4057037204 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f701a6ee0 con 0x7f6f701000b0 2024-04-20T10:31:08.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.940+0000 7f6f67400640 1 -- 172.21.15.42:0/4057037204 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f6f600be440 con 0x7f6f701000b0 2024-04-20T10:31:08.941 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:08.941 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-20T10:31:08.941 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 6 2024-04-20T10:31:08.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.943+0000 7f6f6fe00640 1 -- 172.21.15.42:0/4057037204 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6f540a3bb0 msgr2=0x7f6f540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.943+0000 7f6f6fe00640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6f540a3bb0 0x7f6f540a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f7019fc50 tx=0x7f6f5c005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:08.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.943+0000 7f6f6fe00640 1 -- 172.21.15.42:0/4057037204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f701000b0 msgr2=0x7f6f7019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.943+0000 7f6f6fe00640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f701000b0 0x7f6f7019e740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f60036040 tx=0x7f6f6002c660 comp rx=0 tx=0).stop 2024-04-20T10:31:08.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.943+0000 7f6f6fe00640 1 -- 172.21.15.42:0/4057037204 shutdown_connections 2024-04-20T10:31:08.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.943+0000 7f6f6fe00640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f701017d0 0x7f6f701a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.943+0000 7f6f6fe00640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f70100e10 0x7f6f7019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.943+0000 7f6f6fe00640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6f540a3bb0 0x7f6f540a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.943+0000 7f6f6fe00640 1 --2- 172.21.15.42:0/4057037204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f701000b0 0x7f6f7019e740 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.943+0000 7f6f6fe00640 1 -- 172.21.15.42:0/4057037204 >> 172.21.15.42:0/4057037204 conn(0x7f6f700fb9e0 msgr2=0x7f6f700fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:08.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.943+0000 7f6f6fe00640 1 -- 172.21.15.42:0/4057037204 shutdown_connections 2024-04-20T10:31:08.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:08.943+0000 7f6f6fe00640 1 -- 172.21.15.42:0/4057037204 wait complete. 2024-04-20T10:31:09.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:09 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4057037204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:09.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:09 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/4057037204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:09.399 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:31:09 smithi162 systemd[1]: Starting Ceph mds.f for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:31:09.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:09 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/4057037204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:09.678 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:31:09 smithi162 podman[51920]: 2024-04-20 10:31:09.313594872 +0000 UTC m=+0.017902963 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:31:09.679 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:31:09 smithi162 podman[51920]: 2024-04-20 10:31:09.435094295 +0000 UTC m=+0.139402383 container create 22ceba1b0f8b997926759a89ec09327cfaae81cf28ffc2dcf97dac34a23f973c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-20T10:31:09.679 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:31:09 smithi162 podman[51920]: 2024-04-20 10:31:09.679297199 +0000 UTC m=+0.383605287 container init 22ceba1b0f8b997926759a89ec09327cfaae81cf28ffc2dcf97dac34a23f973c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:31:10.149 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:31:09 smithi162 podman[51920]: 2024-04-20 10:31:09.682047217 +0000 UTC m=+0.386355314 container start 22ceba1b0f8b997926759a89ec09327cfaae81cf28ffc2dcf97dac34a23f973c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-f, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-20T10:31:10.149 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:31:09 smithi162 ceph-mds[51939]: -- 172.21.15.162:0/2518254613 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5589f883ba40 con 0x5589f880ec00 2024-04-20T10:31:10.149 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:31:09 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-f[51935]: starting mds.f at 2024-04-20T10:31:10.149 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:31:09 smithi162 bash[51920]: 22ceba1b0f8b997926759a89ec09327cfaae81cf28ffc2dcf97dac34a23f973c 2024-04-20T10:31:10.149 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:31:09 smithi162 systemd[1]: Started Ceph mds.f for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:31:10.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:10 smithi162 ceph-mon[21854]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:10.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:10.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:10.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:10 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:10.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:10 smithi046 ceph-mon[21574]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:10.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:10.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:10.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:10.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:10.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:10.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:10 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:10.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:10 smithi042 ceph-mon[17078]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:10.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:10.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:10.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:10.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:10.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:10.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:10 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:10.962 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.192+0000 7fddc2400640 1 -- 172.21.15.42:0/1415294425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc1097b0 msgr2=0x7fddbc109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.192+0000 7fddc2400640 1 --2- 172.21.15.42:0/1415294425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc1097b0 0x7fddbc109c30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009900 tx=0x7fdda802bc50 comp rx=0 tx=0).stop 2024-04-20T10:31:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.192+0000 7fddc2400640 1 -- 172.21.15.42:0/1415294425 shutdown_connections 2024-04-20T10:31:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.192+0000 7fddc2400640 1 --2- 172.21.15.42:0/1415294425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddbc10a170 0x7fddbc110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.192+0000 7fddc2400640 1 --2- 172.21.15.42:0/1415294425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc1097b0 0x7fddbc109c30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.192+0000 7fddc2400640 1 --2- 172.21.15.42:0/1415294425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc1085c0 0x7fddbc1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.192+0000 7fddc2400640 1 -- 172.21.15.42:0/1415294425 >> 172.21.15.42:0/1415294425 conn(0x7fddbc077580 msgr2=0x7fddbc0799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.192+0000 7fddc2400640 1 -- 172.21.15.42:0/1415294425 shutdown_connections 2024-04-20T10:31:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.192+0000 7fddc2400640 1 -- 172.21.15.42:0/1415294425 wait complete. 2024-04-20T10:31:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.193+0000 7fddc2400640 1 Processor -- start 2024-04-20T10:31:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.193+0000 7fddc2400640 1 -- start start 2024-04-20T10:31:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.194+0000 7fddc2400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc1085c0 0x7fddbc19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.194+0000 7fddc2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddbc1097b0 0x7fddbc19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.194+0000 7fddc2400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc10a170 0x7fddbc1a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.194+0000 7fddc2400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1142a0 con 0x7fddbc10a170 2024-04-20T10:31:11.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.194+0000 7fddc2400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc114120 con 0x7fddbc1097b0 2024-04-20T10:31:11.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.194+0000 7fddc2400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc114420 con 0x7fddbc1085c0 2024-04-20T10:31:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.194+0000 7fddbb400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc1085c0 0x7fddbc19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.194+0000 7fddbbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc10a170 0x7fddbc1a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.194+0000 7fddbb400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc1085c0 0x7fddbc19e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:36570/0 (socket says 172.21.15.42:36570) 2024-04-20T10:31:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.194+0000 7fddbb400640 1 -- 172.21.15.42:0/4167376635 learned_addr learned my addr 172.21.15.42:0/4167376635 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.194+0000 7fddbaa00640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddbc1097b0 0x7fddbc19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.195+0000 7fddbb400640 1 -- 172.21.15.42:0/4167376635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddbc1097b0 msgr2=0x7fddbc19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.195+0000 7fddbb400640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddbc1097b0 0x7fddbc19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.195+0000 7fddbb400640 1 -- 172.21.15.42:0/4167376635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc10a170 msgr2=0x7fddbc1a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.195+0000 7fddbb400640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc10a170 0x7fddbc1a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.195+0000 7fddbb400640 1 -- 172.21.15.42:0/4167376635 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc1a6530 con 0x7fddbc1085c0 2024-04-20T10:31:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.195+0000 7fddbb400640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc1085c0 0x7fddbc19e910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fddac00e9c0 tx=0x7fddac00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.195+0000 7fddb3e00640 1 -- 172.21.15.42:0/4167376635 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac00cd80 con 0x7fddbc1085c0 2024-04-20T10:31:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.195+0000 7fddc2400640 1 -- 172.21.15.42:0/4167376635 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc1a6820 con 0x7fddbc1085c0 2024-04-20T10:31:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.195+0000 7fddb3e00640 1 -- 172.21.15.42:0/4167376635 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddac0035f0 con 0x7fddbc1085c0 2024-04-20T10:31:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.195+0000 7fddb3e00640 1 -- 172.21.15.42:0/4167376635 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac010690 con 0x7fddbc1085c0 2024-04-20T10:31:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.195+0000 7fddc2400640 1 -- 172.21.15.42:0/4167376635 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc1a6d40 con 0x7fddbc1085c0 2024-04-20T10:31:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.196+0000 7fddc2400640 1 -- 172.21.15.42:0/4167376635 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddbc10f8e0 con 0x7fddbc1085c0 2024-04-20T10:31:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.198+0000 7fddb3e00640 1 -- 172.21.15.42:0/4167376635 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fddac0037a0 con 0x7fddbc1085c0 2024-04-20T10:31:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.200+0000 7fddb3e00640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdda00a3c30 0x7fdda00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.200+0000 7fddb3e00640 1 -- 172.21.15.42:0/4167376635 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fddac0d02c0 con 0x7fddbc1085c0 2024-04-20T10:31:11.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.200+0000 7fddbaa00640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdda00a3c30 0x7fdda00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.201+0000 7fddbaa00640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdda00a3c30 0x7fdda00a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdda802c260 tx=0x7fdda8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.203+0000 7fddb3e00640 1 -- 172.21.15.42:0/4167376635 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fddac09d0a0 con 0x7fddbc1085c0 2024-04-20T10:31:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:11 smithi042 ceph-mon[17078]: Deploying daemon mds.c on smithi162 2024-04-20T10:31:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:11 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.162:6834/2486205910,v1:172.21.15.162:6835/2486205910] up:boot 2024-04-20T10:31:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:11 smithi042 ceph-mon[17078]: fsmap 6 up:standby 2024-04-20T10:31:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:11 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T10:31:11.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.456+0000 7fddc2400640 1 -- 172.21.15.42:0/4167376635 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddbc02d5d0 con 0x7fddbc1085c0 2024-04-20T10:31:11.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.457+0000 7fddb3e00640 1 -- 172.21.15.42:0/4167376635 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7fddac09c9a0 con 0x7fddbc1085c0 2024-04-20T10:31:11.458 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:11.458 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-20T10:31:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 7 2024-04-20T10:31:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.460+0000 7fddc2400640 1 -- 172.21.15.42:0/4167376635 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdda00a3c30 msgr2=0x7fdda00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.460+0000 7fddc2400640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdda00a3c30 0x7fdda00a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdda802c260 tx=0x7fdda8000bc0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.460+0000 7fddc2400640 1 -- 172.21.15.42:0/4167376635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc1085c0 msgr2=0x7fddbc19e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.460+0000 7fddc2400640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc1085c0 0x7fddbc19e910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fddac00e9c0 tx=0x7fddac00ee80 comp rx=0 tx=0).stop 2024-04-20T10:31:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.460+0000 7fddc2400640 1 -- 172.21.15.42:0/4167376635 shutdown_connections 2024-04-20T10:31:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.460+0000 7fddc2400640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddbc10a170 0x7fddbc1a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.460+0000 7fddc2400640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddbc1097b0 0x7fddbc19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.460+0000 7fddc2400640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdda00a3c30 0x7fdda00a60f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.460+0000 7fddc2400640 1 --2- 172.21.15.42:0/4167376635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fddbc1085c0 0x7fddbc19e910 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.460+0000 7fddc2400640 1 -- 172.21.15.42:0/4167376635 >> 172.21.15.42:0/4167376635 conn(0x7fddbc077580 msgr2=0x7fddbc10cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.460+0000 7fddc2400640 1 -- 172.21.15.42:0/4167376635 shutdown_connections 2024-04-20T10:31:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:11.460+0000 7fddc2400640 1 -- 172.21.15.42:0/4167376635 wait complete. 2024-04-20T10:31:11.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:11 smithi046 ceph-mon[21574]: Deploying daemon mds.c on smithi162 2024-04-20T10:31:11.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:11 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.162:6834/2486205910,v1:172.21.15.162:6835/2486205910] up:boot 2024-04-20T10:31:11.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:11 smithi046 ceph-mon[21574]: fsmap 6 up:standby 2024-04-20T10:31:11.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:11 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T10:31:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:11 smithi162 ceph-mon[21854]: Deploying daemon mds.c on smithi162 2024-04-20T10:31:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:11 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.162:6834/2486205910,v1:172.21.15.162:6835/2486205910] up:boot 2024-04-20T10:31:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:11 smithi162 ceph-mon[21854]: fsmap 6 up:standby 2024-04-20T10:31:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:11 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T10:31:12.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:12 smithi162 ceph-mon[21854]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:12.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:12 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/4167376635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:12.400 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:31:12 smithi162 systemd[1]: Starting Ceph mds.c for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:31:12.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:12 smithi046 ceph-mon[21574]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:12.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:12 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/4167376635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:12.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:12 smithi042 ceph-mon[17078]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:12.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:12 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4167376635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:12.842 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:31:12 smithi162 podman[52228]: 2024-04-20 10:31:12.390624084 +0000 UTC m=+0.018341739 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:31:12.842 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:31:12 smithi162 podman[52228]: 2024-04-20 10:31:12.515871383 +0000 UTC m=+0.143589088 container create 669991ee670dd2e0bde5bfd32a0ad2c5e66105ccc4f82e100fa823520fa760a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-c, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-20T10:31:12.842 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:31:12 smithi162 podman[52228]: 2024-04-20 10:31:12.842863136 +0000 UTC m=+0.470580782 container init 669991ee670dd2e0bde5bfd32a0ad2c5e66105ccc4f82e100fa823520fa760a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-c, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-20T10:31:13.149 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:31:12 smithi162 podman[52228]: 2024-04-20 10:31:12.845563484 +0000 UTC m=+0.473281133 container start 669991ee670dd2e0bde5bfd32a0ad2c5e66105ccc4f82e100fa823520fa760a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-20T10:31:13.149 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:31:12 smithi162 ceph-mds[52247]: -- 172.21.15.162:0/1092162725 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56403d7bfa40 con 0x56403d792800 2024-04-20T10:31:13.150 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:31:12 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-c[52243]: starting mds.c at 2024-04-20T10:31:13.150 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:31:13 smithi162 bash[52228]: 669991ee670dd2e0bde5bfd32a0ad2c5e66105ccc4f82e100fa823520fa760a8 2024-04-20T10:31:13.150 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:31:13 smithi162 systemd[1]: Started Ceph mds.c for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:31:13.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.708+0000 7f3a6e600640 1 -- 172.21.15.42:0/2372623528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a68102fc0 msgr2=0x7f3a68103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.708+0000 7f3a6e600640 1 --2- 172.21.15.42:0/2372623528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a68102fc0 0x7f3a68103440 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3a50009900 tx=0x7f3a5002bc40 comp rx=0 tx=0).stop 2024-04-20T10:31:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.708+0000 7f3a6e600640 1 -- 172.21.15.42:0/2372623528 shutdown_connections 2024-04-20T10:31:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.708+0000 7f3a6e600640 1 --2- 172.21.15.42:0/2372623528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68103980 0x7f3a6810a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.708+0000 7f3a6e600640 1 --2- 172.21.15.42:0/2372623528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a68102fc0 0x7f3a68103440 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.708+0000 7f3a6e600640 1 --2- 172.21.15.42:0/2372623528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a68101dd0 0x7f3a681021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.708+0000 7f3a6e600640 1 -- 172.21.15.42:0/2372623528 >> 172.21.15.42:0/2372623528 conn(0x7f3a680fd340 msgr2=0x7f3a680ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.708+0000 7f3a6e600640 1 -- 172.21.15.42:0/2372623528 shutdown_connections 2024-04-20T10:31:13.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.708+0000 7f3a6e600640 1 -- 172.21.15.42:0/2372623528 wait complete. 2024-04-20T10:31:13.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.709+0000 7f3a6e600640 1 Processor -- start 2024-04-20T10:31:13.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.709+0000 7f3a6e600640 1 -- start start 2024-04-20T10:31:13.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.710+0000 7f3a6e600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a68101dd0 0x7f3a68075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:13.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.710+0000 7f3a6e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68102fc0 0x7f3a68074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:13.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.710+0000 7f3a6e600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a68103980 0x7f3a68074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:13.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.710+0000 7f3a6e600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6810da50 con 0x7f3a68103980 2024-04-20T10:31:13.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.710+0000 7f3a6e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6810d8d0 con 0x7f3a68102fc0 2024-04-20T10:31:13.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.710+0000 7f3a6e600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6810dbd0 con 0x7f3a68101dd0 2024-04-20T10:31:13.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.710+0000 7f3a6dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a68103980 0x7f3a68074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:13.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.710+0000 7f3a5fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68102fc0 0x7f3a68074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:13.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.710+0000 7f3a5fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68102fc0 0x7f3a68074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52524/0 (socket says 172.21.15.42:52524) 2024-04-20T10:31:13.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.710+0000 7f3a5fe00640 1 -- 172.21.15.42:0/1596615294 learned_addr learned my addr 172.21.15.42:0/1596615294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.710+0000 7f3a6dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a68103980 0x7f3a68074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33254/0 (socket says 172.21.15.42:33254) 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.710+0000 7f3a6d200640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a68101dd0 0x7f3a68075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.711+0000 7f3a6dc00640 1 -- 172.21.15.42:0/1596615294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a68101dd0 msgr2=0x7f3a68075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.711+0000 7f3a6dc00640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a68101dd0 0x7f3a68075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.711+0000 7f3a6dc00640 1 -- 172.21.15.42:0/1596615294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68102fc0 msgr2=0x7f3a68074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.711+0000 7f3a6dc00640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68102fc0 0x7f3a68074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.711+0000 7f3a6dc00640 1 -- 172.21.15.42:0/1596615294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a68075120 con 0x7f3a68103980 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.711+0000 7f3a5fe00640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68102fc0 0x7f3a68074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.711+0000 7f3a6dc00640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a68103980 0x7f3a68074890 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000e970 tx=0x7f3a6000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.711+0000 7f3a66000640 1 -- 172.21.15.42:0/1596615294 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6000ccc0 con 0x7f3a68103980 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.711+0000 7f3a66000640 1 -- 172.21.15.42:0/1596615294 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a60005020 con 0x7f3a68103980 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.711+0000 7f3a66000640 1 -- 172.21.15.42:0/1596615294 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60004a00 con 0x7f3a68103980 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.711+0000 7f3a6e600640 1 -- 172.21.15.42:0/1596615294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a681a2230 con 0x7f3a68103980 2024-04-20T10:31:13.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.711+0000 7f3a6e600640 1 -- 172.21.15.42:0/1596615294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a681a2700 con 0x7f3a68103980 2024-04-20T10:31:13.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.713+0000 7f3a6e600640 1 -- 172.21.15.42:0/1596615294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a680690f0 con 0x7f3a68103980 2024-04-20T10:31:13.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.715+0000 7f3a66000640 1 -- 172.21.15.42:0/1596615294 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3a600051d0 con 0x7f3a68103980 2024-04-20T10:31:13.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.717+0000 7f3a66000640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3a4c0a3c30 0x7f3a4c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:13.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.717+0000 7f3a66000640 1 -- 172.21.15.42:0/1596615294 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3a600cff40 con 0x7f3a68103980 2024-04-20T10:31:13.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.717+0000 7f3a6d200640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3a4c0a3c30 0x7f3a4c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:13.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.718+0000 7f3a6d200640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3a4c0a3c30 0x7f3a4c0a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3a68102e20 tx=0x7f3a58009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.722+0000 7f3a66000640 1 -- 172.21.15.42:0/1596615294 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3a6009cd20 con 0x7f3a68103980 2024-04-20T10:31:13.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.970+0000 7f3a6e600640 1 -- 172.21.15.42:0/1596615294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a68062b60 con 0x7f3a68103980 2024-04-20T10:31:13.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.971+0000 7f3a66000640 1 -- 172.21.15.42:0/1596615294 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f3a6009c620 con 0x7f3a68103980 2024-04-20T10:31:13.972 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:13.972 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-20T10:31:13.972 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-04-20T10:31:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.974+0000 7f3a6e600640 1 -- 172.21.15.42:0/1596615294 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3a4c0a3c30 msgr2=0x7f3a4c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.974+0000 7f3a6e600640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3a4c0a3c30 0x7f3a4c0a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3a68102e20 tx=0x7f3a58009500 comp rx=0 tx=0).stop 2024-04-20T10:31:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.974+0000 7f3a6e600640 1 -- 172.21.15.42:0/1596615294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a68103980 msgr2=0x7f3a68074890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.974+0000 7f3a6e600640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a68103980 0x7f3a68074890 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000e970 tx=0x7f3a6000ee30 comp rx=0 tx=0).stop 2024-04-20T10:31:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.974+0000 7f3a6e600640 1 -- 172.21.15.42:0/1596615294 shutdown_connections 2024-04-20T10:31:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.974+0000 7f3a6e600640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a68103980 0x7f3a68074890 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.974+0000 7f3a6e600640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a68102fc0 0x7f3a68074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.974+0000 7f3a6e600640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3a4c0a3c30 0x7f3a4c0a60f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.974+0000 7f3a6e600640 1 --2- 172.21.15.42:0/1596615294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a68101dd0 0x7f3a68075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.974+0000 7f3a6e600640 1 -- 172.21.15.42:0/1596615294 >> 172.21.15.42:0/1596615294 conn(0x7f3a680fd340 msgr2=0x7f3a681062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:13.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.975+0000 7f3a6e600640 1 -- 172.21.15.42:0/1596615294 shutdown_connections 2024-04-20T10:31:13.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:13.975+0000 7f3a6e600640 1 -- 172.21.15.42:0/1596615294 wait complete. 2024-04-20T10:31:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:14 smithi042 ceph-mon[17078]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:14.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:14.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:14.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:14 smithi042 ceph-mon[17078]: Deploying daemon mds.h on smithi046 2024-04-20T10:31:14.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:14 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.162:6836/2056054864,v1:172.21.15.162:6837/2056054864] up:boot 2024-04-20T10:31:14.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:14 smithi042 ceph-mon[17078]: fsmap 7 up:standby 2024-04-20T10:31:14.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:14 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T10:31:14.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:14 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1596615294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:14.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:14 smithi162 ceph-mon[21854]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:14.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:14.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:14.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:14.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:14.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:14.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:14.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:14 smithi162 ceph-mon[21854]: Deploying daemon mds.h on smithi046 2024-04-20T10:31:14.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:14 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.162:6836/2056054864,v1:172.21.15.162:6837/2056054864] up:boot 2024-04-20T10:31:14.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:14 smithi162 ceph-mon[21854]: fsmap 7 up:standby 2024-04-20T10:31:14.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:14 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T10:31:14.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:14 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1596615294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:14.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:14 smithi046 ceph-mon[21574]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:14.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:14.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:14.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:14.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:14.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:14.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:14.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:14 smithi046 ceph-mon[21574]: Deploying daemon mds.h on smithi046 2024-04-20T10:31:14.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:14 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.162:6836/2056054864,v1:172.21.15.162:6837/2056054864] up:boot 2024-04-20T10:31:14.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:14 smithi046 ceph-mon[21574]: fsmap 7 up:standby 2024-04-20T10:31:14.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:14 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T10:31:14.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:14 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1596615294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:15.553 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:31:15 smithi046 systemd[1]: Starting Ceph mds.h for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:31:15.858 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:31:15 smithi046 podman[51854]: 2024-04-20 10:31:15.463829961 +0000 UTC m=+0.018213281 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:31:15.858 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:31:15 smithi046 podman[51854]: 2024-04-20 10:31:15.585903568 +0000 UTC m=+0.140286878 container create f397eec9989b20c70ebfd92d06e7317f71bb12ea04b17b1774c49c4151c5e268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-h, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-04-20T10:31:15.858 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:31:15 smithi046 podman[51854]: 2024-04-20 10:31:15.85898378 +0000 UTC m=+0.413367090 container init f397eec9989b20c70ebfd92d06e7317f71bb12ea04b17b1774c49c4151c5e268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-20T10:31:15.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:16.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.216+0000 7fb97e600640 1 -- 172.21.15.42:0/1983127474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9781083e0 msgr2=0x7fb978108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.216+0000 7fb97e600640 1 --2- 172.21.15.42:0/1983127474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9781083e0 0x7fb978108800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb968009900 tx=0x7fb96802bc40 comp rx=0 tx=0).stop 2024-04-20T10:31:16.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.216+0000 7fb97e600640 1 -- 172.21.15.42:0/1983127474 shutdown_connections 2024-04-20T10:31:16.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.216+0000 7fb97e600640 1 --2- 172.21.15.42:0/1983127474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb978109f90 0x7fb978110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.217+0000 7fb97e600640 1 --2- 172.21.15.42:0/1983127474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9781095d0 0x7fb978109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.217+0000 7fb97e600640 1 --2- 172.21.15.42:0/1983127474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9781083e0 0x7fb978108800 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.217+0000 7fb97e600640 1 -- 172.21.15.42:0/1983127474 >> 172.21.15.42:0/1983127474 conn(0x7fb9780773a0 msgr2=0x7fb9780797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:16.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.217+0000 7fb97e600640 1 -- 172.21.15.42:0/1983127474 shutdown_connections 2024-04-20T10:31:16.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.217+0000 7fb97e600640 1 -- 172.21.15.42:0/1983127474 wait complete. 2024-04-20T10:31:16.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.218+0000 7fb97e600640 1 Processor -- start 2024-04-20T10:31:16.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.218+0000 7fb97e600640 1 -- start start 2024-04-20T10:31:16.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb97e600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9781083e0 0x7fb97819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb97e600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9781095d0 0x7fb97819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb97e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb978109f90 0x7fb9781a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb97e600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9781140c0 con 0x7fb9781095d0 2024-04-20T10:31:16.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb97e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb978113f40 con 0x7fb978109f90 2024-04-20T10:31:16.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb97e600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb978114240 con 0x7fb9781083e0 2024-04-20T10:31:16.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb977e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9781095d0 0x7fb97819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb97d200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9781083e0 0x7fb97819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb97d200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9781083e0 0x7fb97819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:36598/0 (socket says 172.21.15.42:36598) 2024-04-20T10:31:16.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb97d200640 1 -- 172.21.15.42:0/3460696854 learned_addr learned my addr 172.21.15.42:0/3460696854 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:16.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb977e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9781095d0 0x7fb97819ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33270/0 (socket says 172.21.15.42:33270) 2024-04-20T10:31:16.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb97dc00640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb978109f90 0x7fb9781a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb977e00640 1 -- 172.21.15.42:0/3460696854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9781083e0 msgr2=0x7fb97819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb977e00640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9781083e0 0x7fb97819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb977e00640 1 -- 172.21.15.42:0/3460696854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb978109f90 msgr2=0x7fb9781a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb977e00640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb978109f90 0x7fb9781a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb977e00640 1 -- 172.21.15.42:0/3460696854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9781a6310 con 0x7fb9781095d0 2024-04-20T10:31:16.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb97d200640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9781083e0 0x7fb97819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:16.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.219+0000 7fb97dc00640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb978109f90 0x7fb9781a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:16.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.220+0000 7fb977e00640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9781095d0 0x7fb97819ec30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb96400e9f0 tx=0x7fb96400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:16.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.220+0000 7fb975600640 1 -- 172.21.15.42:0/3460696854 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96400ccc0 con 0x7fb9781095d0 2024-04-20T10:31:16.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.220+0000 7fb97e600640 1 -- 172.21.15.42:0/3460696854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9781a6600 con 0x7fb9781095d0 2024-04-20T10:31:16.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.220+0000 7fb975600640 1 -- 172.21.15.42:0/3460696854 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb964010910 con 0x7fb9781095d0 2024-04-20T10:31:16.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.220+0000 7fb975600640 1 -- 172.21.15.42:0/3460696854 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964018960 con 0x7fb9781095d0 2024-04-20T10:31:16.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.220+0000 7fb97e600640 1 -- 172.21.15.42:0/3460696854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9781a6ba0 con 0x7fb9781095d0 2024-04-20T10:31:16.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.221+0000 7fb97e600640 1 -- 172.21.15.42:0/3460696854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb97810f700 con 0x7fb9781095d0 2024-04-20T10:31:16.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.224+0000 7fb975600640 1 -- 172.21.15.42:0/3460696854 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb964010ac0 con 0x7fb9781095d0 2024-04-20T10:31:16.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.225+0000 7fb975600640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb95c0a3c30 0x7fb95c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.225+0000 7fb975600640 1 -- 172.21.15.42:0/3460696854 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb9640d0300 con 0x7fb9781095d0 2024-04-20T10:31:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.225+0000 7fb97d200640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb95c0a3c30 0x7fb95c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.226+0000 7fb97d200640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb95c0a3c30 0x7fb95c0a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb968005ab0 tx=0x7fb96802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:16.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.230+0000 7fb975600640 1 -- 172.21.15.42:0/3460696854 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb96409ca90 con 0x7fb9781095d0 2024-04-20T10:31:16.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:16 smithi046 ceph-mon[21574]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:16.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:16.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:16.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:16.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:16.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:16 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:16.304 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:31:15 smithi046 podman[51854]: 2024-04-20 10:31:15.861725674 +0000 UTC m=+0.416109183 container start f397eec9989b20c70ebfd92d06e7317f71bb12ea04b17b1774c49c4151c5e268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-20T10:31:16.304 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:31:15 smithi046 ceph-mds[51873]: -- 172.21.15.46:0/1091489949 <== mon.2 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564251e39a40 con 0x564251e0c800 2024-04-20T10:31:16.304 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:31:15 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-h[51869]: starting mds.h at 2024-04-20T10:31:16.304 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:31:15 smithi046 bash[51854]: f397eec9989b20c70ebfd92d06e7317f71bb12ea04b17b1774c49c4151c5e268 2024-04-20T10:31:16.304 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:31:15 smithi046 systemd[1]: Started Ceph mds.h for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:31:16.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:16 smithi042 ceph-mon[17078]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:16.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:16.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:16.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:16.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:16.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:16 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:16.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:16 smithi162 ceph-mon[21854]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:16.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:16.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:16.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:16.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:16.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:16 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:16.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.480+0000 7fb97e600640 1 -- 172.21.15.42:0/3460696854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb978062b60 con 0x7fb9781095d0 2024-04-20T10:31:16.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.481+0000 7fb975600640 1 -- 172.21.15.42:0/3460696854 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7fb9640a0d90 con 0x7fb9781095d0 2024-04-20T10:31:16.482 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:16.482 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-20T10:31:16.482 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 9 2024-04-20T10:31:16.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.484+0000 7fb97e600640 1 -- 172.21.15.42:0/3460696854 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb95c0a3c30 msgr2=0x7fb95c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.484+0000 7fb97e600640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb95c0a3c30 0x7fb95c0a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb968005ab0 tx=0x7fb96802cf90 comp rx=0 tx=0).stop 2024-04-20T10:31:16.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.484+0000 7fb97e600640 1 -- 172.21.15.42:0/3460696854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9781095d0 msgr2=0x7fb97819ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.484+0000 7fb97e600640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9781095d0 0x7fb97819ec30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb96400e9f0 tx=0x7fb96400eeb0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.484+0000 7fb97e600640 1 -- 172.21.15.42:0/3460696854 shutdown_connections 2024-04-20T10:31:16.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.484+0000 7fb97e600640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb978109f90 0x7fb9781a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.484+0000 7fb97e600640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9781095d0 0x7fb97819ec30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.484+0000 7fb97e600640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb95c0a3c30 0x7fb95c0a60f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.484+0000 7fb97e600640 1 --2- 172.21.15.42:0/3460696854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9781083e0 0x7fb97819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.484+0000 7fb97e600640 1 -- 172.21.15.42:0/3460696854 >> 172.21.15.42:0/3460696854 conn(0x7fb9780773a0 msgr2=0x7fb97810c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:16.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.484+0000 7fb97e600640 1 -- 172.21.15.42:0/3460696854 shutdown_connections 2024-04-20T10:31:16.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:16.484+0000 7fb97e600640 1 -- 172.21.15.42:0/3460696854 wait complete. 2024-04-20T10:31:17.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:17.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:17 smithi042 ceph-mon[17078]: Deploying daemon mds.i on smithi162 2024-04-20T10:31:17.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:17 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.46:6836/3787981189,v1:172.21.15.46:6837/3787981189] up:boot 2024-04-20T10:31:17.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:17 smithi042 ceph-mon[17078]: fsmap 8 up:standby 2024-04-20T10:31:17.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:17 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T10:31:17.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:17 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3460696854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:17.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:17 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:17.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:17 smithi162 ceph-mon[21854]: Deploying daemon mds.i on smithi162 2024-04-20T10:31:17.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:17 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.46:6836/3787981189,v1:172.21.15.46:6837/3787981189] up:boot 2024-04-20T10:31:17.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:17 smithi162 ceph-mon[21854]: fsmap 8 up:standby 2024-04-20T10:31:17.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:17 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T10:31:17.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:17 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3460696854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:17.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:17 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:17.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:17 smithi046 ceph-mon[21574]: Deploying daemon mds.i on smithi162 2024-04-20T10:31:17.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:17 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.46:6836/3787981189,v1:172.21.15.46:6837/3787981189] up:boot 2024-04-20T10:31:17.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:17 smithi046 ceph-mon[21574]: fsmap 8 up:standby 2024-04-20T10:31:17.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:17 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T10:31:17.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:17 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3460696854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:18.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:18 smithi042 ceph-mon[17078]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:18.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:18 smithi162 ceph-mon[21854]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:18.506 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:18.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:18 smithi046 ceph-mon[21574]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:18.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.731+0000 7f392a800640 1 -- 172.21.15.42:0/3145542102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3924077b60 msgr2=0x7f3924077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.731+0000 7f392a800640 1 --2- 172.21.15.42:0/3145542102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3924077b60 0x7f3924077fc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f390c009900 tx=0x7f390c02bc40 comp rx=0 tx=0).stop 2024-04-20T10:31:18.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.731+0000 7f392a800640 1 -- 172.21.15.42:0/3145542102 shutdown_connections 2024-04-20T10:31:18.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.731+0000 7f392a800640 1 --2- 172.21.15.42:0/3145542102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3924078500 0x7f3924078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.731+0000 7f392a800640 1 --2- 172.21.15.42:0/3145542102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3924077b60 0x7f3924077fc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.731+0000 7f392a800640 1 --2- 172.21.15.42:0/3145542102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924076920 0x7f3924076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.731+0000 7f392a800640 1 -- 172.21.15.42:0/3145542102 >> 172.21.15.42:0/3145542102 conn(0x7f39240ff750 msgr2=0x7f3924101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:18.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.731+0000 7f392a800640 1 -- 172.21.15.42:0/3145542102 shutdown_connections 2024-04-20T10:31:18.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.731+0000 7f392a800640 1 -- 172.21.15.42:0/3145542102 wait complete. 2024-04-20T10:31:18.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.732+0000 7f392a800640 1 Processor -- start 2024-04-20T10:31:18.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.732+0000 7f392a800640 1 -- start start 2024-04-20T10:31:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.732+0000 7f392a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3924076920 0x7f392419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f392a800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924077b60 0x7f392419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f392a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3924078500 0x7f39241a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f392a800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3924114250 con 0x7f3924078500 2024-04-20T10:31:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f392a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39241140d0 con 0x7f3924076920 2024-04-20T10:31:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f392a800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f39241143d0 con 0x7f3924077b60 2024-04-20T10:31:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f3922a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924077b60 0x7f392419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f3923e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3924078500 0x7f39241a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f3922a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924077b60 0x7f392419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:36622/0 (socket says 172.21.15.42:36622) 2024-04-20T10:31:18.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f3922a00640 1 -- 172.21.15.42:0/794712653 learned_addr learned my addr 172.21.15.42:0/794712653 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:18.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f3923400640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3924076920 0x7f392419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f3922a00640 1 -- 172.21.15.42:0/794712653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3924076920 msgr2=0x7f392419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f3922a00640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3924076920 0x7f392419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f3922a00640 1 -- 172.21.15.42:0/794712653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3924078500 msgr2=0x7f39241a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f3922a00640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3924078500 0x7f39241a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.733+0000 7f3922a00640 1 -- 172.21.15.42:0/794712653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39241a64e0 con 0x7f3924077b60 2024-04-20T10:31:18.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.734+0000 7f3922a00640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924077b60 0x7f392419ee00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f390c009230 tx=0x7f390c02c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.734+0000 7f391be00640 1 -- 172.21.15.42:0/794712653 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c041070 con 0x7f3924077b60 2024-04-20T10:31:18.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.734+0000 7f391be00640 1 -- 172.21.15.42:0/794712653 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390c004450 con 0x7f3924077b60 2024-04-20T10:31:18.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.734+0000 7f391be00640 1 -- 172.21.15.42:0/794712653 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c003d90 con 0x7f3924077b60 2024-04-20T10:31:18.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.734+0000 7f392a800640 1 -- 172.21.15.42:0/794712653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39241a6770 con 0x7f3924077b60 2024-04-20T10:31:18.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.734+0000 7f392a800640 1 -- 172.21.15.42:0/794712653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39241a6c10 con 0x7f3924077b60 2024-04-20T10:31:18.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.736+0000 7f392a800640 1 -- 172.21.15.42:0/794712653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3924110d10 con 0x7f3924077b60 2024-04-20T10:31:18.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.737+0000 7f391be00640 1 -- 172.21.15.42:0/794712653 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f390c044070 con 0x7f3924077b60 2024-04-20T10:31:18.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.739+0000 7f391be00640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f39080a3bb0 0x7f39080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.739+0000 7f391be00640 1 -- 172.21.15.42:0/794712653 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f390c0f1b80 con 0x7f3924077b60 2024-04-20T10:31:18.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.739+0000 7f3923400640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f39080a3bb0 0x7f39080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.740+0000 7f3923400640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f39080a3bb0 0x7f39080a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f39140095f0 tx=0x7f3914005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:18.743+0000 7f391be00640 1 -- 172.21.15.42:0/794712653 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f390c0be960 con 0x7f3924077b60 2024-04-20T10:31:19.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.004+0000 7f392a800640 1 -- 172.21.15.42:0/794712653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3924062b60 con 0x7f3924077b60 2024-04-20T10:31:19.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.005+0000 7f391be00640 1 -- 172.21.15.42:0/794712653 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f390c0be260 con 0x7f3924077b60 2024-04-20T10:31:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:19.006 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-20T10:31:19.006 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 9 2024-04-20T10:31:19.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.008+0000 7f392a800640 1 -- 172.21.15.42:0/794712653 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f39080a3bb0 msgr2=0x7f39080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.008+0000 7f392a800640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f39080a3bb0 0x7f39080a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f39140095f0 tx=0x7f3914005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:19.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.008+0000 7f392a800640 1 -- 172.21.15.42:0/794712653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924077b60 msgr2=0x7f392419ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.008+0000 7f392a800640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924077b60 0x7f392419ee00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f390c009230 tx=0x7f390c02c660 comp rx=0 tx=0).stop 2024-04-20T10:31:19.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.008+0000 7f392a800640 1 -- 172.21.15.42:0/794712653 shutdown_connections 2024-04-20T10:31:19.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.008+0000 7f392a800640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3924078500 0x7f39241a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.008+0000 7f392a800640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f39080a3bb0 0x7f39080a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.008+0000 7f392a800640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3924077b60 0x7f392419ee00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.008+0000 7f392a800640 1 --2- 172.21.15.42:0/794712653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3924076920 0x7f392419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.008+0000 7f392a800640 1 -- 172.21.15.42:0/794712653 >> 172.21.15.42:0/794712653 conn(0x7f39240ff750 msgr2=0x7f3924100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:19.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.008+0000 7f392a800640 1 -- 172.21.15.42:0/794712653 shutdown_connections 2024-04-20T10:31:19.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:19.009+0000 7f392a800640 1 -- 172.21.15.42:0/794712653 wait complete. 2024-04-20T10:31:19.086 INFO:journalctl@ceph.mds.i.smithi162.stdout:Apr 20 10:31:18 smithi162 systemd[1]: Started Ceph mds.i for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:31:19.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:19 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/794712653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:19.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:19.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:19.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:19.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:19.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:19.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:19 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:19.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:19 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/794712653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:19.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:19.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:19.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:19.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:19.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:19.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:19 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:19.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:19 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/794712653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:19.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:19.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:19.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:19.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:19.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:19.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:19 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:20.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:20 smithi042 ceph-mon[17078]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:20.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:20 smithi042 ceph-mon[17078]: Deploying daemon mds.g on smithi042 2024-04-20T10:31:20.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:20 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.162:6838/3710674499,v1:172.21.15.162:6839/3710674499] up:boot 2024-04-20T10:31:20.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:20 smithi042 ceph-mon[17078]: fsmap 9 up:standby 2024-04-20T10:31:20.344 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:20 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T10:31:20.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:20 smithi162 ceph-mon[21854]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:20.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:20 smithi162 ceph-mon[21854]: Deploying daemon mds.g on smithi042 2024-04-20T10:31:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:20 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.162:6838/3710674499,v1:172.21.15.162:6839/3710674499] up:boot 2024-04-20T10:31:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:20 smithi162 ceph-mon[21854]: fsmap 9 up:standby 2024-04-20T10:31:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:20 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T10:31:20.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:20 smithi046 ceph-mon[21574]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:20.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:20 smithi046 ceph-mon[21574]: Deploying daemon mds.g on smithi042 2024-04-20T10:31:20.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:20 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.162:6838/3710674499,v1:172.21.15.162:6839/3710674499] up:boot 2024-04-20T10:31:20.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:20 smithi046 ceph-mon[21574]: fsmap 9 up:standby 2024-04-20T10:31:20.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:20 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T10:31:21.033 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:21.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.260+0000 7f1075c00640 1 -- 172.21.15.42:0/3478497945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10701083e0 msgr2=0x7f1070108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.260+0000 7f1075c00640 1 --2- 172.21.15.42:0/3478497945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10701083e0 0x7f1070108800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1060009900 tx=0x7f106002bc30 comp rx=0 tx=0).stop 2024-04-20T10:31:21.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.260+0000 7f1075c00640 1 -- 172.21.15.42:0/3478497945 shutdown_connections 2024-04-20T10:31:21.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.260+0000 7f1075c00640 1 --2- 172.21.15.42:0/3478497945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1070109f90 0x7f1070110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.260+0000 7f1075c00640 1 --2- 172.21.15.42:0/3478497945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10701095d0 0x7f1070109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.260+0000 7f1075c00640 1 --2- 172.21.15.42:0/3478497945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10701083e0 0x7f1070108800 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.260+0000 7f1075c00640 1 -- 172.21.15.42:0/3478497945 >> 172.21.15.42:0/3478497945 conn(0x7f10700773a0 msgr2=0x7f10700797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:21.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.260+0000 7f1075c00640 1 -- 172.21.15.42:0/3478497945 shutdown_connections 2024-04-20T10:31:21.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.260+0000 7f1075c00640 1 -- 172.21.15.42:0/3478497945 wait complete. 2024-04-20T10:31:21.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.261+0000 7f1075c00640 1 Processor -- start 2024-04-20T10:31:21.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.261+0000 7f1075c00640 1 -- start start 2024-04-20T10:31:21.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.261+0000 7f1075c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10701083e0 0x7f107019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.261+0000 7f1075c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10701095d0 0x7f107019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.261+0000 7f1075c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1070109f90 0x7f10701a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.261+0000 7f1075c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f10701140c0 con 0x7f10701095d0 2024-04-20T10:31:21.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.261+0000 7f1075c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1070113f40 con 0x7f10701083e0 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.261+0000 7f1075c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1070114240 con 0x7f1070109f90 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.262+0000 7f106f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10701095d0 0x7f107019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.262+0000 7f106f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10701095d0 0x7f107019ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34416/0 (socket says 172.21.15.42:34416) 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.262+0000 7f106f400640 1 -- 172.21.15.42:0/3144054972 learned_addr learned my addr 172.21.15.42:0/3144054972 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.262+0000 7f106fe00640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10701083e0 0x7f107019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.262+0000 7f1075200640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1070109f90 0x7f10701a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.262+0000 7f106f400640 1 -- 172.21.15.42:0/3144054972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1070109f90 msgr2=0x7f10701a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.262+0000 7f106f400640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1070109f90 0x7f10701a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.262+0000 7f106f400640 1 -- 172.21.15.42:0/3144054972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10701083e0 msgr2=0x7f107019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.262+0000 7f106f400640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10701083e0 0x7f107019e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.262+0000 7f106f400640 1 -- 172.21.15.42:0/3144054972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10701a6300 con 0x7f10701095d0 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.262+0000 7f106f400640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10701095d0 0x7f107019ec20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f105c00eb90 tx=0x7f105c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.263+0000 7f106cc00640 1 -- 172.21.15.42:0/3144054972 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c004d20 con 0x7f10701095d0 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.263+0000 7f106cc00640 1 -- 172.21.15.42:0/3144054972 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f105c00ee90 con 0x7f10701095d0 2024-04-20T10:31:21.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.263+0000 7f106cc00640 1 -- 172.21.15.42:0/3144054972 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c010830 con 0x7f10701095d0 2024-04-20T10:31:21.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.263+0000 7f1075c00640 1 -- 172.21.15.42:0/3144054972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10701a65f0 con 0x7f10701095d0 2024-04-20T10:31:21.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.263+0000 7f1075c00640 1 -- 172.21.15.42:0/3144054972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10701a6af0 con 0x7f10701095d0 2024-04-20T10:31:21.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.264+0000 7f1075c00640 1 -- 172.21.15.42:0/3144054972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107010f700 con 0x7f10701095d0 2024-04-20T10:31:21.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.267+0000 7f106cc00640 1 -- 172.21.15.42:0/3144054972 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f105c005020 con 0x7f10701095d0 2024-04-20T10:31:21.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.269+0000 7f106cc00640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f10540a3c30 0x7f10540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.269+0000 7f106cc00640 1 -- 172.21.15.42:0/3144054972 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f105c0d05b0 con 0x7f10701095d0 2024-04-20T10:31:21.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.269+0000 7f106fe00640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f10540a3c30 0x7f10540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.270+0000 7f106fe00640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f10540a3c30 0x7f10540a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f106002c170 tx=0x7f106002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:21.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.274+0000 7f106cc00640 1 -- 172.21.15.42:0/3144054972 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f105c09d390 con 0x7f10701095d0 2024-04-20T10:31:21.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.527+0000 7f1075c00640 1 -- 172.21.15.42:0/3144054972 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f107002d5d0 con 0x7f10701095d0 2024-04-20T10:31:21.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.528+0000 7f106cc00640 1 -- 172.21.15.42:0/3144054972 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f105c09cc90 con 0x7f10701095d0 2024-04-20T10:31:21.529 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:21.529 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24568,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T10:31:21.530 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 10 2024-04-20T10:31:21.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.531+0000 7f1075c00640 1 -- 172.21.15.42:0/3144054972 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f10540a3c30 msgr2=0x7f10540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.531+0000 7f1075c00640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f10540a3c30 0x7f10540a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f106002c170 tx=0x7f106002df90 comp rx=0 tx=0).stop 2024-04-20T10:31:21.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.531+0000 7f1075c00640 1 -- 172.21.15.42:0/3144054972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10701095d0 msgr2=0x7f107019ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.531+0000 7f1075c00640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10701095d0 0x7f107019ec20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f105c00eb90 tx=0x7f105c00c6a0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.531+0000 7f1075c00640 1 -- 172.21.15.42:0/3144054972 shutdown_connections 2024-04-20T10:31:21.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.531+0000 7f1075c00640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f10540a3c30 0x7f10540a60f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.531+0000 7f1075c00640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1070109f90 0x7f10701a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.531+0000 7f1075c00640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10701095d0 0x7f107019ec20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.531+0000 7f1075c00640 1 --2- 172.21.15.42:0/3144054972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10701083e0 0x7f107019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.531+0000 7f1075c00640 1 -- 172.21.15.42:0/3144054972 >> 172.21.15.42:0/3144054972 conn(0x7f10700773a0 msgr2=0x7f107010c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:21.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.531+0000 7f1075c00640 1 -- 172.21.15.42:0/3144054972 shutdown_connections 2024-04-20T10:31:21.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:21.532+0000 7f1075c00640 1 -- 172.21.15.42:0/3144054972 wait complete. 2024-04-20T10:31:22.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:22 smithi042 ceph-mon[17078]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:22.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:22 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3144054972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:22.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:22 smithi162 ceph-mon[21854]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:22.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:22 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3144054972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:22.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:22 smithi046 ceph-mon[21574]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:22.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:22 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3144054972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:23.084 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:31:22 smithi042 podman[58056]: 2024-04-20 10:31:22.821217497 +0000 UTC m=+0.145855165 container create 45239af92595ee9df7e10eb5096c3acbbedbc3816bb69f3723c9c73a55be0ba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-g, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-20T10:31:23.085 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:31:23 smithi042 podman[58056]: 2024-04-20 10:31:23.08351042 +0000 UTC m=+0.408148116 container init 45239af92595ee9df7e10eb5096c3acbbedbc3816bb69f3723c9c73a55be0ba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2) 2024-04-20T10:31:23.354 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:31:23 smithi042 podman[58056]: 2024-04-20 10:31:23.094338062 +0000 UTC m=+0.418975710 container start 45239af92595ee9df7e10eb5096c3acbbedbc3816bb69f3723c9c73a55be0ba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-g, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-20T10:31:23.354 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:31:23 smithi042 ceph-mds[58079]: -- 172.21.15.42:0/2289908780 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5563045fba40 con 0x5563045cf000 2024-04-20T10:31:23.355 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:31:23 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-g[58075]: starting mds.g at 2024-04-20T10:31:23.355 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:31:23 smithi042 bash[58056]: 45239af92595ee9df7e10eb5096c3acbbedbc3816bb69f3723c9c73a55be0ba1 2024-04-20T10:31:23.355 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:31:23 smithi042 systemd[1]: Started Ceph mds.g for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:31:23.553 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:23.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.730+0000 7f14eb200640 1 -- 172.21.15.42:0/645317105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec1039b0 msgr2=0x7f14ec10a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:23.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.730+0000 7f14eb200640 1 --2- 172.21.15.42:0/645317105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec1039b0 0x7f14ec10a240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009ca0 tx=0x7f14e402c0f0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.730+0000 7f14eb200640 1 -- 172.21.15.42:0/645317105 shutdown_connections 2024-04-20T10:31:23.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.730+0000 7f14eb200640 1 --2- 172.21.15.42:0/645317105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec1039b0 0x7f14ec10a240 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.730+0000 7f14eb200640 1 --2- 172.21.15.42:0/645317105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ec102ff0 0x7f14ec103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.730+0000 7f14eb200640 1 --2- 172.21.15.42:0/645317105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec101e00 0x7f14ec102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.730+0000 7f14eb200640 1 -- 172.21.15.42:0/645317105 >> 172.21.15.42:0/645317105 conn(0x7f14ec0fd370 msgr2=0x7f14ec0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:23.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.731+0000 7f14eb200640 1 -- 172.21.15.42:0/645317105 shutdown_connections 2024-04-20T10:31:23.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.731+0000 7f14eb200640 1 -- 172.21.15.42:0/645317105 wait complete. 2024-04-20T10:31:23.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.732+0000 7f14eb200640 1 Processor -- start 2024-04-20T10:31:23.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.732+0000 7f14eb200640 1 -- start start 2024-04-20T10:31:23.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.732+0000 7f14eb200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ec101e00 0x7f14ec19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:23.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.732+0000 7f14eb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec102ff0 0x7f14ec19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:23.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.732+0000 7f14eb200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec1039b0 0x7f14ec1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:23.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.732+0000 7f14eb200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec10da80 con 0x7f14ec1039b0 2024-04-20T10:31:23.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.732+0000 7f14eb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec10d900 con 0x7f14ec102ff0 2024-04-20T10:31:23.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.732+0000 7f14eb200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec10dc00 con 0x7f14ec101e00 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.732+0000 7f14e9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec102ff0 0x7f14ec19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.732+0000 7f14e9e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ec101e00 0x7f14ec19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.732+0000 7f14e9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec102ff0 0x7f14ec19a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:35642/0 (socket says 172.21.15.42:35642) 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.733+0000 7f14e9400640 1 -- 172.21.15.42:0/1090619240 learned_addr learned my addr 172.21.15.42:0/1090619240 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.733+0000 7f14ea800640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec1039b0 0x7f14ec1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.733+0000 7f14e9400640 1 -- 172.21.15.42:0/1090619240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ec101e00 msgr2=0x7f14ec19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.733+0000 7f14e9400640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ec101e00 0x7f14ec19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.733+0000 7f14e9400640 1 -- 172.21.15.42:0/1090619240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec1039b0 msgr2=0x7f14ec1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.733+0000 7f14e9400640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec1039b0 0x7f14ec1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.733+0000 7f14e9400640 1 -- 172.21.15.42:0/1090619240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec1a1ed0 con 0x7f14ec102ff0 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.733+0000 7f14e9400640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec102ff0 0x7f14ec19a7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14d800e9f0 tx=0x7f14d800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.733+0000 7f14e2a00640 1 -- 172.21.15.42:0/1090619240 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d800cd80 con 0x7f14ec102ff0 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.734+0000 7f14eb200640 1 -- 172.21.15.42:0/1090619240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ec1a21c0 con 0x7f14ec102ff0 2024-04-20T10:31:23.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.734+0000 7f14e2a00640 1 -- 172.21.15.42:0/1090619240 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d8005500 con 0x7f14ec102ff0 2024-04-20T10:31:23.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.734+0000 7f14e2a00640 1 -- 172.21.15.42:0/1090619240 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8010690 con 0x7f14ec102ff0 2024-04-20T10:31:23.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.734+0000 7f14eb200640 1 -- 172.21.15.42:0/1090619240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ec1a26c0 con 0x7f14ec102ff0 2024-04-20T10:31:23.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.735+0000 7f14eb200640 1 -- 172.21.15.42:0/1090619240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ec1a2980 con 0x7f14ec102ff0 2024-04-20T10:31:23.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.737+0000 7f14e2a00640 1 -- 172.21.15.42:0/1090619240 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f14d80046c0 con 0x7f14ec102ff0 2024-04-20T10:31:23.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.738+0000 7f14e2a00640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f14d00a3bb0 0x7f14d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:23.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.738+0000 7f14e2a00640 1 -- 172.21.15.42:0/1090619240 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f14d80d0380 con 0x7f14ec102ff0 2024-04-20T10:31:23.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.738+0000 7f14e9e00640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f14d00a3bb0 0x7f14d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:23.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.739+0000 7f14e9e00640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f14d00a3bb0 0x7f14d00a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14dc005fd0 tx=0x7f14dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:23.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:23.743+0000 7f14e2a00640 1 -- 172.21.15.42:0/1090619240 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f14d809d160 con 0x7f14ec102ff0 2024-04-20T10:31:24.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.022+0000 7f14eb200640 1 -- 172.21.15.42:0/1090619240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14ec1a2b50 con 0x7f14ec102ff0 2024-04-20T10:31:24.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.023+0000 7f14e2a00640 1 -- 172.21.15.42:0/1090619240 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f14d809ca60 con 0x7f14ec102ff0 2024-04-20T10:31:24.023 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:24.024 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24568,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T10:31:24.024 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 10 2024-04-20T10:31:24.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.025+0000 7f14eb200640 1 -- 172.21.15.42:0/1090619240 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f14d00a3bb0 msgr2=0x7f14d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:24.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.025+0000 7f14eb200640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f14d00a3bb0 0x7f14d00a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14dc005fd0 tx=0x7f14dc00b410 comp rx=0 tx=0).stop 2024-04-20T10:31:24.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.026+0000 7f14eb200640 1 -- 172.21.15.42:0/1090619240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec102ff0 msgr2=0x7f14ec19a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:24.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.026+0000 7f14eb200640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec102ff0 0x7f14ec19a7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14d800e9f0 tx=0x7f14d800edf0 comp rx=0 tx=0).stop 2024-04-20T10:31:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.026+0000 7f14eb200640 1 -- 172.21.15.42:0/1090619240 shutdown_connections 2024-04-20T10:31:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.026+0000 7f14eb200640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14ec1039b0 0x7f14ec1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.026+0000 7f14eb200640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec102ff0 0x7f14ec19a7f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.026+0000 7f14eb200640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f14d00a3bb0 0x7f14d00a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.026+0000 7f14eb200640 1 --2- 172.21.15.42:0/1090619240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ec101e00 0x7f14ec19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.026+0000 7f14eb200640 1 -- 172.21.15.42:0/1090619240 >> 172.21.15.42:0/1090619240 conn(0x7f14ec0fd370 msgr2=0x7f14ec074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.026+0000 7f14eb200640 1 -- 172.21.15.42:0/1090619240 shutdown_connections 2024-04-20T10:31:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:24.026+0000 7f14eb200640 1 -- 172.21.15.42:0/1090619240 wait complete. 2024-04-20T10:31:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:24 smithi042 ceph-mon[17078]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:24 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:24 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:24 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:24 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:24 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:24 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:24.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:24 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1090619240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:24.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:24 smithi162 ceph-mon[21854]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:24.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:24 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:24.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:24 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:24.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:24 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:24.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:24 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:24.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:24 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:24.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:24 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:24.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:24 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1090619240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:24.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:24 smithi046 ceph-mon[21574]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:24.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:24 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:24.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:24 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:24.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:24 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:24.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:24 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:24.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:24 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:24.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:24 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:24.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:24 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1090619240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:25.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:25 smithi162 ceph-mon[21854]: Deploying daemon mds.a on smithi042 2024-04-20T10:31:25.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:25 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.42:6838/2523412756,v1:172.21.15.42:6839/2523412756] up:boot 2024-04-20T10:31:25.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:25 smithi162 ceph-mon[21854]: fsmap 10 up:standby 2024-04-20T10:31:25.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:25 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T10:31:25.431 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:25 smithi042 ceph-mon[17078]: Deploying daemon mds.a on smithi042 2024-04-20T10:31:25.431 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:25 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.42:6838/2523412756,v1:172.21.15.42:6839/2523412756] up:boot 2024-04-20T10:31:25.431 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:25 smithi042 ceph-mon[17078]: fsmap 10 up:standby 2024-04-20T10:31:25.431 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:25 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T10:31:25.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:25 smithi046 ceph-mon[21574]: Deploying daemon mds.a on smithi042 2024-04-20T10:31:25.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:25 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.42:6838/2523412756,v1:172.21.15.42:6839/2523412756] up:boot 2024-04-20T10:31:25.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:25 smithi046 ceph-mon[21574]: fsmap 10 up:standby 2024-04-20T10:31:25.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:25 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T10:31:26.045 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:26.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.343+0000 7fe74a600640 1 -- 172.21.15.42:0/3975023033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe744101fd0 msgr2=0x7fe7441023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:26.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.343+0000 7fe74a600640 1 --2- 172.21.15.42:0/3975023033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe744101fd0 0x7fe7441023f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe734009860 tx=0x7fe73402bb70 comp rx=0 tx=0).stop 2024-04-20T10:31:26.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.343+0000 7fe74a600640 1 -- 172.21.15.42:0/3975023033 shutdown_connections 2024-04-20T10:31:26.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.343+0000 7fe74a600640 1 --2- 172.21.15.42:0/3975023033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe744103b80 0x7fe74410a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:26.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.343+0000 7fe74a600640 1 --2- 172.21.15.42:0/3975023033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7441031c0 0x7fe744103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:26.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.343+0000 7fe74a600640 1 --2- 172.21.15.42:0/3975023033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe744101fd0 0x7fe7441023f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:26.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.343+0000 7fe74a600640 1 -- 172.21.15.42:0/3975023033 >> 172.21.15.42:0/3975023033 conn(0x7fe7440fd540 msgr2=0x7fe7440ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:26.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.344+0000 7fe74a600640 1 -- 172.21.15.42:0/3975023033 shutdown_connections 2024-04-20T10:31:26.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.344+0000 7fe74a600640 1 -- 172.21.15.42:0/3975023033 wait complete. 2024-04-20T10:31:26.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.345+0000 7fe74a600640 1 Processor -- start 2024-04-20T10:31:26.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.345+0000 7fe74a600640 1 -- start start 2024-04-20T10:31:26.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.345+0000 7fe74a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe744101fd0 0x7fe7441982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.345+0000 7fe74a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7441031c0 0x7fe744198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.345+0000 7fe74a600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe744103b80 0x7fe74419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.345+0000 7fe74a600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe74410dc20 con 0x7fe744101fd0 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.345+0000 7fe74a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe74410daa0 con 0x7fe7441031c0 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.345+0000 7fe74a600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe74410dda0 con 0x7fe744103b80 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.346+0000 7fe743e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe744103b80 0x7fe74419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.346+0000 7fe743400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe744101fd0 0x7fe7441982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.346+0000 7fe742a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7441031c0 0x7fe744198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.346+0000 7fe743e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe744103b80 0x7fe74419f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:57224/0 (socket says 172.21.15.42:57224) 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.346+0000 7fe743e00640 1 -- 172.21.15.42:0/3206102994 learned_addr learned my addr 172.21.15.42:0/3206102994 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.346+0000 7fe743400640 1 -- 172.21.15.42:0/3206102994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe744103b80 msgr2=0x7fe74419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.346+0000 7fe743400640 1 --2- 172.21.15.42:0/3206102994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe744103b80 0x7fe74419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.346+0000 7fe743400640 1 -- 172.21.15.42:0/3206102994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7441031c0 msgr2=0x7fe744198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.346+0000 7fe743400640 1 --2- 172.21.15.42:0/3206102994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7441031c0 0x7fe744198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.346+0000 7fe743400640 1 -- 172.21.15.42:0/3206102994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74419ff10 con 0x7fe744101fd0 2024-04-20T10:31:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.346+0000 7fe743400640 1 --2- 172.21.15.42:0/3206102994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe744101fd0 0x7fe7441982f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe73402c0b0 tx=0x7fe7340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.347+0000 7fe73be00640 1 -- 172.21.15.42:0/3206102994 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734041020 con 0x7fe744101fd0 2024-04-20T10:31:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.347+0000 7fe74a600640 1 -- 172.21.15.42:0/3206102994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7441a01a0 con 0x7fe744101fd0 2024-04-20T10:31:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.347+0000 7fe73be00640 1 -- 172.21.15.42:0/3206102994 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7340347b0 con 0x7fe744101fd0 2024-04-20T10:31:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.347+0000 7fe73be00640 1 -- 172.21.15.42:0/3206102994 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734039040 con 0x7fe744101fd0 2024-04-20T10:31:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.347+0000 7fe74a600640 1 -- 172.21.15.42:0/3206102994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7441a0690 con 0x7fe744101fd0 2024-04-20T10:31:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.348+0000 7fe74a600640 1 -- 172.21.15.42:0/3206102994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe744109410 con 0x7fe744101fd0 2024-04-20T10:31:26.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.351+0000 7fe73be00640 1 -- 172.21.15.42:0/3206102994 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe734044020 con 0x7fe744101fd0 2024-04-20T10:31:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.352+0000 7fe73be00640 1 --2- 172.21.15.42:0/3206102994 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe7280a3bb0 0x7fe7280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.352+0000 7fe73be00640 1 -- 172.21.15.42:0/3206102994 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe7340f19c0 con 0x7fe744101fd0 2024-04-20T10:31:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.353+0000 7fe742a00640 1 --2- 172.21.15.42:0/3206102994 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe7280a3bb0 0x7fe7280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.353+0000 7fe742a00640 1 --2- 172.21.15.42:0/3206102994 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe7280a3bb0 0x7fe7280a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe744199800 tx=0x7fe73000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:26.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.362+0000 7fe73be00640 1 -- 172.21.15.42:0/3206102994 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe73404b020 con 0x7fe744101fd0 2024-04-20T10:31:26.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:26 smithi162 ceph-mon[21854]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:26.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:26 smithi046 ceph-mon[21574]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:26.591 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:26 smithi042 ceph-mon[17078]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:26.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.640+0000 7fe74a600640 1 -- 172.21.15.42:0/3206102994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe74402d5d0 con 0x7fe744101fd0 2024-04-20T10:31:26.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.641+0000 7fe73be00640 1 -- 172.21.15.42:0/3206102994 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7fe7340be7a0 con 0x7fe744101fd0 2024-04-20T10:31:26.642 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:26.642 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24568,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T10:31:26.642 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 11 2024-04-20T10:31:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.644+0000 7fe74a600640 1 -- 172.21.15.42:0/3206102994 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe7280a3bb0 msgr2=0x7fe7280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.644+0000 7fe74a600640 1 --2- 172.21.15.42:0/3206102994 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe7280a3bb0 0x7fe7280a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe744199800 tx=0x7fe73000b410 comp rx=0 tx=0).stop 2024-04-20T10:31:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.644+0000 7fe74a600640 1 -- 172.21.15.42:0/3206102994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe744101fd0 msgr2=0x7fe7441982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.644+0000 7fe74a600640 1 --2- 172.21.15.42:0/3206102994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe744101fd0 0x7fe7441982f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe73402c0b0 tx=0x7fe7340042f0 comp rx=0 tx=0).stop 2024-04-20T10:31:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.644+0000 7fe74a600640 1 -- 172.21.15.42:0/3206102994 shutdown_connections 2024-04-20T10:31:26.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.644+0000 7fe74a600640 1 --2- 172.21.15.42:0/3206102994 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe7280a3bb0 0x7fe7280a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:26.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.644+0000 7fe74a600640 1 --2- 172.21.15.42:0/3206102994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe744103b80 0x7fe74419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:26.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.644+0000 7fe74a600640 1 --2- 172.21.15.42:0/3206102994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7441031c0 0x7fe744198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:26.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.644+0000 7fe74a600640 1 --2- 172.21.15.42:0/3206102994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe744101fd0 0x7fe7441982f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:26.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.644+0000 7fe74a600640 1 -- 172.21.15.42:0/3206102994 >> 172.21.15.42:0/3206102994 conn(0x7fe7440fd540 msgr2=0x7fe7441065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:26.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.644+0000 7fe74a600640 1 -- 172.21.15.42:0/3206102994 shutdown_connections 2024-04-20T10:31:26.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:26.644+0000 7fe74a600640 1 -- 172.21.15.42:0/3206102994 wait complete. 2024-04-20T10:31:27.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:27 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3206102994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:27.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:27 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3206102994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:27.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:27 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3206102994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:27.604 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 20 10:31:27 smithi042 systemd[1]: Started Ceph mds.a for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:31:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:28 smithi162 ceph-mon[21854]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:28 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:28 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:28 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:28 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:28 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:28 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:28.453 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:28 smithi046 ceph-mon[21574]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:28.453 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:28 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:28.453 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:28 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:28.453 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:28 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:28.453 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:28 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:28.453 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:28 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:28.453 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:28 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:28.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:28 smithi042 ceph-mon[17078]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:28.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:28 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:28.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:28 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:28.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:28 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:28.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:28 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T10:31:28.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:28 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T10:31:28.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:28 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:28.664 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.890+0000 7fcd6a000640 1 -- 172.21.15.42:0/2654621061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6410a170 msgr2=0x7fcd64110a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.890+0000 7fcd6a000640 1 --2- 172.21.15.42:0/2654621061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6410a170 0x7fcd64110a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009ca0 tx=0x7fcd5c02c0f0 comp rx=0 tx=0).stop 2024-04-20T10:31:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.891+0000 7fcd6a000640 1 -- 172.21.15.42:0/2654621061 shutdown_connections 2024-04-20T10:31:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.891+0000 7fcd6a000640 1 --2- 172.21.15.42:0/2654621061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6410a170 0x7fcd64110a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.891+0000 7fcd6a000640 1 --2- 172.21.15.42:0/2654621061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd641097b0 0x7fcd64109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.891+0000 7fcd6a000640 1 --2- 172.21.15.42:0/2654621061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd641085c0 0x7fcd641089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.891+0000 7fcd6a000640 1 -- 172.21.15.42:0/2654621061 >> 172.21.15.42:0/2654621061 conn(0x7fcd64077580 msgr2=0x7fcd640799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.891+0000 7fcd6a000640 1 -- 172.21.15.42:0/2654621061 shutdown_connections 2024-04-20T10:31:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.891+0000 7fcd6a000640 1 -- 172.21.15.42:0/2654621061 wait complete. 2024-04-20T10:31:28.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.892+0000 7fcd6a000640 1 Processor -- start 2024-04-20T10:31:28.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.892+0000 7fcd6a000640 1 -- start start 2024-04-20T10:31:28.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.892+0000 7fcd6a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd641085c0 0x7fcd6419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:28.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.892+0000 7fcd6a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd641097b0 0x7fcd6419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:28.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.892+0000 7fcd6a000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd6410a170 0x7fcd641a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:28.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.892+0000 7fcd6a000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcd64114240 con 0x7fcd641097b0 2024-04-20T10:31:28.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.892+0000 7fcd6a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd641140c0 con 0x7fcd641085c0 2024-04-20T10:31:28.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.892+0000 7fcd6a000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fcd641143c0 con 0x7fcd6410a170 2024-04-20T10:31:28.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.892+0000 7fcd63400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd641085c0 0x7fcd6419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:28.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.893+0000 7fcd62a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd641097b0 0x7fcd6419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:28.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.893+0000 7fcd63400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd641085c0 0x7fcd6419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42228/0 (socket says 172.21.15.42:42228) 2024-04-20T10:31:28.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.893+0000 7fcd63400640 1 -- 172.21.15.42:0/1746738943 learned_addr learned my addr 172.21.15.42:0/1746738943 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:28.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.893+0000 7fcd63e00640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd6410a170 0x7fcd641a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:28.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.893+0000 7fcd63400640 1 -- 172.21.15.42:0/1746738943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd6410a170 msgr2=0x7fcd641a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:28.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.893+0000 7fcd63400640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd6410a170 0x7fcd641a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.893+0000 7fcd63400640 1 -- 172.21.15.42:0/1746738943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd641097b0 msgr2=0x7fcd6419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.893+0000 7fcd63400640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd641097b0 0x7fcd6419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.893+0000 7fcd63400640 1 -- 172.21.15.42:0/1746738943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd641a6480 con 0x7fcd641085c0 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.893+0000 7fcd63400640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd641085c0 0x7fcd6419e860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400e9c0 tx=0x7fcd5400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.893+0000 7fcd5be00640 1 -- 172.21.15.42:0/1746738943 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5400cd80 con 0x7fcd641085c0 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.893+0000 7fcd5be00640 1 -- 172.21.15.42:0/1746738943 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd540035f0 con 0x7fcd641085c0 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.894+0000 7fcd6a000640 1 -- 172.21.15.42:0/1746738943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd641a6770 con 0x7fcd641085c0 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.894+0000 7fcd5be00640 1 -- 172.21.15.42:0/1746738943 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54010690 con 0x7fcd641085c0 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.894+0000 7fcd6a000640 1 -- 172.21.15.42:0/1746738943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd641a6c70 con 0x7fcd641085c0 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.895+0000 7fcd6a000640 1 -- 172.21.15.42:0/1746738943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd6410f8e0 con 0x7fcd641085c0 2024-04-20T10:31:28.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.897+0000 7fcd5be00640 1 -- 172.21.15.42:0/1746738943 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcd540037a0 con 0x7fcd641085c0 2024-04-20T10:31:28.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.898+0000 7fcd5be00640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fcd480a3bb0 0x7fcd480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:28.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.898+0000 7fcd5be00640 1 -- 172.21.15.42:0/1746738943 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcd540d03f0 con 0x7fcd641085c0 2024-04-20T10:31:28.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.899+0000 7fcd62a00640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fcd480a3bb0 0x7fcd480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:28.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.899+0000 7fcd62a00640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fcd480a3bb0 0x7fcd480a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcd6419fd70 tx=0x7fcd50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:28.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:28.900+0000 7fcd5be00640 1 -- 172.21.15.42:0/1746738943 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcd5409d1d0 con 0x7fcd641085c0 2024-04-20T10:31:29.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.151+0000 7fcd6a000640 1 -- 172.21.15.42:0/1746738943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd6402d5d0 con 0x7fcd641085c0 2024-04-20T10:31:29.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.153+0000 7fcd5be00640 1 -- 172.21.15.42:0/1746738943 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7fcd5409cad0 con 0x7fcd641085c0 2024-04-20T10:31:29.154 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:29.154 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24568,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T10:31:29.154 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 12 2024-04-20T10:31:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.156+0000 7fcd6a000640 1 -- 172.21.15.42:0/1746738943 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fcd480a3bb0 msgr2=0x7fcd480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.156+0000 7fcd6a000640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fcd480a3bb0 0x7fcd480a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcd6419fd70 tx=0x7fcd50005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.156+0000 7fcd6a000640 1 -- 172.21.15.42:0/1746738943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd641085c0 msgr2=0x7fcd6419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.156+0000 7fcd6a000640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd641085c0 0x7fcd6419e860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400e9c0 tx=0x7fcd5400ee80 comp rx=0 tx=0).stop 2024-04-20T10:31:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.156+0000 7fcd6a000640 1 -- 172.21.15.42:0/1746738943 shutdown_connections 2024-04-20T10:31:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.156+0000 7fcd6a000640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fcd480a3bb0 0x7fcd480a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.156+0000 7fcd6a000640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd6410a170 0x7fcd641a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.156+0000 7fcd6a000640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd641097b0 0x7fcd6419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.156+0000 7fcd6a000640 1 --2- 172.21.15.42:0/1746738943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd641085c0 0x7fcd6419e860 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.156+0000 7fcd6a000640 1 -- 172.21.15.42:0/1746738943 >> 172.21.15.42:0/1746738943 conn(0x7fcd64077580 msgr2=0x7fcd6410cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.156+0000 7fcd6a000640 1 -- 172.21.15.42:0/1746738943 shutdown_connections 2024-04-20T10:31:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:29.156+0000 7fcd6a000640 1 -- 172.21.15.42:0/1746738943 wait complete. 2024-04-20T10:31:29.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:29 smithi162 ceph-mon[21854]: Deploying daemon mds.b on smithi046 2024-04-20T10:31:29.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:29 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.42:6840/3615312070,v1:172.21.15.42:6841/3615312070] up:boot 2024-04-20T10:31:29.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:29 smithi162 ceph-mon[21854]: fsmap 11 up:standby 2024-04-20T10:31:29.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:29 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T10:31:29.530 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:29 smithi046 ceph-mon[21574]: Deploying daemon mds.b on smithi046 2024-04-20T10:31:29.530 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:29 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.42:6840/3615312070,v1:172.21.15.42:6841/3615312070] up:boot 2024-04-20T10:31:29.530 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:29 smithi046 ceph-mon[21574]: fsmap 11 up:standby 2024-04-20T10:31:29.530 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:29 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T10:31:29.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:29 smithi042 ceph-mon[17078]: Deploying daemon mds.b on smithi046 2024-04-20T10:31:29.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:29 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.42:6840/3615312070,v1:172.21.15.42:6841/3615312070] up:boot 2024-04-20T10:31:29.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:29 smithi042 ceph-mon[17078]: fsmap 11 up:standby 2024-04-20T10:31:29.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:29 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T10:31:29.803 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:31:29 smithi046 systemd[1]: Starting Ceph mds.b for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:31:30.206 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:30 smithi046 ceph-mon[21574]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:30.206 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:30 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1746738943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:30.206 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:31:29 smithi046 podman[52171]: 2024-04-20 10:31:29.854899952 +0000 UTC m=+0.018599917 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:31:30.206 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:31:29 smithi046 podman[52171]: 2024-04-20 10:31:29.961126323 +0000 UTC m=+0.124826265 container create bc113dea996017e7fc033c439d1d7b5b87c67ca8c3364bbe1ddea7dfdb5ef9c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:31:30.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:30 smithi162 ceph-mon[21854]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:30.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:30 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1746738943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:30.553 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:31:30 smithi046 podman[52171]: 2024-04-20 10:31:30.206894462 +0000 UTC m=+0.370594411 container init bc113dea996017e7fc033c439d1d7b5b87c67ca8c3364bbe1ddea7dfdb5ef9c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-20T10:31:30.554 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:31:30 smithi046 podman[52171]: 2024-04-20 10:31:30.210061066 +0000 UTC m=+0.373761006 container start bc113dea996017e7fc033c439d1d7b5b87c67ca8c3364bbe1ddea7dfdb5ef9c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-20T10:31:30.554 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:31:30 smithi046 ceph-mds[52190]: -- 172.21.15.46:0/4080528466 <== mon.2 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577e6d25a40 con 0x5577e6cf9000 2024-04-20T10:31:30.554 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:31:30 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b[52186]: starting mds.b at 2024-04-20T10:31:30.554 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:31:30 smithi046 bash[52171]: bc113dea996017e7fc033c439d1d7b5b87c67ca8c3364bbe1ddea7dfdb5ef9c2 2024-04-20T10:31:30.554 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:31:30 smithi046 systemd[1]: Started Ceph mds.b for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:31:30.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:30 smithi042 ceph-mon[17078]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:30.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:30 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1746738943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:31.177 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:31.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.387+0000 7f573ac00640 1 -- 172.21.15.42:0/4047017984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57341051f0 msgr2=0x7f5734105670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:31.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.387+0000 7f573ac00640 1 --2- 172.21.15.42:0/4047017984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57341051f0 0x7f5734105670 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5720009900 tx=0x7f572002bc30 comp rx=0 tx=0).stop 2024-04-20T10:31:31.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.387+0000 7f573ac00640 1 -- 172.21.15.42:0/4047017984 shutdown_connections 2024-04-20T10:31:31.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.387+0000 7f573ac00640 1 --2- 172.21.15.42:0/4047017984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5734105bb0 0x7f573410c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.387+0000 7f573ac00640 1 --2- 172.21.15.42:0/4047017984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57341051f0 0x7f5734105670 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.387+0000 7f573ac00640 1 --2- 172.21.15.42:0/4047017984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734104000 0x7f5734104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.387+0000 7f573ac00640 1 -- 172.21.15.42:0/4047017984 >> 172.21.15.42:0/4047017984 conn(0x7f57340ff570 msgr2=0x7f57341019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:31.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.387+0000 7f573ac00640 1 -- 172.21.15.42:0/4047017984 shutdown_connections 2024-04-20T10:31:31.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.387+0000 7f573ac00640 1 -- 172.21.15.42:0/4047017984 wait complete. 2024-04-20T10:31:31.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.388+0000 7f573ac00640 1 Processor -- start 2024-04-20T10:31:31.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.388+0000 7f573ac00640 1 -- start start 2024-04-20T10:31:31.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.389+0000 7f573ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5734104000 0x7f5734079d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:31.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.389+0000 7f573ac00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57341051f0 0x7f573407a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:31.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.389+0000 7f573ac00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734105bb0 0x7f5734076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.389+0000 7f573ac00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f573410fce0 con 0x7f5734105bb0 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.389+0000 7f573ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f573410fb60 con 0x7f5734104000 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.389+0000 7f573ac00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f573410fe60 con 0x7f57341051f0 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.389+0000 7f5739800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5734104000 0x7f5734079d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.389+0000 7f5739800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5734104000 0x7f5734079d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42238/0 (socket says 172.21.15.42:42238) 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.389+0000 7f5739800640 1 -- 172.21.15.42:0/2365534619 learned_addr learned my addr 172.21.15.42:0/2365534619 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.389+0000 7f5738e00640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57341051f0 0x7f573407a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.389+0000 7f573a200640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734105bb0 0x7f5734076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.390+0000 7f5739800640 1 -- 172.21.15.42:0/2365534619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57341051f0 msgr2=0x7f573407a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.390+0000 7f5739800640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57341051f0 0x7f573407a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.390+0000 7f5739800640 1 -- 172.21.15.42:0/2365534619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734105bb0 msgr2=0x7f5734076880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.390+0000 7f5739800640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734105bb0 0x7f5734076880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.390+0000 7f5739800640 1 -- 172.21.15.42:0/2365534619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5734077140 con 0x7f5734104000 2024-04-20T10:31:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.390+0000 7f5739800640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5734104000 0x7f5734079d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f572800e9c0 tx=0x7f572800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:31.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.390+0000 7f5732000640 1 -- 172.21.15.42:0/2365534619 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572800cde0 con 0x7f5734104000 2024-04-20T10:31:31.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.390+0000 7f573ac00640 1 -- 172.21.15.42:0/2365534619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57341a67c0 con 0x7f5734104000 2024-04-20T10:31:31.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.390+0000 7f5732000640 1 -- 172.21.15.42:0/2365534619 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5728010910 con 0x7f5734104000 2024-04-20T10:31:31.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.390+0000 7f5732000640 1 -- 172.21.15.42:0/2365534619 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728018a10 con 0x7f5734104000 2024-04-20T10:31:31.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.390+0000 7f573ac00640 1 -- 172.21.15.42:0/2365534619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57341a6c20 con 0x7f5734104000 2024-04-20T10:31:31.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.392+0000 7f573ac00640 1 -- 172.21.15.42:0/2365534619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f573410b2c0 con 0x7f5734104000 2024-04-20T10:31:31.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.394+0000 7f5732000640 1 -- 172.21.15.42:0/2365534619 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f572801f020 con 0x7f5734104000 2024-04-20T10:31:31.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.396+0000 7f5732000640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f571c0a3c30 0x7f571c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:31.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.396+0000 7f5732000640 1 -- 172.21.15.42:0/2365534619 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f57280d01d0 con 0x7f5734104000 2024-04-20T10:31:31.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.396+0000 7f5738e00640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f571c0a3c30 0x7f571c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:31.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.397+0000 7f5738e00640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f571c0a3c30 0x7f571c0a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5720000c00 tx=0x7f572002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:31.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.400+0000 7f5732000640 1 -- 172.21.15.42:0/2365534619 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f572809cfb0 con 0x7f5734104000 2024-04-20T10:31:31.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:31 smithi042 ceph-mon[17078]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:31.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:31:31.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:31 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] up:boot 2024-04-20T10:31:31.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:31 smithi042 ceph-mon[17078]: fsmap 12 up:standby 2024-04-20T10:31:31.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T10:31:31.619 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.619 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.619 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.619 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.619 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.619 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:31 smithi162 ceph-mon[21854]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:31.620 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.620 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:31:31.620 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:31 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] up:boot 2024-04-20T10:31:31.620 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:31 smithi162 ceph-mon[21854]: fsmap 12 up:standby 2024-04-20T10:31:31.620 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T10:31:31.636 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.636 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.636 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.636 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.636 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.636 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:31 smithi046 ceph-mon[21574]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:31.637 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:31.637 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:31:31.637 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:31 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] up:boot 2024-04-20T10:31:31.637 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:31 smithi046 ceph-mon[21574]: fsmap 12 up:standby 2024-04-20T10:31:31.637 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T10:31:31.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.668+0000 7f573ac00640 1 -- 172.21.15.42:0/2365534619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5734062c30 con 0x7f5734104000 2024-04-20T10:31:31.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.669+0000 7f5732000640 1 -- 172.21.15.42:0/2365534619 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7f572809c8b0 con 0x7f5734104000 2024-04-20T10:31:31.670 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:31.670 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24568,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-20T10:31:31.671 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-04-20T10:31:31.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.672+0000 7f573ac00640 1 -- 172.21.15.42:0/2365534619 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f571c0a3c30 msgr2=0x7f571c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:31.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.672+0000 7f573ac00640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f571c0a3c30 0x7f571c0a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5720000c00 tx=0x7f572002df90 comp rx=0 tx=0).stop 2024-04-20T10:31:31.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.672+0000 7f573ac00640 1 -- 172.21.15.42:0/2365534619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5734104000 msgr2=0x7f5734079d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:31.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.672+0000 7f573ac00640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5734104000 0x7f5734079d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f572800e9c0 tx=0x7f572800ee80 comp rx=0 tx=0).stop 2024-04-20T10:31:31.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.673+0000 7f573ac00640 1 -- 172.21.15.42:0/2365534619 shutdown_connections 2024-04-20T10:31:31.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.673+0000 7f573ac00640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734105bb0 0x7f5734076880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.673+0000 7f573ac00640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f571c0a3c30 0x7f571c0a60f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.673+0000 7f573ac00640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57341051f0 0x7f573407a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.673+0000 7f573ac00640 1 --2- 172.21.15.42:0/2365534619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5734104000 0x7f5734079d10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.673+0000 7f573ac00640 1 -- 172.21.15.42:0/2365534619 >> 172.21.15.42:0/2365534619 conn(0x7f57340ff570 msgr2=0x7f5734108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.673+0000 7f573ac00640 1 -- 172.21.15.42:0/2365534619 shutdown_connections 2024-04-20T10:31:31.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.673+0000 7f573ac00640 1 -- 172.21.15.42:0/2365534619 wait complete. 2024-04-20T10:31:31.693 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:31:31.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.885+0000 7efce5600640 1 -- 172.21.15.42:0/1269921209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce01041e0 msgr2=0x7efce0104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:31.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.885+0000 7efce5600640 1 --2- 172.21.15.42:0/1269921209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce01041e0 0x7efce0104600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009900 tx=0x7efcd002bc30 comp rx=0 tx=0).stop 2024-04-20T10:31:31.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.885+0000 7efce5600640 1 -- 172.21.15.42:0/1269921209 shutdown_connections 2024-04-20T10:31:31.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.885+0000 7efce5600640 1 --2- 172.21.15.42:0/1269921209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0105d90 0x7efce010c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.885+0000 7efce5600640 1 --2- 172.21.15.42:0/1269921209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce01053d0 0x7efce0105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.885+0000 7efce5600640 1 --2- 172.21.15.42:0/1269921209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce01041e0 0x7efce0104600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.885+0000 7efce5600640 1 -- 172.21.15.42:0/1269921209 >> 172.21.15.42:0/1269921209 conn(0x7efce00ff750 msgr2=0x7efce0101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:31.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.885+0000 7efce5600640 1 -- 172.21.15.42:0/1269921209 shutdown_connections 2024-04-20T10:31:31.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.885+0000 7efce5600640 1 -- 172.21.15.42:0/1269921209 wait complete. 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.886+0000 7efce5600640 1 Processor -- start 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.886+0000 7efce5600640 1 -- start start 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efce5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce01041e0 0x7efce019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efce5600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce01053d0 0x7efce019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efce5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0105d90 0x7efce01a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efce5600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efce010fec0 con 0x7efce01041e0 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efce5600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efce010fd40 con 0x7efce0105d90 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efce5600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efce0110040 con 0x7efce01053d0 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efcde000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce01053d0 0x7efce019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efcdea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce01041e0 0x7efce019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efcde000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce01053d0 0x7efce019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:44840/0 (socket says 172.21.15.42:44840) 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efcde000640 1 -- 172.21.15.42:0/1144921904 learned_addr learned my addr 172.21.15.42:0/1144921904 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:31.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efcdf400640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0105d90 0x7efce01a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:31.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efcde000640 1 -- 172.21.15.42:0/1144921904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0105d90 msgr2=0x7efce01a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:31.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efcde000640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0105d90 0x7efce01a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efcde000640 1 -- 172.21.15.42:0/1144921904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce01041e0 msgr2=0x7efce019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:31.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efcde000640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce01041e0 0x7efce019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:31.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efcde000640 1 -- 172.21.15.42:0/1144921904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce01a20e0 con 0x7efce01053d0 2024-04-20T10:31:31.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.887+0000 7efcdf400640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0105d90 0x7efce01a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:31:31.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.888+0000 7efcde000640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce01053d0 0x7efce019aa00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efccc00e9f0 tx=0x7efccc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:31.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.888+0000 7efcd7400640 1 -- 172.21.15.42:0/1144921904 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc00cd80 con 0x7efce01053d0 2024-04-20T10:31:31.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.888+0000 7efcd7400640 1 -- 172.21.15.42:0/1144921904 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efccc010910 con 0x7efce01053d0 2024-04-20T10:31:31.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.888+0000 7efce5600640 1 -- 172.21.15.42:0/1144921904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce01a23d0 con 0x7efce01053d0 2024-04-20T10:31:31.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.888+0000 7efcd7400640 1 -- 172.21.15.42:0/1144921904 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc018990 con 0x7efce01053d0 2024-04-20T10:31:31.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.888+0000 7efce5600640 1 -- 172.21.15.42:0/1144921904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce01a2970 con 0x7efce01053d0 2024-04-20T10:31:31.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.889+0000 7efce5600640 1 -- 172.21.15.42:0/1144921904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce010b500 con 0x7efce01053d0 2024-04-20T10:31:31.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.892+0000 7efcd7400640 1 -- 172.21.15.42:0/1144921904 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efccc010ac0 con 0x7efce01053d0 2024-04-20T10:31:31.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.893+0000 7efcd7400640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7efcc40a3c30 0x7efcc40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:31.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.893+0000 7efcd7400640 1 -- 172.21.15.42:0/1144921904 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efccc0d0320 con 0x7efce01053d0 2024-04-20T10:31:31.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.893+0000 7efcdea00640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7efcc40a3c30 0x7efcc40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:31.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.894+0000 7efcdea00640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7efcc40a3c30 0x7efcc40a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efcd002c170 tx=0x7efcd002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:31.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:31.897+0000 7efcd7400640 1 -- 172.21.15.42:0/1144921904 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efccc09d100 con 0x7efce01053d0 2024-04-20T10:31:32.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.105+0000 7efce5600640 1 -- 172.21.15.42:0/1144921904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efce0062b60 con 0x7efce01053d0 2024-04-20T10:31:32.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.106+0000 7efcd7400640 1 -- 172.21.15.42:0/1144921904 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18434 (secure 0 0 0) 0x7efccc09ca00 con 0x7efce01053d0 2024-04-20T10:31:32.107 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:32.107 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:30:37.958499+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:31:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.109+0000 7efce5600640 1 -- 172.21.15.42:0/1144921904 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7efcc40a3c30 msgr2=0x7efcc40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.109+0000 7efce5600640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7efcc40a3c30 0x7efcc40a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efcd002c170 tx=0x7efcd002df90 comp rx=0 tx=0).stop 2024-04-20T10:31:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.109+0000 7efce5600640 1 -- 172.21.15.42:0/1144921904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce01053d0 msgr2=0x7efce019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.109+0000 7efce5600640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce01053d0 0x7efce019aa00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efccc00e9f0 tx=0x7efccc00eeb0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.109+0000 7efce5600640 1 -- 172.21.15.42:0/1144921904 shutdown_connections 2024-04-20T10:31:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.109+0000 7efce5600640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0105d90 0x7efce01a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.109+0000 7efce5600640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7efcc40a3c30 0x7efcc40a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.109+0000 7efce5600640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efce01053d0 0x7efce019aa00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.109+0000 7efce5600640 1 --2- 172.21.15.42:0/1144921904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce01041e0 0x7efce019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.109+0000 7efce5600640 1 -- 172.21.15.42:0/1144921904 >> 172.21.15.42:0/1144921904 conn(0x7efce00ff750 msgr2=0x7efce01086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.109+0000 7efce5600640 1 -- 172.21.15.42:0/1144921904 shutdown_connections 2024-04-20T10:31:32.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.109+0000 7efce5600640 1 -- 172.21.15.42:0/1144921904 wait complete. 2024-04-20T10:31:32.130 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:26:16.537072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:31:32.131 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:31:32.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.325+0000 7f025be00640 1 -- 172.21.15.42:0/1203120392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c105130 msgr2=0x7f025c107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:32.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.325+0000 7f025be00640 1 --2- 172.21.15.42:0/1203120392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c105130 0x7f025c107520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0248009900 tx=0x7f024802bc40 comp rx=0 tx=0).stop 2024-04-20T10:31:32.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.325+0000 7f025be00640 1 -- 172.21.15.42:0/1203120392 shutdown_connections 2024-04-20T10:31:32.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.325+0000 7f025be00640 1 --2- 172.21.15.42:0/1203120392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f025c107a60 0x7f025c109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.325+0000 7f025be00640 1 --2- 172.21.15.42:0/1203120392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c105130 0x7f025c107520 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.325+0000 7f025be00640 1 --2- 172.21.15.42:0/1203120392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f025c102800 0x7f025c104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.325+0000 7f025be00640 1 -- 172.21.15.42:0/1203120392 >> 172.21.15.42:0/1203120392 conn(0x7f025c0fc230 msgr2=0x7f025c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:32.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.325+0000 7f025be00640 1 -- 172.21.15.42:0/1203120392 shutdown_connections 2024-04-20T10:31:32.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.325+0000 7f025be00640 1 -- 172.21.15.42:0/1203120392 wait complete. 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.326+0000 7f025be00640 1 Processor -- start 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.326+0000 7f025be00640 1 -- start start 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f025c102800 0x7f025c196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c105130 0x7f025c196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f025c107a60 0x7f025c193620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f025c10da60 con 0x7f025c107a60 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f025c10d8e0 con 0x7f025c105130 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025be00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f025c10dbe0 con 0x7f025c102800 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025aa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f025c102800 0x7f025c196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c105130 0x7f025c196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025aa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f025c102800 0x7f025c196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:44868/0 (socket says 172.21.15.42:44868) 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025aa00640 1 -- 172.21.15.42:0/220106003 learned_addr learned my addr 172.21.15.42:0/220106003 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025b400640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f025c107a60 0x7f025c193620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025a000640 1 -- 172.21.15.42:0/220106003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f025c102800 msgr2=0x7f025c196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:32.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025a000640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f025c102800 0x7f025c196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025a000640 1 -- 172.21.15.42:0/220106003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f025c107a60 msgr2=0x7f025c193620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:32.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025a000640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f025c107a60 0x7f025c193620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.327+0000 7f025a000640 1 -- 172.21.15.42:0/220106003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c193ee0 con 0x7f025c105130 2024-04-20T10:31:32.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.328+0000 7f025a000640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c105130 0x7f025c196ff0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0248009230 tx=0x7f024802c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:32.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.328+0000 7f0253400640 1 -- 172.21.15.42:0/220106003 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248041070 con 0x7f025c105130 2024-04-20T10:31:32.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.328+0000 7f0253400640 1 -- 172.21.15.42:0/220106003 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02480054e0 con 0x7f025c105130 2024-04-20T10:31:32.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.328+0000 7f025be00640 1 -- 172.21.15.42:0/220106003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c1a6700 con 0x7f025c105130 2024-04-20T10:31:32.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.328+0000 7f0253400640 1 -- 172.21.15.42:0/220106003 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248003e40 con 0x7f025c105130 2024-04-20T10:31:32.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.328+0000 7f025be00640 1 -- 172.21.15.42:0/220106003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c1a6ab0 con 0x7f025c105130 2024-04-20T10:31:32.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.329+0000 7f025be00640 1 -- 172.21.15.42:0/220106003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025c1095b0 con 0x7f025c105130 2024-04-20T10:31:32.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.331+0000 7f0253400640 1 -- 172.21.15.42:0/220106003 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0248044070 con 0x7f025c105130 2024-04-20T10:31:32.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.333+0000 7f0253400640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f02400a3c30 0x7f02400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:32.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.333+0000 7f0253400640 1 -- 172.21.15.42:0/220106003 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f02480f1960 con 0x7f025c105130 2024-04-20T10:31:32.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.333+0000 7f025aa00640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f02400a3c30 0x7f02400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:32.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.334+0000 7f025aa00640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f02400a3c30 0x7f02400a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f024c009cd0 tx=0x7f024c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:32.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.337+0000 7f0253400640 1 -- 172.21.15.42:0/220106003 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f02480be0f0 con 0x7f025c105130 2024-04-20T10:31:32.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.545+0000 7f025be00640 1 -- 172.21.15.42:0/220106003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f025c062b60 con 0x7f025c105130 2024-04-20T10:31:32.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.547+0000 7f0253400640 1 -- 172.21.15.42:0/220106003 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f02480c23f0 con 0x7f025c105130 2024-04-20T10:31:32.548 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-20T10:31:32.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.550+0000 7f025be00640 1 -- 172.21.15.42:0/220106003 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f02400a3c30 msgr2=0x7f02400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:32.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.550+0000 7f025be00640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f02400a3c30 0x7f02400a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f024c009cd0 tx=0x7f024c005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:32.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.550+0000 7f025be00640 1 -- 172.21.15.42:0/220106003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c105130 msgr2=0x7f025c196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:32.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.550+0000 7f025be00640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c105130 0x7f025c196ff0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0248009230 tx=0x7f024802c660 comp rx=0 tx=0).stop 2024-04-20T10:31:32.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.550+0000 7f025be00640 1 -- 172.21.15.42:0/220106003 shutdown_connections 2024-04-20T10:31:32.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.550+0000 7f025be00640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f025c107a60 0x7f025c193620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.550+0000 7f025be00640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f025c105130 0x7f025c196ff0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.550+0000 7f025be00640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f02400a3c30 0x7f02400a60f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.550+0000 7f025be00640 1 --2- 172.21.15.42:0/220106003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f025c102800 0x7f025c196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.550+0000 7f025be00640 1 -- 172.21.15.42:0/220106003 >> 172.21.15.42:0/220106003 conn(0x7f025c0fc230 msgr2=0x7f025c101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:32.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.550+0000 7f025be00640 1 -- 172.21.15.42:0/220106003 shutdown_connections 2024-04-20T10:31:32.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.550+0000 7f025be00640 1 -- 172.21.15.42:0/220106003 wait complete. 2024-04-20T10:31:32.575 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T10:31:32.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.782+0000 7f79b2200640 1 -- 172.21.15.42:0/3459595211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79ac1051d0 msgr2=0x7f79ac105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:32.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.782+0000 7f79b2200640 1 --2- 172.21.15.42:0/3459595211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79ac1051d0 0x7f79ac105650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7998009900 tx=0x7f799802bc30 comp rx=0 tx=0).stop 2024-04-20T10:31:32.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.783+0000 7f79b2200640 1 -- 172.21.15.42:0/3459595211 shutdown_connections 2024-04-20T10:31:32.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.783+0000 7f79b2200640 1 --2- 172.21.15.42:0/3459595211 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ac105b90 0x7f79ac10c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.783+0000 7f79b2200640 1 --2- 172.21.15.42:0/3459595211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79ac1051d0 0x7f79ac105650 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.783+0000 7f79b2200640 1 --2- 172.21.15.42:0/3459595211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79ac103fe0 0x7f79ac104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.783+0000 7f79b2200640 1 -- 172.21.15.42:0/3459595211 >> 172.21.15.42:0/3459595211 conn(0x7f79ac0ff570 msgr2=0x7f79ac101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:32.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.783+0000 7f79b2200640 1 -- 172.21.15.42:0/3459595211 shutdown_connections 2024-04-20T10:31:32.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.783+0000 7f79b2200640 1 -- 172.21.15.42:0/3459595211 wait complete. 2024-04-20T10:31:32.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.784+0000 7f79b2200640 1 Processor -- start 2024-04-20T10:31:32.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.784+0000 7f79b2200640 1 -- start start 2024-04-20T10:31:32.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.784+0000 7f79b2200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79ac103fe0 0x7f79ac19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:32.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.784+0000 7f79b2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79ac1051d0 0x7f79ac19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:32.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.784+0000 7f79b2200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ac105b90 0x7f79ac1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:32.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.784+0000 7f79b2200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac10fcc0 con 0x7f79ac105b90 2024-04-20T10:31:32.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.784+0000 7f79b2200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac10fb40 con 0x7f79ac1051d0 2024-04-20T10:31:32.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.784+0000 7f79b2200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac10fe40 con 0x7f79ac103fe0 2024-04-20T10:31:32.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.785+0000 7f79abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79ac1051d0 0x7f79ac19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:32.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.785+0000 7f79b0e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79ac103fe0 0x7f79ac19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:32.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.785+0000 7f79abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79ac1051d0 0x7f79ac19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42288/0 (socket says 172.21.15.42:42288) 2024-04-20T10:31:32.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.785+0000 7f79abe00640 1 -- 172.21.15.42:0/2442402109 learned_addr learned my addr 172.21.15.42:0/2442402109 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:32.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.785+0000 7f79b0e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79ac103fe0 0x7f79ac19a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:44892/0 (socket says 172.21.15.42:44892) 2024-04-20T10:31:32.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.785+0000 7f79b1800640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ac105b90 0x7f79ac1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:32.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.785+0000 7f79abe00640 1 -- 172.21.15.42:0/2442402109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79ac103fe0 msgr2=0x7f79ac19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:32.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.785+0000 7f79abe00640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79ac103fe0 0x7f79ac19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.785+0000 7f79abe00640 1 -- 172.21.15.42:0/2442402109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ac105b90 msgr2=0x7f79ac1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:32.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.785+0000 7f79abe00640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ac105b90 0x7f79ac1a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:32.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.785+0000 7f79abe00640 1 -- 172.21.15.42:0/2442402109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ac1a1f00 con 0x7f79ac1051d0 2024-04-20T10:31:32.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.785+0000 7f79b0e00640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79ac103fe0 0x7f79ac19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:32.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.786+0000 7f79abe00640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79ac1051d0 0x7f79ac19a820 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7998005ab0 tx=0x7f799802c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:32.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.786+0000 7f79a9600640 1 -- 172.21.15.42:0/2442402109 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998043020 con 0x7f79ac1051d0 2024-04-20T10:31:32.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.786+0000 7f79a9600640 1 -- 172.21.15.42:0/2442402109 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79980054e0 con 0x7f79ac1051d0 2024-04-20T10:31:32.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.786+0000 7f79b2200640 1 -- 172.21.15.42:0/2442402109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ac1a2190 con 0x7f79ac1051d0 2024-04-20T10:31:32.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.786+0000 7f79a9600640 1 -- 172.21.15.42:0/2442402109 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998003ce0 con 0x7f79ac1051d0 2024-04-20T10:31:32.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.786+0000 7f79b2200640 1 -- 172.21.15.42:0/2442402109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ac1a2730 con 0x7f79ac1051d0 2024-04-20T10:31:32.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.788+0000 7f79b2200640 1 -- 172.21.15.42:0/2442402109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79ac10b2a0 con 0x7f79ac1051d0 2024-04-20T10:31:32.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.789+0000 7f79a9600640 1 -- 172.21.15.42:0/2442402109 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7998046020 con 0x7f79ac1051d0 2024-04-20T10:31:32.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.790+0000 7f79a9600640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f79940a3c30 0x7f79940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:32.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.791+0000 7f79a9600640 1 -- 172.21.15.42:0/2442402109 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f79980f1fe0 con 0x7f79ac1051d0 2024-04-20T10:31:32.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.791+0000 7f79b0e00640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f79940a3c30 0x7f79940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:32.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.791+0000 7f79b0e00640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f79940a3c30 0x7f79940a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f79ac105030 tx=0x7f79a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:32.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:32.794+0000 7f79a9600640 1 -- 172.21.15.42:0/2442402109 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f799804d020 con 0x7f79ac1051d0 2024-04-20T10:31:32.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:32 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2365534619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:32.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:32 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1144921904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:32.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:32 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2365534619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:32.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:32 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1144921904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:32.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:32 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2365534619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:32.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:32 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1144921904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:33.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.026+0000 7f79b2200640 1 -- 172.21.15.42:0/2442402109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f79ac062b60 con 0x7f79ac1051d0 2024-04-20T10:31:33.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.026+0000 7f79a9600640 1 -- 172.21.15.42:0/2442402109 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f79980bee00 con 0x7f79ac1051d0 2024-04-20T10:31:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:33.027 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-04-20T10:31:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:31:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-04-20T10:31:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-04-20T10:31:33.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:31:33.027 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-04-20T10:31:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.029+0000 7f79b2200640 1 -- 172.21.15.42:0/2442402109 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f79940a3c30 msgr2=0x7f79940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.029+0000 7f79b2200640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f79940a3c30 0x7f79940a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f79ac105030 tx=0x7f79a0005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.029+0000 7f79b2200640 1 -- 172.21.15.42:0/2442402109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79ac1051d0 msgr2=0x7f79ac19a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.029+0000 7f79b2200640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79ac1051d0 0x7f79ac19a820 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7998005ab0 tx=0x7f799802c6c0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.029+0000 7f79b2200640 1 -- 172.21.15.42:0/2442402109 shutdown_connections 2024-04-20T10:31:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.029+0000 7f79b2200640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79ac105b90 0x7f79ac1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.029+0000 7f79b2200640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79ac1051d0 0x7f79ac19a820 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.029+0000 7f79b2200640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f79940a3c30 0x7f79940a60f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.029+0000 7f79b2200640 1 --2- 172.21.15.42:0/2442402109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79ac103fe0 0x7f79ac19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.029+0000 7f79b2200640 1 -- 172.21.15.42:0/2442402109 >> 172.21.15.42:0/2442402109 conn(0x7f79ac0ff570 msgr2=0x7f79ac108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.029+0000 7f79b2200640 1 -- 172.21.15.42:0/2442402109 shutdown_connections 2024-04-20T10:31:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.029+0000 7f79b2200640 1 -- 172.21.15.42:0/2442402109 wait complete. 2024-04-20T10:31:33.051 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T10:31:33.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T10:31:33.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.253+0000 7f6241c00640 1 -- 172.21.15.42:0/1886895180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f623c109e70 msgr2=0x7f623c10c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:33.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.253+0000 7f6241c00640 1 --2- 172.21.15.42:0/1886895180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f623c109e70 0x7f623c10c300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6234009ca0 tx=0x7f623402c0f0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.253+0000 7f6241c00640 1 -- 172.21.15.42:0/1886895180 shutdown_connections 2024-04-20T10:31:33.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.253+0000 7f6241c00640 1 --2- 172.21.15.42:0/1886895180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f623c109e70 0x7f623c10c300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.253+0000 7f6241c00640 1 --2- 172.21.15.42:0/1886895180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f623c0749c0 0x7f623c074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.253+0000 7f6241c00640 1 --2- 172.21.15.42:0/1886895180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c075fa0 0x7f623c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.253+0000 7f6241c00640 1 -- 172.21.15.42:0/1886895180 >> 172.21.15.42:0/1886895180 conn(0x7f623c0fd540 msgr2=0x7f623c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:33.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.253+0000 7f6241c00640 1 -- 172.21.15.42:0/1886895180 shutdown_connections 2024-04-20T10:31:33.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.253+0000 7f6241c00640 1 -- 172.21.15.42:0/1886895180 wait complete. 2024-04-20T10:31:33.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.254+0000 7f6241c00640 1 Processor -- start 2024-04-20T10:31:33.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.254+0000 7f6241c00640 1 -- start start 2024-04-20T10:31:33.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f6241c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f623c0749c0 0x7f623c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:33.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f6241c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f623c075fa0 0x7f623c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:33.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f6241c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c109e70 0x7f623c1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f6241c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f623c10fe30 con 0x7f623c0749c0 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f6241c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f623c10fcb0 con 0x7f623c109e70 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f6241c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f623c10ffb0 con 0x7f623c075fa0 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f623aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f623c0749c0 0x7f623c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f623aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f623c0749c0 0x7f623c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53790/0 (socket says 172.21.15.42:53790) 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f623aa00640 1 -- 172.21.15.42:0/2502546992 learned_addr learned my addr 172.21.15.42:0/2502546992 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f623b400640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c109e70 0x7f623c1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f623a000640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f623c075fa0 0x7f623c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f623b400640 1 -- 172.21.15.42:0/2502546992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f623c075fa0 msgr2=0x7f623c19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f623b400640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f623c075fa0 0x7f623c19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f623b400640 1 -- 172.21.15.42:0/2502546992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f623c0749c0 msgr2=0x7f623c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f623b400640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f623c0749c0 0x7f623c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f623b400640 1 -- 172.21.15.42:0/2502546992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c1a2040 con 0x7f623c109e70 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f623a000640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f623c075fa0 0x7f623c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.255+0000 7f623aa00640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f623c0749c0 0x7f623c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:33.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.256+0000 7f623b400640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c109e70 0x7f623c1a19d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6234037040 tx=0x7f6234032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:33.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.257+0000 7f6233400640 1 -- 172.21.15.42:0/2502546992 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234041020 con 0x7f623c109e70 2024-04-20T10:31:33.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.257+0000 7f6233400640 1 -- 172.21.15.42:0/2502546992 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6234034cd0 con 0x7f623c109e70 2024-04-20T10:31:33.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.257+0000 7f6233400640 1 -- 172.21.15.42:0/2502546992 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234039040 con 0x7f623c109e70 2024-04-20T10:31:33.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.257+0000 7f6241c00640 1 -- 172.21.15.42:0/2502546992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623c1a22d0 con 0x7f623c109e70 2024-04-20T10:31:33.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.257+0000 7f6241c00640 1 -- 172.21.15.42:0/2502546992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623c1a2740 con 0x7f623c109e70 2024-04-20T10:31:33.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.258+0000 7f6241c00640 1 -- 172.21.15.42:0/2502546992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623c108ba0 con 0x7f623c109e70 2024-04-20T10:31:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.260+0000 7f6233400640 1 -- 172.21.15.42:0/2502546992 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6234044020 con 0x7f623c109e70 2024-04-20T10:31:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.261+0000 7f6233400640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f62200a3c30 0x7f62200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.261+0000 7f6233400640 1 -- 172.21.15.42:0/2502546992 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f62340f1e50 con 0x7f623c109e70 2024-04-20T10:31:33.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.262+0000 7f623aa00640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f62200a3c30 0x7f62200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:33.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.262+0000 7f623aa00640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f62200a3c30 0x7f62200a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f622c011550 tx=0x7f622c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:33.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.266+0000 7f6233400640 1 -- 172.21.15.42:0/2502546992 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f623404b020 con 0x7f623c109e70 2024-04-20T10:31:33.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.508+0000 7f6241c00640 1 -- 172.21.15.42:0/2502546992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f623c062b60 con 0x7f623c109e70 2024-04-20T10:31:33.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.575+0000 7f6233400640 1 -- 172.21.15.42:0/2502546992 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f62340bec30 con 0x7f623c109e70 2024-04-20T10:31:33.578 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-04-20T10:31:33.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.583+0000 7f6241c00640 1 -- 172.21.15.42:0/2502546992 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f62200a3c30 msgr2=0x7f62200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:33.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.583+0000 7f6241c00640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f62200a3c30 0x7f62200a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f622c011550 tx=0x7f622c005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:33.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.583+0000 7f6241c00640 1 -- 172.21.15.42:0/2502546992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c109e70 msgr2=0x7f623c1a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:33.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.583+0000 7f6241c00640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c109e70 0x7f623c1a19d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6234037040 tx=0x7f6234032ea0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.583+0000 7f6241c00640 1 -- 172.21.15.42:0/2502546992 shutdown_connections 2024-04-20T10:31:33.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.583+0000 7f6241c00640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f623c109e70 0x7f623c1a19d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.583+0000 7f6241c00640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f62200a3c30 0x7f62200a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.583+0000 7f6241c00640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f623c075fa0 0x7f623c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.583+0000 7f6241c00640 1 --2- 172.21.15.42:0/2502546992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f623c0749c0 0x7f623c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.583+0000 7f6241c00640 1 -- 172.21.15.42:0/2502546992 >> 172.21.15.42:0/2502546992 conn(0x7f623c0fd540 msgr2=0x7f623c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:33.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.583+0000 7f6241c00640 1 -- 172.21.15.42:0/2502546992 shutdown_connections 2024-04-20T10:31:33.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.584+0000 7f6241c00640 1 -- 172.21.15.42:0/2502546992 wait complete. 2024-04-20T10:31:33.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T10:31:33.657 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:33 smithi042 ceph-mon[17078]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:33.657 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:33 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/220106003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:31:33.657 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:33 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:33.657 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:33 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:33.657 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:33 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:33.657 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:33 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:33.657 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:33 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2442402109' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-20T10:31:33.657 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:33 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2502546992' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T10:31:33.657 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:33 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T10:31:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.790+0000 7f7db7800640 1 -- 172.21.15.42:0/2168270454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db0076920 msgr2=0x7f7db0076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.790+0000 7f7db7800640 1 --2- 172.21.15.42:0/2168270454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db0076920 0x7f7db0076d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009900 tx=0x7f7da402bc30 comp rx=0 tx=0).stop 2024-04-20T10:31:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.791+0000 7f7db7800640 1 -- 172.21.15.42:0/2168270454 shutdown_connections 2024-04-20T10:31:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.791+0000 7f7db7800640 1 --2- 172.21.15.42:0/2168270454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7db0078500 0x7f7db0078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.791+0000 7f7db7800640 1 --2- 172.21.15.42:0/2168270454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7db0077b60 0x7f7db0077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.791+0000 7f7db7800640 1 --2- 172.21.15.42:0/2168270454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db0076920 0x7f7db0076d40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.791+0000 7f7db7800640 1 -- 172.21.15.42:0/2168270454 >> 172.21.15.42:0/2168270454 conn(0x7f7db00ff710 msgr2=0x7f7db0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:33.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.791+0000 7f7db7800640 1 -- 172.21.15.42:0/2168270454 shutdown_connections 2024-04-20T10:31:33.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.791+0000 7f7db7800640 1 -- 172.21.15.42:0/2168270454 wait complete. 2024-04-20T10:31:33.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.792+0000 7f7db7800640 1 Processor -- start 2024-04-20T10:31:33.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.792+0000 7f7db7800640 1 -- start start 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.792+0000 7f7db7800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db0076920 0x7f7db019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.792+0000 7f7db7800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7db0077b60 0x7f7db019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.792+0000 7f7db7800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7db0078500 0x7f7db01a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.792+0000 7f7db7800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0114290 con 0x7f7db0077b60 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.792+0000 7f7db7800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0114110 con 0x7f7db0076920 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.792+0000 7f7db7800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0114410 con 0x7f7db0078500 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.792+0000 7f7dafe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7db0077b60 0x7f7db019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.792+0000 7f7db4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db0076920 0x7f7db019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.793+0000 7f7db4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db0076920 0x7f7db019e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42320/0 (socket says 172.21.15.42:42320) 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.793+0000 7f7db4e00640 1 -- 172.21.15.42:0/1431429157 learned_addr learned my addr 172.21.15.42:0/1431429157 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.793+0000 7f7db5800640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7db0078500 0x7f7db01a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.793+0000 7f7db4e00640 1 -- 172.21.15.42:0/1431429157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7db0078500 msgr2=0x7f7db01a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.793+0000 7f7db4e00640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7db0078500 0x7f7db01a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.793+0000 7f7db4e00640 1 -- 172.21.15.42:0/1431429157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7db0077b60 msgr2=0x7f7db019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.793+0000 7f7db4e00640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7db0077b60 0x7f7db019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.793+0000 7f7db4e00640 1 -- 172.21.15.42:0/1431429157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db01a6500 con 0x7f7db0076920 2024-04-20T10:31:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.793+0000 7f7dafe00640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7db0077b60 0x7f7db019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.793+0000 7f7db4e00640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db0076920 0x7f7db019e8e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7da4000c00 tx=0x7f7da402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.794+0000 7f7dad600640 1 -- 172.21.15.42:0/1431429157 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4043020 con 0x7f7db0076920 2024-04-20T10:31:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.794+0000 7f7dad600640 1 -- 172.21.15.42:0/1431429157 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da40054e0 con 0x7f7db0076920 2024-04-20T10:31:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.794+0000 7f7db7800640 1 -- 172.21.15.42:0/1431429157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db01a6790 con 0x7f7db0076920 2024-04-20T10:31:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.794+0000 7f7dad600640 1 -- 172.21.15.42:0/1431429157 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4003c40 con 0x7f7db0076920 2024-04-20T10:31:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.794+0000 7f7db7800640 1 -- 172.21.15.42:0/1431429157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db01a6d30 con 0x7f7db0076920 2024-04-20T10:31:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.795+0000 7f7db7800640 1 -- 172.21.15.42:0/1431429157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db0110c60 con 0x7f7db0076920 2024-04-20T10:31:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.797+0000 7f7dad600640 1 -- 172.21.15.42:0/1431429157 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7da4046020 con 0x7f7db0076920 2024-04-20T10:31:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.798+0000 7f7dad600640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f7d980a3c30 0x7f7d980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.798+0000 7f7dad600640 1 -- 172.21.15.42:0/1431429157 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f7da40f2110 con 0x7f7db0076920 2024-04-20T10:31:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.798+0000 7f7dafe00640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f7d980a3c30 0x7f7d980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.799+0000 7f7dafe00640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f7d980a3c30 0x7f7d980a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7db019fdf0 tx=0x7f7d9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:33.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:33.802+0000 7f7dad600640 1 -- 172.21.15.42:0/1431429157 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7da404d020 con 0x7f7db0076920 2024-04-20T10:31:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:33 smithi046 ceph-mon[21574]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:33 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/220106003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:31:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:33 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:33 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:33 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:33 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:33 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2442402109' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-20T10:31:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:33 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2502546992' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T10:31:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:33 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T10:31:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:33 smithi162 ceph-mon[21854]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:33 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/220106003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:31:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:33 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:33 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:33 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:33 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:33 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2442402109' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-20T10:31:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:33 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2502546992' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T10:31:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:33 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T10:31:34.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.003+0000 7f7db7800640 1 -- 172.21.15.42:0/1431429157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7db002d5d0 con 0x7f7db0076920 2024-04-20T10:31:34.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.579+0000 7f7dad600640 1 -- 172.21.15.42:0/1431429157 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f7da40bed80 con 0x7f7db0076920 2024-04-20T10:31:34.581 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-04-20T10:31:34.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.585+0000 7f7db7800640 1 -- 172.21.15.42:0/1431429157 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f7d980a3c30 msgr2=0x7f7d980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:34.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.585+0000 7f7db7800640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f7d980a3c30 0x7f7d980a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7db019fdf0 tx=0x7f7d9c005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:34.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.585+0000 7f7db7800640 1 -- 172.21.15.42:0/1431429157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db0076920 msgr2=0x7f7db019e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.585+0000 7f7db7800640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db0076920 0x7f7db019e8e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7da4000c00 tx=0x7f7da402c650 comp rx=0 tx=0).stop 2024-04-20T10:31:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.586+0000 7f7db7800640 1 -- 172.21.15.42:0/1431429157 shutdown_connections 2024-04-20T10:31:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.586+0000 7f7db7800640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f7d980a3c30 0x7f7d980a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.586+0000 7f7db7800640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7db0078500 0x7f7db01a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.586+0000 7f7db7800640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7db0077b60 0x7f7db019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.586+0000 7f7db7800640 1 --2- 172.21.15.42:0/1431429157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7db0076920 0x7f7db019e8e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.586+0000 7f7db7800640 1 -- 172.21.15.42:0/1431429157 >> 172.21.15.42:0/1431429157 conn(0x7f7db00ff710 msgr2=0x7f7db0100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.586+0000 7f7db7800640 1 -- 172.21.15.42:0/1431429157 shutdown_connections 2024-04-20T10:31:34.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.587+0000 7f7db7800640 1 -- 172.21.15.42:0/1431429157 wait complete. 2024-04-20T10:31:34.619 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T10:31:34.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.802+0000 7f9b58a00640 1 -- 172.21.15.42:0/1287624135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54107b50 msgr2=0x7f9b54109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:34.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.802+0000 7f9b58a00640 1 --2- 172.21.15.42:0/1287624135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54107b50 0x7f9b54109fa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009ca0 tx=0x7f9b4c02c0f0 comp rx=0 tx=0).stop 2024-04-20T10:31:34.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.802+0000 7f9b58a00640 1 -- 172.21.15.42:0/1287624135 shutdown_connections 2024-04-20T10:31:34.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.802+0000 7f9b58a00640 1 --2- 172.21.15.42:0/1287624135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54107b50 0x7f9b54109fa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:34.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.802+0000 7f9b58a00640 1 --2- 172.21.15.42:0/1287624135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b54105220 0x7f9b54107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:34.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.802+0000 7f9b58a00640 1 --2- 172.21.15.42:0/1287624135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b541028f0 0x7f9b54104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:34.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.802+0000 7f9b58a00640 1 -- 172.21.15.42:0/1287624135 >> 172.21.15.42:0/1287624135 conn(0x7f9b540fc300 msgr2=0x7f9b540fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:34.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.802+0000 7f9b58a00640 1 -- 172.21.15.42:0/1287624135 shutdown_connections 2024-04-20T10:31:34.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.802+0000 7f9b58a00640 1 -- 172.21.15.42:0/1287624135 wait complete. 2024-04-20T10:31:34.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.803+0000 7f9b58a00640 1 Processor -- start 2024-04-20T10:31:34.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.803+0000 7f9b58a00640 1 -- start start 2024-04-20T10:31:34.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.803+0000 7f9b58a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541028f0 0x7f9b54198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.803+0000 7f9b58a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b54105220 0x7f9b54198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.803+0000 7f9b58a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b54107b50 0x7f9b5419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.803+0000 7f9b58a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5410db10 con 0x7f9b54105220 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.803+0000 7f9b58a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5410d990 con 0x7f9b541028f0 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.803+0000 7f9b58a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5410dc90 con 0x7f9b54107b50 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.803+0000 7f9b52800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541028f0 0x7f9b54198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b52800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541028f0 0x7f9b54198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42340/0 (socket says 172.21.15.42:42340) 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b52800640 1 -- 172.21.15.42:0/4040630750 learned_addr learned my addr 172.21.15.42:0/4040630750 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b49e00640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b54105220 0x7f9b54198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b53200640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b54107b50 0x7f9b5419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b52800640 1 -- 172.21.15.42:0/4040630750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b54107b50 msgr2=0x7f9b5419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b52800640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b54107b50 0x7f9b5419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:34.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b52800640 1 -- 172.21.15.42:0/4040630750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b54105220 msgr2=0x7f9b54198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:34.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b52800640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b54105220 0x7f9b54198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:34.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b52800640 1 -- 172.21.15.42:0/4040630750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5419fd30 con 0x7f9b541028f0 2024-04-20T10:31:34.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b53200640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b54107b50 0x7f9b5419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:31:34.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b52800640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541028f0 0x7f9b54198110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400e9f0 tx=0x7f9b4400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:34.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b4be00640 1 -- 172.21.15.42:0/4040630750 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4400cd80 con 0x7f9b541028f0 2024-04-20T10:31:34.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b4be00640 1 -- 172.21.15.42:0/4040630750 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b44010910 con 0x7f9b541028f0 2024-04-20T10:31:34.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.804+0000 7f9b4be00640 1 -- 172.21.15.42:0/4040630750 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44018990 con 0x7f9b541028f0 2024-04-20T10:31:34.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.805+0000 7f9b58a00640 1 -- 172.21.15.42:0/4040630750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5419ffc0 con 0x7f9b541028f0 2024-04-20T10:31:34.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.805+0000 7f9b58a00640 1 -- 172.21.15.42:0/4040630750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b541a0430 con 0x7f9b541028f0 2024-04-20T10:31:34.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.806+0000 7f9b58a00640 1 -- 172.21.15.42:0/4040630750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b540fddf0 con 0x7f9b541028f0 2024-04-20T10:31:34.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.808+0000 7f9b4be00640 1 -- 172.21.15.42:0/4040630750 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9b44010ac0 con 0x7f9b541028f0 2024-04-20T10:31:34.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.809+0000 7f9b4be00640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9b380a3c30 0x7f9b380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:34.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.809+0000 7f9b4be00640 1 -- 172.21.15.42:0/4040630750 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f9b440d0610 con 0x7f9b541028f0 2024-04-20T10:31:34.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.809+0000 7f9b49e00640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9b380a3c30 0x7f9b380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:34.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.810+0000 7f9b49e00640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9b380a3c30 0x7f9b380a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b54199620 tx=0x7f9b3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:34.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:34.813+0000 7f9b4be00640 1 -- 172.21.15.42:0/4040630750 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9b4409d100 con 0x7f9b541028f0 2024-04-20T10:31:34.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:34 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-20T10:31:34.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:34 smithi042 ceph-mon[17078]: osdmap e82: 12 total, 12 up, 12 in 2024-04-20T10:31:34.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:34.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:34.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:34.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:31:34.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:34.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:34 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:31:34.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:34 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1431429157' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T10:31:34.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:34 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T10:31:34.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:34 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-20T10:31:34.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:34 smithi162 ceph-mon[21854]: osdmap e82: 12 total, 12 up, 12 in 2024-04-20T10:31:34.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:34.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:34.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:34.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:31:34.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:34.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:34 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:31:34.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:34 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1431429157' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T10:31:34.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:34 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T10:31:34.928 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:34 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-20T10:31:34.928 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:34 smithi046 ceph-mon[21574]: osdmap e82: 12 total, 12 up, 12 in 2024-04-20T10:31:34.928 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:34.928 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:34.928 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:34.928 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:31:34.928 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:34.928 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:34 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:31:34.928 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:34 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1431429157' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T10:31:34.928 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:34 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T10:31:35.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.064+0000 7f9b58a00640 1 -- 172.21.15.42:0/4040630750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9b5402d5d0 con 0x7f9b541028f0 2024-04-20T10:31:35.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.724+0000 7f9b4be00640 1 -- 172.21.15.42:0/4040630750 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f9b4409ca00 con 0x7f9b541028f0 2024-04-20T10:31:35.725 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 2 and data pool 3 2024-04-20T10:31:35.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.729+0000 7f9b58a00640 1 -- 172.21.15.42:0/4040630750 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9b380a3c30 msgr2=0x7f9b380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:35.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.729+0000 7f9b58a00640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9b380a3c30 0x7f9b380a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b54199620 tx=0x7f9b3c005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:35.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.729+0000 7f9b58a00640 1 -- 172.21.15.42:0/4040630750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541028f0 msgr2=0x7f9b54198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:35.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.729+0000 7f9b58a00640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541028f0 0x7f9b54198110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400e9f0 tx=0x7f9b4400eeb0 comp rx=0 tx=0).stop 2024-04-20T10:31:35.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.730+0000 7f9b58a00640 1 -- 172.21.15.42:0/4040630750 shutdown_connections 2024-04-20T10:31:35.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.730+0000 7f9b58a00640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9b380a3c30 0x7f9b380a60f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:35.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.730+0000 7f9b58a00640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b54107b50 0x7f9b5419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:35.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.730+0000 7f9b58a00640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b54105220 0x7f9b54198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:35.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.730+0000 7f9b58a00640 1 --2- 172.21.15.42:0/4040630750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b541028f0 0x7f9b54198110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:35.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.730+0000 7f9b58a00640 1 -- 172.21.15.42:0/4040630750 >> 172.21.15.42:0/4040630750 conn(0x7f9b540fc300 msgr2=0x7f9b54103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:35.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.730+0000 7f9b58a00640 1 -- 172.21.15.42:0/4040630750 shutdown_connections 2024-04-20T10:31:35.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.731+0000 7f9b58a00640 1 -- 172.21.15.42:0/4040630750 wait complete. 2024-04-20T10:31:35.754 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:31:35.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:35 smithi042 ceph-mon[17078]: pgmap v314: 33 pgs: 2 creating+peering, 1 active+clean, 30 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:35.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:35 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-20T10:31:35.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:35 smithi042 ceph-mon[17078]: osdmap e83: 12 total, 12 up, 12 in 2024-04-20T10:31:35.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:35 smithi042 ceph-mon[17078]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T10:31:35.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:35 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4040630750' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T10:31:35.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:35 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T10:31:35.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:35 smithi042 ceph-mon[17078]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-20T10:31:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:35 smithi162 ceph-mon[21854]: pgmap v314: 33 pgs: 2 creating+peering, 1 active+clean, 30 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:35 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-20T10:31:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:35 smithi162 ceph-mon[21854]: osdmap e83: 12 total, 12 up, 12 in 2024-04-20T10:31:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:35 smithi162 ceph-mon[21854]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T10:31:35.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:35 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/4040630750' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T10:31:35.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:35 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T10:31:35.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:35 smithi162 ceph-mon[21854]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-20T10:31:35.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.936+0000 7f46b2000640 1 -- 172.21.15.42:0/695209227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46ac1041c0 msgr2=0x7f46ac1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:35.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.936+0000 7f46b2000640 1 --2- 172.21.15.42:0/695209227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46ac1041c0 0x7f46ac1045e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f469c009900 tx=0x7f469c02bc30 comp rx=0 tx=0).stop 2024-04-20T10:31:35.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.937+0000 7f46b2000640 1 -- 172.21.15.42:0/695209227 shutdown_connections 2024-04-20T10:31:35.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.937+0000 7f46b2000640 1 --2- 172.21.15.42:0/695209227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ac105d70 0x7f46ac10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:35.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.937+0000 7f46b2000640 1 --2- 172.21.15.42:0/695209227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46ac1053b0 0x7f46ac105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:35.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.937+0000 7f46b2000640 1 --2- 172.21.15.42:0/695209227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46ac1041c0 0x7f46ac1045e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:35.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.937+0000 7f46b2000640 1 -- 172.21.15.42:0/695209227 >> 172.21.15.42:0/695209227 conn(0x7f46ac0ff750 msgr2=0x7f46ac101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:35.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.937+0000 7f46b2000640 1 -- 172.21.15.42:0/695209227 shutdown_connections 2024-04-20T10:31:35.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.937+0000 7f46b2000640 1 -- 172.21.15.42:0/695209227 wait complete. 2024-04-20T10:31:35.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.938+0000 7f46b2000640 1 Processor -- start 2024-04-20T10:31:35.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.938+0000 7f46b2000640 1 -- start start 2024-04-20T10:31:35.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.938+0000 7f46b2000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46ac1041c0 0x7f46ac19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:35.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.938+0000 7f46b2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46ac1053b0 0x7f46ac19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.938+0000 7f46b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ac105d70 0x7f46ac1a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.938+0000 7f46b2000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac10fe70 con 0x7f46ac1053b0 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.938+0000 7f46b2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac10fcf0 con 0x7f46ac105d70 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.938+0000 7f46b2000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac10fff0 con 0x7f46ac1041c0 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.939+0000 7f46ab400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46ac1041c0 0x7f46ac19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.939+0000 7f46ab400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46ac1041c0 0x7f46ac19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:44968/0 (socket says 172.21.15.42:44968) 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.939+0000 7f46aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46ac1053b0 0x7f46ac19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.939+0000 7f46ab400640 1 -- 172.21.15.42:0/2538902023 learned_addr learned my addr 172.21.15.42:0/2538902023 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.939+0000 7f46abe00640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ac105d70 0x7f46ac1a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.939+0000 7f46ab400640 1 -- 172.21.15.42:0/2538902023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ac105d70 msgr2=0x7f46ac1a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.939+0000 7f46ab400640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ac105d70 0x7f46ac1a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.939+0000 7f46ab400640 1 -- 172.21.15.42:0/2538902023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46ac1053b0 msgr2=0x7f46ac19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.939+0000 7f46ab400640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46ac1053b0 0x7f46ac19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:35.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.939+0000 7f46ab400640 1 -- 172.21.15.42:0/2538902023 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac1a2130 con 0x7f46ac1041c0 2024-04-20T10:31:35.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.939+0000 7f46ab400640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46ac1041c0 0x7f46ac19a510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f469c000c00 tx=0x7f469c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:35.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.940+0000 7f46a3e00640 1 -- 172.21.15.42:0/2538902023 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c043020 con 0x7f46ac1041c0 2024-04-20T10:31:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.940+0000 7f46a3e00640 1 -- 172.21.15.42:0/2538902023 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f469c0054e0 con 0x7f46ac1041c0 2024-04-20T10:31:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.940+0000 7f46b2000640 1 -- 172.21.15.42:0/2538902023 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ac1a23c0 con 0x7f46ac1041c0 2024-04-20T10:31:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.940+0000 7f46a3e00640 1 -- 172.21.15.42:0/2538902023 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c003c40 con 0x7f46ac1041c0 2024-04-20T10:31:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.940+0000 7f46b2000640 1 -- 172.21.15.42:0/2538902023 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ac1a2910 con 0x7f46ac1041c0 2024-04-20T10:31:35.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.941+0000 7f46b2000640 1 -- 172.21.15.42:0/2538902023 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ac10b4e0 con 0x7f46ac1041c0 2024-04-20T10:31:35.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.943+0000 7f46a3e00640 1 -- 172.21.15.42:0/2538902023 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f469c046020 con 0x7f46ac1041c0 2024-04-20T10:31:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.944+0000 7f46a3e00640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f46900a3bb0 0x7f46900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:35.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.944+0000 7f46a3e00640 1 -- 172.21.15.42:0/2538902023 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f469c0f2320 con 0x7f46ac1041c0 2024-04-20T10:31:35.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.944+0000 7f46aaa00640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f46900a3bb0 0x7f46900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:35.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.945+0000 7f46aaa00640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f46900a3bb0 0x7f46900a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f46ac19ba20 tx=0x7f4698005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:35.948+0000 7f46a3e00640 1 -- 172.21.15.42:0/2538902023 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f469c04d020 con 0x7f46ac1041c0 2024-04-20T10:31:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:35 smithi046 ceph-mon[21574]: pgmap v314: 33 pgs: 2 creating+peering, 1 active+clean, 30 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:31:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:35 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-20T10:31:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:35 smithi046 ceph-mon[21574]: osdmap e83: 12 total, 12 up, 12 in 2024-04-20T10:31:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:35 smithi046 ceph-mon[21574]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T10:31:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:35 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/4040630750' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T10:31:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:35 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T10:31:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:35 smithi046 ceph-mon[21574]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-20T10:31:36.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.186+0000 7f46b2000640 1 -- 172.21.15.42:0/2538902023 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46ac062b60 con 0x7f46ac1041c0 2024-04-20T10:31:36.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.188+0000 7f46a3e00640 1 -- 172.21.15.42:0/2538902023 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22210 (secure 0 0 0) 0x7f469c0bedb0 con 0x7f46ac1041c0 2024-04-20T10:31:36.189 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:36.189 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":84,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:31:35.067567+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:31:36.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.190+0000 7f46b2000640 1 -- 172.21.15.42:0/2538902023 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f46900a3bb0 msgr2=0x7f46900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:36.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.190+0000 7f46b2000640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f46900a3bb0 0x7f46900a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f46ac19ba20 tx=0x7f4698005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:36.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.190+0000 7f46b2000640 1 -- 172.21.15.42:0/2538902023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46ac1041c0 msgr2=0x7f46ac19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:36.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.190+0000 7f46b2000640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46ac1041c0 0x7f46ac19a510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f469c000c00 tx=0x7f469c02c650 comp rx=0 tx=0).stop 2024-04-20T10:31:36.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.190+0000 7f46b2000640 1 -- 172.21.15.42:0/2538902023 shutdown_connections 2024-04-20T10:31:36.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.191+0000 7f46b2000640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ac105d70 0x7f46ac1a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.191+0000 7f46b2000640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46ac1053b0 0x7f46ac19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.191+0000 7f46b2000640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f46900a3bb0 0x7f46900a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.191+0000 7f46b2000640 1 --2- 172.21.15.42:0/2538902023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46ac1041c0 0x7f46ac19a510 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.191+0000 7f46b2000640 1 -- 172.21.15.42:0/2538902023 >> 172.21.15.42:0/2538902023 conn(0x7f46ac0ff750 msgr2=0x7f46ac1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:36.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.191+0000 7f46b2000640 1 -- 172.21.15.42:0/2538902023 shutdown_connections 2024-04-20T10:31:36.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.191+0000 7f46b2000640 1 -- 172.21.15.42:0/2538902023 wait complete. 2024-04-20T10:31:36.208 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:31:36.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.387+0000 7fbc94000640 1 -- 172.21.15.42:0/1817988997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc8c1041e0 msgr2=0x7fbc8c104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:36.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.387+0000 7fbc94000640 1 --2- 172.21.15.42:0/1817988997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc8c1041e0 0x7fbc8c104600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009900 tx=0x7fbc8002bc30 comp rx=0 tx=0).stop 2024-04-20T10:31:36.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.387+0000 7fbc94000640 1 -- 172.21.15.42:0/1817988997 shutdown_connections 2024-04-20T10:31:36.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.387+0000 7fbc94000640 1 --2- 172.21.15.42:0/1817988997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8c105d90 0x7fbc8c10c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.387+0000 7fbc94000640 1 --2- 172.21.15.42:0/1817988997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc8c1053d0 0x7fbc8c105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.387+0000 7fbc94000640 1 --2- 172.21.15.42:0/1817988997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc8c1041e0 0x7fbc8c104600 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.387+0000 7fbc94000640 1 -- 172.21.15.42:0/1817988997 >> 172.21.15.42:0/1817988997 conn(0x7fbc8c0ff750 msgr2=0x7fbc8c101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:36.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.387+0000 7fbc94000640 1 -- 172.21.15.42:0/1817988997 shutdown_connections 2024-04-20T10:31:36.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.387+0000 7fbc94000640 1 -- 172.21.15.42:0/1817988997 wait complete. 2024-04-20T10:31:36.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.388+0000 7fbc94000640 1 Processor -- start 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.388+0000 7fbc94000640 1 -- start start 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.388+0000 7fbc94000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8c1041e0 0x7fbc8c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.388+0000 7fbc94000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc8c1053d0 0x7fbc8c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.388+0000 7fbc94000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc8c105d90 0x7fbc8c1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.388+0000 7fbc94000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c10fec0 con 0x7fbc8c105d90 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc94000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c10fd40 con 0x7fbc8c1041e0 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc94000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c110040 con 0x7fbc8c1053d0 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc90c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc8c1053d0 0x7fbc8c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc91600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8c1041e0 0x7fbc8c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc92000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc8c105d90 0x7fbc8c1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc91600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8c1041e0 0x7fbc8c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42384/0 (socket says 172.21.15.42:42384) 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc90c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc8c1053d0 0x7fbc8c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:44974/0 (socket says 172.21.15.42:44974) 2024-04-20T10:31:36.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc91600640 1 -- 172.21.15.42:0/1199142700 learned_addr learned my addr 172.21.15.42:0/1199142700 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:36.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc90c00640 1 -- 172.21.15.42:0/1199142700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8c1041e0 msgr2=0x7fbc8c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:36.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc90c00640 1 --2- 172.21.15.42:0/1199142700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8c1041e0 0x7fbc8c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc90c00640 1 -- 172.21.15.42:0/1199142700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc8c105d90 msgr2=0x7fbc8c1a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:36.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc90c00640 1 --2- 172.21.15.42:0/1199142700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc8c105d90 0x7fbc8c1a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.389+0000 7fbc90c00640 1 -- 172.21.15.42:0/1199142700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c1a20e0 con 0x7fbc8c1053d0 2024-04-20T10:31:36.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.390+0000 7fbc90c00640 1 --2- 172.21.15.42:0/1199142700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc8c1053d0 0x7fbc8c19aa00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800e9f0 tx=0x7fbc7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:36.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.390+0000 7fbc8a000640 1 -- 172.21.15.42:0/1199142700 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800cd80 con 0x7fbc8c1053d0 2024-04-20T10:31:36.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.390+0000 7fbc94000640 1 -- 172.21.15.42:0/1199142700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c1a23d0 con 0x7fbc8c1053d0 2024-04-20T10:31:36.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.390+0000 7fbc8a000640 1 -- 172.21.15.42:0/1199142700 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc780035f0 con 0x7fbc8c1053d0 2024-04-20T10:31:36.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.390+0000 7fbc8a000640 1 -- 172.21.15.42:0/1199142700 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78010670 con 0x7fbc8c1053d0 2024-04-20T10:31:36.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.390+0000 7fbc94000640 1 -- 172.21.15.42:0/1199142700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c1a2970 con 0x7fbc8c1053d0 2024-04-20T10:31:36.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.391+0000 7fbc94000640 1 -- 172.21.15.42:0/1199142700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8c10b500 con 0x7fbc8c1053d0 2024-04-20T10:31:36.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.394+0000 7fbc8a000640 1 -- 172.21.15.42:0/1199142700 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbc780037a0 con 0x7fbc8c1053d0 2024-04-20T10:31:36.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.395+0000 7fbc8a000640 1 --2- 172.21.15.42:0/1199142700 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fbc740a3c30 0x7fbc740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:36.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.395+0000 7fbc8a000640 1 -- 172.21.15.42:0/1199142700 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fbc780d05a0 con 0x7fbc8c1053d0 2024-04-20T10:31:36.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.395+0000 7fbc91600640 1 --2- 172.21.15.42:0/1199142700 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fbc740a3c30 0x7fbc740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:36.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.396+0000 7fbc91600640 1 --2- 172.21.15.42:0/1199142700 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fbc740a3c30 0x7fbc740a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc80006010 tx=0x7fbc8002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:36.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.399+0000 7fbc8a000640 1 -- 172.21.15.42:0/1199142700 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbc7809d030 con 0x7fbc8c1053d0 2024-04-20T10:31:36.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.626+0000 7fbc94000640 1 -- 172.21.15.42:0/1199142700 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc8c062b60 con 0x7fbc8c1053d0 2024-04-20T10:31:36.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.628+0000 7fbc8a000640 1 -- 172.21.15.42:0/1199142700 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22197 (secure 0 0 0) 0x7fbc7809c930 con 0x7fbc8c1053d0 2024-04-20T10:31:36.628 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:36.629 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":85,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:31:36.594768+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:31:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.630+0000 7fbc94000640 1 -- 172.21.15.42:0/1199142700 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fbc740a3c30 msgr2=0x7fbc740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.630+0000 7fbc94000640 1 --2- 172.21.15.42:0/1199142700 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fbc740a3c30 0x7fbc740a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc80006010 tx=0x7fbc8002df90 comp rx=0 tx=0).stop 2024-04-20T10:31:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.630+0000 7fbc94000640 1 -- 172.21.15.42:0/1199142700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc8c1053d0 msgr2=0x7fbc8c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.630+0000 7fbc94000640 1 --2- 172.21.15.42:0/1199142700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc8c1053d0 0x7fbc8c19aa00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800e9f0 tx=0x7fbc7800eeb0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.630+0000 7fbc94000640 1 -- 172.21.15.42:0/1199142700 shutdown_connections 2024-04-20T10:31:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.630+0000 7fbc94000640 1 --2- 172.21.15.42:0/1199142700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc8c105d90 0x7fbc8c1a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.630+0000 7fbc94000640 1 --2- 172.21.15.42:0/1199142700 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fbc740a3c30 0x7fbc740a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.630+0000 7fbc94000640 1 --2- 172.21.15.42:0/1199142700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc8c1053d0 0x7fbc8c19aa00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.630+0000 7fbc94000640 1 --2- 172.21.15.42:0/1199142700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8c1041e0 0x7fbc8c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.630+0000 7fbc94000640 1 -- 172.21.15.42:0/1199142700 >> 172.21.15.42:0/1199142700 conn(0x7fbc8c0ff750 msgr2=0x7fbc8c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.630+0000 7fbc94000640 1 -- 172.21.15.42:0/1199142700 shutdown_connections 2024-04-20T10:31:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.630+0000 7fbc94000640 1 -- 172.21.15.42:0/1199142700 wait complete. 2024-04-20T10:31:36.649 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T10:31:36.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.832+0000 7f9092400640 1 -- 172.21.15.42:0/2817367365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f908c101ea0 msgr2=0x7f908c1022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:36.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.832+0000 7f9092400640 1 --2- 172.21.15.42:0/2817367365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f908c101ea0 0x7f908c1022c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9080009860 tx=0x7f908002bb70 comp rx=0 tx=0).stop 2024-04-20T10:31:36.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.832+0000 7f9092400640 1 -- 172.21.15.42:0/2817367365 shutdown_connections 2024-04-20T10:31:36.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.832+0000 7f9092400640 1 --2- 172.21.15.42:0/2817367365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f908c103a40 0x7f908c10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.832+0000 7f9092400640 1 --2- 172.21.15.42:0/2817367365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f908c103080 0x7f908c103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.832+0000 7f9092400640 1 --2- 172.21.15.42:0/2817367365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f908c101ea0 0x7f908c1022c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.832+0000 7f9092400640 1 -- 172.21.15.42:0/2817367365 >> 172.21.15.42:0/2817367365 conn(0x7f908c0fd460 msgr2=0x7f908c0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:36.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.832+0000 7f9092400640 1 -- 172.21.15.42:0/2817367365 shutdown_connections 2024-04-20T10:31:36.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.832+0000 7f9092400640 1 -- 172.21.15.42:0/2817367365 wait complete. 2024-04-20T10:31:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.833+0000 7f9092400640 1 Processor -- start 2024-04-20T10:31:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.833+0000 7f9092400640 1 -- start start 2024-04-20T10:31:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.833+0000 7f9092400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f908c101ea0 0x7f908c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.833+0000 7f9092400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f908c103080 0x7f908c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9092400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f908c103a40 0x7f908c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9092400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f908c10db10 con 0x7f908c103a40 2024-04-20T10:31:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9092400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f908c10d990 con 0x7f908c101ea0 2024-04-20T10:31:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9092400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f908c10dc90 con 0x7f908c103080 2024-04-20T10:31:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9091a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f908c103a40 0x7f908c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9091000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f908c101ea0 0x7f908c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9091a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f908c103a40 0x7f908c1a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53882/0 (socket says 172.21.15.42:53882) 2024-04-20T10:31:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9091a00640 1 -- 172.21.15.42:0/1181492871 learned_addr learned my addr 172.21.15.42:0/1181492871 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f908be00640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f908c103080 0x7f908c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9091000640 1 -- 172.21.15.42:0/1181492871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f908c103080 msgr2=0x7f908c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9091000640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f908c103080 0x7f908c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9091000640 1 -- 172.21.15.42:0/1181492871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f908c103a40 msgr2=0x7f908c1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9091000640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f908c103a40 0x7f908c1a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9091000640 1 -- 172.21.15.42:0/1181492871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c1a1f90 con 0x7f908c101ea0 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.834+0000 7f9091a00640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f908c103a40 0x7f908c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.835+0000 7f9091000640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f908c101ea0 0x7f908c19a370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9080009780 tx=0x7f9080032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.835+0000 7f9089600640 1 -- 172.21.15.42:0/1181492871 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080041020 con 0x7f908c101ea0 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.835+0000 7f9092400640 1 -- 172.21.15.42:0/1181492871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908c1a2220 con 0x7f908c101ea0 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.835+0000 7f9089600640 1 -- 172.21.15.42:0/1181492871 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908002c9f0 con 0x7f908c101ea0 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.835+0000 7f9089600640 1 -- 172.21.15.42:0/1181492871 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080039040 con 0x7f908c101ea0 2024-04-20T10:31:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.835+0000 7f9092400640 1 -- 172.21.15.42:0/1181492871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908c1a2730 con 0x7f908c101ea0 2024-04-20T10:31:36.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.836+0000 7f9092400640 1 -- 172.21.15.42:0/1181492871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908c1a2990 con 0x7f908c101ea0 2024-04-20T10:31:36.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.838+0000 7f9089600640 1 -- 172.21.15.42:0/1181492871 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9080044020 con 0x7f908c101ea0 2024-04-20T10:31:36.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.840+0000 7f9089600640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f90740a3bb0 0x7f90740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:36.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.840+0000 7f9089600640 1 -- 172.21.15.42:0/1181492871 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f90800f1f90 con 0x7f908c101ea0 2024-04-20T10:31:36.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.840+0000 7f908be00640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f90740a3bb0 0x7f90740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:36.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.841+0000 7f908be00640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f90740a3bb0 0x7f90740a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f908c19b880 tx=0x7f9078009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:36.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:36.843+0000 7f9089600640 1 -- 172.21.15.42:0/1181492871 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f908004b020 con 0x7f908c101ea0 2024-04-20T10:31:36.852 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:36 smithi042 ceph-mon[17078]: osdmap e84: 12 total, 12 up, 12 in 2024-04-20T10:31:36.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:36 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:36.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:36 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:36.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:36 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:36.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:36 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:36.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:36 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-20T10:31:36.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:36 smithi042 ceph-mon[17078]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-20T10:31:36.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:36 smithi042 ceph-mon[17078]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-20T10:31:36.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:36 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2538902023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:36.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:36 smithi162 ceph-mon[21854]: osdmap e84: 12 total, 12 up, 12 in 2024-04-20T10:31:36.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:36 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:36.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:36 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:36.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:36 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:36.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:36 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:36.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:36 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-20T10:31:36.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:36 smithi162 ceph-mon[21854]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-20T10:31:36.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:36 smithi162 ceph-mon[21854]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-20T10:31:36.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:36 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2538902023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:37.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:36 smithi046 ceph-mon[21574]: osdmap e84: 12 total, 12 up, 12 in 2024-04-20T10:31:37.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:36 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:37.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:36 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:37.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:36 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:37.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:36 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:37.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:36 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-20T10:31:37.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:36 smithi046 ceph-mon[21574]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-20T10:31:37.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:36 smithi046 ceph-mon[21574]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-20T10:31:37.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:36 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2538902023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:37.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.094+0000 7f9092400640 1 -- 172.21.15.42:0/1181492871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f908c1a2b60 con 0x7f908c101ea0 2024-04-20T10:31:37.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.748+0000 7f9089600640 1 -- 172.21.15.42:0/1181492871 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f90800bea20 con 0x7f908c101ea0 2024-04-20T10:31:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.751+0000 7f9092400640 1 -- 172.21.15.42:0/1181492871 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f90740a3bb0 msgr2=0x7f90740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.752+0000 7f9092400640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f90740a3bb0 0x7f90740a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f908c19b880 tx=0x7f9078009260 comp rx=0 tx=0).stop 2024-04-20T10:31:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.752+0000 7f9092400640 1 -- 172.21.15.42:0/1181492871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f908c101ea0 msgr2=0x7f908c19a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.752+0000 7f9092400640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f908c101ea0 0x7f908c19a370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9080009780 tx=0x7f9080032430 comp rx=0 tx=0).stop 2024-04-20T10:31:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.752+0000 7f9092400640 1 -- 172.21.15.42:0/1181492871 shutdown_connections 2024-04-20T10:31:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.752+0000 7f9092400640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f908c103a40 0x7f908c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.752+0000 7f9092400640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f90740a3bb0 0x7f90740a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.752+0000 7f9092400640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f908c103080 0x7f908c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.752+0000 7f9092400640 1 --2- 172.21.15.42:0/1181492871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f908c101ea0 0x7f908c19a370 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.752+0000 7f9092400640 1 -- 172.21.15.42:0/1181492871 >> 172.21.15.42:0/1181492871 conn(0x7f908c0fd460 msgr2=0x7f908c074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.752+0000 7f9092400640 1 -- 172.21.15.42:0/1181492871 shutdown_connections 2024-04-20T10:31:37.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.752+0000 7f9092400640 1 -- 172.21.15.42:0/1181492871 wait complete. 2024-04-20T10:31:37.778 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-20T10:31:37.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-20T10:31:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: pgmap v317: 97 pgs: 5 creating+peering, 24 active+clean, 68 unknown; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-04-20T10:31:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: osdmap e85: 12 total, 12 up, 12 in 2024-04-20T10:31:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1199142700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T10:31:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: Cluster is now healthy 2024-04-20T10:31:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] up:active 2024-04-20T10:31:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T10:31:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T10:31:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:31:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1181492871' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T10:31:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:37 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T10:31:37.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.997+0000 7f782ce00640 1 -- 172.21.15.42:0/2036725163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f782810a050 msgr2=0x7f78281108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:37.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.997+0000 7f782ce00640 1 --2- 172.21.15.42:0/2036725163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f782810a050 0x7f78281108e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7820009ca0 tx=0x7f782002c0f0 comp rx=0 tx=0).stop 2024-04-20T10:31:37.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.997+0000 7f782ce00640 1 -- 172.21.15.42:0/2036725163 shutdown_connections 2024-04-20T10:31:37.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.997+0000 7f782ce00640 1 --2- 172.21.15.42:0/2036725163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f782810a050 0x7f78281108e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:37.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.997+0000 7f782ce00640 1 --2- 172.21.15.42:0/2036725163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7828109690 0x7f7828109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:37.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.997+0000 7f782ce00640 1 --2- 172.21.15.42:0/2036725163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78281084a0 0x7f78281088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:37.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.997+0000 7f782ce00640 1 -- 172.21.15.42:0/2036725163 >> 172.21.15.42:0/2036725163 conn(0x7f7828077430 msgr2=0x7f7828079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:37.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.997+0000 7f782ce00640 1 -- 172.21.15.42:0/2036725163 shutdown_connections 2024-04-20T10:31:37.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.997+0000 7f782ce00640 1 -- 172.21.15.42:0/2036725163 wait complete. 2024-04-20T10:31:38.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.998+0000 7f782ce00640 1 Processor -- start 2024-04-20T10:31:38.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.998+0000 7f782ce00640 1 -- start start 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.998+0000 7f782ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78281084a0 0x7f782819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.998+0000 7f782ce00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7828109690 0x7f782819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.998+0000 7f782ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f782810a050 0x7f78281a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.998+0000 7f782ce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7828114130 con 0x7f782810a050 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.998+0000 7f782ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7828113fb0 con 0x7f78281084a0 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.998+0000 7f782ce00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f78281142b0 con 0x7f7828109690 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.999+0000 7f7827e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f782810a050 0x7f78281a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.999+0000 7f7827400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78281084a0 0x7f782819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.999+0000 7f7827400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78281084a0 0x7f782819e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42426/0 (socket says 172.21.15.42:42426) 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.999+0000 7f7827400640 1 -- 172.21.15.42:0/3650052238 learned_addr learned my addr 172.21.15.42:0/3650052238 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.999+0000 7f781fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7828109690 0x7f782819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.999+0000 7f7827e00640 1 -- 172.21.15.42:0/3650052238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7828109690 msgr2=0x7f782819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.999+0000 7f7827e00640 1 --2- 172.21.15.42:0/3650052238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7828109690 0x7f782819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.999+0000 7f7827e00640 1 -- 172.21.15.42:0/3650052238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78281084a0 msgr2=0x7f782819e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:38.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.999+0000 7f7827e00640 1 --2- 172.21.15.42:0/3650052238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78281084a0 0x7f782819e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:38.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.999+0000 7f7827e00640 1 -- 172.21.15.42:0/3650052238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78281a6300 con 0x7f782810a050 2024-04-20T10:31:38.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:37.999+0000 7f7827e00640 1 --2- 172.21.15.42:0/3650052238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f782810a050 0x7f78281a5c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f782002c630 tx=0x7f7820032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:38.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.000+0000 7f7824c00640 1 -- 172.21.15.42:0/3650052238 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820041020 con 0x7f782810a050 2024-04-20T10:31:38.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.000+0000 7f7824c00640 1 -- 172.21.15.42:0/3650052238 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7820034cd0 con 0x7f782810a050 2024-04-20T10:31:38.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.000+0000 7f782ce00640 1 -- 172.21.15.42:0/3650052238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78281a6590 con 0x7f782810a050 2024-04-20T10:31:38.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.000+0000 7f7824c00640 1 -- 172.21.15.42:0/3650052238 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820039040 con 0x7f782810a050 2024-04-20T10:31:38.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.000+0000 7f782ce00640 1 -- 172.21.15.42:0/3650052238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78281a6a50 con 0x7f782810a050 2024-04-20T10:31:38.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.001+0000 7f782ce00640 1 -- 172.21.15.42:0/3650052238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f782810f760 con 0x7f782810a050 2024-04-20T10:31:38.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.004+0000 7f7824c00640 1 -- 172.21.15.42:0/3650052238 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7820044020 con 0x7f782810a050 2024-04-20T10:31:38.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.005+0000 7f7824c00640 1 --2- 172.21.15.42:0/3650052238 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f780c0a3bb0 0x7f780c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:38.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.005+0000 7f7824c00640 1 -- 172.21.15.42:0/3650052238 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f78200f24f0 con 0x7f782810a050 2024-04-20T10:31:38.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.006+0000 7f7827400640 1 --2- 172.21.15.42:0/3650052238 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f780c0a3bb0 0x7f780c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:38.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.006+0000 7f7827400640 1 --2- 172.21.15.42:0/3650052238 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f780c0a3bb0 0x7f780c0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f78180058c0 tx=0x7f7818005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:38.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.008+0000 7f7824c00640 1 -- 172.21.15.42:0/3650052238 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f782004b020 con 0x7f782810a050 2024-04-20T10:31:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: pgmap v317: 97 pgs: 5 creating+peering, 24 active+clean, 68 unknown; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-04-20T10:31:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: osdmap e85: 12 total, 12 up, 12 in 2024-04-20T10:31:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1199142700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T10:31:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: Cluster is now healthy 2024-04-20T10:31:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] up:active 2024-04-20T10:31:38.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T10:31:38.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:38.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T10:31:38.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:31:38.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:38.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1181492871' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T10:31:38.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:37 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: pgmap v317: 97 pgs: 5 creating+peering, 24 active+clean, 68 unknown; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: osdmap e85: 12 total, 12 up, 12 in 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1199142700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: Cluster is now healthy 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] up:active 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:31:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:31:38.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1181492871' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T10:31:38.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:37 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T10:31:38.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.259+0000 7f782ce00640 1 -- 172.21.15.42:0/3650052238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f782802d5d0 con 0x7f782810a050 2024-04-20T10:31:38.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.743+0000 7f7824c00640 1 -- 172.21.15.42:0/3650052238 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f78200bf010 con 0x7f782810a050 2024-04-20T10:31:38.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.747+0000 7f782ce00640 1 -- 172.21.15.42:0/3650052238 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f780c0a3bb0 msgr2=0x7f780c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:38.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.747+0000 7f782ce00640 1 --2- 172.21.15.42:0/3650052238 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f780c0a3bb0 0x7f780c0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f78180058c0 tx=0x7f7818005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:38.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.747+0000 7f782ce00640 1 -- 172.21.15.42:0/3650052238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f782810a050 msgr2=0x7f78281a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:38.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.747+0000 7f782ce00640 1 --2- 172.21.15.42:0/3650052238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f782810a050 0x7f78281a5c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f782002c630 tx=0x7f7820032f20 comp rx=0 tx=0).stop 2024-04-20T10:31:38.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.747+0000 7f782ce00640 1 -- 172.21.15.42:0/3650052238 shutdown_connections 2024-04-20T10:31:38.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.747+0000 7f782ce00640 1 --2- 172.21.15.42:0/3650052238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f782810a050 0x7f78281a5c90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:38.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.747+0000 7f782ce00640 1 --2- 172.21.15.42:0/3650052238 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f780c0a3bb0 0x7f780c0a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:38.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.747+0000 7f782ce00640 1 --2- 172.21.15.42:0/3650052238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7828109690 0x7f782819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:38.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.747+0000 7f782ce00640 1 --2- 172.21.15.42:0/3650052238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78281084a0 0x7f782819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:38.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.747+0000 7f782ce00640 1 -- 172.21.15.42:0/3650052238 >> 172.21.15.42:0/3650052238 conn(0x7f7828077430 msgr2=0x7f782810c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:38.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.748+0000 7f782ce00640 1 -- 172.21.15.42:0/3650052238 shutdown_connections 2024-04-20T10:31:38.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.748+0000 7f782ce00640 1 -- 172.21.15.42:0/3650052238 wait complete. 2024-04-20T10:31:38.777 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-20T10:31:38.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.995+0000 7fe243e00640 1 -- 172.21.15.42:0/186122900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441040a0 msgr2=0x7fe2441044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:38.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.995+0000 7fe243e00640 1 --2- 172.21.15.42:0/186122900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441040a0 0x7fe2441044c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe234009900 tx=0x7fe23402bc30 comp rx=0 tx=0).stop 2024-04-20T10:31:38.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.995+0000 7fe243e00640 1 -- 172.21.15.42:0/186122900 shutdown_connections 2024-04-20T10:31:38.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.995+0000 7fe243e00640 1 --2- 172.21.15.42:0/186122900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe244105c50 0x7fe24410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:38.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.995+0000 7fe243e00640 1 --2- 172.21.15.42:0/186122900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe244105290 0x7fe244105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:38.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.995+0000 7fe243e00640 1 --2- 172.21.15.42:0/186122900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441040a0 0x7fe2441044c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:38.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.995+0000 7fe243e00640 1 -- 172.21.15.42:0/186122900 >> 172.21.15.42:0/186122900 conn(0x7fe2440ff610 msgr2=0x7fe244101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:38.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.995+0000 7fe243e00640 1 -- 172.21.15.42:0/186122900 shutdown_connections 2024-04-20T10:31:38.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.995+0000 7fe243e00640 1 -- 172.21.15.42:0/186122900 wait complete. 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.996+0000 7fe243e00640 1 Processor -- start 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.996+0000 7fe243e00640 1 -- start start 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.997+0000 7fe243e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441040a0 0x7fe24419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.997+0000 7fe243e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe244105290 0x7fe24419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.997+0000 7fe243e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe244105c50 0x7fe2441a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.997+0000 7fe243e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe24410fd40 con 0x7fe2441040a0 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.997+0000 7fe243e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe24410fbc0 con 0x7fe244105c50 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.997+0000 7fe243e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe24410fec0 con 0x7fe244105290 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.997+0000 7fe242a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441040a0 0x7fe24419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.997+0000 7fe242a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441040a0 0x7fe24419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51574/0 (socket says 172.21.15.42:51574) 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.997+0000 7fe242a00640 1 -- 172.21.15.42:0/2194569496 learned_addr learned my addr 172.21.15.42:0/2194569496 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.997+0000 7fe242000640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe244105290 0x7fe24419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.997+0000 7fe243400640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe244105c50 0x7fe2441a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.998+0000 7fe242a00640 1 -- 172.21.15.42:0/2194569496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe244105290 msgr2=0x7fe24419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.998+0000 7fe242a00640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe244105290 0x7fe24419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.998+0000 7fe242a00640 1 -- 172.21.15.42:0/2194569496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe244105c50 msgr2=0x7fe2441a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.998+0000 7fe242a00640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe244105c50 0x7fe2441a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.998+0000 7fe242a00640 1 -- 172.21.15.42:0/2194569496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2441a1fe0 con 0x7fe2441040a0 2024-04-20T10:31:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.998+0000 7fe242a00640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441040a0 0x7fe24419a3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe234000c00 tx=0x7fe23402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.998+0000 7fe23b400640 1 -- 172.21.15.42:0/2194569496 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234043020 con 0x7fe2441040a0 2024-04-20T10:31:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.998+0000 7fe23b400640 1 -- 172.21.15.42:0/2194569496 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2340054e0 con 0x7fe2441040a0 2024-04-20T10:31:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.998+0000 7fe243e00640 1 -- 172.21.15.42:0/2194569496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2441a2270 con 0x7fe2441040a0 2024-04-20T10:31:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.998+0000 7fe23b400640 1 -- 172.21.15.42:0/2194569496 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234003c40 con 0x7fe2441040a0 2024-04-20T10:31:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:38.999+0000 7fe243e00640 1 -- 172.21.15.42:0/2194569496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2441a2810 con 0x7fe2441040a0 2024-04-20T10:31:39.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.000+0000 7fe243e00640 1 -- 172.21.15.42:0/2194569496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe24410b3e0 con 0x7fe2441040a0 2024-04-20T10:31:39.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.002+0000 7fe23b400640 1 -- 172.21.15.42:0/2194569496 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe234046020 con 0x7fe2441040a0 2024-04-20T10:31:39.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.003+0000 7fe23b400640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe2280a3c30 0x7fe2280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:39.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.003+0000 7fe23b400640 1 -- 172.21.15.42:0/2194569496 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe2340f22c0 con 0x7fe2441040a0 2024-04-20T10:31:39.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.003+0000 7fe242000640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe2280a3c30 0x7fe2280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:39.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.004+0000 7fe242000640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe2280a3c30 0x7fe2280a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe24419b8d0 tx=0x7fe230005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.008+0000 7fe23b400640 1 -- 172.21.15.42:0/2194569496 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe23404d020 con 0x7fe2441040a0 2024-04-20T10:31:39.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:38 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-20T10:31:39.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:38 smithi046 ceph-mon[21574]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T10:31:39.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:38 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3650052238' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T10:31:39.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:38 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3650052238' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-20T10:31:39.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:38 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-20T10:31:39.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:38 smithi042 ceph-mon[17078]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T10:31:39.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:38 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3650052238' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T10:31:39.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:38 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3650052238' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-20T10:31:39.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:38 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-20T10:31:39.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:38 smithi162 ceph-mon[21854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T10:31:39.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:38 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3650052238' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T10:31:39.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:38 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3650052238' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-20T10:31:39.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.261+0000 7fe243e00640 1 -- 172.21.15.42:0/2194569496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fe24402d7f0 con 0x7fe2441040a0 2024-04-20T10:31:39.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.776+0000 7fe23b400640 1 -- 172.21.15.42:0/2194569496 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v19) ==== 115+0+0 (secure 0 0 0) 0x7fe2340bed50 con 0x7fe2441040a0 2024-04-20T10:31:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.781+0000 7fe243e00640 1 -- 172.21.15.42:0/2194569496 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe2280a3c30 msgr2=0x7fe2280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.781+0000 7fe243e00640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe2280a3c30 0x7fe2280a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe24419b8d0 tx=0x7fe230005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.781+0000 7fe243e00640 1 -- 172.21.15.42:0/2194569496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441040a0 msgr2=0x7fe24419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.781+0000 7fe243e00640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441040a0 0x7fe24419a3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe234000c00 tx=0x7fe23402c650 comp rx=0 tx=0).stop 2024-04-20T10:31:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.782+0000 7fe243e00640 1 -- 172.21.15.42:0/2194569496 shutdown_connections 2024-04-20T10:31:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.782+0000 7fe243e00640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe244105c50 0x7fe2441a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.782+0000 7fe243e00640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe2280a3c30 0x7fe2280a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.782+0000 7fe243e00640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe244105290 0x7fe24419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.782+0000 7fe243e00640 1 --2- 172.21.15.42:0/2194569496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441040a0 0x7fe24419a3c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.782+0000 7fe243e00640 1 -- 172.21.15.42:0/2194569496 >> 172.21.15.42:0/2194569496 conn(0x7fe2440ff610 msgr2=0x7fe2441085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.782+0000 7fe243e00640 1 -- 172.21.15.42:0/2194569496 shutdown_connections 2024-04-20T10:31:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:39.782+0000 7fe243e00640 1 -- 172.21.15.42:0/2194569496 wait complete. 2024-04-20T10:31:39.807 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-20T10:31:39.807 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-20T10:31:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.032+0000 7ff2e3200640 1 -- 172.21.15.42:0/3833447031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e4078500 msgr2=0x7ff2e4078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.032+0000 7ff2e3200640 1 --2- 172.21.15.42:0/3833447031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e4078500 0x7ff2e4078da0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009be0 tx=0x7ff2dc02bef0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.032+0000 7ff2e3200640 1 -- 172.21.15.42:0/3833447031 shutdown_connections 2024-04-20T10:31:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.032+0000 7ff2e3200640 1 --2- 172.21.15.42:0/3833447031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e4078500 0x7ff2e4078da0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.032+0000 7ff2e3200640 1 --2- 172.21.15.42:0/3833447031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2e4077b60 0x7ff2e4077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.032+0000 7ff2e3200640 1 --2- 172.21.15.42:0/3833447031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2e4076920 0x7ff2e4076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.032+0000 7ff2e3200640 1 -- 172.21.15.42:0/3833447031 >> 172.21.15.42:0/3833447031 conn(0x7ff2e40ff610 msgr2=0x7ff2e4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.032+0000 7ff2e3200640 1 -- 172.21.15.42:0/3833447031 shutdown_connections 2024-04-20T10:31:40.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.032+0000 7ff2e3200640 1 -- 172.21.15.42:0/3833447031 wait complete. 2024-04-20T10:31:40.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.033+0000 7ff2e3200640 1 Processor -- start 2024-04-20T10:31:40.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.034+0000 7ff2e3200640 1 -- start start 2024-04-20T10:31:40.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.034+0000 7ff2e3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2e4076920 0x7ff2e419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:40.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.034+0000 7ff2e3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e4077b60 0x7ff2e419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:40.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.034+0000 7ff2e3200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2e4078500 0x7ff2e41a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:40.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.034+0000 7ff2e3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e4114170 con 0x7ff2e4077b60 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.034+0000 7ff2e3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e4113ff0 con 0x7ff2e4076920 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.034+0000 7ff2e3200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41142f0 con 0x7ff2e4078500 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.034+0000 7ff2dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e4077b60 0x7ff2e419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.034+0000 7ff2e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2e4076920 0x7ff2e419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.034+0000 7ff2dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e4077b60 0x7ff2e419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51600/0 (socket says 172.21.15.42:51600) 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.034+0000 7ff2dbe00640 1 -- 172.21.15.42:0/1792001074 learned_addr learned my addr 172.21.15.42:0/1792001074 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.034+0000 7ff2e2800640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2e4078500 0x7ff2e41a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.035+0000 7ff2e1e00640 1 -- 172.21.15.42:0/1792001074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2e4078500 msgr2=0x7ff2e41a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.035+0000 7ff2e1e00640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2e4078500 0x7ff2e41a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.035+0000 7ff2e1e00640 1 -- 172.21.15.42:0/1792001074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e4077b60 msgr2=0x7ff2e419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.035+0000 7ff2e1e00640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e4077b60 0x7ff2e419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.035+0000 7ff2e1e00640 1 -- 172.21.15.42:0/1792001074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e41a6350 con 0x7ff2e4076920 2024-04-20T10:31:40.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.035+0000 7ff2e1e00640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2e4076920 0x7ff2e419e730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4005fd0 tx=0x7ff2d400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.035+0000 7ff2daa00640 1 -- 172.21.15.42:0/1792001074 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400cd00 con 0x7ff2e4076920 2024-04-20T10:31:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.035+0000 7ff2daa00640 1 -- 172.21.15.42:0/1792001074 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2d40035f0 con 0x7ff2e4076920 2024-04-20T10:31:40.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.035+0000 7ff2e3200640 1 -- 172.21.15.42:0/1792001074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e41a6640 con 0x7ff2e4076920 2024-04-20T10:31:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.035+0000 7ff2daa00640 1 -- 172.21.15.42:0/1792001074 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4010670 con 0x7ff2e4076920 2024-04-20T10:31:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.035+0000 7ff2e3200640 1 -- 172.21.15.42:0/1792001074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e41a6be0 con 0x7ff2e4076920 2024-04-20T10:31:40.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.037+0000 7ff2e3200640 1 -- 172.21.15.42:0/1792001074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e4110a10 con 0x7ff2e4076920 2024-04-20T10:31:40.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.039+0000 7ff2daa00640 1 -- 172.21.15.42:0/1792001074 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff2d40037a0 con 0x7ff2e4076920 2024-04-20T10:31:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.040+0000 7ff2daa00640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff2c80a3c30 0x7ff2c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.040+0000 7ff2daa00640 1 -- 172.21.15.42:0/1792001074 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff2d40d0630 con 0x7ff2e4076920 2024-04-20T10:31:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.040+0000 7ff2dbe00640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff2c80a3c30 0x7ff2c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:40.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.041+0000 7ff2dbe00640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff2c80a3c30 0x7ff2c80a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff2e419fc40 tx=0x7ff2cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:40.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.042+0000 7ff2daa00640 1 -- 172.21.15.42:0/1792001074 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff2d4009510 con 0x7ff2e4076920 2024-04-20T10:31:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:39 smithi046 ceph-mon[21574]: pgmap v319: 97 pgs: 5 creating+peering, 82 active+clean, 10 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-04-20T10:31:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:39 smithi046 ceph-mon[21574]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-20T10:31:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:39 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2194569496' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T10:31:40.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:39 smithi042 ceph-mon[17078]: pgmap v319: 97 pgs: 5 creating+peering, 82 active+clean, 10 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-04-20T10:31:40.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:39 smithi042 ceph-mon[17078]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-20T10:31:40.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:39 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2194569496' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T10:31:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:39 smithi162 ceph-mon[21854]: pgmap v319: 97 pgs: 5 creating+peering, 82 active+clean, 10 unknown; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-04-20T10:31:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:39 smithi162 ceph-mon[21854]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-20T10:31:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:39 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2194569496' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T10:31:40.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.279+0000 7ff2e3200640 1 -- 172.21.15.42:0/1792001074 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff2e40623a0 con 0x7ff2c80a3c30 2024-04-20T10:31:40.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.313+0000 7ff2daa00640 1 -- 172.21.15.42:0/1792001074 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff2e40623a0 con 0x7ff2c80a3c30 2024-04-20T10:31:40.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.315+0000 7ff2e3200640 1 -- 172.21.15.42:0/1792001074 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff2c80a3c30 msgr2=0x7ff2c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:40.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.315+0000 7ff2e3200640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff2c80a3c30 0x7ff2c80a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff2e419fc40 tx=0x7ff2cc005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:40.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.315+0000 7ff2e3200640 1 -- 172.21.15.42:0/1792001074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2e4076920 msgr2=0x7ff2e419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:40.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.315+0000 7ff2e3200640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2e4076920 0x7ff2e419e730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4005fd0 tx=0x7ff2d400ee50 comp rx=0 tx=0).stop 2024-04-20T10:31:40.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.316+0000 7ff2e3200640 1 -- 172.21.15.42:0/1792001074 shutdown_connections 2024-04-20T10:31:40.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.316+0000 7ff2e3200640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff2c80a3c30 0x7ff2c80a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.316+0000 7ff2e3200640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2e4078500 0x7ff2e41a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.316+0000 7ff2e3200640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e4077b60 0x7ff2e419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.316+0000 7ff2e3200640 1 --2- 172.21.15.42:0/1792001074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2e4076920 0x7ff2e419e730 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.316+0000 7ff2e3200640 1 -- 172.21.15.42:0/1792001074 >> 172.21.15.42:0/1792001074 conn(0x7ff2e40ff610 msgr2=0x7ff2e4101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:40.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.316+0000 7ff2e3200640 1 -- 172.21.15.42:0/1792001074 shutdown_connections 2024-04-20T10:31:40.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.316+0000 7ff2e3200640 1 -- 172.21.15.42:0/1792001074 wait complete. 2024-04-20T10:31:40.336 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-20T10:31:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.538+0000 7fb6db600640 1 -- 172.21.15.42:0/2616463786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6d4075f40 msgr2=0x7fb6d4074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.538+0000 7fb6db600640 1 --2- 172.21.15.42:0/2616463786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6d4075f40 0x7fb6d4074390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009ca0 tx=0x7fb6c402c180 comp rx=0 tx=0).stop 2024-04-20T10:31:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.538+0000 7fb6db600640 1 -- 172.21.15.42:0/2616463786 shutdown_connections 2024-04-20T10:31:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.538+0000 7fb6db600640 1 --2- 172.21.15.42:0/2616463786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d4109e40 0x7fb6d410c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.538+0000 7fb6db600640 1 --2- 172.21.15.42:0/2616463786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d4074960 0x7fb6d4074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.538+0000 7fb6db600640 1 --2- 172.21.15.42:0/2616463786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6d4075f40 0x7fb6d4074390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.538+0000 7fb6db600640 1 -- 172.21.15.42:0/2616463786 >> 172.21.15.42:0/2616463786 conn(0x7fb6d40fd540 msgr2=0x7fb6d40ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.538+0000 7fb6db600640 1 -- 172.21.15.42:0/2616463786 shutdown_connections 2024-04-20T10:31:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.538+0000 7fb6db600640 1 -- 172.21.15.42:0/2616463786 wait complete. 2024-04-20T10:31:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.539+0000 7fb6db600640 1 Processor -- start 2024-04-20T10:31:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.539+0000 7fb6db600640 1 -- start start 2024-04-20T10:31:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.539+0000 7fb6db600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d4074960 0x7fb6d419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6db600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d4075f40 0x7fb6d419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6db600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6d4109e40 0x7fb6d41a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6db600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d410fe40 con 0x7fb6d4109e40 2024-04-20T10:31:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6db600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d410fcc0 con 0x7fb6d4075f40 2024-04-20T10:31:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6db600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d410ffc0 con 0x7fb6d4074960 2024-04-20T10:31:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6d9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6d4109e40 0x7fb6d41a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6d9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6d4109e40 0x7fb6d41a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51628/0 (socket says 172.21.15.42:51628) 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6d9600640 1 -- 172.21.15.42:0/1283043737 learned_addr learned my addr 172.21.15.42:0/1283043737 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6d8c00640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d4074960 0x7fb6d419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6d3e00640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d4075f40 0x7fb6d419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6d9600640 1 -- 172.21.15.42:0/1283043737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d4074960 msgr2=0x7fb6d419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6d9600640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d4074960 0x7fb6d419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6d9600640 1 -- 172.21.15.42:0/1283043737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d4075f40 msgr2=0x7fb6d419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6d9600640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d4075f40 0x7fb6d419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.540+0000 7fb6d9600640 1 -- 172.21.15.42:0/1283043737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d41a2130 con 0x7fb6d4109e40 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.541+0000 7fb6d9600640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6d4109e40 0x7fb6d41a1ac0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00e970 tx=0x7fb6cc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.541+0000 7fb6d1600640 1 -- 172.21.15.42:0/1283043737 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc00ccc0 con 0x7fb6d4109e40 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.541+0000 7fb6d1600640 1 -- 172.21.15.42:0/1283043737 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6cc00ce60 con 0x7fb6d4109e40 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.541+0000 7fb6d1600640 1 -- 172.21.15.42:0/1283043737 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc010490 con 0x7fb6d4109e40 2024-04-20T10:31:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.541+0000 7fb6db600640 1 -- 172.21.15.42:0/1283043737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d41a2420 con 0x7fb6d4109e40 2024-04-20T10:31:40.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.541+0000 7fb6db600640 1 -- 172.21.15.42:0/1283043737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d41a2920 con 0x7fb6d4109e40 2024-04-20T10:31:40.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.542+0000 7fb6db600640 1 -- 172.21.15.42:0/1283043737 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d4108c00 con 0x7fb6d4109e40 2024-04-20T10:31:40.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.544+0000 7fb6d1600640 1 -- 172.21.15.42:0/1283043737 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb6cc005020 con 0x7fb6d4109e40 2024-04-20T10:31:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.546+0000 7fb6d1600640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb6b80a3c30 0x7fb6b80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.546+0000 7fb6d1600640 1 -- 172.21.15.42:0/1283043737 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb6cc0d07f0 con 0x7fb6d4109e40 2024-04-20T10:31:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.546+0000 7fb6d8c00640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb6b80a3c30 0x7fb6b80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.547+0000 7fb6d8c00640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb6b80a3c30 0x7fb6b80a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb6c402c790 tx=0x7fb6c4005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:40.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.550+0000 7fb6d1600640 1 -- 172.21.15.42:0/1283043737 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb6cc09d280 con 0x7fb6d4109e40 2024-04-20T10:31:40.604 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:31:40 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:31:40.294+0000 7f3066000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:31:40.604 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:31:40 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:31:40.294+0000 7f3066000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:31:40.604 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:31:40 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:31:40.294+0000 7f3066000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:31:40.604 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:31:40 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:31:40.294+0000 7f3066000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:31:40.604 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:31:40 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:31:40.294+0000 7f3066000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:31:40.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.794+0000 7fb6db600640 1 -- 172.21.15.42:0/1283043737 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb6d40623a0 con 0x7fb6b80a3c30 2024-04-20T10:31:40.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.809+0000 7fb6d1600640 1 -- 172.21.15.42:0/1283043737 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb6cc0991f0 con 0x7fb6d4109e40 2024-04-20T10:31:40.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.837+0000 7fb6d1600640 1 -- 172.21.15.42:0/1283043737 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb6d40623a0 con 0x7fb6b80a3c30 2024-04-20T10:31:40.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.841+0000 7fb6db600640 1 -- 172.21.15.42:0/1283043737 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb6b80a3c30 msgr2=0x7fb6b80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:40.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.841+0000 7fb6db600640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb6b80a3c30 0x7fb6b80a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb6c402c790 tx=0x7fb6c4005f70 comp rx=0 tx=0).stop 2024-04-20T10:31:40.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.841+0000 7fb6db600640 1 -- 172.21.15.42:0/1283043737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6d4109e40 msgr2=0x7fb6d41a1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:40.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.841+0000 7fb6db600640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6d4109e40 0x7fb6d41a1ac0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00e970 tx=0x7fb6cc00ed70 comp rx=0 tx=0).stop 2024-04-20T10:31:40.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.841+0000 7fb6db600640 1 -- 172.21.15.42:0/1283043737 shutdown_connections 2024-04-20T10:31:40.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.841+0000 7fb6db600640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6d4109e40 0x7fb6d41a1ac0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.841+0000 7fb6db600640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d4075f40 0x7fb6d419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.841+0000 7fb6db600640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb6b80a3c30 0x7fb6b80a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.841+0000 7fb6db600640 1 --2- 172.21.15.42:0/1283043737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d4074960 0x7fb6d419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:40.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.841+0000 7fb6db600640 1 -- 172.21.15.42:0/1283043737 >> 172.21.15.42:0/1283043737 conn(0x7fb6d40fd540 msgr2=0x7fb6d4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:40.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.841+0000 7fb6db600640 1 -- 172.21.15.42:0/1283043737 shutdown_connections 2024-04-20T10:31:40.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:40.841+0000 7fb6db600640 1 -- 172.21.15.42:0/1283043737 wait complete. 2024-04-20T10:31:40.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-20T10:31:41.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:40 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2194569496' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-20T10:31:41.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:40 smithi046 ceph-mon[21574]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-20T10:31:41.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.063+0000 7f55f3000640 1 -- 172.21.15.42:0/3999293717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ec075fa0 msgr2=0x7f55ec0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:41.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.063+0000 7f55f3000640 1 --2- 172.21.15.42:0/3999293717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ec075fa0 0x7f55ec0743f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009ca0 tx=0x7f55dc02c100 comp rx=0 tx=0).stop 2024-04-20T10:31:41.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.063+0000 7f55f3000640 1 -- 172.21.15.42:0/3999293717 shutdown_connections 2024-04-20T10:31:41.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.063+0000 7f55f3000640 1 --2- 172.21.15.42:0/3999293717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ec109e60 0x7f55ec10c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.063+0000 7f55f3000640 1 --2- 172.21.15.42:0/3999293717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55ec0749c0 0x7f55ec074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.063+0000 7f55f3000640 1 --2- 172.21.15.42:0/3999293717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ec075fa0 0x7f55ec0743f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.063+0000 7f55f3000640 1 -- 172.21.15.42:0/3999293717 >> 172.21.15.42:0/3999293717 conn(0x7f55ec0fd510 msgr2=0x7f55ec0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:41.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.063+0000 7f55f3000640 1 -- 172.21.15.42:0/3999293717 shutdown_connections 2024-04-20T10:31:41.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.063+0000 7f55f3000640 1 -- 172.21.15.42:0/3999293717 wait complete. 2024-04-20T10:31:41.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.064+0000 7f55f3000640 1 Processor -- start 2024-04-20T10:31:41.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.064+0000 7f55f3000640 1 -- start start 2024-04-20T10:31:41.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.065+0000 7f55f3000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ec0749c0 0x7f55ec19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:41.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.065+0000 7f55f3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ec075fa0 0x7f55ec19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:41.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.065+0000 7f55f3000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55ec109e60 0x7f55ec1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:41.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.065+0000 7f55f3000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec10fdf0 con 0x7f55ec109e60 2024-04-20T10:31:41.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.065+0000 7f55f3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec10fc70 con 0x7f55ec075fa0 2024-04-20T10:31:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.065+0000 7f55f3000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec10ff70 con 0x7f55ec0749c0 2024-04-20T10:31:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.065+0000 7f55ebe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ec0749c0 0x7f55ec19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.065+0000 7f55f1000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55ec109e60 0x7f55ec1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.065+0000 7f55ebe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ec0749c0 0x7f55ec19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:36010/0 (socket says 172.21.15.42:36010) 2024-04-20T10:31:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.065+0000 7f55ebe00640 1 -- 172.21.15.42:0/3752071362 learned_addr learned my addr 172.21.15.42:0/3752071362 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.065+0000 7f55eb400640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ec075fa0 0x7f55ec19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.066+0000 7f55ebe00640 1 -- 172.21.15.42:0/3752071362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ec075fa0 msgr2=0x7f55ec19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.066+0000 7f55ebe00640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ec075fa0 0x7f55ec19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.066+0000 7f55ebe00640 1 -- 172.21.15.42:0/3752071362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55ec109e60 msgr2=0x7f55ec1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.066+0000 7f55ebe00640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55ec109e60 0x7f55ec1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.066+0000 7f55ebe00640 1 -- 172.21.15.42:0/3752071362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ec1a20c0 con 0x7f55ec0749c0 2024-04-20T10:31:41.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.066+0000 7f55ebe00640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ec0749c0 0x7f55ec19a4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0325f0 tx=0x7f55dc035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:41.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.066+0000 7f55e8c00640 1 -- 172.21.15.42:0/3752071362 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc039040 con 0x7f55ec0749c0 2024-04-20T10:31:41.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.066+0000 7f55e8c00640 1 -- 172.21.15.42:0/3752071362 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55dc044070 con 0x7f55ec0749c0 2024-04-20T10:31:41.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.066+0000 7f55f3000640 1 -- 172.21.15.42:0/3752071362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ec1a2350 con 0x7f55ec0749c0 2024-04-20T10:31:41.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.066+0000 7f55e8c00640 1 -- 172.21.15.42:0/3752071362 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc004610 con 0x7f55ec0749c0 2024-04-20T10:31:41.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.067+0000 7f55f3000640 1 -- 172.21.15.42:0/3752071362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ec1a2810 con 0x7f55ec0749c0 2024-04-20T10:31:41.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.068+0000 7f55f3000640 1 -- 172.21.15.42:0/3752071362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55ec108b90 con 0x7f55ec0749c0 2024-04-20T10:31:41.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.070+0000 7f55e8c00640 1 -- 172.21.15.42:0/3752071362 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f55dc003970 con 0x7f55ec0749c0 2024-04-20T10:31:41.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.071+0000 7f55e8c00640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f55d00a3c40 0x7f55d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:41.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.071+0000 7f55e8c00640 1 -- 172.21.15.42:0/3752071362 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f55dc0f2a60 con 0x7f55ec0749c0 2024-04-20T10:31:41.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.071+0000 7f55eb400640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f55d00a3c40 0x7f55d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:41.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.072+0000 7f55eb400640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f55d00a3c40 0x7f55d00a6100 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f55ec19b9b0 tx=0x7f55d800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:41.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.075+0000 7f55e8c00640 1 -- 172.21.15.42:0/3752071362 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f55dc0bf520 con 0x7f55ec0749c0 2024-04-20T10:31:41.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:40 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2194569496' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-20T10:31:41.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:40 smithi042 ceph-mon[17078]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-20T10:31:41.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:40 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2194569496' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-20T10:31:41.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:40 smithi162 ceph-mon[21854]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-20T10:31:41.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.309+0000 7f55f3000640 1 -- 172.21.15.42:0/3752071362 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f55ec062430 con 0x7f55d00a3c40 2024-04-20T10:31:41.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.319+0000 7f55e8c00640 1 -- 172.21.15.42:0/3752071362 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f55ec062430 con 0x7f55d00a3c40 2024-04-20T10:31:41.320 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/qa/sv_0/faf2b45c-ccb8-4d6e-b730-3a16894aba7a 2024-04-20T10:31:41.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.322+0000 7f55f3000640 1 -- 172.21.15.42:0/3752071362 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f55d00a3c40 msgr2=0x7f55d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:41.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.322+0000 7f55f3000640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f55d00a3c40 0x7f55d00a6100 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f55ec19b9b0 tx=0x7f55d800b390 comp rx=0 tx=0).stop 2024-04-20T10:31:41.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.322+0000 7f55f3000640 1 -- 172.21.15.42:0/3752071362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ec0749c0 msgr2=0x7f55ec19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:41.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.322+0000 7f55f3000640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ec0749c0 0x7f55ec19a4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0325f0 tx=0x7f55dc035d10 comp rx=0 tx=0).stop 2024-04-20T10:31:41.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.322+0000 7f55f3000640 1 -- 172.21.15.42:0/3752071362 shutdown_connections 2024-04-20T10:31:41.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.322+0000 7f55f3000640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55ec109e60 0x7f55ec1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.322+0000 7f55f3000640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55ec075fa0 0x7f55ec19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.322+0000 7f55f3000640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f55d00a3c40 0x7f55d00a6100 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.322+0000 7f55f3000640 1 --2- 172.21.15.42:0/3752071362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55ec0749c0 0x7f55ec19a4f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.322+0000 7f55f3000640 1 -- 172.21.15.42:0/3752071362 >> 172.21.15.42:0/3752071362 conn(0x7f55ec0fd510 msgr2=0x7f55ec105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:41.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.322+0000 7f55f3000640 1 -- 172.21.15.42:0/3752071362 shutdown_connections 2024-04-20T10:31:41.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.322+0000 7f55f3000640 1 -- 172.21.15.42:0/3752071362 wait complete. 2024-04-20T10:31:41.340 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-20T10:31:41.553 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:31:41 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b[52186]: 2024-04-20T10:31:41.162+0000 7fadd9665700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:31:41.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.554+0000 7f780b000640 1 -- 172.21.15.42:0/3591307912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78041040a0 msgr2=0x7f78041044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:41.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.554+0000 7f780b000640 1 --2- 172.21.15.42:0/3591307912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78041040a0 0x7f78041044c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009900 tx=0x7f77f802bc30 comp rx=0 tx=0).stop 2024-04-20T10:31:41.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.554+0000 7f780b000640 1 -- 172.21.15.42:0/3591307912 shutdown_connections 2024-04-20T10:31:41.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.554+0000 7f780b000640 1 --2- 172.21.15.42:0/3591307912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7804105c50 0x7f780410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.554+0000 7f780b000640 1 --2- 172.21.15.42:0/3591307912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7804105290 0x7f7804105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.554+0000 7f780b000640 1 --2- 172.21.15.42:0/3591307912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78041040a0 0x7f78041044c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.554+0000 7f780b000640 1 -- 172.21.15.42:0/3591307912 >> 172.21.15.42:0/3591307912 conn(0x7f78040ff610 msgr2=0x7f7804101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:41.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.554+0000 7f780b000640 1 -- 172.21.15.42:0/3591307912 shutdown_connections 2024-04-20T10:31:41.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.554+0000 7f780b000640 1 -- 172.21.15.42:0/3591307912 wait complete. 2024-04-20T10:31:41.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.555+0000 7f780b000640 1 Processor -- start 2024-04-20T10:31:41.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f780b000640 1 -- start start 2024-04-20T10:31:41.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f780b000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78041040a0 0x7f780419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:41.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f780b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7804105290 0x7f780419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:41.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f780b000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7804105c50 0x7f78041a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f780b000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f780410fd30 con 0x7f7804105c50 2024-04-20T10:31:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f780b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f780410fbb0 con 0x7f7804105290 2024-04-20T10:31:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f780b000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f780410feb0 con 0x7f78041040a0 2024-04-20T10:31:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f7809c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78041040a0 0x7f780419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f7809200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7804105290 0x7f780419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f780a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7804105c50 0x7f78041a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f7809c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78041040a0 0x7f780419a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:36038/0 (socket says 172.21.15.42:36038) 2024-04-20T10:31:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f7809200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7804105290 0x7f780419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:55258/0 (socket says 172.21.15.42:55258) 2024-04-20T10:31:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.556+0000 7f7809c00640 1 -- 172.21.15.42:0/1940228842 learned_addr learned my addr 172.21.15.42:0/1940228842 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.557+0000 7f7809200640 1 -- 172.21.15.42:0/1940228842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78041040a0 msgr2=0x7f780419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:41.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.557+0000 7f7809200640 1 --2- 172.21.15.42:0/1940228842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78041040a0 0x7f780419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.557+0000 7f7809200640 1 -- 172.21.15.42:0/1940228842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7804105c50 msgr2=0x7f78041a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:41.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.557+0000 7f7809200640 1 --2- 172.21.15.42:0/1940228842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7804105c50 0x7f78041a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.557+0000 7f7809200640 1 -- 172.21.15.42:0/1940228842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78041a1ff0 con 0x7f7804105290 2024-04-20T10:31:41.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.557+0000 7f7809200640 1 --2- 172.21.15.42:0/1940228842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7804105290 0x7f780419a910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77f000e9f0 tx=0x7f77f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:41.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.557+0000 7f7802000640 1 -- 172.21.15.42:0/1940228842 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f000ccc0 con 0x7f7804105290 2024-04-20T10:31:41.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.557+0000 7f780b000640 1 -- 172.21.15.42:0/1940228842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78041a22e0 con 0x7f7804105290 2024-04-20T10:31:41.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.557+0000 7f7802000640 1 -- 172.21.15.42:0/1940228842 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f0005020 con 0x7f7804105290 2024-04-20T10:31:41.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.557+0000 7f7802000640 1 -- 172.21.15.42:0/1940228842 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f00049e0 con 0x7f7804105290 2024-04-20T10:31:41.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.558+0000 7f780b000640 1 -- 172.21.15.42:0/1940228842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78041a27e0 con 0x7f7804105290 2024-04-20T10:31:41.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.559+0000 7f780b000640 1 -- 172.21.15.42:0/1940228842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780410b490 con 0x7f7804105290 2024-04-20T10:31:41.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.562+0000 7f7802000640 1 -- 172.21.15.42:0/1940228842 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f77f0005560 con 0x7f7804105290 2024-04-20T10:31:41.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.564+0000 7f7802000640 1 --2- 172.21.15.42:0/1940228842 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f77ec0a3c40 0x7f77ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:41.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.564+0000 7f7802000640 1 -- 172.21.15.42:0/1940228842 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f77f00d05c0 con 0x7f7804105290 2024-04-20T10:31:41.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.564+0000 7f7809c00640 1 --2- 172.21.15.42:0/1940228842 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f77ec0a3c40 0x7f77ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:41.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.565+0000 7f7802000640 1 -- 172.21.15.42:0/1940228842 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f77f00051d0 con 0x7f7804105290 2024-04-20T10:31:41.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.565+0000 7f7809c00640 1 --2- 172.21.15.42:0/1940228842 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f77ec0a3c40 0x7f77ec0a6100 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f77f802c8e0 tx=0x7f77f8005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:41.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.808+0000 7f780b000640 1 -- 172.21.15.42:0/1940228842 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f78040623a0 con 0x7f77ec0a3c40 2024-04-20T10:31:41.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.842+0000 7f7802000640 1 -- 172.21.15.42:0/1940228842 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f78040623a0 con 0x7f77ec0a3c40 2024-04-20T10:31:41.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.844+0000 7f780b000640 1 -- 172.21.15.42:0/1940228842 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f77ec0a3c40 msgr2=0x7f77ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:41.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.844+0000 7f780b000640 1 --2- 172.21.15.42:0/1940228842 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f77ec0a3c40 0x7f77ec0a6100 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f77f802c8e0 tx=0x7f77f8005b50 comp rx=0 tx=0).stop 2024-04-20T10:31:41.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.844+0000 7f780b000640 1 -- 172.21.15.42:0/1940228842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7804105290 msgr2=0x7f780419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:41.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.844+0000 7f780b000640 1 --2- 172.21.15.42:0/1940228842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7804105290 0x7f780419a910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77f000e9f0 tx=0x7f77f000eeb0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.845+0000 7f780b000640 1 -- 172.21.15.42:0/1940228842 shutdown_connections 2024-04-20T10:31:41.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.845+0000 7f780b000640 1 --2- 172.21.15.42:0/1940228842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7804105c50 0x7f78041a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.845+0000 7f780b000640 1 --2- 172.21.15.42:0/1940228842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7804105290 0x7f780419a910 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.845+0000 7f780b000640 1 --2- 172.21.15.42:0/1940228842 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f77ec0a3c40 0x7f77ec0a6100 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.845+0000 7f780b000640 1 --2- 172.21.15.42:0/1940228842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78041040a0 0x7f780419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:41.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.845+0000 7f780b000640 1 -- 172.21.15.42:0/1940228842 >> 172.21.15.42:0/1940228842 conn(0x7f78040ff610 msgr2=0x7f7804108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:41.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.845+0000 7f780b000640 1 -- 172.21.15.42:0/1940228842 shutdown_connections 2024-04-20T10:31:41.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:41.845+0000 7f780b000640 1 -- 172.21.15.42:0/1940228842 wait complete. 2024-04-20T10:31:41.865 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-20T10:31:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:41 smithi046 ceph-mon[21574]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:41 smithi046 ceph-mon[21574]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-20T10:31:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:41 smithi046 ceph-mon[21574]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:41 smithi046 ceph-mon[21574]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-20T10:31:42.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.077+0000 7f84a5400640 1 -- 172.21.15.42:0/895847275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84a01006e0 msgr2=0x7f84a0100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:42.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.077+0000 7f84a5400640 1 --2- 172.21.15.42:0/895847275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84a01006e0 0x7f84a0100b00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8490009900 tx=0x7f849002bc30 comp rx=0 tx=0).stop 2024-04-20T10:31:42.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.078+0000 7f84a5400640 1 -- 172.21.15.42:0/895847275 shutdown_connections 2024-04-20T10:31:42.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.078+0000 7f84a5400640 1 --2- 172.21.15.42:0/895847275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a01019e0 0x7f84a0101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.078+0000 7f84a5400640 1 --2- 172.21.15.42:0/895847275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a0101040 0x7f84a01014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.078+0000 7f84a5400640 1 --2- 172.21.15.42:0/895847275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84a01006e0 0x7f84a0100b00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.078+0000 7f84a5400640 1 -- 172.21.15.42:0/895847275 >> 172.21.15.42:0/895847275 conn(0x7f84a00fc300 msgr2=0x7f84a00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:42.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.078+0000 7f84a5400640 1 -- 172.21.15.42:0/895847275 shutdown_connections 2024-04-20T10:31:42.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.078+0000 7f84a5400640 1 -- 172.21.15.42:0/895847275 wait complete. 2024-04-20T10:31:42.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.079+0000 7f84a5400640 1 Processor -- start 2024-04-20T10:31:42.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.079+0000 7f84a5400640 1 -- start start 2024-04-20T10:31:42.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.079+0000 7f84a5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a01006e0 0x7f84a019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:42.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f84a5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a0101040 0x7f84a019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:42.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f84a5400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84a01019e0 0x7f84a01a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:42.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f84a5400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f84a010fd10 con 0x7f84a01006e0 2024-04-20T10:31:42.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f84a5400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84a010fb90 con 0x7f84a0101040 2024-04-20T10:31:42.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f84a5400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f84a010fe90 con 0x7f84a01019e0 2024-04-20T10:31:42.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f849fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a01006e0 0x7f84a019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:42.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f849fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a01006e0 0x7f84a019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51700/0 (socket says 172.21.15.42:51700) 2024-04-20T10:31:42.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f849fe00640 1 -- 172.21.15.42:0/757432290 learned_addr learned my addr 172.21.15.42:0/757432290 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:42.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f849f400640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a0101040 0x7f84a019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:42.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f84a4a00640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84a01019e0 0x7f84a01a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:42.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f849fe00640 1 -- 172.21.15.42:0/757432290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84a01019e0 msgr2=0x7f84a01a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:42.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f849fe00640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84a01019e0 0x7f84a01a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f849fe00640 1 -- 172.21.15.42:0/757432290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a0101040 msgr2=0x7f84a019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:42.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f849fe00640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a0101040 0x7f84a019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f849fe00640 1 -- 172.21.15.42:0/757432290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a01a1f80 con 0x7f84a01006e0 2024-04-20T10:31:42.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.080+0000 7f849f400640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a0101040 0x7f84a019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:42.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.081+0000 7f849fe00640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a01006e0 0x7f84a019a360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8490009230 tx=0x7f849002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:42.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.081+0000 7f849cc00640 1 -- 172.21.15.42:0/757432290 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490041070 con 0x7f84a01006e0 2024-04-20T10:31:42.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.081+0000 7f84a5400640 1 -- 172.21.15.42:0/757432290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a01a2210 con 0x7f84a01006e0 2024-04-20T10:31:42.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.081+0000 7f849cc00640 1 -- 172.21.15.42:0/757432290 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8490004450 con 0x7f84a01006e0 2024-04-20T10:31:42.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.081+0000 7f849cc00640 1 -- 172.21.15.42:0/757432290 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490003db0 con 0x7f84a01006e0 2024-04-20T10:31:42.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.081+0000 7f84a5400640 1 -- 172.21.15.42:0/757432290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a01a2700 con 0x7f84a01006e0 2024-04-20T10:31:42.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.082+0000 7f84a5400640 1 -- 172.21.15.42:0/757432290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a010b430 con 0x7f84a01006e0 2024-04-20T10:31:42.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.084+0000 7f849cc00640 1 -- 172.21.15.42:0/757432290 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8490044070 con 0x7f84a01006e0 2024-04-20T10:31:42.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.086+0000 7f849cc00640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f84840a3c40 0x7f84840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:42.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.086+0000 7f849cc00640 1 -- 172.21.15.42:0/757432290 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f84900f1f30 con 0x7f84a01006e0 2024-04-20T10:31:42.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.086+0000 7f849f400640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f84840a3c40 0x7f84840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:42.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.087+0000 7f849f400640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f84840a3c40 0x7f84840a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f848c0095f0 tx=0x7f848c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:42.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.090+0000 7f849cc00640 1 -- 172.21.15.42:0/757432290 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f84900be970 con 0x7f84a01006e0 2024-04-20T10:31:42.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:41 smithi042 ceph-mon[17078]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:42.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:41 smithi042 ceph-mon[17078]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-20T10:31:42.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:41 smithi042 ceph-mon[17078]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:42.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:41 smithi042 ceph-mon[17078]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-20T10:31:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:41 smithi162 ceph-mon[21854]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:41 smithi162 ceph-mon[21854]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-20T10:31:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:41 smithi162 ceph-mon[21854]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:42.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:41 smithi162 ceph-mon[21854]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-20T10:31:42.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.328+0000 7f84a5400640 1 -- 172.21.15.42:0/757432290 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f84a0062430 con 0x7f84840a3c40 2024-04-20T10:31:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.336+0000 7f849cc00640 1 -- 172.21.15.42:0/757432290 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f84a0062430 con 0x7f84840a3c40 2024-04-20T10:31:42.339 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/qa/sv_1/bb321639-ad29-40c2-98d4-8ea3b9b7ace4 2024-04-20T10:31:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.338+0000 7f84a5400640 1 -- 172.21.15.42:0/757432290 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f84840a3c40 msgr2=0x7f84840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.338+0000 7f84a5400640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f84840a3c40 0x7f84840a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f848c0095f0 tx=0x7f848c005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.338+0000 7f84a5400640 1 -- 172.21.15.42:0/757432290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a01006e0 msgr2=0x7f84a019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.338+0000 7f84a5400640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a01006e0 0x7f84a019a360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8490009230 tx=0x7f849002c650 comp rx=0 tx=0).stop 2024-04-20T10:31:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.339+0000 7f84a5400640 1 -- 172.21.15.42:0/757432290 shutdown_connections 2024-04-20T10:31:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.339+0000 7f84a5400640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f84840a3c40 0x7f84840a6100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.339+0000 7f84a5400640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84a01019e0 0x7f84a01a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.339+0000 7f84a5400640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84a0101040 0x7f84a019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.339+0000 7f84a5400640 1 --2- 172.21.15.42:0/757432290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a01006e0 0x7f84a019a360 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.339+0000 7f84a5400640 1 -- 172.21.15.42:0/757432290 >> 172.21.15.42:0/757432290 conn(0x7f84a00fc300 msgr2=0x7f84a00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.339+0000 7f84a5400640 1 -- 172.21.15.42:0/757432290 shutdown_connections 2024-04-20T10:31:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.339+0000 7f84a5400640 1 -- 172.21.15.42:0/757432290 wait complete. 2024-04-20T10:31:42.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:42.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.572+0000 7f47fa200640 1 -- 172.21.15.42:0/3473432694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47f41051d0 msgr2=0x7f47f4105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:42.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.572+0000 7f47fa200640 1 --2- 172.21.15.42:0/3473432694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47f41051d0 0x7f47f4105650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009860 tx=0x7f47e002bb70 comp rx=0 tx=0).stop 2024-04-20T10:31:42.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.572+0000 7f47fa200640 1 -- 172.21.15.42:0/3473432694 shutdown_connections 2024-04-20T10:31:42.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.572+0000 7f47fa200640 1 --2- 172.21.15.42:0/3473432694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47f4105b90 0x7f47f410c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.572+0000 7f47fa200640 1 --2- 172.21.15.42:0/3473432694 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47f41051d0 0x7f47f4105650 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.572+0000 7f47fa200640 1 --2- 172.21.15.42:0/3473432694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f4103fe0 0x7f47f4104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.572+0000 7f47fa200640 1 -- 172.21.15.42:0/3473432694 >> 172.21.15.42:0/3473432694 conn(0x7f47f40ff570 msgr2=0x7f47f4101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:42.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.573+0000 7f47fa200640 1 -- 172.21.15.42:0/3473432694 shutdown_connections 2024-04-20T10:31:42.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.573+0000 7f47fa200640 1 -- 172.21.15.42:0/3473432694 wait complete. 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.574+0000 7f47fa200640 1 Processor -- start 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.574+0000 7f47fa200640 1 -- start start 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.574+0000 7f47fa200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f4103fe0 0x7f47f419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.574+0000 7f47fa200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47f41051d0 0x7f47f419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.574+0000 7f47fa200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47f4105b90 0x7f47f41a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.574+0000 7f47fa200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f47f410fc70 con 0x7f47f4103fe0 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.574+0000 7f47fa200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f47f410faf0 con 0x7f47f41051d0 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.575+0000 7f47fa200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f47f410fdf0 con 0x7f47f4105b90 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.575+0000 7f47f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f4103fe0 0x7f47f419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.575+0000 7f47f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f4103fe0 0x7f47f419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51728/0 (socket says 172.21.15.42:51728) 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.575+0000 7f47f8e00640 1 -- 172.21.15.42:0/3352165529 learned_addr learned my addr 172.21.15.42:0/3352165529 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.575+0000 7f47f9800640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47f4105b90 0x7f47f41a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.575+0000 7f47f3e00640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47f41051d0 0x7f47f419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:42.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.575+0000 7f47f8e00640 1 -- 172.21.15.42:0/3352165529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47f4105b90 msgr2=0x7f47f41a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:42.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.575+0000 7f47f8e00640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47f4105b90 0x7f47f41a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.575+0000 7f47f8e00640 1 -- 172.21.15.42:0/3352165529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47f41051d0 msgr2=0x7f47f419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:42.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.575+0000 7f47f8e00640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47f41051d0 0x7f47f419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.575+0000 7f47f8e00640 1 -- 172.21.15.42:0/3352165529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f41a1ee0 con 0x7f47f4103fe0 2024-04-20T10:31:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.576+0000 7f47f8e00640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f4103fe0 0x7f47f419a2c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f47e800e9c0 tx=0x7f47e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.576+0000 7f47f1600640 1 -- 172.21.15.42:0/3352165529 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e800cd80 con 0x7f47f4103fe0 2024-04-20T10:31:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.576+0000 7f47f1600640 1 -- 172.21.15.42:0/3352165529 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e80035f0 con 0x7f47f4103fe0 2024-04-20T10:31:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.576+0000 7f47fa200640 1 -- 172.21.15.42:0/3352165529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f41a21d0 con 0x7f47f4103fe0 2024-04-20T10:31:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.576+0000 7f47f1600640 1 -- 172.21.15.42:0/3352165529 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8010690 con 0x7f47f4103fe0 2024-04-20T10:31:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.576+0000 7f47fa200640 1 -- 172.21.15.42:0/3352165529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f41a26f0 con 0x7f47f4103fe0 2024-04-20T10:31:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.577+0000 7f47fa200640 1 -- 172.21.15.42:0/3352165529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f410b300 con 0x7f47f4103fe0 2024-04-20T10:31:42.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.580+0000 7f47f1600640 1 -- 172.21.15.42:0/3352165529 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f47e80037a0 con 0x7f47f4103fe0 2024-04-20T10:31:42.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.581+0000 7f47f1600640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f47dc0a3c40 0x7f47dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:42.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.581+0000 7f47f1600640 1 -- 172.21.15.42:0/3352165529 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f47e80d08e0 con 0x7f47f4103fe0 2024-04-20T10:31:42.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.581+0000 7f47f3e00640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f47dc0a3c40 0x7f47dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:42.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.582+0000 7f47f3e00640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f47dc0a3c40 0x7f47dc0a6100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009830 tx=0x7f47e0034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:42.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.585+0000 7f47f1600640 1 -- 172.21.15.42:0/3352165529 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f47e809d210 con 0x7f47f4103fe0 2024-04-20T10:31:42.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.839+0000 7f47fa200640 1 -- 172.21.15.42:0/3352165529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47f402d5d0 con 0x7f47f4103fe0 2024-04-20T10:31:42.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.841+0000 7f47f1600640 1 -- 172.21.15.42:0/3352165529 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f47e809cb10 con 0x7f47f4103fe0 2024-04-20T10:31:42.842 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:42.842 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:31:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:31:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.844+0000 7f47fa200640 1 -- 172.21.15.42:0/3352165529 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f47dc0a3c40 msgr2=0x7f47dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.844+0000 7f47fa200640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f47dc0a3c40 0x7f47dc0a6100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009830 tx=0x7f47e0034ab0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.844+0000 7f47fa200640 1 -- 172.21.15.42:0/3352165529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f4103fe0 msgr2=0x7f47f419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.844+0000 7f47fa200640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f4103fe0 0x7f47f419a2c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f47e800e9c0 tx=0x7f47e800ee80 comp rx=0 tx=0).stop 2024-04-20T10:31:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.844+0000 7f47fa200640 1 -- 172.21.15.42:0/3352165529 shutdown_connections 2024-04-20T10:31:42.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.844+0000 7f47fa200640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f47dc0a3c40 0x7f47dc0a6100 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.844+0000 7f47fa200640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47f4105b90 0x7f47f41a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.844+0000 7f47fa200640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47f41051d0 0x7f47f419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.844+0000 7f47fa200640 1 --2- 172.21.15.42:0/3352165529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f4103fe0 0x7f47f419a2c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.844+0000 7f47fa200640 1 -- 172.21.15.42:0/3352165529 >> 172.21.15.42:0/3352165529 conn(0x7f47f40ff570 msgr2=0x7f47f41084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.845+0000 7f47fa200640 1 -- 172.21.15.42:0/3352165529 shutdown_connections 2024-04-20T10:31:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:42.845+0000 7f47fa200640 1 -- 172.21.15.42:0/3352165529 wait complete. 2024-04-20T10:31:42.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:31:43.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.074+0000 7f17d5800640 1 -- 172.21.15.42:0/2854929944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17d0105190 msgr2=0x7f17d0107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:43.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.074+0000 7f17d5800640 1 --2- 172.21.15.42:0/2854929944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17d0105190 0x7f17d0107580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009900 tx=0x7f17b802bc40 comp rx=0 tx=0).stop 2024-04-20T10:31:43.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.074+0000 7f17d5800640 1 -- 172.21.15.42:0/2854929944 shutdown_connections 2024-04-20T10:31:43.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.074+0000 7f17d5800640 1 --2- 172.21.15.42:0/2854929944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17d0107ac0 0x7f17d0109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.074+0000 7f17d5800640 1 --2- 172.21.15.42:0/2854929944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17d0105190 0x7f17d0107580 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.074+0000 7f17d5800640 1 --2- 172.21.15.42:0/2854929944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0102860 0x7f17d0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.074+0000 7f17d5800640 1 -- 172.21.15.42:0/2854929944 >> 172.21.15.42:0/2854929944 conn(0x7f17d00fc230 msgr2=0x7f17d00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:43.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.074+0000 7f17d5800640 1 -- 172.21.15.42:0/2854929944 shutdown_connections 2024-04-20T10:31:43.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.074+0000 7f17d5800640 1 -- 172.21.15.42:0/2854929944 wait complete. 2024-04-20T10:31:43.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.076+0000 7f17d5800640 1 Processor -- start 2024-04-20T10:31:43.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.076+0000 7f17d5800640 1 -- start start 2024-04-20T10:31:43.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.076+0000 7f17d5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17d0102860 0x7f17d01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:43.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.076+0000 7f17d5800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17d0105190 0x7f17d0198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:43.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.076+0000 7f17d5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0107ac0 0x7f17d019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.076+0000 7f17d5800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17d010da50 con 0x7f17d0102860 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.076+0000 7f17d5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17d010d8d0 con 0x7f17d0107ac0 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.076+0000 7f17d5800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f17d010dbd0 con 0x7f17d0105190 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.076+0000 7f17cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17d0102860 0x7f17d01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.077+0000 7f17cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17d0102860 0x7f17d01980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51754/0 (socket says 172.21.15.42:51754) 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.077+0000 7f17cfe00640 1 -- 172.21.15.42:0/1340706152 learned_addr learned my addr 172.21.15.42:0/1340706152 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.077+0000 7f17cf400640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17d0105190 0x7f17d0198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.077+0000 7f17d4e00640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0107ac0 0x7f17d019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.077+0000 7f17cf400640 1 -- 172.21.15.42:0/1340706152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0107ac0 msgr2=0x7f17d019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.077+0000 7f17cf400640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0107ac0 0x7f17d019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.077+0000 7f17cf400640 1 -- 172.21.15.42:0/1340706152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17d0102860 msgr2=0x7f17d01980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.077+0000 7f17cf400640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17d0102860 0x7f17d01980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.077+0000 7f17cf400640 1 -- 172.21.15.42:0/1340706152 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d019fcf0 con 0x7f17d0105190 2024-04-20T10:31:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.077+0000 7f17cfe00640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17d0102860 0x7f17d01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:43.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.077+0000 7f17cf400640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17d0105190 0x7f17d0198610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f17b802c180 tx=0x7f17b802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:43.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.078+0000 7f17ccc00640 1 -- 172.21.15.42:0/1340706152 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8041070 con 0x7f17d0105190 2024-04-20T10:31:43.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.078+0000 7f17ccc00640 1 -- 172.21.15.42:0/1340706152 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b8004450 con 0x7f17d0105190 2024-04-20T10:31:43.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.078+0000 7f17d5800640 1 -- 172.21.15.42:0/1340706152 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d019ff80 con 0x7f17d0105190 2024-04-20T10:31:43.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.078+0000 7f17ccc00640 1 -- 172.21.15.42:0/1340706152 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8034440 con 0x7f17d0105190 2024-04-20T10:31:43.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.078+0000 7f17d5800640 1 -- 172.21.15.42:0/1340706152 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d01a03e0 con 0x7f17d0105190 2024-04-20T10:31:43.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.079+0000 7f17d5800640 1 -- 172.21.15.42:0/1340706152 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d00fdd20 con 0x7f17d0105190 2024-04-20T10:31:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.081+0000 7f17ccc00640 1 -- 172.21.15.42:0/1340706152 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f17b8044070 con 0x7f17d0105190 2024-04-20T10:31:43.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.083+0000 7f17ccc00640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f17b40a3c40 0x7f17b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:43.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.083+0000 7f17ccc00640 1 -- 172.21.15.42:0/1340706152 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f17b80f2040 con 0x7f17d0105190 2024-04-20T10:31:43.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.083+0000 7f17cfe00640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f17b40a3c40 0x7f17b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:43.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.084+0000 7f17cfe00640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f17b40a3c40 0x7f17b40a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f17c000f7d0 tx=0x7f17c0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:43.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.085+0000 7f17ccc00640 1 -- 172.21.15.42:0/1340706152 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f17b80be430 con 0x7f17d0105190 2024-04-20T10:31:43.102 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:42 smithi042 ceph-mon[17078]: from='client.24700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:43.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:42 smithi042 ceph-mon[17078]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:43.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:42 smithi162 ceph-mon[21854]: from='client.24700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:43.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:42 smithi162 ceph-mon[21854]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:43.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.297+0000 7f17d5800640 1 -- 172.21.15.42:0/1340706152 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17d002d820 con 0x7f17d0105190 2024-04-20T10:31:43.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.299+0000 7f17ccc00640 1 -- 172.21.15.42:0/1340706152 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22197 (secure 0 0 0) 0x7f17b80c2730 con 0x7f17d0105190 2024-04-20T10:31:43.300 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:43.301 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":85,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:31:36.594768+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:31:43.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.302+0000 7f17d5800640 1 -- 172.21.15.42:0/1340706152 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f17b40a3c40 msgr2=0x7f17b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:43.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.302+0000 7f17d5800640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f17b40a3c40 0x7f17b40a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f17c000f7d0 tx=0x7f17c0009500 comp rx=0 tx=0).stop 2024-04-20T10:31:43.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.302+0000 7f17d5800640 1 -- 172.21.15.42:0/1340706152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17d0105190 msgr2=0x7f17d0198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:43.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.302+0000 7f17d5800640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17d0105190 0x7f17d0198610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f17b802c180 tx=0x7f17b802c660 comp rx=0 tx=0).stop 2024-04-20T10:31:43.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.302+0000 7f17d5800640 1 -- 172.21.15.42:0/1340706152 shutdown_connections 2024-04-20T10:31:43.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.302+0000 7f17d5800640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0107ac0 0x7f17d019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.302+0000 7f17d5800640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f17b40a3c40 0x7f17b40a6100 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.302+0000 7f17d5800640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17d0105190 0x7f17d0198610 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.302+0000 7f17d5800640 1 --2- 172.21.15.42:0/1340706152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17d0102860 0x7f17d01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.302+0000 7f17d5800640 1 -- 172.21.15.42:0/1340706152 >> 172.21.15.42:0/1340706152 conn(0x7f17d00fc230 msgr2=0x7f17d01032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:43.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.303+0000 7f17d5800640 1 -- 172.21.15.42:0/1340706152 shutdown_connections 2024-04-20T10:31:43.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.303+0000 7f17d5800640 1 -- 172.21.15.42:0/1340706152 wait complete. 2024-04-20T10:31:43.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:42 smithi046 ceph-mon[21574]: from='client.24700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:43.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:42 smithi046 ceph-mon[21574]: from='client.24686 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:43.323 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T10:31:43.323 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:31:43.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.534+0000 7f0248a00640 1 -- 172.21.15.42:0/4237671847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0244109f90 msgr2=0x7f0244110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:43.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.534+0000 7f0248a00640 1 --2- 172.21.15.42:0/4237671847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0244109f90 0x7f0244110820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f023c009ca0 tx=0x7f023c02c100 comp rx=0 tx=0).stop 2024-04-20T10:31:43.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.534+0000 7f0248a00640 1 -- 172.21.15.42:0/4237671847 shutdown_connections 2024-04-20T10:31:43.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.534+0000 7f0248a00640 1 --2- 172.21.15.42:0/4237671847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0244109f90 0x7f0244110820 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.534+0000 7f0248a00640 1 --2- 172.21.15.42:0/4237671847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02441095d0 0x7f0244109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.534+0000 7f0248a00640 1 --2- 172.21.15.42:0/4237671847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02441083e0 0x7f0244108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.534+0000 7f0248a00640 1 -- 172.21.15.42:0/4237671847 >> 172.21.15.42:0/4237671847 conn(0x7f02440773a0 msgr2=0x7f02440797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:43.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.534+0000 7f0248a00640 1 -- 172.21.15.42:0/4237671847 shutdown_connections 2024-04-20T10:31:43.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.534+0000 7f0248a00640 1 -- 172.21.15.42:0/4237671847 wait complete. 2024-04-20T10:31:43.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.535+0000 7f0248a00640 1 Processor -- start 2024-04-20T10:31:43.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.535+0000 7f0248a00640 1 -- start start 2024-04-20T10:31:43.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0248a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02441083e0 0x7f024419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:43.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0248a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02441095d0 0x7f024419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:43.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0248a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0244109f90 0x7f02441a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:43.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0248a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0244114070 con 0x7f02441095d0 2024-04-20T10:31:43.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0248a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0244113ef0 con 0x7f0244109f90 2024-04-20T10:31:43.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0248a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f02441141f0 con 0x7f02441083e0 2024-04-20T10:31:43.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0242800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02441083e0 0x7f024419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:43.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0241e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02441095d0 0x7f024419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:43.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0242800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02441083e0 0x7f024419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:36120/0 (socket says 172.21.15.42:36120) 2024-04-20T10:31:43.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0243200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0244109f90 0x7f02441a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0242800640 1 -- 172.21.15.42:0/615974039 learned_addr learned my addr 172.21.15.42:0/615974039 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0242800640 1 -- 172.21.15.42:0/615974039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0244109f90 msgr2=0x7f02441a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0242800640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0244109f90 0x7f02441a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0242800640 1 -- 172.21.15.42:0/615974039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02441095d0 msgr2=0x7f024419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.536+0000 7f0242800640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02441095d0 0x7f024419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.537+0000 7f0242800640 1 -- 172.21.15.42:0/615974039 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02441a6310 con 0x7f02441083e0 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.537+0000 7f0241e00640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02441095d0 0x7f024419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.537+0000 7f0242800640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02441083e0 0x7f024419e6f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f023400e9c0 tx=0x7f023400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.537+0000 7f023b400640 1 -- 172.21.15.42:0/615974039 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023400cd80 con 0x7f02441083e0 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.537+0000 7f0248a00640 1 -- 172.21.15.42:0/615974039 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02441a6600 con 0x7f02441083e0 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.537+0000 7f023b400640 1 -- 172.21.15.42:0/615974039 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02340035f0 con 0x7f02441083e0 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.537+0000 7f023b400640 1 -- 172.21.15.42:0/615974039 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0234010690 con 0x7f02441083e0 2024-04-20T10:31:43.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.537+0000 7f0248a00640 1 -- 172.21.15.42:0/615974039 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02441a6b50 con 0x7f02441083e0 2024-04-20T10:31:43.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.538+0000 7f0248a00640 1 -- 172.21.15.42:0/615974039 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f024410f700 con 0x7f02441083e0 2024-04-20T10:31:43.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.540+0000 7f023b400640 1 -- 172.21.15.42:0/615974039 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f02340037a0 con 0x7f02441083e0 2024-04-20T10:31:43.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.542+0000 7f023b400640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f02280a3c40 0x7f02280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:43.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.542+0000 7f023b400640 1 -- 172.21.15.42:0/615974039 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f02340d08c0 con 0x7f02441083e0 2024-04-20T10:31:43.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.542+0000 7f0241e00640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f02280a3c40 0x7f02280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:43.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.543+0000 7f0241e00640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f02280a3c40 0x7f02280a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f024419fc00 tx=0x7f0230005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:43.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.546+0000 7f023b400640 1 -- 172.21.15.42:0/615974039 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f023409d1f0 con 0x7f02441083e0 2024-04-20T10:31:43.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.776+0000 7f0248a00640 1 -- 172.21.15.42:0/615974039 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02440623a0 con 0x7f02280a3c40 2024-04-20T10:31:43.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.802+0000 7f023b400640 1 -- 172.21.15.42:0/615974039 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258860 (secure 0 0 0) 0x7f02440623a0 con 0x7f02280a3c40 2024-04-20T10:31:43.808 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:43.809 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-04-20T10:31:43.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.809+0000 7f0248a00640 1 -- 172.21.15.42:0/615974039 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f02280a3c40 msgr2=0x7f02280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:43.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.809+0000 7f0248a00640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f02280a3c40 0x7f02280a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f024419fc00 tx=0x7f0230005f90 comp rx=0 tx=0).stop 2024-04-20T10:31:43.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.809+0000 7f0248a00640 1 -- 172.21.15.42:0/615974039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02441083e0 msgr2=0x7f024419e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:43.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.809+0000 7f0248a00640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02441083e0 0x7f024419e6f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f023400e9c0 tx=0x7f023400ee80 comp rx=0 tx=0).stop 2024-04-20T10:31:43.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.810+0000 7f0248a00640 1 -- 172.21.15.42:0/615974039 shutdown_connections 2024-04-20T10:31:43.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.810+0000 7f0248a00640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0244109f90 0x7f02441a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.810+0000 7f0248a00640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02441095d0 0x7f024419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.810+0000 7f0248a00640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f02280a3c40 0x7f02280a6100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.810+0000 7f0248a00640 1 --2- 172.21.15.42:0/615974039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02441083e0 0x7f024419e6f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:43.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.810+0000 7f0248a00640 1 -- 172.21.15.42:0/615974039 >> 172.21.15.42:0/615974039 conn(0x7f02440773a0 msgr2=0x7f024410c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:43.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.810+0000 7f0248a00640 1 -- 172.21.15.42:0/615974039 shutdown_connections 2024-04-20T10:31:43.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:43.810+0000 7f0248a00640 1 -- 172.21.15.42:0/615974039 wait complete. 2024-04-20T10:31:43.831 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-20T10:31:42.511684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":612178,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":148,"num_read_kb":134,"num_write":234,"num_write_kb":2974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":184,"ondisk_log_size":184,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364352,"kb_used_data":10964,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577472,"statfs":{"total":1151940427776,"available":1151567331328,"internally_reserved":0,"allocated":11227136,"data_stored":7066469,"data_compressed":107397,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21810,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":12,"num_read_kb":15,"num_write":53,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.935999"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131353+0000","last_change":"2024-04-20T10:31:35.649585+0000","last_active":"2024-04-20T10:31:37.131353+0000","last_peered":"2024-04-20T10:31:37.131353+0000","last_clean":"2024-04-20T10:31:37.131353+0000","last_became_active":"2024-04-20T10:31:35.649282+0000","last_became_peered":"2024-04-20T10:31:35.649282+0000","last_unstale":"2024-04-20T10:31:37.131353+0000","last_undegraded":"2024-04-20T10:31:37.131353+0000","last_fullsized":"2024-04-20T10:31:37.131353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:01:00.517383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.132035+0000","last_change":"2024-04-20T10:31:35.648299+0000","last_active":"2024-04-20T10:31:37.132035+0000","last_peered":"2024-04-20T10:31:37.132035+0000","last_clean":"2024-04-20T10:31:37.132035+0000","last_became_active":"2024-04-20T10:31:35.647283+0000","last_became_peered":"2024-04-20T10:31:35.647283+0000","last_unstale":"2024-04-20T10:31:37.132035+0000","last_undegraded":"2024-04-20T10:31:37.132035+0000","last_fullsized":"2024-04-20T10:31:37.132035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:59.006234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.132022+0000","last_change":"2024-04-20T10:31:35.647029+0000","last_active":"2024-04-20T10:31:37.132022+0000","last_peered":"2024-04-20T10:31:37.132022+0000","last_clean":"2024-04-20T10:31:37.132022+0000","last_became_active":"2024-04-20T10:31:35.646649+0000","last_became_peered":"2024-04-20T10:31:35.646649+0000","last_unstale":"2024-04-20T10:31:37.132022+0000","last_undegraded":"2024-04-20T10:31:37.132022+0000","last_fullsized":"2024-04-20T10:31:37.132022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:03:23.420078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129750+0000","last_change":"2024-04-20T10:31:35.624628+0000","last_active":"2024-04-20T10:31:37.129750+0000","last_peered":"2024-04-20T10:31:37.129750+0000","last_clean":"2024-04-20T10:31:37.129750+0000","last_became_active":"2024-04-20T10:31:35.623607+0000","last_became_peered":"2024-04-20T10:31:35.623607+0000","last_unstale":"2024-04-20T10:31:37.129750+0000","last_undegraded":"2024-04-20T10:31:37.129750+0000","last_fullsized":"2024-04-20T10:31:37.129750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:44:46.768584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129707+0000","last_change":"2024-04-20T10:31:35.638293+0000","last_active":"2024-04-20T10:31:37.129707+0000","last_peered":"2024-04-20T10:31:37.129707+0000","last_clean":"2024-04-20T10:31:37.129707+0000","last_became_active":"2024-04-20T10:31:35.637967+0000","last_became_peered":"2024-04-20T10:31:35.637967+0000","last_unstale":"2024-04-20T10:31:37.129707+0000","last_undegraded":"2024-04-20T10:31:37.129707+0000","last_fullsized":"2024-04-20T10:31:37.129707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:58:00.765015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130472+0000","last_change":"2024-04-20T10:31:35.634565+0000","last_active":"2024-04-20T10:31:37.130472+0000","last_peered":"2024-04-20T10:31:37.130472+0000","last_clean":"2024-04-20T10:31:37.130472+0000","last_became_active":"2024-04-20T10:31:35.634200+0000","last_became_peered":"2024-04-20T10:31:35.634200+0000","last_unstale":"2024-04-20T10:31:37.130472+0000","last_undegraded":"2024-04-20T10:31:37.130472+0000","last_fullsized":"2024-04-20T10:31:37.130472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:14.298732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031087+0000","last_change":"2024-04-20T10:31:35.637417+0000","last_active":"2024-04-20T10:31:37.031087+0000","last_peered":"2024-04-20T10:31:37.031087+0000","last_clean":"2024-04-20T10:31:37.031087+0000","last_became_active":"2024-04-20T10:31:35.637053+0000","last_became_peered":"2024-04-20T10:31:35.637053+0000","last_unstale":"2024-04-20T10:31:37.031087+0000","last_undegraded":"2024-04-20T10:31:37.031087+0000","last_fullsized":"2024-04-20T10:31:37.031087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:41:49.264677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600368+0000","last_change":"2024-04-20T10:31:35.631996+0000","last_active":"2024-04-20T10:31:36.600368+0000","last_peered":"2024-04-20T10:31:36.600368+0000","last_clean":"2024-04-20T10:31:36.600368+0000","last_became_active":"2024-04-20T10:31:35.631576+0000","last_became_peered":"2024-04-20T10:31:35.631576+0000","last_unstale":"2024-04-20T10:31:36.600368+0000","last_undegraded":"2024-04-20T10:31:36.600368+0000","last_fullsized":"2024-04-20T10:31:36.600368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:51.448025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600173+0000","last_change":"2024-04-20T10:31:35.624430+0000","last_active":"2024-04-20T10:31:36.600173+0000","last_peered":"2024-04-20T10:31:36.600173+0000","last_clean":"2024-04-20T10:31:36.600173+0000","last_became_active":"2024-04-20T10:31:35.623018+0000","last_became_peered":"2024-04-20T10:31:35.623018+0000","last_unstale":"2024-04-20T10:31:36.600173+0000","last_undegraded":"2024-04-20T10:31:36.600173+0000","last_fullsized":"2024-04-20T10:31:36.600173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:43.100925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130486+0000","last_change":"2024-04-20T10:31:35.619038+0000","last_active":"2024-04-20T10:31:37.130486+0000","last_peered":"2024-04-20T10:31:37.130486+0000","last_clean":"2024-04-20T10:31:37.130486+0000","last_became_active":"2024-04-20T10:31:35.618699+0000","last_became_peered":"2024-04-20T10:31:35.618699+0000","last_unstale":"2024-04-20T10:31:37.130486+0000","last_undegraded":"2024-04-20T10:31:37.130486+0000","last_fullsized":"2024-04-20T10:31:37.130486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:27.205634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031330+0000","last_change":"2024-04-20T10:31:35.619908+0000","last_active":"2024-04-20T10:31:37.031330+0000","last_peered":"2024-04-20T10:31:37.031330+0000","last_clean":"2024-04-20T10:31:37.031330+0000","last_became_active":"2024-04-20T10:31:35.619580+0000","last_became_peered":"2024-04-20T10:31:35.619580+0000","last_unstale":"2024-04-20T10:31:37.031330+0000","last_undegraded":"2024-04-20T10:31:37.031330+0000","last_fullsized":"2024-04-20T10:31:37.031330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:54:48.999925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129708+0000","last_change":"2024-04-20T10:31:35.623677+0000","last_active":"2024-04-20T10:31:37.129708+0000","last_peered":"2024-04-20T10:31:37.129708+0000","last_clean":"2024-04-20T10:31:37.129708+0000","last_became_active":"2024-04-20T10:31:35.621544+0000","last_became_peered":"2024-04-20T10:31:35.621544+0000","last_unstale":"2024-04-20T10:31:37.129708+0000","last_undegraded":"2024-04-20T10:31:37.129708+0000","last_fullsized":"2024-04-20T10:31:37.129708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:50:17.902954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.603679+0000","last_change":"2024-04-20T10:31:35.619594+0000","last_active":"2024-04-20T10:31:36.603679+0000","last_peered":"2024-04-20T10:31:36.603679+0000","last_clean":"2024-04-20T10:31:36.603679+0000","last_became_active":"2024-04-20T10:31:35.618831+0000","last_became_peered":"2024-04-20T10:31:35.618831+0000","last_unstale":"2024-04-20T10:31:36.603679+0000","last_undegraded":"2024-04-20T10:31:36.603679+0000","last_fullsized":"2024-04-20T10:31:36.603679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:07.625225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131522+0000","last_change":"2024-04-20T10:31:35.647476+0000","last_active":"2024-04-20T10:31:37.131522+0000","last_peered":"2024-04-20T10:31:37.131522+0000","last_clean":"2024-04-20T10:31:37.131522+0000","last_became_active":"2024-04-20T10:31:35.647009+0000","last_became_peered":"2024-04-20T10:31:35.647009+0000","last_unstale":"2024-04-20T10:31:37.131522+0000","last_undegraded":"2024-04-20T10:31:37.131522+0000","last_fullsized":"2024-04-20T10:31:37.131522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:11:37.526115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131729+0000","last_change":"2024-04-20T10:31:35.648641+0000","last_active":"2024-04-20T10:31:37.131729+0000","last_peered":"2024-04-20T10:31:37.131729+0000","last_clean":"2024-04-20T10:31:37.131729+0000","last_became_active":"2024-04-20T10:31:35.648150+0000","last_became_peered":"2024-04-20T10:31:35.648150+0000","last_unstale":"2024-04-20T10:31:37.131729+0000","last_undegraded":"2024-04-20T10:31:37.131729+0000","last_fullsized":"2024-04-20T10:31:37.131729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:17:44.253934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.599556+0000","last_change":"2024-04-20T10:31:35.643142+0000","last_active":"2024-04-20T10:31:36.599556+0000","last_peered":"2024-04-20T10:31:36.599556+0000","last_clean":"2024-04-20T10:31:36.599556+0000","last_became_active":"2024-04-20T10:31:35.642742+0000","last_became_peered":"2024-04-20T10:31:35.642742+0000","last_unstale":"2024-04-20T10:31:36.599556+0000","last_undegraded":"2024-04-20T10:31:36.599556+0000","last_fullsized":"2024-04-20T10:31:36.599556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:40.818808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131594+0000","last_change":"2024-04-20T10:31:35.650333+0000","last_active":"2024-04-20T10:31:37.131594+0000","last_peered":"2024-04-20T10:31:37.131594+0000","last_clean":"2024-04-20T10:31:37.131594+0000","last_became_active":"2024-04-20T10:31:35.649933+0000","last_became_peered":"2024-04-20T10:31:35.649933+0000","last_unstale":"2024-04-20T10:31:37.131594+0000","last_undegraded":"2024-04-20T10:31:37.131594+0000","last_fullsized":"2024-04-20T10:31:37.131594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:34.513990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131620+0000","last_change":"2024-04-20T10:31:35.637315+0000","last_active":"2024-04-20T10:31:37.131620+0000","last_peered":"2024-04-20T10:31:37.131620+0000","last_clean":"2024-04-20T10:31:37.131620+0000","last_became_active":"2024-04-20T10:31:35.635909+0000","last_became_peered":"2024-04-20T10:31:35.635909+0000","last_unstale":"2024-04-20T10:31:37.131620+0000","last_undegraded":"2024-04-20T10:31:37.131620+0000","last_fullsized":"2024-04-20T10:31:37.131620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:24:07.400032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131027+0000","last_change":"2024-04-20T10:31:35.638888+0000","last_active":"2024-04-20T10:31:37.131027+0000","last_peered":"2024-04-20T10:31:37.131027+0000","last_clean":"2024-04-20T10:31:37.131027+0000","last_became_active":"2024-04-20T10:31:35.638568+0000","last_became_peered":"2024-04-20T10:31:35.638568+0000","last_unstale":"2024-04-20T10:31:37.131027+0000","last_undegraded":"2024-04-20T10:31:37.131027+0000","last_fullsized":"2024-04-20T10:31:37.131027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:48.363874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130999+0000","last_change":"2024-04-20T10:31:34.597814+0000","last_active":"2024-04-20T10:31:37.130999+0000","last_peered":"2024-04-20T10:31:37.130999+0000","last_clean":"2024-04-20T10:31:37.130999+0000","last_became_active":"2024-04-20T10:31:34.597480+0000","last_became_peered":"2024-04-20T10:31:34.597480+0000","last_unstale":"2024-04-20T10:31:37.130999+0000","last_undegraded":"2024-04-20T10:31:37.130999+0000","last_fullsized":"2024-04-20T10:31:37.130999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:48:26.102787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.601222+0000","last_change":"2024-04-20T10:31:35.637189+0000","last_active":"2024-04-20T10:31:36.601222+0000","last_peered":"2024-04-20T10:31:36.601222+0000","last_clean":"2024-04-20T10:31:36.601222+0000","last_became_active":"2024-04-20T10:31:35.636324+0000","last_became_peered":"2024-04-20T10:31:35.636324+0000","last_unstale":"2024-04-20T10:31:36.601222+0000","last_undegraded":"2024-04-20T10:31:36.601222+0000","last_fullsized":"2024-04-20T10:31:36.601222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:46.692986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600931+0000","last_change":"2024-04-20T10:31:34.609942+0000","last_active":"2024-04-20T10:31:36.600931+0000","last_peered":"2024-04-20T10:31:36.600931+0000","last_clean":"2024-04-20T10:31:36.600931+0000","last_became_active":"2024-04-20T10:31:34.608994+0000","last_became_peered":"2024-04-20T10:31:34.608994+0000","last_unstale":"2024-04-20T10:31:36.600931+0000","last_undegraded":"2024-04-20T10:31:36.600931+0000","last_fullsized":"2024-04-20T10:31:36.600931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:26:53.488091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.132750+0000","last_change":"2024-04-20T10:31:35.647710+0000","last_active":"2024-04-20T10:31:37.132750+0000","last_peered":"2024-04-20T10:31:37.132750+0000","last_clean":"2024-04-20T10:31:37.132750+0000","last_became_active":"2024-04-20T10:31:35.646037+0000","last_became_peered":"2024-04-20T10:31:35.646037+0000","last_unstale":"2024-04-20T10:31:37.132750+0000","last_undegraded":"2024-04-20T10:31:37.132750+0000","last_fullsized":"2024-04-20T10:31:37.132750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:50:21.092892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130762+0000","last_change":"2024-04-20T10:31:34.593529+0000","last_active":"2024-04-20T10:31:37.130762+0000","last_peered":"2024-04-20T10:31:37.130762+0000","last_clean":"2024-04-20T10:31:37.130762+0000","last_became_active":"2024-04-20T10:31:34.592583+0000","last_became_peered":"2024-04-20T10:31:34.592583+0000","last_unstale":"2024-04-20T10:31:37.130762+0000","last_undegraded":"2024-04-20T10:31:37.130762+0000","last_fullsized":"2024-04-20T10:31:37.130762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:51:20.570499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.599687+0000","last_change":"2024-04-20T10:31:35.630739+0000","last_active":"2024-04-20T10:31:36.599687+0000","last_peered":"2024-04-20T10:31:36.599687+0000","last_clean":"2024-04-20T10:31:36.599687+0000","last_became_active":"2024-04-20T10:31:35.630424+0000","last_became_peered":"2024-04-20T10:31:35.630424+0000","last_unstale":"2024-04-20T10:31:36.599687+0000","last_undegraded":"2024-04-20T10:31:36.599687+0000","last_fullsized":"2024-04-20T10:31:36.599687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:48:17.485677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129985+0000","last_change":"2024-04-20T10:31:34.591652+0000","last_active":"2024-04-20T10:31:37.129985+0000","last_peered":"2024-04-20T10:31:37.129985+0000","last_clean":"2024-04-20T10:31:37.129985+0000","last_became_active":"2024-04-20T10:31:34.588945+0000","last_became_peered":"2024-04-20T10:31:34.588945+0000","last_unstale":"2024-04-20T10:31:37.129985+0000","last_undegraded":"2024-04-20T10:31:37.129985+0000","last_fullsized":"2024-04-20T10:31:37.129985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:15.137046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130324+0000","last_change":"2024-04-20T10:31:35.622462+0000","last_active":"2024-04-20T10:31:37.130324+0000","last_peered":"2024-04-20T10:31:37.130324+0000","last_clean":"2024-04-20T10:31:37.130324+0000","last_became_active":"2024-04-20T10:31:35.622125+0000","last_became_peered":"2024-04-20T10:31:35.622125+0000","last_unstale":"2024-04-20T10:31:37.130324+0000","last_undegraded":"2024-04-20T10:31:37.130324+0000","last_fullsized":"2024-04-20T10:31:37.130324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:27.894564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130040+0000","last_change":"2024-04-20T10:31:34.595202+0000","last_active":"2024-04-20T10:31:37.130040+0000","last_peered":"2024-04-20T10:31:37.130040+0000","last_clean":"2024-04-20T10:31:37.130040+0000","last_became_active":"2024-04-20T10:31:34.594784+0000","last_became_peered":"2024-04-20T10:31:34.594784+0000","last_unstale":"2024-04-20T10:31:37.130040+0000","last_undegraded":"2024-04-20T10:31:37.130040+0000","last_fullsized":"2024-04-20T10:31:37.130040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:04:09.559326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130279+0000","last_change":"2024-04-20T10:31:35.637040+0000","last_active":"2024-04-20T10:31:37.130279+0000","last_peered":"2024-04-20T10:31:37.130279+0000","last_clean":"2024-04-20T10:31:37.130279+0000","last_became_active":"2024-04-20T10:31:35.636682+0000","last_became_peered":"2024-04-20T10:31:35.636682+0000","last_unstale":"2024-04-20T10:31:37.130279+0000","last_undegraded":"2024-04-20T10:31:37.130279+0000","last_fullsized":"2024-04-20T10:31:37.130279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:40:11.569674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130086+0000","last_change":"2024-04-20T10:31:34.612038+0000","last_active":"2024-04-20T10:31:37.130086+0000","last_peered":"2024-04-20T10:31:37.130086+0000","last_clean":"2024-04-20T10:31:37.130086+0000","last_became_active":"2024-04-20T10:31:34.611671+0000","last_became_peered":"2024-04-20T10:31:34.611671+0000","last_unstale":"2024-04-20T10:31:37.130086+0000","last_undegraded":"2024-04-20T10:31:37.130086+0000","last_fullsized":"2024-04-20T10:31:37.130086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:47.862702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130251+0000","last_change":"2024-04-20T10:31:35.643930+0000","last_active":"2024-04-20T10:31:37.130251+0000","last_peered":"2024-04-20T10:31:37.130251+0000","last_clean":"2024-04-20T10:31:37.130251+0000","last_became_active":"2024-04-20T10:31:35.643470+0000","last_became_peered":"2024-04-20T10:31:35.643470+0000","last_unstale":"2024-04-20T10:31:37.130251+0000","last_undegraded":"2024-04-20T10:31:37.130251+0000","last_fullsized":"2024-04-20T10:31:37.130251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:55.570176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.602632+0000","last_change":"2024-04-20T10:31:34.613224+0000","last_active":"2024-04-20T10:31:36.602632+0000","last_peered":"2024-04-20T10:31:36.602632+0000","last_clean":"2024-04-20T10:31:36.602632+0000","last_became_active":"2024-04-20T10:31:34.612871+0000","last_became_peered":"2024-04-20T10:31:34.612871+0000","last_unstale":"2024-04-20T10:31:36.602632+0000","last_undegraded":"2024-04-20T10:31:36.602632+0000","last_fullsized":"2024-04-20T10:31:36.602632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:05:57.284870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600158+0000","last_change":"2024-04-20T10:31:35.643824+0000","last_active":"2024-04-20T10:31:36.600158+0000","last_peered":"2024-04-20T10:31:36.600158+0000","last_clean":"2024-04-20T10:31:36.600158+0000","last_became_active":"2024-04-20T10:31:35.643425+0000","last_became_peered":"2024-04-20T10:31:35.643425+0000","last_unstale":"2024-04-20T10:31:36.600158+0000","last_undegraded":"2024-04-20T10:31:36.600158+0000","last_fullsized":"2024-04-20T10:31:36.600158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:34.008374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129648+0000","last_change":"2024-04-20T10:31:34.593071+0000","last_active":"2024-04-20T10:31:37.129648+0000","last_peered":"2024-04-20T10:31:37.129648+0000","last_clean":"2024-04-20T10:31:37.129648+0000","last_became_active":"2024-04-20T10:31:34.592721+0000","last_became_peered":"2024-04-20T10:31:34.592721+0000","last_unstale":"2024-04-20T10:31:37.129648+0000","last_undegraded":"2024-04-20T10:31:37.129648+0000","last_fullsized":"2024-04-20T10:31:37.129648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:23:43.749586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130978+0000","last_change":"2024-04-20T10:31:35.647524+0000","last_active":"2024-04-20T10:31:37.130978+0000","last_peered":"2024-04-20T10:31:37.130978+0000","last_clean":"2024-04-20T10:31:37.130978+0000","last_became_active":"2024-04-20T10:31:35.647122+0000","last_became_peered":"2024-04-20T10:31:35.647122+0000","last_unstale":"2024-04-20T10:31:37.130978+0000","last_undegraded":"2024-04-20T10:31:37.130978+0000","last_fullsized":"2024-04-20T10:31:37.130978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130381+0000","last_change":"2024-04-20T10:31:34.604500+0000","last_active":"2024-04-20T10:31:37.130381+0000","last_peered":"2024-04-20T10:31:37.130381+0000","last_clean":"2024-04-20T10:31:37.130381+0000","last_became_active":"2024-04-20T10:31:34.604148+0000","last_became_peered":"2024-04-20T10:31:34.604148+0000","last_unstale":"2024-04-20T10:31:37.130381+0000","last_undegraded":"2024-04-20T10:31:37.130381+0000","last_fullsized":"2024-04-20T10:31:37.130381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:49:15.174414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600172+0000","last_change":"2024-04-20T10:31:35.636751+0000","last_active":"2024-04-20T10:31:36.600172+0000","last_peered":"2024-04-20T10:31:36.600172+0000","last_clean":"2024-04-20T10:31:36.600172+0000","last_became_active":"2024-04-20T10:31:35.636103+0000","last_became_peered":"2024-04-20T10:31:35.636103+0000","last_unstale":"2024-04-20T10:31:36.600172+0000","last_undegraded":"2024-04-20T10:31:36.600172+0000","last_fullsized":"2024-04-20T10:31:36.600172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:22:40.580512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031232+0000","last_change":"2024-04-20T10:31:34.595207+0000","last_active":"2024-04-20T10:31:37.031232+0000","last_peered":"2024-04-20T10:31:37.031232+0000","last_clean":"2024-04-20T10:31:37.031232+0000","last_became_active":"2024-04-20T10:31:34.594860+0000","last_became_peered":"2024-04-20T10:31:34.594860+0000","last_unstale":"2024-04-20T10:31:37.031232+0000","last_undegraded":"2024-04-20T10:31:37.031232+0000","last_fullsized":"2024-04-20T10:31:37.031232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:50.054454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.132706+0000","last_change":"2024-04-20T10:31:35.648086+0000","last_active":"2024-04-20T10:31:37.132706+0000","last_peered":"2024-04-20T10:31:37.132706+0000","last_clean":"2024-04-20T10:31:37.132706+0000","last_became_active":"2024-04-20T10:31:35.647275+0000","last_became_peered":"2024-04-20T10:31:35.647275+0000","last_unstale":"2024-04-20T10:31:37.132706+0000","last_undegraded":"2024-04-20T10:31:37.132706+0000","last_fullsized":"2024-04-20T10:31:37.132706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:27.669008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130336+0000","last_change":"2024-04-20T10:31:34.601674+0000","last_active":"2024-04-20T10:31:37.130336+0000","last_peered":"2024-04-20T10:31:37.130336+0000","last_clean":"2024-04-20T10:31:37.130336+0000","last_became_active":"2024-04-20T10:31:34.601255+0000","last_became_peered":"2024-04-20T10:31:34.601255+0000","last_unstale":"2024-04-20T10:31:37.130336+0000","last_undegraded":"2024-04-20T10:31:37.130336+0000","last_fullsized":"2024-04-20T10:31:37.130336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:57.821635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600043+0000","last_change":"2024-04-20T10:31:35.621888+0000","last_active":"2024-04-20T10:31:36.600043+0000","last_peered":"2024-04-20T10:31:36.600043+0000","last_clean":"2024-04-20T10:31:36.600043+0000","last_became_active":"2024-04-20T10:31:35.621503+0000","last_became_peered":"2024-04-20T10:31:35.621503+0000","last_unstale":"2024-04-20T10:31:36.600043+0000","last_undegraded":"2024-04-20T10:31:36.600043+0000","last_fullsized":"2024-04-20T10:31:36.600043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:15:21.118638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'13","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:40.823399+0000","last_change":"2024-04-20T10:31:34.590558+0000","last_active":"2024-04-20T10:31:40.823399+0000","last_peered":"2024-04-20T10:31:40.823399+0000","last_clean":"2024-04-20T10:31:40.823399+0000","last_became_active":"2024-04-20T10:31:34.590140+0000","last_became_peered":"2024-04-20T10:31:34.590140+0000","last_unstale":"2024-04-20T10:31:40.823399+0000","last_undegraded":"2024-04-20T10:31:40.823399+0000","last_fullsized":"2024-04-20T10:31:40.823399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:15:18.591748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":12,"num_write":13,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130485+0000","last_change":"2024-04-20T10:31:35.620780+0000","last_active":"2024-04-20T10:31:37.130485+0000","last_peered":"2024-04-20T10:31:37.130485+0000","last_clean":"2024-04-20T10:31:37.130485+0000","last_became_active":"2024-04-20T10:31:35.620429+0000","last_became_peered":"2024-04-20T10:31:35.620429+0000","last_unstale":"2024-04-20T10:31:37.130485+0000","last_undegraded":"2024-04-20T10:31:37.130485+0000","last_fullsized":"2024-04-20T10:31:37.130485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:47:45.587920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131388+0000","last_change":"2024-04-20T10:31:34.602135+0000","last_active":"2024-04-20T10:31:37.131388+0000","last_peered":"2024-04-20T10:31:37.131388+0000","last_clean":"2024-04-20T10:31:37.131388+0000","last_became_active":"2024-04-20T10:31:34.601460+0000","last_became_peered":"2024-04-20T10:31:34.601460+0000","last_unstale":"2024-04-20T10:31:37.131388+0000","last_undegraded":"2024-04-20T10:31:37.131388+0000","last_fullsized":"2024-04-20T10:31:37.131388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:00:00.832702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.603279+0000","last_change":"2024-04-20T10:31:35.635526+0000","last_active":"2024-04-20T10:31:36.603279+0000","last_peered":"2024-04-20T10:31:36.603279+0000","last_clean":"2024-04-20T10:31:36.603279+0000","last_became_active":"2024-04-20T10:31:35.635143+0000","last_became_peered":"2024-04-20T10:31:35.635143+0000","last_unstale":"2024-04-20T10:31:36.603279+0000","last_undegraded":"2024-04-20T10:31:36.603279+0000","last_fullsized":"2024-04-20T10:31:36.603279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:57:24.509318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129948+0000","last_change":"2024-04-20T10:31:34.604561+0000","last_active":"2024-04-20T10:31:37.129948+0000","last_peered":"2024-04-20T10:31:37.129948+0000","last_clean":"2024-04-20T10:31:37.129948+0000","last_became_active":"2024-04-20T10:31:34.604232+0000","last_became_peered":"2024-04-20T10:31:34.604232+0000","last_unstale":"2024-04-20T10:31:37.129948+0000","last_undegraded":"2024-04-20T10:31:37.129948+0000","last_fullsized":"2024-04-20T10:31:37.129948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:39.317473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031364+0000","last_change":"2024-04-20T10:31:35.632840+0000","last_active":"2024-04-20T10:31:37.031364+0000","last_peered":"2024-04-20T10:31:37.031364+0000","last_clean":"2024-04-20T10:31:37.031364+0000","last_became_active":"2024-04-20T10:31:35.632471+0000","last_became_peered":"2024-04-20T10:31:35.632471+0000","last_unstale":"2024-04-20T10:31:37.031364+0000","last_undegraded":"2024-04-20T10:31:37.031364+0000","last_fullsized":"2024-04-20T10:31:37.031364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:06.679967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129253+0000","last_change":"2024-04-20T10:31:34.591696+0000","last_active":"2024-04-20T10:31:37.129253+0000","last_peered":"2024-04-20T10:31:37.129253+0000","last_clean":"2024-04-20T10:31:37.129253+0000","last_became_active":"2024-04-20T10:31:34.591330+0000","last_became_peered":"2024-04-20T10:31:34.591330+0000","last_unstale":"2024-04-20T10:31:37.129253+0000","last_undegraded":"2024-04-20T10:31:37.129253+0000","last_fullsized":"2024-04-20T10:31:37.129253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:44:36.385963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.030932+0000","last_change":"2024-04-20T10:31:35.627201+0000","last_active":"2024-04-20T10:31:37.030932+0000","last_peered":"2024-04-20T10:31:37.030932+0000","last_clean":"2024-04-20T10:31:37.030932+0000","last_became_active":"2024-04-20T10:31:35.626880+0000","last_became_peered":"2024-04-20T10:31:35.626880+0000","last_unstale":"2024-04-20T10:31:37.030932+0000","last_undegraded":"2024-04-20T10:31:37.030932+0000","last_fullsized":"2024-04-20T10:31:37.030932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:44.562723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129588+0000","last_change":"2024-04-20T10:31:34.600683+0000","last_active":"2024-04-20T10:31:37.129588+0000","last_peered":"2024-04-20T10:31:37.129588+0000","last_clean":"2024-04-20T10:31:37.129588+0000","last_became_active":"2024-04-20T10:31:34.600305+0000","last_became_peered":"2024-04-20T10:31:34.600305+0000","last_unstale":"2024-04-20T10:31:37.129588+0000","last_undegraded":"2024-04-20T10:31:37.129588+0000","last_fullsized":"2024-04-20T10:31:37.129588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:05.441158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129612+0000","last_change":"2024-04-20T10:31:35.619835+0000","last_active":"2024-04-20T10:31:37.129612+0000","last_peered":"2024-04-20T10:31:37.129612+0000","last_clean":"2024-04-20T10:31:37.129612+0000","last_became_active":"2024-04-20T10:31:35.619554+0000","last_became_peered":"2024-04-20T10:31:35.619554+0000","last_unstale":"2024-04-20T10:31:37.129612+0000","last_undegraded":"2024-04-20T10:31:37.129612+0000","last_fullsized":"2024-04-20T10:31:37.129612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:33.161563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130198+0000","last_change":"2024-04-20T10:31:34.591128+0000","last_active":"2024-04-20T10:31:37.130198+0000","last_peered":"2024-04-20T10:31:37.130198+0000","last_clean":"2024-04-20T10:31:37.130198+0000","last_became_active":"2024-04-20T10:31:34.587772+0000","last_became_peered":"2024-04-20T10:31:34.587772+0000","last_unstale":"2024-04-20T10:31:37.130198+0000","last_undegraded":"2024-04-20T10:31:37.130198+0000","last_fullsized":"2024-04-20T10:31:37.130198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:12:42.541111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130659+0000","last_change":"2024-04-20T10:31:35.616743+0000","last_active":"2024-04-20T10:31:37.130659+0000","last_peered":"2024-04-20T10:31:37.130659+0000","last_clean":"2024-04-20T10:31:37.130659+0000","last_became_active":"2024-04-20T10:31:35.616419+0000","last_became_peered":"2024-04-20T10:31:35.616419+0000","last_unstale":"2024-04-20T10:31:37.130659+0000","last_undegraded":"2024-04-20T10:31:37.130659+0000","last_fullsized":"2024-04-20T10:31:37.130659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:37.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130529+0000","last_change":"2024-04-20T10:31:34.603734+0000","last_active":"2024-04-20T10:31:37.130529+0000","last_peered":"2024-04-20T10:31:37.130529+0000","last_clean":"2024-04-20T10:31:37.130529+0000","last_became_active":"2024-04-20T10:31:34.603388+0000","last_became_peered":"2024-04-20T10:31:34.603388+0000","last_unstale":"2024-04-20T10:31:37.130529+0000","last_undegraded":"2024-04-20T10:31:37.130529+0000","last_fullsized":"2024-04-20T10:31:37.130529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:58:57.199173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130603+0000","last_change":"2024-04-20T10:31:35.638996+0000","last_active":"2024-04-20T10:31:37.130603+0000","last_peered":"2024-04-20T10:31:37.130603+0000","last_clean":"2024-04-20T10:31:37.130603+0000","last_became_active":"2024-04-20T10:31:35.638571+0000","last_became_peered":"2024-04-20T10:31:35.638571+0000","last_unstale":"2024-04-20T10:31:37.130603+0000","last_undegraded":"2024-04-20T10:31:37.130603+0000","last_fullsized":"2024-04-20T10:31:37.130603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:59.541738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.030911+0000","last_change":"2024-04-20T10:31:34.612501+0000","last_active":"2024-04-20T10:31:37.030911+0000","last_peered":"2024-04-20T10:31:37.030911+0000","last_clean":"2024-04-20T10:31:37.030911+0000","last_became_active":"2024-04-20T10:31:34.612150+0000","last_became_peered":"2024-04-20T10:31:34.612150+0000","last_unstale":"2024-04-20T10:31:37.030911+0000","last_undegraded":"2024-04-20T10:31:37.030911+0000","last_fullsized":"2024-04-20T10:31:37.030911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:28.172851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130490+0000","last_change":"2024-04-20T10:31:35.643814+0000","last_active":"2024-04-20T10:31:37.130490+0000","last_peered":"2024-04-20T10:31:37.130490+0000","last_clean":"2024-04-20T10:31:37.130490+0000","last_became_active":"2024-04-20T10:31:35.643458+0000","last_became_peered":"2024-04-20T10:31:35.643458+0000","last_unstale":"2024-04-20T10:31:37.130490+0000","last_undegraded":"2024-04-20T10:31:37.130490+0000","last_fullsized":"2024-04-20T10:31:37.130490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:45.149598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129776+0000","last_change":"2024-04-20T10:31:35.617775+0000","last_active":"2024-04-20T10:31:37.129776+0000","last_peered":"2024-04-20T10:31:37.129776+0000","last_clean":"2024-04-20T10:31:37.129776+0000","last_became_active":"2024-04-20T10:31:35.617465+0000","last_became_peered":"2024-04-20T10:31:35.617465+0000","last_unstale":"2024-04-20T10:31:37.129776+0000","last_undegraded":"2024-04-20T10:31:37.129776+0000","last_fullsized":"2024-04-20T10:31:37.129776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:37.830953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.602324+0000","last_change":"2024-04-20T10:31:34.615044+0000","last_active":"2024-04-20T10:31:36.602324+0000","last_peered":"2024-04-20T10:31:36.602324+0000","last_clean":"2024-04-20T10:31:36.602324+0000","last_became_active":"2024-04-20T10:31:34.612875+0000","last_became_peered":"2024-04-20T10:31:34.612875+0000","last_unstale":"2024-04-20T10:31:36.602324+0000","last_undegraded":"2024-04-20T10:31:36.602324+0000","last_fullsized":"2024-04-20T10:31:36.602324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:59.867501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.601075+0000","last_change":"2024-04-20T10:31:35.630909+0000","last_active":"2024-04-20T10:31:36.601075+0000","last_peered":"2024-04-20T10:31:36.601075+0000","last_clean":"2024-04-20T10:31:36.601075+0000","last_became_active":"2024-04-20T10:31:35.630455+0000","last_became_peered":"2024-04-20T10:31:35.630455+0000","last_unstale":"2024-04-20T10:31:36.601075+0000","last_undegraded":"2024-04-20T10:31:36.601075+0000","last_fullsized":"2024-04-20T10:31:36.601075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:02.522485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129773+0000","last_change":"2024-04-20T10:31:34.602466+0000","last_active":"2024-04-20T10:31:37.129773+0000","last_peered":"2024-04-20T10:31:37.129773+0000","last_clean":"2024-04-20T10:31:37.129773+0000","last_became_active":"2024-04-20T10:31:34.602089+0000","last_became_peered":"2024-04-20T10:31:34.602089+0000","last_unstale":"2024-04-20T10:31:37.129773+0000","last_undegraded":"2024-04-20T10:31:37.129773+0000","last_fullsized":"2024-04-20T10:31:37.129773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:19:09.849840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.030759+0000","last_change":"2024-04-20T10:30:14.724347+0000","last_active":"2024-04-20T10:31:37.030759+0000","last_peered":"2024-04-20T10:31:37.030759+0000","last_clean":"2024-04-20T10:31:37.030759+0000","last_became_active":"2024-04-20T10:30:14.723877+0000","last_became_peered":"2024-04-20T10:30:14.723877+0000","last_unstale":"2024-04-20T10:31:37.030759+0000","last_undegraded":"2024-04-20T10:31:37.030759+0000","last_fullsized":"2024-04-20T10:31:37.030759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_clean_scrub_stamp":"2024-04-20T10:26:16.803986+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:30.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.601697+0000","last_change":"2024-04-20T10:31:35.626505+0000","last_active":"2024-04-20T10:31:36.601697+0000","last_peered":"2024-04-20T10:31:36.601697+0000","last_clean":"2024-04-20T10:31:36.601697+0000","last_became_active":"2024-04-20T10:31:35.626141+0000","last_became_peered":"2024-04-20T10:31:35.626141+0000","last_unstale":"2024-04-20T10:31:36.601697+0000","last_undegraded":"2024-04-20T10:31:36.601697+0000","last_fullsized":"2024-04-20T10:31:36.601697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:29.827225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031139+0000","last_change":"2024-04-20T10:31:34.596936+0000","last_active":"2024-04-20T10:31:37.031139+0000","last_peered":"2024-04-20T10:31:37.031139+0000","last_clean":"2024-04-20T10:31:37.031139+0000","last_became_active":"2024-04-20T10:31:34.596692+0000","last_became_peered":"2024-04-20T10:31:34.596692+0000","last_unstale":"2024-04-20T10:31:37.031139+0000","last_undegraded":"2024-04-20T10:31:37.031139+0000","last_fullsized":"2024-04-20T10:31:37.031139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:02:39.819096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130580+0000","last_change":"2024-04-20T10:31:34.601407+0000","last_active":"2024-04-20T10:31:37.130580+0000","last_peered":"2024-04-20T10:31:37.130580+0000","last_clean":"2024-04-20T10:31:37.130580+0000","last_became_active":"2024-04-20T10:31:34.601007+0000","last_became_peered":"2024-04-20T10:31:34.601007+0000","last_unstale":"2024-04-20T10:31:37.130580+0000","last_undegraded":"2024-04-20T10:31:37.130580+0000","last_fullsized":"2024-04-20T10:31:37.130580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:05:03.037841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.604823+0000","last_change":"2024-04-20T10:31:35.641046+0000","last_active":"2024-04-20T10:31:36.604823+0000","last_peered":"2024-04-20T10:31:36.604823+0000","last_clean":"2024-04-20T10:31:36.604823+0000","last_became_active":"2024-04-20T10:31:35.640666+0000","last_became_peered":"2024-04-20T10:31:35.640666+0000","last_unstale":"2024-04-20T10:31:36.604823+0000","last_undegraded":"2024-04-20T10:31:36.604823+0000","last_fullsized":"2024-04-20T10:31:36.604823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:42.496037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130770+0000","last_change":"2024-04-20T10:31:34.604643+0000","last_active":"2024-04-20T10:31:37.130770+0000","last_peered":"2024-04-20T10:31:37.130770+0000","last_clean":"2024-04-20T10:31:37.130770+0000","last_became_active":"2024-04-20T10:31:34.604210+0000","last_became_peered":"2024-04-20T10:31:34.604210+0000","last_unstale":"2024-04-20T10:31:37.130770+0000","last_undegraded":"2024-04-20T10:31:37.130770+0000","last_fullsized":"2024-04-20T10:31:37.130770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:24:09.748012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600855+0000","last_change":"2024-04-20T10:31:35.641981+0000","last_active":"2024-04-20T10:31:36.600855+0000","last_peered":"2024-04-20T10:31:36.600855+0000","last_clean":"2024-04-20T10:31:36.600855+0000","last_became_active":"2024-04-20T10:31:35.641607+0000","last_became_peered":"2024-04-20T10:31:35.641607+0000","last_unstale":"2024-04-20T10:31:36.600855+0000","last_undegraded":"2024-04-20T10:31:36.600855+0000","last_fullsized":"2024-04-20T10:31:36.600855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:48.520813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129572+0000","last_change":"2024-04-20T10:31:34.611218+0000","last_active":"2024-04-20T10:31:37.129572+0000","last_peered":"2024-04-20T10:31:37.129572+0000","last_clean":"2024-04-20T10:31:37.129572+0000","last_became_active":"2024-04-20T10:31:34.610950+0000","last_became_peered":"2024-04-20T10:31:34.610950+0000","last_unstale":"2024-04-20T10:31:37.129572+0000","last_undegraded":"2024-04-20T10:31:37.129572+0000","last_fullsized":"2024-04-20T10:31:37.129572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:14:11.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.132241+0000","last_change":"2024-04-20T10:31:35.647531+0000","last_active":"2024-04-20T10:31:37.132241+0000","last_peered":"2024-04-20T10:31:37.132241+0000","last_clean":"2024-04-20T10:31:37.132241+0000","last_became_active":"2024-04-20T10:31:35.646389+0000","last_became_peered":"2024-04-20T10:31:35.646389+0000","last_unstale":"2024-04-20T10:31:37.132241+0000","last_undegraded":"2024-04-20T10:31:37.132241+0000","last_fullsized":"2024-04-20T10:31:37.132241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:50.141385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.599584+0000","last_change":"2024-04-20T10:31:34.595565+0000","last_active":"2024-04-20T10:31:36.599584+0000","last_peered":"2024-04-20T10:31:36.599584+0000","last_clean":"2024-04-20T10:31:36.599584+0000","last_became_active":"2024-04-20T10:31:34.595271+0000","last_became_peered":"2024-04-20T10:31:34.595271+0000","last_unstale":"2024-04-20T10:31:36.599584+0000","last_undegraded":"2024-04-20T10:31:36.599584+0000","last_fullsized":"2024-04-20T10:31:36.599584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:55.024192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129572+0000","last_change":"2024-04-20T10:31:35.616751+0000","last_active":"2024-04-20T10:31:37.129572+0000","last_peered":"2024-04-20T10:31:37.129572+0000","last_clean":"2024-04-20T10:31:37.129572+0000","last_became_active":"2024-04-20T10:31:35.616432+0000","last_became_peered":"2024-04-20T10:31:35.616432+0000","last_unstale":"2024-04-20T10:31:37.129572+0000","last_undegraded":"2024-04-20T10:31:37.129572+0000","last_fullsized":"2024-04-20T10:31:37.129572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:25.052140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600621+0000","last_change":"2024-04-20T10:31:34.609139+0000","last_active":"2024-04-20T10:31:36.600621+0000","last_peered":"2024-04-20T10:31:36.600621+0000","last_clean":"2024-04-20T10:31:36.600621+0000","last_became_active":"2024-04-20T10:31:34.608818+0000","last_became_peered":"2024-04-20T10:31:34.608818+0000","last_unstale":"2024-04-20T10:31:36.600621+0000","last_undegraded":"2024-04-20T10:31:36.600621+0000","last_fullsized":"2024-04-20T10:31:36.600621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:42:39.581059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130186+0000","last_change":"2024-04-20T10:31:35.643791+0000","last_active":"2024-04-20T10:31:37.130186+0000","last_peered":"2024-04-20T10:31:37.130186+0000","last_clean":"2024-04-20T10:31:37.130186+0000","last_became_active":"2024-04-20T10:31:35.643390+0000","last_became_peered":"2024-04-20T10:31:35.643390+0000","last_unstale":"2024-04-20T10:31:37.130186+0000","last_undegraded":"2024-04-20T10:31:37.130186+0000","last_fullsized":"2024-04-20T10:31:37.130186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:10.149979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130993+0000","last_change":"2024-04-20T10:31:34.606178+0000","last_active":"2024-04-20T10:31:37.130993+0000","last_peered":"2024-04-20T10:31:37.130993+0000","last_clean":"2024-04-20T10:31:37.130993+0000","last_became_active":"2024-04-20T10:31:34.605817+0000","last_became_peered":"2024-04-20T10:31:34.605817+0000","last_unstale":"2024-04-20T10:31:37.130993+0000","last_undegraded":"2024-04-20T10:31:37.130993+0000","last_fullsized":"2024-04-20T10:31:37.130993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:15.929905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.601575+0000","last_change":"2024-04-20T10:31:35.649487+0000","last_active":"2024-04-20T10:31:36.601575+0000","last_peered":"2024-04-20T10:31:36.601575+0000","last_clean":"2024-04-20T10:31:36.601575+0000","last_became_active":"2024-04-20T10:31:35.648884+0000","last_became_peered":"2024-04-20T10:31:35.648884+0000","last_unstale":"2024-04-20T10:31:36.601575+0000","last_undegraded":"2024-04-20T10:31:36.601575+0000","last_fullsized":"2024-04-20T10:31:36.601575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:07.389100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129827+0000","last_change":"2024-04-20T10:31:34.595718+0000","last_active":"2024-04-20T10:31:37.129827+0000","last_peered":"2024-04-20T10:31:37.129827+0000","last_clean":"2024-04-20T10:31:37.129827+0000","last_became_active":"2024-04-20T10:31:34.595405+0000","last_became_peered":"2024-04-20T10:31:34.595405+0000","last_unstale":"2024-04-20T10:31:37.129827+0000","last_undegraded":"2024-04-20T10:31:37.129827+0000","last_fullsized":"2024-04-20T10:31:37.129827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:16:02.772345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031172+0000","last_change":"2024-04-20T10:31:35.630256+0000","last_active":"2024-04-20T10:31:37.031172+0000","last_peered":"2024-04-20T10:31:37.031172+0000","last_clean":"2024-04-20T10:31:37.031172+0000","last_became_active":"2024-04-20T10:31:35.629879+0000","last_became_peered":"2024-04-20T10:31:35.629879+0000","last_unstale":"2024-04-20T10:31:37.031172+0000","last_undegraded":"2024-04-20T10:31:37.031172+0000","last_fullsized":"2024-04-20T10:31:37.031172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:51.601923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131371+0000","last_change":"2024-04-20T10:31:34.595149+0000","last_active":"2024-04-20T10:31:37.131371+0000","last_peered":"2024-04-20T10:31:37.131371+0000","last_clean":"2024-04-20T10:31:37.131371+0000","last_became_active":"2024-04-20T10:31:34.594808+0000","last_became_peered":"2024-04-20T10:31:34.594808+0000","last_unstale":"2024-04-20T10:31:37.131371+0000","last_undegraded":"2024-04-20T10:31:37.131371+0000","last_fullsized":"2024-04-20T10:31:37.131371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:51.707710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031390+0000","last_change":"2024-04-20T10:31:35.620612+0000","last_active":"2024-04-20T10:31:37.031390+0000","last_peered":"2024-04-20T10:31:37.031390+0000","last_clean":"2024-04-20T10:31:37.031390+0000","last_became_active":"2024-04-20T10:31:35.620312+0000","last_became_peered":"2024-04-20T10:31:35.620312+0000","last_unstale":"2024-04-20T10:31:37.031390+0000","last_undegraded":"2024-04-20T10:31:37.031390+0000","last_fullsized":"2024-04-20T10:31:37.031390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:37:50.756950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130209+0000","last_change":"2024-04-20T10:31:34.604613+0000","last_active":"2024-04-20T10:31:37.130209+0000","last_peered":"2024-04-20T10:31:37.130209+0000","last_clean":"2024-04-20T10:31:37.130209+0000","last_became_active":"2024-04-20T10:31:34.604246+0000","last_became_peered":"2024-04-20T10:31:34.604246+0000","last_unstale":"2024-04-20T10:31:37.130209+0000","last_undegraded":"2024-04-20T10:31:37.130209+0000","last_fullsized":"2024-04-20T10:31:37.130209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:26.521769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031114+0000","last_change":"2024-04-20T10:31:35.639396+0000","last_active":"2024-04-20T10:31:37.031114+0000","last_peered":"2024-04-20T10:31:37.031114+0000","last_clean":"2024-04-20T10:31:37.031114+0000","last_became_active":"2024-04-20T10:31:35.638979+0000","last_became_peered":"2024-04-20T10:31:35.638979+0000","last_unstale":"2024-04-20T10:31:37.031114+0000","last_undegraded":"2024-04-20T10:31:37.031114+0000","last_fullsized":"2024-04-20T10:31:37.031114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:47.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.604649+0000","last_change":"2024-04-20T10:31:34.607420+0000","last_active":"2024-04-20T10:31:36.604649+0000","last_peered":"2024-04-20T10:31:36.604649+0000","last_clean":"2024-04-20T10:31:36.604649+0000","last_became_active":"2024-04-20T10:31:34.602587+0000","last_became_peered":"2024-04-20T10:31:34.602587+0000","last_unstale":"2024-04-20T10:31:36.604649+0000","last_undegraded":"2024-04-20T10:31:36.604649+0000","last_fullsized":"2024-04-20T10:31:36.604649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:56.447944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031381+0000","last_change":"2024-04-20T10:31:35.624903+0000","last_active":"2024-04-20T10:31:37.031381+0000","last_peered":"2024-04-20T10:31:37.031381+0000","last_clean":"2024-04-20T10:31:37.031381+0000","last_became_active":"2024-04-20T10:31:35.619479+0000","last_became_peered":"2024-04-20T10:31:35.619479+0000","last_unstale":"2024-04-20T10:31:37.031381+0000","last_undegraded":"2024-04-20T10:31:37.031381+0000","last_fullsized":"2024-04-20T10:31:37.031381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:25:40.601983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130405+0000","last_change":"2024-04-20T10:31:35.620351+0000","last_active":"2024-04-20T10:31:37.130405+0000","last_peered":"2024-04-20T10:31:37.130405+0000","last_clean":"2024-04-20T10:31:37.130405+0000","last_became_active":"2024-04-20T10:31:35.620028+0000","last_became_peered":"2024-04-20T10:31:35.620028+0000","last_unstale":"2024-04-20T10:31:37.130405+0000","last_undegraded":"2024-04-20T10:31:37.130405+0000","last_fullsized":"2024-04-20T10:31:37.130405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:51:53.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131233+0000","last_change":"2024-04-20T10:31:35.615917+0000","last_active":"2024-04-20T10:31:37.131233+0000","last_peered":"2024-04-20T10:31:37.131233+0000","last_clean":"2024-04-20T10:31:37.131233+0000","last_became_active":"2024-04-20T10:31:35.615582+0000","last_became_peered":"2024-04-20T10:31:35.615582+0000","last_unstale":"2024-04-20T10:31:37.131233+0000","last_undegraded":"2024-04-20T10:31:37.131233+0000","last_fullsized":"2024-04-20T10:31:37.131233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:54:59.754819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131232+0000","last_change":"2024-04-20T10:31:35.650006+0000","last_active":"2024-04-20T10:31:37.131232+0000","last_peered":"2024-04-20T10:31:37.131232+0000","last_clean":"2024-04-20T10:31:37.131232+0000","last_became_active":"2024-04-20T10:31:35.649683+0000","last_became_peered":"2024-04-20T10:31:35.649683+0000","last_unstale":"2024-04-20T10:31:37.131232+0000","last_undegraded":"2024-04-20T10:31:37.131232+0000","last_fullsized":"2024-04-20T10:31:37.131232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:10:01.484809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.132307+0000","last_change":"2024-04-20T10:31:35.646591+0000","last_active":"2024-04-20T10:31:37.132307+0000","last_peered":"2024-04-20T10:31:37.132307+0000","last_clean":"2024-04-20T10:31:37.132307+0000","last_became_active":"2024-04-20T10:31:35.646188+0000","last_became_peered":"2024-04-20T10:31:35.646188+0000","last_unstale":"2024-04-20T10:31:37.132307+0000","last_undegraded":"2024-04-20T10:31:37.132307+0000","last_fullsized":"2024-04-20T10:31:37.132307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:23.412022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.604578+0000","last_change":"2024-04-20T10:31:35.650879+0000","last_active":"2024-04-20T10:31:36.604578+0000","last_peered":"2024-04-20T10:31:36.604578+0000","last_clean":"2024-04-20T10:31:36.604578+0000","last_became_active":"2024-04-20T10:31:35.650486+0000","last_became_peered":"2024-04-20T10:31:35.650486+0000","last_unstale":"2024-04-20T10:31:36.604578+0000","last_undegraded":"2024-04-20T10:31:36.604578+0000","last_fullsized":"2024-04-20T10:31:36.604578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:51.893446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131065+0000","last_change":"2024-04-20T10:31:35.619883+0000","last_active":"2024-04-20T10:31:37.131065+0000","last_peered":"2024-04-20T10:31:37.131065+0000","last_clean":"2024-04-20T10:31:37.131065+0000","last_became_active":"2024-04-20T10:31:35.619628+0000","last_became_peered":"2024-04-20T10:31:35.619628+0000","last_unstale":"2024-04-20T10:31:37.131065+0000","last_undegraded":"2024-04-20T10:31:37.131065+0000","last_fullsized":"2024-04-20T10:31:37.131065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:45.018417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131531+0000","last_change":"2024-04-20T10:31:35.620773+0000","last_active":"2024-04-20T10:31:37.131531+0000","last_peered":"2024-04-20T10:31:37.131531+0000","last_clean":"2024-04-20T10:31:37.131531+0000","last_became_active":"2024-04-20T10:31:35.620467+0000","last_became_peered":"2024-04-20T10:31:35.620467+0000","last_unstale":"2024-04-20T10:31:37.131531+0000","last_undegraded":"2024-04-20T10:31:37.131531+0000","last_fullsized":"2024-04-20T10:31:37.131531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:31:48.009168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.601683+0000","last_change":"2024-04-20T10:31:35.644618+0000","last_active":"2024-04-20T10:31:36.601683+0000","last_peered":"2024-04-20T10:31:36.601683+0000","last_clean":"2024-04-20T10:31:36.601683+0000","last_became_active":"2024-04-20T10:31:35.644068+0000","last_became_peered":"2024-04-20T10:31:35.644068+0000","last_unstale":"2024-04-20T10:31:36.601683+0000","last_undegraded":"2024-04-20T10:31:36.601683+0000","last_fullsized":"2024-04-20T10:31:36.601683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:51.987584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129630+0000","last_change":"2024-04-20T10:31:35.637588+0000","last_active":"2024-04-20T10:31:37.129630+0000","last_peered":"2024-04-20T10:31:37.129630+0000","last_clean":"2024-04-20T10:31:37.129630+0000","last_became_active":"2024-04-20T10:31:35.637319+0000","last_became_peered":"2024-04-20T10:31:35.637319+0000","last_unstale":"2024-04-20T10:31:37.129630+0000","last_undegraded":"2024-04-20T10:31:37.129630+0000","last_fullsized":"2024-04-20T10:31:37.129630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:54.070894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600498+0000","last_change":"2024-04-20T10:31:35.629551+0000","last_active":"2024-04-20T10:31:36.600498+0000","last_peered":"2024-04-20T10:31:36.600498+0000","last_clean":"2024-04-20T10:31:36.600498+0000","last_became_active":"2024-04-20T10:31:35.628710+0000","last_became_peered":"2024-04-20T10:31:35.628710+0000","last_unstale":"2024-04-20T10:31:36.600498+0000","last_undegraded":"2024-04-20T10:31:36.600498+0000","last_fullsized":"2024-04-20T10:31:36.600498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:31.883788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.599716+0000","last_change":"2024-04-20T10:31:35.629978+0000","last_active":"2024-04-20T10:31:36.599716+0000","last_peered":"2024-04-20T10:31:36.599716+0000","last_clean":"2024-04-20T10:31:36.599716+0000","last_became_active":"2024-04-20T10:31:35.629596+0000","last_became_peered":"2024-04-20T10:31:35.629596+0000","last_unstale":"2024-04-20T10:31:36.599716+0000","last_undegraded":"2024-04-20T10:31:36.599716+0000","last_fullsized":"2024-04-20T10:31:36.599716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:02.523099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.603850+0000","last_change":"2024-04-20T10:31:35.652481+0000","last_active":"2024-04-20T10:31:36.603850+0000","last_peered":"2024-04-20T10:31:36.603850+0000","last_clean":"2024-04-20T10:31:36.603850+0000","last_became_active":"2024-04-20T10:31:35.651837+0000","last_became_peered":"2024-04-20T10:31:35.651837+0000","last_unstale":"2024-04-20T10:31:36.603850+0000","last_undegraded":"2024-04-20T10:31:36.603850+0000","last_fullsized":"2024-04-20T10:31:36.603850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:53.206954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130588+0000","last_change":"2024-04-20T10:31:35.622228+0000","last_active":"2024-04-20T10:31:37.130588+0000","last_peered":"2024-04-20T10:31:37.130588+0000","last_clean":"2024-04-20T10:31:37.130588+0000","last_became_active":"2024-04-20T10:31:35.621910+0000","last_became_peered":"2024-04-20T10:31:35.621910+0000","last_unstale":"2024-04-20T10:31:37.130588+0000","last_undegraded":"2024-04-20T10:31:37.130588+0000","last_fullsized":"2024-04-20T10:31:37.130588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:47:34.319839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281463,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":21693,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":15,"num_write":52,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":65079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":1037295,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":439125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":966656,"data_stored":1029471,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":459416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":440167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1029071,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34156,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34156,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34156,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":20929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":21089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:31:43.835 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:31:44.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.088+0000 7fd723200640 1 -- 172.21.15.42:0/3979246601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c105d90 msgr2=0x7fd71c10c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:44.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.088+0000 7fd723200640 1 --2- 172.21.15.42:0/3979246601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c105d90 0x7fd71c10c620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd714009be0 tx=0x7fd71402bef0 comp rx=0 tx=0).stop 2024-04-20T10:31:44.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.088+0000 7fd723200640 1 -- 172.21.15.42:0/3979246601 shutdown_connections 2024-04-20T10:31:44.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.088+0000 7fd723200640 1 --2- 172.21.15.42:0/3979246601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c105d90 0x7fd71c10c620 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:44.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.088+0000 7fd723200640 1 --2- 172.21.15.42:0/3979246601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd71c1053d0 0x7fd71c105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:44.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.088+0000 7fd723200640 1 --2- 172.21.15.42:0/3979246601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd71c1041e0 0x7fd71c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:44.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.088+0000 7fd723200640 1 -- 172.21.15.42:0/3979246601 >> 172.21.15.42:0/3979246601 conn(0x7fd71c0ff750 msgr2=0x7fd71c101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:44.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.088+0000 7fd723200640 1 -- 172.21.15.42:0/3979246601 shutdown_connections 2024-04-20T10:31:44.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.088+0000 7fd723200640 1 -- 172.21.15.42:0/3979246601 wait complete. 2024-04-20T10:31:44.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.089+0000 7fd723200640 1 Processor -- start 2024-04-20T10:31:44.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.089+0000 7fd723200640 1 -- start start 2024-04-20T10:31:44.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.090+0000 7fd723200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c1041e0 0x7fd71c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:44.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.090+0000 7fd723200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd71c1053d0 0x7fd71c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:44.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.090+0000 7fd723200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd71c105d90 0x7fd71c1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:44.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.090+0000 7fd723200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c10fec0 con 0x7fd71c1053d0 2024-04-20T10:31:44.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.090+0000 7fd723200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c10fd40 con 0x7fd71c1041e0 2024-04-20T10:31:44.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.090+0000 7fd723200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c110040 con 0x7fd71c105d90 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.090+0000 7fd721200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd71c105d90 0x7fd71c1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.090+0000 7fd721200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd71c105d90 0x7fd71c1a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:36134/0 (socket says 172.21.15.42:36134) 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.090+0000 7fd721200640 1 -- 172.21.15.42:0/3667936427 learned_addr learned my addr 172.21.15.42:0/3667936427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.090+0000 7fd71be00640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c1041e0 0x7fd71c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.090+0000 7fd71b400640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd71c1053d0 0x7fd71c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.091+0000 7fd721200640 1 -- 172.21.15.42:0/3667936427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c1041e0 msgr2=0x7fd71c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.091+0000 7fd721200640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c1041e0 0x7fd71c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.091+0000 7fd721200640 1 -- 172.21.15.42:0/3667936427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd71c1053d0 msgr2=0x7fd71c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.091+0000 7fd721200640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd71c1053d0 0x7fd71c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.091+0000 7fd721200640 1 -- 172.21.15.42:0/3667936427 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c1a20e0 con 0x7fd71c105d90 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.091+0000 7fd71be00640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c1041e0 0x7fd71c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.091+0000 7fd721200640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd71c105d90 0x7fd71c1a1a70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd714036040 tx=0x7fd714033580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.091+0000 7fd718c00640 1 -- 172.21.15.42:0/3667936427 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714043020 con 0x7fd71c105d90 2024-04-20T10:31:44.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.091+0000 7fd723200640 1 -- 172.21.15.42:0/3667936427 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71c1a2370 con 0x7fd71c105d90 2024-04-20T10:31:44.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.091+0000 7fd718c00640 1 -- 172.21.15.42:0/3667936427 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd714003e50 con 0x7fd71c105d90 2024-04-20T10:31:44.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.091+0000 7fd718c00640 1 -- 172.21.15.42:0/3667936427 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714034690 con 0x7fd71c105d90 2024-04-20T10:31:44.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.092+0000 7fd723200640 1 -- 172.21.15.42:0/3667936427 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd71c1a28c0 con 0x7fd71c105d90 2024-04-20T10:31:44.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.093+0000 7fd723200640 1 -- 172.21.15.42:0/3667936427 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd71c10b4a0 con 0x7fd71c105d90 2024-04-20T10:31:44.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.095+0000 7fd718c00640 1 -- 172.21.15.42:0/3667936427 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd714046020 con 0x7fd71c105d90 2024-04-20T10:31:44.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.096+0000 7fd718c00640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd7000a3c40 0x7fd7000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:44.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.096+0000 7fd718c00640 1 -- 172.21.15.42:0/3667936427 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd7140f2ac0 con 0x7fd71c105d90 2024-04-20T10:31:44.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.096+0000 7fd71be00640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd7000a3c40 0x7fd7000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:44.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.097+0000 7fd71be00640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd7000a3c40 0x7fd7000a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd71c105230 tx=0x7fd70c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:44.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.100+0000 7fd718c00640 1 -- 172.21.15.42:0/3667936427 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd71404d020 con 0x7fd71c105d90 2024-04-20T10:31:44.102 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:43 smithi042 ceph-mon[17078]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:44.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:43 smithi042 ceph-mon[17078]: pgmap v321: 97 pgs: 97 active+clean; 598 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-20T10:31:44.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:43 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3352165529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:44.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:43 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1340706152' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:43 smithi162 ceph-mon[21854]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:43 smithi162 ceph-mon[21854]: pgmap v321: 97 pgs: 97 active+clean; 598 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-20T10:31:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:43 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3352165529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:43 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1340706152' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:44.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:43 smithi046 ceph-mon[21574]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:31:44.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:43 smithi046 ceph-mon[21574]: pgmap v321: 97 pgs: 97 active+clean; 598 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-20T10:31:44.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:43 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3352165529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:31:44.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:43 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1340706152' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:44.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.303+0000 7fd723200640 1 -- 172.21.15.42:0/3667936427 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd71c0623a0 con 0x7fd7000a3c40 2024-04-20T10:31:44.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.317+0000 7fd718c00640 1 -- 172.21.15.42:0/3667936427 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258860 (secure 0 0 0) 0x7fd71c0623a0 con 0x7fd7000a3c40 2024-04-20T10:31:44.321 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:44.323 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-04-20T10:31:44.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.323+0000 7fd723200640 1 -- 172.21.15.42:0/3667936427 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd7000a3c40 msgr2=0x7fd7000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:44.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.323+0000 7fd723200640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd7000a3c40 0x7fd7000a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd71c105230 tx=0x7fd70c009260 comp rx=0 tx=0).stop 2024-04-20T10:31:44.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.323+0000 7fd723200640 1 -- 172.21.15.42:0/3667936427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd71c105d90 msgr2=0x7fd71c1a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:44.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.323+0000 7fd723200640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd71c105d90 0x7fd71c1a1a70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd714036040 tx=0x7fd714033580 comp rx=0 tx=0).stop 2024-04-20T10:31:44.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.323+0000 7fd723200640 1 -- 172.21.15.42:0/3667936427 shutdown_connections 2024-04-20T10:31:44.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.323+0000 7fd723200640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd7000a3c40 0x7fd7000a6100 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:44.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.323+0000 7fd723200640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd71c105d90 0x7fd71c1a1a70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:44.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.323+0000 7fd723200640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd71c1053d0 0x7fd71c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:44.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.323+0000 7fd723200640 1 --2- 172.21.15.42:0/3667936427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c1041e0 0x7fd71c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:44.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.323+0000 7fd723200640 1 -- 172.21.15.42:0/3667936427 >> 172.21.15.42:0/3667936427 conn(0x7fd71c0ff750 msgr2=0x7fd71c108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:44.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.323+0000 7fd723200640 1 -- 172.21.15.42:0/3667936427 shutdown_connections 2024-04-20T10:31:44.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:44.324+0000 7fd723200640 1 -- 172.21.15.42:0/3667936427 wait complete. 2024-04-20T10:31:44.345 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-20T10:31:42.511684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":612178,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":148,"num_read_kb":134,"num_write":234,"num_write_kb":2974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":184,"ondisk_log_size":184,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364352,"kb_used_data":10964,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577472,"statfs":{"total":1151940427776,"available":1151567331328,"internally_reserved":0,"allocated":11227136,"data_stored":7066469,"data_compressed":107397,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21810,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":12,"num_read_kb":15,"num_write":53,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.935999"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131353+0000","last_change":"2024-04-20T10:31:35.649585+0000","last_active":"2024-04-20T10:31:37.131353+0000","last_peered":"2024-04-20T10:31:37.131353+0000","last_clean":"2024-04-20T10:31:37.131353+0000","last_became_active":"2024-04-20T10:31:35.649282+0000","last_became_peered":"2024-04-20T10:31:35.649282+0000","last_unstale":"2024-04-20T10:31:37.131353+0000","last_undegraded":"2024-04-20T10:31:37.131353+0000","last_fullsized":"2024-04-20T10:31:37.131353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:01:00.517383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.132035+0000","last_change":"2024-04-20T10:31:35.648299+0000","last_active":"2024-04-20T10:31:37.132035+0000","last_peered":"2024-04-20T10:31:37.132035+0000","last_clean":"2024-04-20T10:31:37.132035+0000","last_became_active":"2024-04-20T10:31:35.647283+0000","last_became_peered":"2024-04-20T10:31:35.647283+0000","last_unstale":"2024-04-20T10:31:37.132035+0000","last_undegraded":"2024-04-20T10:31:37.132035+0000","last_fullsized":"2024-04-20T10:31:37.132035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:59.006234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.132022+0000","last_change":"2024-04-20T10:31:35.647029+0000","last_active":"2024-04-20T10:31:37.132022+0000","last_peered":"2024-04-20T10:31:37.132022+0000","last_clean":"2024-04-20T10:31:37.132022+0000","last_became_active":"2024-04-20T10:31:35.646649+0000","last_became_peered":"2024-04-20T10:31:35.646649+0000","last_unstale":"2024-04-20T10:31:37.132022+0000","last_undegraded":"2024-04-20T10:31:37.132022+0000","last_fullsized":"2024-04-20T10:31:37.132022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:03:23.420078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129750+0000","last_change":"2024-04-20T10:31:35.624628+0000","last_active":"2024-04-20T10:31:37.129750+0000","last_peered":"2024-04-20T10:31:37.129750+0000","last_clean":"2024-04-20T10:31:37.129750+0000","last_became_active":"2024-04-20T10:31:35.623607+0000","last_became_peered":"2024-04-20T10:31:35.623607+0000","last_unstale":"2024-04-20T10:31:37.129750+0000","last_undegraded":"2024-04-20T10:31:37.129750+0000","last_fullsized":"2024-04-20T10:31:37.129750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:44:46.768584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129707+0000","last_change":"2024-04-20T10:31:35.638293+0000","last_active":"2024-04-20T10:31:37.129707+0000","last_peered":"2024-04-20T10:31:37.129707+0000","last_clean":"2024-04-20T10:31:37.129707+0000","last_became_active":"2024-04-20T10:31:35.637967+0000","last_became_peered":"2024-04-20T10:31:35.637967+0000","last_unstale":"2024-04-20T10:31:37.129707+0000","last_undegraded":"2024-04-20T10:31:37.129707+0000","last_fullsized":"2024-04-20T10:31:37.129707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:58:00.765015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130472+0000","last_change":"2024-04-20T10:31:35.634565+0000","last_active":"2024-04-20T10:31:37.130472+0000","last_peered":"2024-04-20T10:31:37.130472+0000","last_clean":"2024-04-20T10:31:37.130472+0000","last_became_active":"2024-04-20T10:31:35.634200+0000","last_became_peered":"2024-04-20T10:31:35.634200+0000","last_unstale":"2024-04-20T10:31:37.130472+0000","last_undegraded":"2024-04-20T10:31:37.130472+0000","last_fullsized":"2024-04-20T10:31:37.130472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:14.298732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031087+0000","last_change":"2024-04-20T10:31:35.637417+0000","last_active":"2024-04-20T10:31:37.031087+0000","last_peered":"2024-04-20T10:31:37.031087+0000","last_clean":"2024-04-20T10:31:37.031087+0000","last_became_active":"2024-04-20T10:31:35.637053+0000","last_became_peered":"2024-04-20T10:31:35.637053+0000","last_unstale":"2024-04-20T10:31:37.031087+0000","last_undegraded":"2024-04-20T10:31:37.031087+0000","last_fullsized":"2024-04-20T10:31:37.031087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:41:49.264677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600368+0000","last_change":"2024-04-20T10:31:35.631996+0000","last_active":"2024-04-20T10:31:36.600368+0000","last_peered":"2024-04-20T10:31:36.600368+0000","last_clean":"2024-04-20T10:31:36.600368+0000","last_became_active":"2024-04-20T10:31:35.631576+0000","last_became_peered":"2024-04-20T10:31:35.631576+0000","last_unstale":"2024-04-20T10:31:36.600368+0000","last_undegraded":"2024-04-20T10:31:36.600368+0000","last_fullsized":"2024-04-20T10:31:36.600368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:51.448025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600173+0000","last_change":"2024-04-20T10:31:35.624430+0000","last_active":"2024-04-20T10:31:36.600173+0000","last_peered":"2024-04-20T10:31:36.600173+0000","last_clean":"2024-04-20T10:31:36.600173+0000","last_became_active":"2024-04-20T10:31:35.623018+0000","last_became_peered":"2024-04-20T10:31:35.623018+0000","last_unstale":"2024-04-20T10:31:36.600173+0000","last_undegraded":"2024-04-20T10:31:36.600173+0000","last_fullsized":"2024-04-20T10:31:36.600173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:43.100925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130486+0000","last_change":"2024-04-20T10:31:35.619038+0000","last_active":"2024-04-20T10:31:37.130486+0000","last_peered":"2024-04-20T10:31:37.130486+0000","last_clean":"2024-04-20T10:31:37.130486+0000","last_became_active":"2024-04-20T10:31:35.618699+0000","last_became_peered":"2024-04-20T10:31:35.618699+0000","last_unstale":"2024-04-20T10:31:37.130486+0000","last_undegraded":"2024-04-20T10:31:37.130486+0000","last_fullsized":"2024-04-20T10:31:37.130486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:27.205634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031330+0000","last_change":"2024-04-20T10:31:35.619908+0000","last_active":"2024-04-20T10:31:37.031330+0000","last_peered":"2024-04-20T10:31:37.031330+0000","last_clean":"2024-04-20T10:31:37.031330+0000","last_became_active":"2024-04-20T10:31:35.619580+0000","last_became_peered":"2024-04-20T10:31:35.619580+0000","last_unstale":"2024-04-20T10:31:37.031330+0000","last_undegraded":"2024-04-20T10:31:37.031330+0000","last_fullsized":"2024-04-20T10:31:37.031330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:54:48.999925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129708+0000","last_change":"2024-04-20T10:31:35.623677+0000","last_active":"2024-04-20T10:31:37.129708+0000","last_peered":"2024-04-20T10:31:37.129708+0000","last_clean":"2024-04-20T10:31:37.129708+0000","last_became_active":"2024-04-20T10:31:35.621544+0000","last_became_peered":"2024-04-20T10:31:35.621544+0000","last_unstale":"2024-04-20T10:31:37.129708+0000","last_undegraded":"2024-04-20T10:31:37.129708+0000","last_fullsized":"2024-04-20T10:31:37.129708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:50:17.902954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.603679+0000","last_change":"2024-04-20T10:31:35.619594+0000","last_active":"2024-04-20T10:31:36.603679+0000","last_peered":"2024-04-20T10:31:36.603679+0000","last_clean":"2024-04-20T10:31:36.603679+0000","last_became_active":"2024-04-20T10:31:35.618831+0000","last_became_peered":"2024-04-20T10:31:35.618831+0000","last_unstale":"2024-04-20T10:31:36.603679+0000","last_undegraded":"2024-04-20T10:31:36.603679+0000","last_fullsized":"2024-04-20T10:31:36.603679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:07.625225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131522+0000","last_change":"2024-04-20T10:31:35.647476+0000","last_active":"2024-04-20T10:31:37.131522+0000","last_peered":"2024-04-20T10:31:37.131522+0000","last_clean":"2024-04-20T10:31:37.131522+0000","last_became_active":"2024-04-20T10:31:35.647009+0000","last_became_peered":"2024-04-20T10:31:35.647009+0000","last_unstale":"2024-04-20T10:31:37.131522+0000","last_undegraded":"2024-04-20T10:31:37.131522+0000","last_fullsized":"2024-04-20T10:31:37.131522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:11:37.526115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131729+0000","last_change":"2024-04-20T10:31:35.648641+0000","last_active":"2024-04-20T10:31:37.131729+0000","last_peered":"2024-04-20T10:31:37.131729+0000","last_clean":"2024-04-20T10:31:37.131729+0000","last_became_active":"2024-04-20T10:31:35.648150+0000","last_became_peered":"2024-04-20T10:31:35.648150+0000","last_unstale":"2024-04-20T10:31:37.131729+0000","last_undegraded":"2024-04-20T10:31:37.131729+0000","last_fullsized":"2024-04-20T10:31:37.131729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:17:44.253934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.599556+0000","last_change":"2024-04-20T10:31:35.643142+0000","last_active":"2024-04-20T10:31:36.599556+0000","last_peered":"2024-04-20T10:31:36.599556+0000","last_clean":"2024-04-20T10:31:36.599556+0000","last_became_active":"2024-04-20T10:31:35.642742+0000","last_became_peered":"2024-04-20T10:31:35.642742+0000","last_unstale":"2024-04-20T10:31:36.599556+0000","last_undegraded":"2024-04-20T10:31:36.599556+0000","last_fullsized":"2024-04-20T10:31:36.599556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:40.818808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131594+0000","last_change":"2024-04-20T10:31:35.650333+0000","last_active":"2024-04-20T10:31:37.131594+0000","last_peered":"2024-04-20T10:31:37.131594+0000","last_clean":"2024-04-20T10:31:37.131594+0000","last_became_active":"2024-04-20T10:31:35.649933+0000","last_became_peered":"2024-04-20T10:31:35.649933+0000","last_unstale":"2024-04-20T10:31:37.131594+0000","last_undegraded":"2024-04-20T10:31:37.131594+0000","last_fullsized":"2024-04-20T10:31:37.131594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:34.513990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131620+0000","last_change":"2024-04-20T10:31:35.637315+0000","last_active":"2024-04-20T10:31:37.131620+0000","last_peered":"2024-04-20T10:31:37.131620+0000","last_clean":"2024-04-20T10:31:37.131620+0000","last_became_active":"2024-04-20T10:31:35.635909+0000","last_became_peered":"2024-04-20T10:31:35.635909+0000","last_unstale":"2024-04-20T10:31:37.131620+0000","last_undegraded":"2024-04-20T10:31:37.131620+0000","last_fullsized":"2024-04-20T10:31:37.131620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:24:07.400032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131027+0000","last_change":"2024-04-20T10:31:35.638888+0000","last_active":"2024-04-20T10:31:37.131027+0000","last_peered":"2024-04-20T10:31:37.131027+0000","last_clean":"2024-04-20T10:31:37.131027+0000","last_became_active":"2024-04-20T10:31:35.638568+0000","last_became_peered":"2024-04-20T10:31:35.638568+0000","last_unstale":"2024-04-20T10:31:37.131027+0000","last_undegraded":"2024-04-20T10:31:37.131027+0000","last_fullsized":"2024-04-20T10:31:37.131027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:48.363874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130999+0000","last_change":"2024-04-20T10:31:34.597814+0000","last_active":"2024-04-20T10:31:37.130999+0000","last_peered":"2024-04-20T10:31:37.130999+0000","last_clean":"2024-04-20T10:31:37.130999+0000","last_became_active":"2024-04-20T10:31:34.597480+0000","last_became_peered":"2024-04-20T10:31:34.597480+0000","last_unstale":"2024-04-20T10:31:37.130999+0000","last_undegraded":"2024-04-20T10:31:37.130999+0000","last_fullsized":"2024-04-20T10:31:37.130999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:48:26.102787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.601222+0000","last_change":"2024-04-20T10:31:35.637189+0000","last_active":"2024-04-20T10:31:36.601222+0000","last_peered":"2024-04-20T10:31:36.601222+0000","last_clean":"2024-04-20T10:31:36.601222+0000","last_became_active":"2024-04-20T10:31:35.636324+0000","last_became_peered":"2024-04-20T10:31:35.636324+0000","last_unstale":"2024-04-20T10:31:36.601222+0000","last_undegraded":"2024-04-20T10:31:36.601222+0000","last_fullsized":"2024-04-20T10:31:36.601222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:46.692986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600931+0000","last_change":"2024-04-20T10:31:34.609942+0000","last_active":"2024-04-20T10:31:36.600931+0000","last_peered":"2024-04-20T10:31:36.600931+0000","last_clean":"2024-04-20T10:31:36.600931+0000","last_became_active":"2024-04-20T10:31:34.608994+0000","last_became_peered":"2024-04-20T10:31:34.608994+0000","last_unstale":"2024-04-20T10:31:36.600931+0000","last_undegraded":"2024-04-20T10:31:36.600931+0000","last_fullsized":"2024-04-20T10:31:36.600931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:26:53.488091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.132750+0000","last_change":"2024-04-20T10:31:35.647710+0000","last_active":"2024-04-20T10:31:37.132750+0000","last_peered":"2024-04-20T10:31:37.132750+0000","last_clean":"2024-04-20T10:31:37.132750+0000","last_became_active":"2024-04-20T10:31:35.646037+0000","last_became_peered":"2024-04-20T10:31:35.646037+0000","last_unstale":"2024-04-20T10:31:37.132750+0000","last_undegraded":"2024-04-20T10:31:37.132750+0000","last_fullsized":"2024-04-20T10:31:37.132750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:50:21.092892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130762+0000","last_change":"2024-04-20T10:31:34.593529+0000","last_active":"2024-04-20T10:31:37.130762+0000","last_peered":"2024-04-20T10:31:37.130762+0000","last_clean":"2024-04-20T10:31:37.130762+0000","last_became_active":"2024-04-20T10:31:34.592583+0000","last_became_peered":"2024-04-20T10:31:34.592583+0000","last_unstale":"2024-04-20T10:31:37.130762+0000","last_undegraded":"2024-04-20T10:31:37.130762+0000","last_fullsized":"2024-04-20T10:31:37.130762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:51:20.570499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.599687+0000","last_change":"2024-04-20T10:31:35.630739+0000","last_active":"2024-04-20T10:31:36.599687+0000","last_peered":"2024-04-20T10:31:36.599687+0000","last_clean":"2024-04-20T10:31:36.599687+0000","last_became_active":"2024-04-20T10:31:35.630424+0000","last_became_peered":"2024-04-20T10:31:35.630424+0000","last_unstale":"2024-04-20T10:31:36.599687+0000","last_undegraded":"2024-04-20T10:31:36.599687+0000","last_fullsized":"2024-04-20T10:31:36.599687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:48:17.485677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129985+0000","last_change":"2024-04-20T10:31:34.591652+0000","last_active":"2024-04-20T10:31:37.129985+0000","last_peered":"2024-04-20T10:31:37.129985+0000","last_clean":"2024-04-20T10:31:37.129985+0000","last_became_active":"2024-04-20T10:31:34.588945+0000","last_became_peered":"2024-04-20T10:31:34.588945+0000","last_unstale":"2024-04-20T10:31:37.129985+0000","last_undegraded":"2024-04-20T10:31:37.129985+0000","last_fullsized":"2024-04-20T10:31:37.129985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:15.137046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130324+0000","last_change":"2024-04-20T10:31:35.622462+0000","last_active":"2024-04-20T10:31:37.130324+0000","last_peered":"2024-04-20T10:31:37.130324+0000","last_clean":"2024-04-20T10:31:37.130324+0000","last_became_active":"2024-04-20T10:31:35.622125+0000","last_became_peered":"2024-04-20T10:31:35.622125+0000","last_unstale":"2024-04-20T10:31:37.130324+0000","last_undegraded":"2024-04-20T10:31:37.130324+0000","last_fullsized":"2024-04-20T10:31:37.130324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:27.894564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130040+0000","last_change":"2024-04-20T10:31:34.595202+0000","last_active":"2024-04-20T10:31:37.130040+0000","last_peered":"2024-04-20T10:31:37.130040+0000","last_clean":"2024-04-20T10:31:37.130040+0000","last_became_active":"2024-04-20T10:31:34.594784+0000","last_became_peered":"2024-04-20T10:31:34.594784+0000","last_unstale":"2024-04-20T10:31:37.130040+0000","last_undegraded":"2024-04-20T10:31:37.130040+0000","last_fullsized":"2024-04-20T10:31:37.130040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:04:09.559326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130279+0000","last_change":"2024-04-20T10:31:35.637040+0000","last_active":"2024-04-20T10:31:37.130279+0000","last_peered":"2024-04-20T10:31:37.130279+0000","last_clean":"2024-04-20T10:31:37.130279+0000","last_became_active":"2024-04-20T10:31:35.636682+0000","last_became_peered":"2024-04-20T10:31:35.636682+0000","last_unstale":"2024-04-20T10:31:37.130279+0000","last_undegraded":"2024-04-20T10:31:37.130279+0000","last_fullsized":"2024-04-20T10:31:37.130279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:40:11.569674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130086+0000","last_change":"2024-04-20T10:31:34.612038+0000","last_active":"2024-04-20T10:31:37.130086+0000","last_peered":"2024-04-20T10:31:37.130086+0000","last_clean":"2024-04-20T10:31:37.130086+0000","last_became_active":"2024-04-20T10:31:34.611671+0000","last_became_peered":"2024-04-20T10:31:34.611671+0000","last_unstale":"2024-04-20T10:31:37.130086+0000","last_undegraded":"2024-04-20T10:31:37.130086+0000","last_fullsized":"2024-04-20T10:31:37.130086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:47.862702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130251+0000","last_change":"2024-04-20T10:31:35.643930+0000","last_active":"2024-04-20T10:31:37.130251+0000","last_peered":"2024-04-20T10:31:37.130251+0000","last_clean":"2024-04-20T10:31:37.130251+0000","last_became_active":"2024-04-20T10:31:35.643470+0000","last_became_peered":"2024-04-20T10:31:35.643470+0000","last_unstale":"2024-04-20T10:31:37.130251+0000","last_undegraded":"2024-04-20T10:31:37.130251+0000","last_fullsized":"2024-04-20T10:31:37.130251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:55.570176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.602632+0000","last_change":"2024-04-20T10:31:34.613224+0000","last_active":"2024-04-20T10:31:36.602632+0000","last_peered":"2024-04-20T10:31:36.602632+0000","last_clean":"2024-04-20T10:31:36.602632+0000","last_became_active":"2024-04-20T10:31:34.612871+0000","last_became_peered":"2024-04-20T10:31:34.612871+0000","last_unstale":"2024-04-20T10:31:36.602632+0000","last_undegraded":"2024-04-20T10:31:36.602632+0000","last_fullsized":"2024-04-20T10:31:36.602632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:05:57.284870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600158+0000","last_change":"2024-04-20T10:31:35.643824+0000","last_active":"2024-04-20T10:31:36.600158+0000","last_peered":"2024-04-20T10:31:36.600158+0000","last_clean":"2024-04-20T10:31:36.600158+0000","last_became_active":"2024-04-20T10:31:35.643425+0000","last_became_peered":"2024-04-20T10:31:35.643425+0000","last_unstale":"2024-04-20T10:31:36.600158+0000","last_undegraded":"2024-04-20T10:31:36.600158+0000","last_fullsized":"2024-04-20T10:31:36.600158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:34.008374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129648+0000","last_change":"2024-04-20T10:31:34.593071+0000","last_active":"2024-04-20T10:31:37.129648+0000","last_peered":"2024-04-20T10:31:37.129648+0000","last_clean":"2024-04-20T10:31:37.129648+0000","last_became_active":"2024-04-20T10:31:34.592721+0000","last_became_peered":"2024-04-20T10:31:34.592721+0000","last_unstale":"2024-04-20T10:31:37.129648+0000","last_undegraded":"2024-04-20T10:31:37.129648+0000","last_fullsized":"2024-04-20T10:31:37.129648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:23:43.749586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130978+0000","last_change":"2024-04-20T10:31:35.647524+0000","last_active":"2024-04-20T10:31:37.130978+0000","last_peered":"2024-04-20T10:31:37.130978+0000","last_clean":"2024-04-20T10:31:37.130978+0000","last_became_active":"2024-04-20T10:31:35.647122+0000","last_became_peered":"2024-04-20T10:31:35.647122+0000","last_unstale":"2024-04-20T10:31:37.130978+0000","last_undegraded":"2024-04-20T10:31:37.130978+0000","last_fullsized":"2024-04-20T10:31:37.130978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130381+0000","last_change":"2024-04-20T10:31:34.604500+0000","last_active":"2024-04-20T10:31:37.130381+0000","last_peered":"2024-04-20T10:31:37.130381+0000","last_clean":"2024-04-20T10:31:37.130381+0000","last_became_active":"2024-04-20T10:31:34.604148+0000","last_became_peered":"2024-04-20T10:31:34.604148+0000","last_unstale":"2024-04-20T10:31:37.130381+0000","last_undegraded":"2024-04-20T10:31:37.130381+0000","last_fullsized":"2024-04-20T10:31:37.130381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:49:15.174414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600172+0000","last_change":"2024-04-20T10:31:35.636751+0000","last_active":"2024-04-20T10:31:36.600172+0000","last_peered":"2024-04-20T10:31:36.600172+0000","last_clean":"2024-04-20T10:31:36.600172+0000","last_became_active":"2024-04-20T10:31:35.636103+0000","last_became_peered":"2024-04-20T10:31:35.636103+0000","last_unstale":"2024-04-20T10:31:36.600172+0000","last_undegraded":"2024-04-20T10:31:36.600172+0000","last_fullsized":"2024-04-20T10:31:36.600172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:22:40.580512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031232+0000","last_change":"2024-04-20T10:31:34.595207+0000","last_active":"2024-04-20T10:31:37.031232+0000","last_peered":"2024-04-20T10:31:37.031232+0000","last_clean":"2024-04-20T10:31:37.031232+0000","last_became_active":"2024-04-20T10:31:34.594860+0000","last_became_peered":"2024-04-20T10:31:34.594860+0000","last_unstale":"2024-04-20T10:31:37.031232+0000","last_undegraded":"2024-04-20T10:31:37.031232+0000","last_fullsized":"2024-04-20T10:31:37.031232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:50.054454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.132706+0000","last_change":"2024-04-20T10:31:35.648086+0000","last_active":"2024-04-20T10:31:37.132706+0000","last_peered":"2024-04-20T10:31:37.132706+0000","last_clean":"2024-04-20T10:31:37.132706+0000","last_became_active":"2024-04-20T10:31:35.647275+0000","last_became_peered":"2024-04-20T10:31:35.647275+0000","last_unstale":"2024-04-20T10:31:37.132706+0000","last_undegraded":"2024-04-20T10:31:37.132706+0000","last_fullsized":"2024-04-20T10:31:37.132706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:27.669008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130336+0000","last_change":"2024-04-20T10:31:34.601674+0000","last_active":"2024-04-20T10:31:37.130336+0000","last_peered":"2024-04-20T10:31:37.130336+0000","last_clean":"2024-04-20T10:31:37.130336+0000","last_became_active":"2024-04-20T10:31:34.601255+0000","last_became_peered":"2024-04-20T10:31:34.601255+0000","last_unstale":"2024-04-20T10:31:37.130336+0000","last_undegraded":"2024-04-20T10:31:37.130336+0000","last_fullsized":"2024-04-20T10:31:37.130336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:57.821635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600043+0000","last_change":"2024-04-20T10:31:35.621888+0000","last_active":"2024-04-20T10:31:36.600043+0000","last_peered":"2024-04-20T10:31:36.600043+0000","last_clean":"2024-04-20T10:31:36.600043+0000","last_became_active":"2024-04-20T10:31:35.621503+0000","last_became_peered":"2024-04-20T10:31:35.621503+0000","last_unstale":"2024-04-20T10:31:36.600043+0000","last_undegraded":"2024-04-20T10:31:36.600043+0000","last_fullsized":"2024-04-20T10:31:36.600043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:15:21.118638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'13","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:40.823399+0000","last_change":"2024-04-20T10:31:34.590558+0000","last_active":"2024-04-20T10:31:40.823399+0000","last_peered":"2024-04-20T10:31:40.823399+0000","last_clean":"2024-04-20T10:31:40.823399+0000","last_became_active":"2024-04-20T10:31:34.590140+0000","last_became_peered":"2024-04-20T10:31:34.590140+0000","last_unstale":"2024-04-20T10:31:40.823399+0000","last_undegraded":"2024-04-20T10:31:40.823399+0000","last_fullsized":"2024-04-20T10:31:40.823399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:15:18.591748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":12,"num_write":13,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130485+0000","last_change":"2024-04-20T10:31:35.620780+0000","last_active":"2024-04-20T10:31:37.130485+0000","last_peered":"2024-04-20T10:31:37.130485+0000","last_clean":"2024-04-20T10:31:37.130485+0000","last_became_active":"2024-04-20T10:31:35.620429+0000","last_became_peered":"2024-04-20T10:31:35.620429+0000","last_unstale":"2024-04-20T10:31:37.130485+0000","last_undegraded":"2024-04-20T10:31:37.130485+0000","last_fullsized":"2024-04-20T10:31:37.130485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:47:45.587920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131388+0000","last_change":"2024-04-20T10:31:34.602135+0000","last_active":"2024-04-20T10:31:37.131388+0000","last_peered":"2024-04-20T10:31:37.131388+0000","last_clean":"2024-04-20T10:31:37.131388+0000","last_became_active":"2024-04-20T10:31:34.601460+0000","last_became_peered":"2024-04-20T10:31:34.601460+0000","last_unstale":"2024-04-20T10:31:37.131388+0000","last_undegraded":"2024-04-20T10:31:37.131388+0000","last_fullsized":"2024-04-20T10:31:37.131388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:00:00.832702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.603279+0000","last_change":"2024-04-20T10:31:35.635526+0000","last_active":"2024-04-20T10:31:36.603279+0000","last_peered":"2024-04-20T10:31:36.603279+0000","last_clean":"2024-04-20T10:31:36.603279+0000","last_became_active":"2024-04-20T10:31:35.635143+0000","last_became_peered":"2024-04-20T10:31:35.635143+0000","last_unstale":"2024-04-20T10:31:36.603279+0000","last_undegraded":"2024-04-20T10:31:36.603279+0000","last_fullsized":"2024-04-20T10:31:36.603279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:57:24.509318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129948+0000","last_change":"2024-04-20T10:31:34.604561+0000","last_active":"2024-04-20T10:31:37.129948+0000","last_peered":"2024-04-20T10:31:37.129948+0000","last_clean":"2024-04-20T10:31:37.129948+0000","last_became_active":"2024-04-20T10:31:34.604232+0000","last_became_peered":"2024-04-20T10:31:34.604232+0000","last_unstale":"2024-04-20T10:31:37.129948+0000","last_undegraded":"2024-04-20T10:31:37.129948+0000","last_fullsized":"2024-04-20T10:31:37.129948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:39.317473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031364+0000","last_change":"2024-04-20T10:31:35.632840+0000","last_active":"2024-04-20T10:31:37.031364+0000","last_peered":"2024-04-20T10:31:37.031364+0000","last_clean":"2024-04-20T10:31:37.031364+0000","last_became_active":"2024-04-20T10:31:35.632471+0000","last_became_peered":"2024-04-20T10:31:35.632471+0000","last_unstale":"2024-04-20T10:31:37.031364+0000","last_undegraded":"2024-04-20T10:31:37.031364+0000","last_fullsized":"2024-04-20T10:31:37.031364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:06.679967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129253+0000","last_change":"2024-04-20T10:31:34.591696+0000","last_active":"2024-04-20T10:31:37.129253+0000","last_peered":"2024-04-20T10:31:37.129253+0000","last_clean":"2024-04-20T10:31:37.129253+0000","last_became_active":"2024-04-20T10:31:34.591330+0000","last_became_peered":"2024-04-20T10:31:34.591330+0000","last_unstale":"2024-04-20T10:31:37.129253+0000","last_undegraded":"2024-04-20T10:31:37.129253+0000","last_fullsized":"2024-04-20T10:31:37.129253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:44:36.385963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.030932+0000","last_change":"2024-04-20T10:31:35.627201+0000","last_active":"2024-04-20T10:31:37.030932+0000","last_peered":"2024-04-20T10:31:37.030932+0000","last_clean":"2024-04-20T10:31:37.030932+0000","last_became_active":"2024-04-20T10:31:35.626880+0000","last_became_peered":"2024-04-20T10:31:35.626880+0000","last_unstale":"2024-04-20T10:31:37.030932+0000","last_undegraded":"2024-04-20T10:31:37.030932+0000","last_fullsized":"2024-04-20T10:31:37.030932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:44.562723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129588+0000","last_change":"2024-04-20T10:31:34.600683+0000","last_active":"2024-04-20T10:31:37.129588+0000","last_peered":"2024-04-20T10:31:37.129588+0000","last_clean":"2024-04-20T10:31:37.129588+0000","last_became_active":"2024-04-20T10:31:34.600305+0000","last_became_peered":"2024-04-20T10:31:34.600305+0000","last_unstale":"2024-04-20T10:31:37.129588+0000","last_undegraded":"2024-04-20T10:31:37.129588+0000","last_fullsized":"2024-04-20T10:31:37.129588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:05.441158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129612+0000","last_change":"2024-04-20T10:31:35.619835+0000","last_active":"2024-04-20T10:31:37.129612+0000","last_peered":"2024-04-20T10:31:37.129612+0000","last_clean":"2024-04-20T10:31:37.129612+0000","last_became_active":"2024-04-20T10:31:35.619554+0000","last_became_peered":"2024-04-20T10:31:35.619554+0000","last_unstale":"2024-04-20T10:31:37.129612+0000","last_undegraded":"2024-04-20T10:31:37.129612+0000","last_fullsized":"2024-04-20T10:31:37.129612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:33.161563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130198+0000","last_change":"2024-04-20T10:31:34.591128+0000","last_active":"2024-04-20T10:31:37.130198+0000","last_peered":"2024-04-20T10:31:37.130198+0000","last_clean":"2024-04-20T10:31:37.130198+0000","last_became_active":"2024-04-20T10:31:34.587772+0000","last_became_peered":"2024-04-20T10:31:34.587772+0000","last_unstale":"2024-04-20T10:31:37.130198+0000","last_undegraded":"2024-04-20T10:31:37.130198+0000","last_fullsized":"2024-04-20T10:31:37.130198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:12:42.541111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130659+0000","last_change":"2024-04-20T10:31:35.616743+0000","last_active":"2024-04-20T10:31:37.130659+0000","last_peered":"2024-04-20T10:31:37.130659+0000","last_clean":"2024-04-20T10:31:37.130659+0000","last_became_active":"2024-04-20T10:31:35.616419+0000","last_became_peered":"2024-04-20T10:31:35.616419+0000","last_unstale":"2024-04-20T10:31:37.130659+0000","last_undegraded":"2024-04-20T10:31:37.130659+0000","last_fullsized":"2024-04-20T10:31:37.130659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:37.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130529+0000","last_change":"2024-04-20T10:31:34.603734+0000","last_active":"2024-04-20T10:31:37.130529+0000","last_peered":"2024-04-20T10:31:37.130529+0000","last_clean":"2024-04-20T10:31:37.130529+0000","last_became_active":"2024-04-20T10:31:34.603388+0000","last_became_peered":"2024-04-20T10:31:34.603388+0000","last_unstale":"2024-04-20T10:31:37.130529+0000","last_undegraded":"2024-04-20T10:31:37.130529+0000","last_fullsized":"2024-04-20T10:31:37.130529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:58:57.199173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130603+0000","last_change":"2024-04-20T10:31:35.638996+0000","last_active":"2024-04-20T10:31:37.130603+0000","last_peered":"2024-04-20T10:31:37.130603+0000","last_clean":"2024-04-20T10:31:37.130603+0000","last_became_active":"2024-04-20T10:31:35.638571+0000","last_became_peered":"2024-04-20T10:31:35.638571+0000","last_unstale":"2024-04-20T10:31:37.130603+0000","last_undegraded":"2024-04-20T10:31:37.130603+0000","last_fullsized":"2024-04-20T10:31:37.130603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:59.541738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.030911+0000","last_change":"2024-04-20T10:31:34.612501+0000","last_active":"2024-04-20T10:31:37.030911+0000","last_peered":"2024-04-20T10:31:37.030911+0000","last_clean":"2024-04-20T10:31:37.030911+0000","last_became_active":"2024-04-20T10:31:34.612150+0000","last_became_peered":"2024-04-20T10:31:34.612150+0000","last_unstale":"2024-04-20T10:31:37.030911+0000","last_undegraded":"2024-04-20T10:31:37.030911+0000","last_fullsized":"2024-04-20T10:31:37.030911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:28.172851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130490+0000","last_change":"2024-04-20T10:31:35.643814+0000","last_active":"2024-04-20T10:31:37.130490+0000","last_peered":"2024-04-20T10:31:37.130490+0000","last_clean":"2024-04-20T10:31:37.130490+0000","last_became_active":"2024-04-20T10:31:35.643458+0000","last_became_peered":"2024-04-20T10:31:35.643458+0000","last_unstale":"2024-04-20T10:31:37.130490+0000","last_undegraded":"2024-04-20T10:31:37.130490+0000","last_fullsized":"2024-04-20T10:31:37.130490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:45.149598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129776+0000","last_change":"2024-04-20T10:31:35.617775+0000","last_active":"2024-04-20T10:31:37.129776+0000","last_peered":"2024-04-20T10:31:37.129776+0000","last_clean":"2024-04-20T10:31:37.129776+0000","last_became_active":"2024-04-20T10:31:35.617465+0000","last_became_peered":"2024-04-20T10:31:35.617465+0000","last_unstale":"2024-04-20T10:31:37.129776+0000","last_undegraded":"2024-04-20T10:31:37.129776+0000","last_fullsized":"2024-04-20T10:31:37.129776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:37.830953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.602324+0000","last_change":"2024-04-20T10:31:34.615044+0000","last_active":"2024-04-20T10:31:36.602324+0000","last_peered":"2024-04-20T10:31:36.602324+0000","last_clean":"2024-04-20T10:31:36.602324+0000","last_became_active":"2024-04-20T10:31:34.612875+0000","last_became_peered":"2024-04-20T10:31:34.612875+0000","last_unstale":"2024-04-20T10:31:36.602324+0000","last_undegraded":"2024-04-20T10:31:36.602324+0000","last_fullsized":"2024-04-20T10:31:36.602324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:59.867501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.601075+0000","last_change":"2024-04-20T10:31:35.630909+0000","last_active":"2024-04-20T10:31:36.601075+0000","last_peered":"2024-04-20T10:31:36.601075+0000","last_clean":"2024-04-20T10:31:36.601075+0000","last_became_active":"2024-04-20T10:31:35.630455+0000","last_became_peered":"2024-04-20T10:31:35.630455+0000","last_unstale":"2024-04-20T10:31:36.601075+0000","last_undegraded":"2024-04-20T10:31:36.601075+0000","last_fullsized":"2024-04-20T10:31:36.601075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:02.522485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129773+0000","last_change":"2024-04-20T10:31:34.602466+0000","last_active":"2024-04-20T10:31:37.129773+0000","last_peered":"2024-04-20T10:31:37.129773+0000","last_clean":"2024-04-20T10:31:37.129773+0000","last_became_active":"2024-04-20T10:31:34.602089+0000","last_became_peered":"2024-04-20T10:31:34.602089+0000","last_unstale":"2024-04-20T10:31:37.129773+0000","last_undegraded":"2024-04-20T10:31:37.129773+0000","last_fullsized":"2024-04-20T10:31:37.129773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:19:09.849840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.030759+0000","last_change":"2024-04-20T10:30:14.724347+0000","last_active":"2024-04-20T10:31:37.030759+0000","last_peered":"2024-04-20T10:31:37.030759+0000","last_clean":"2024-04-20T10:31:37.030759+0000","last_became_active":"2024-04-20T10:30:14.723877+0000","last_became_peered":"2024-04-20T10:30:14.723877+0000","last_unstale":"2024-04-20T10:31:37.030759+0000","last_undegraded":"2024-04-20T10:31:37.030759+0000","last_fullsized":"2024-04-20T10:31:37.030759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_clean_scrub_stamp":"2024-04-20T10:26:16.803986+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:30.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.601697+0000","last_change":"2024-04-20T10:31:35.626505+0000","last_active":"2024-04-20T10:31:36.601697+0000","last_peered":"2024-04-20T10:31:36.601697+0000","last_clean":"2024-04-20T10:31:36.601697+0000","last_became_active":"2024-04-20T10:31:35.626141+0000","last_became_peered":"2024-04-20T10:31:35.626141+0000","last_unstale":"2024-04-20T10:31:36.601697+0000","last_undegraded":"2024-04-20T10:31:36.601697+0000","last_fullsized":"2024-04-20T10:31:36.601697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:29.827225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031139+0000","last_change":"2024-04-20T10:31:34.596936+0000","last_active":"2024-04-20T10:31:37.031139+0000","last_peered":"2024-04-20T10:31:37.031139+0000","last_clean":"2024-04-20T10:31:37.031139+0000","last_became_active":"2024-04-20T10:31:34.596692+0000","last_became_peered":"2024-04-20T10:31:34.596692+0000","last_unstale":"2024-04-20T10:31:37.031139+0000","last_undegraded":"2024-04-20T10:31:37.031139+0000","last_fullsized":"2024-04-20T10:31:37.031139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:02:39.819096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130580+0000","last_change":"2024-04-20T10:31:34.601407+0000","last_active":"2024-04-20T10:31:37.130580+0000","last_peered":"2024-04-20T10:31:37.130580+0000","last_clean":"2024-04-20T10:31:37.130580+0000","last_became_active":"2024-04-20T10:31:34.601007+0000","last_became_peered":"2024-04-20T10:31:34.601007+0000","last_unstale":"2024-04-20T10:31:37.130580+0000","last_undegraded":"2024-04-20T10:31:37.130580+0000","last_fullsized":"2024-04-20T10:31:37.130580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:05:03.037841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.604823+0000","last_change":"2024-04-20T10:31:35.641046+0000","last_active":"2024-04-20T10:31:36.604823+0000","last_peered":"2024-04-20T10:31:36.604823+0000","last_clean":"2024-04-20T10:31:36.604823+0000","last_became_active":"2024-04-20T10:31:35.640666+0000","last_became_peered":"2024-04-20T10:31:35.640666+0000","last_unstale":"2024-04-20T10:31:36.604823+0000","last_undegraded":"2024-04-20T10:31:36.604823+0000","last_fullsized":"2024-04-20T10:31:36.604823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:42.496037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130770+0000","last_change":"2024-04-20T10:31:34.604643+0000","last_active":"2024-04-20T10:31:37.130770+0000","last_peered":"2024-04-20T10:31:37.130770+0000","last_clean":"2024-04-20T10:31:37.130770+0000","last_became_active":"2024-04-20T10:31:34.604210+0000","last_became_peered":"2024-04-20T10:31:34.604210+0000","last_unstale":"2024-04-20T10:31:37.130770+0000","last_undegraded":"2024-04-20T10:31:37.130770+0000","last_fullsized":"2024-04-20T10:31:37.130770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:24:09.748012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600855+0000","last_change":"2024-04-20T10:31:35.641981+0000","last_active":"2024-04-20T10:31:36.600855+0000","last_peered":"2024-04-20T10:31:36.600855+0000","last_clean":"2024-04-20T10:31:36.600855+0000","last_became_active":"2024-04-20T10:31:35.641607+0000","last_became_peered":"2024-04-20T10:31:35.641607+0000","last_unstale":"2024-04-20T10:31:36.600855+0000","last_undegraded":"2024-04-20T10:31:36.600855+0000","last_fullsized":"2024-04-20T10:31:36.600855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:48.520813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129572+0000","last_change":"2024-04-20T10:31:34.611218+0000","last_active":"2024-04-20T10:31:37.129572+0000","last_peered":"2024-04-20T10:31:37.129572+0000","last_clean":"2024-04-20T10:31:37.129572+0000","last_became_active":"2024-04-20T10:31:34.610950+0000","last_became_peered":"2024-04-20T10:31:34.610950+0000","last_unstale":"2024-04-20T10:31:37.129572+0000","last_undegraded":"2024-04-20T10:31:37.129572+0000","last_fullsized":"2024-04-20T10:31:37.129572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:14:11.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.132241+0000","last_change":"2024-04-20T10:31:35.647531+0000","last_active":"2024-04-20T10:31:37.132241+0000","last_peered":"2024-04-20T10:31:37.132241+0000","last_clean":"2024-04-20T10:31:37.132241+0000","last_became_active":"2024-04-20T10:31:35.646389+0000","last_became_peered":"2024-04-20T10:31:35.646389+0000","last_unstale":"2024-04-20T10:31:37.132241+0000","last_undegraded":"2024-04-20T10:31:37.132241+0000","last_fullsized":"2024-04-20T10:31:37.132241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:50.141385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.599584+0000","last_change":"2024-04-20T10:31:34.595565+0000","last_active":"2024-04-20T10:31:36.599584+0000","last_peered":"2024-04-20T10:31:36.599584+0000","last_clean":"2024-04-20T10:31:36.599584+0000","last_became_active":"2024-04-20T10:31:34.595271+0000","last_became_peered":"2024-04-20T10:31:34.595271+0000","last_unstale":"2024-04-20T10:31:36.599584+0000","last_undegraded":"2024-04-20T10:31:36.599584+0000","last_fullsized":"2024-04-20T10:31:36.599584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:55.024192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129572+0000","last_change":"2024-04-20T10:31:35.616751+0000","last_active":"2024-04-20T10:31:37.129572+0000","last_peered":"2024-04-20T10:31:37.129572+0000","last_clean":"2024-04-20T10:31:37.129572+0000","last_became_active":"2024-04-20T10:31:35.616432+0000","last_became_peered":"2024-04-20T10:31:35.616432+0000","last_unstale":"2024-04-20T10:31:37.129572+0000","last_undegraded":"2024-04-20T10:31:37.129572+0000","last_fullsized":"2024-04-20T10:31:37.129572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:25.052140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600621+0000","last_change":"2024-04-20T10:31:34.609139+0000","last_active":"2024-04-20T10:31:36.600621+0000","last_peered":"2024-04-20T10:31:36.600621+0000","last_clean":"2024-04-20T10:31:36.600621+0000","last_became_active":"2024-04-20T10:31:34.608818+0000","last_became_peered":"2024-04-20T10:31:34.608818+0000","last_unstale":"2024-04-20T10:31:36.600621+0000","last_undegraded":"2024-04-20T10:31:36.600621+0000","last_fullsized":"2024-04-20T10:31:36.600621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:42:39.581059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130186+0000","last_change":"2024-04-20T10:31:35.643791+0000","last_active":"2024-04-20T10:31:37.130186+0000","last_peered":"2024-04-20T10:31:37.130186+0000","last_clean":"2024-04-20T10:31:37.130186+0000","last_became_active":"2024-04-20T10:31:35.643390+0000","last_became_peered":"2024-04-20T10:31:35.643390+0000","last_unstale":"2024-04-20T10:31:37.130186+0000","last_undegraded":"2024-04-20T10:31:37.130186+0000","last_fullsized":"2024-04-20T10:31:37.130186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:10.149979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130993+0000","last_change":"2024-04-20T10:31:34.606178+0000","last_active":"2024-04-20T10:31:37.130993+0000","last_peered":"2024-04-20T10:31:37.130993+0000","last_clean":"2024-04-20T10:31:37.130993+0000","last_became_active":"2024-04-20T10:31:34.605817+0000","last_became_peered":"2024-04-20T10:31:34.605817+0000","last_unstale":"2024-04-20T10:31:37.130993+0000","last_undegraded":"2024-04-20T10:31:37.130993+0000","last_fullsized":"2024-04-20T10:31:37.130993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:15.929905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.601575+0000","last_change":"2024-04-20T10:31:35.649487+0000","last_active":"2024-04-20T10:31:36.601575+0000","last_peered":"2024-04-20T10:31:36.601575+0000","last_clean":"2024-04-20T10:31:36.601575+0000","last_became_active":"2024-04-20T10:31:35.648884+0000","last_became_peered":"2024-04-20T10:31:35.648884+0000","last_unstale":"2024-04-20T10:31:36.601575+0000","last_undegraded":"2024-04-20T10:31:36.601575+0000","last_fullsized":"2024-04-20T10:31:36.601575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:07.389100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129827+0000","last_change":"2024-04-20T10:31:34.595718+0000","last_active":"2024-04-20T10:31:37.129827+0000","last_peered":"2024-04-20T10:31:37.129827+0000","last_clean":"2024-04-20T10:31:37.129827+0000","last_became_active":"2024-04-20T10:31:34.595405+0000","last_became_peered":"2024-04-20T10:31:34.595405+0000","last_unstale":"2024-04-20T10:31:37.129827+0000","last_undegraded":"2024-04-20T10:31:37.129827+0000","last_fullsized":"2024-04-20T10:31:37.129827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:16:02.772345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031172+0000","last_change":"2024-04-20T10:31:35.630256+0000","last_active":"2024-04-20T10:31:37.031172+0000","last_peered":"2024-04-20T10:31:37.031172+0000","last_clean":"2024-04-20T10:31:37.031172+0000","last_became_active":"2024-04-20T10:31:35.629879+0000","last_became_peered":"2024-04-20T10:31:35.629879+0000","last_unstale":"2024-04-20T10:31:37.031172+0000","last_undegraded":"2024-04-20T10:31:37.031172+0000","last_fullsized":"2024-04-20T10:31:37.031172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:51.601923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131371+0000","last_change":"2024-04-20T10:31:34.595149+0000","last_active":"2024-04-20T10:31:37.131371+0000","last_peered":"2024-04-20T10:31:37.131371+0000","last_clean":"2024-04-20T10:31:37.131371+0000","last_became_active":"2024-04-20T10:31:34.594808+0000","last_became_peered":"2024-04-20T10:31:34.594808+0000","last_unstale":"2024-04-20T10:31:37.131371+0000","last_undegraded":"2024-04-20T10:31:37.131371+0000","last_fullsized":"2024-04-20T10:31:37.131371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:51.707710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031390+0000","last_change":"2024-04-20T10:31:35.620612+0000","last_active":"2024-04-20T10:31:37.031390+0000","last_peered":"2024-04-20T10:31:37.031390+0000","last_clean":"2024-04-20T10:31:37.031390+0000","last_became_active":"2024-04-20T10:31:35.620312+0000","last_became_peered":"2024-04-20T10:31:35.620312+0000","last_unstale":"2024-04-20T10:31:37.031390+0000","last_undegraded":"2024-04-20T10:31:37.031390+0000","last_fullsized":"2024-04-20T10:31:37.031390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:37:50.756950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130209+0000","last_change":"2024-04-20T10:31:34.604613+0000","last_active":"2024-04-20T10:31:37.130209+0000","last_peered":"2024-04-20T10:31:37.130209+0000","last_clean":"2024-04-20T10:31:37.130209+0000","last_became_active":"2024-04-20T10:31:34.604246+0000","last_became_peered":"2024-04-20T10:31:34.604246+0000","last_unstale":"2024-04-20T10:31:37.130209+0000","last_undegraded":"2024-04-20T10:31:37.130209+0000","last_fullsized":"2024-04-20T10:31:37.130209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:26.521769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031114+0000","last_change":"2024-04-20T10:31:35.639396+0000","last_active":"2024-04-20T10:31:37.031114+0000","last_peered":"2024-04-20T10:31:37.031114+0000","last_clean":"2024-04-20T10:31:37.031114+0000","last_became_active":"2024-04-20T10:31:35.638979+0000","last_became_peered":"2024-04-20T10:31:35.638979+0000","last_unstale":"2024-04-20T10:31:37.031114+0000","last_undegraded":"2024-04-20T10:31:37.031114+0000","last_fullsized":"2024-04-20T10:31:37.031114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:47.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.604649+0000","last_change":"2024-04-20T10:31:34.607420+0000","last_active":"2024-04-20T10:31:36.604649+0000","last_peered":"2024-04-20T10:31:36.604649+0000","last_clean":"2024-04-20T10:31:36.604649+0000","last_became_active":"2024-04-20T10:31:34.602587+0000","last_became_peered":"2024-04-20T10:31:34.602587+0000","last_unstale":"2024-04-20T10:31:36.604649+0000","last_undegraded":"2024-04-20T10:31:36.604649+0000","last_fullsized":"2024-04-20T10:31:36.604649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:56.447944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.031381+0000","last_change":"2024-04-20T10:31:35.624903+0000","last_active":"2024-04-20T10:31:37.031381+0000","last_peered":"2024-04-20T10:31:37.031381+0000","last_clean":"2024-04-20T10:31:37.031381+0000","last_became_active":"2024-04-20T10:31:35.619479+0000","last_became_peered":"2024-04-20T10:31:35.619479+0000","last_unstale":"2024-04-20T10:31:37.031381+0000","last_undegraded":"2024-04-20T10:31:37.031381+0000","last_fullsized":"2024-04-20T10:31:37.031381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:25:40.601983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130405+0000","last_change":"2024-04-20T10:31:35.620351+0000","last_active":"2024-04-20T10:31:37.130405+0000","last_peered":"2024-04-20T10:31:37.130405+0000","last_clean":"2024-04-20T10:31:37.130405+0000","last_became_active":"2024-04-20T10:31:35.620028+0000","last_became_peered":"2024-04-20T10:31:35.620028+0000","last_unstale":"2024-04-20T10:31:37.130405+0000","last_undegraded":"2024-04-20T10:31:37.130405+0000","last_fullsized":"2024-04-20T10:31:37.130405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:51:53.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131233+0000","last_change":"2024-04-20T10:31:35.615917+0000","last_active":"2024-04-20T10:31:37.131233+0000","last_peered":"2024-04-20T10:31:37.131233+0000","last_clean":"2024-04-20T10:31:37.131233+0000","last_became_active":"2024-04-20T10:31:35.615582+0000","last_became_peered":"2024-04-20T10:31:35.615582+0000","last_unstale":"2024-04-20T10:31:37.131233+0000","last_undegraded":"2024-04-20T10:31:37.131233+0000","last_fullsized":"2024-04-20T10:31:37.131233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:54:59.754819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131232+0000","last_change":"2024-04-20T10:31:35.650006+0000","last_active":"2024-04-20T10:31:37.131232+0000","last_peered":"2024-04-20T10:31:37.131232+0000","last_clean":"2024-04-20T10:31:37.131232+0000","last_became_active":"2024-04-20T10:31:35.649683+0000","last_became_peered":"2024-04-20T10:31:35.649683+0000","last_unstale":"2024-04-20T10:31:37.131232+0000","last_undegraded":"2024-04-20T10:31:37.131232+0000","last_fullsized":"2024-04-20T10:31:37.131232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:10:01.484809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.132307+0000","last_change":"2024-04-20T10:31:35.646591+0000","last_active":"2024-04-20T10:31:37.132307+0000","last_peered":"2024-04-20T10:31:37.132307+0000","last_clean":"2024-04-20T10:31:37.132307+0000","last_became_active":"2024-04-20T10:31:35.646188+0000","last_became_peered":"2024-04-20T10:31:35.646188+0000","last_unstale":"2024-04-20T10:31:37.132307+0000","last_undegraded":"2024-04-20T10:31:37.132307+0000","last_fullsized":"2024-04-20T10:31:37.132307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:23.412022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.604578+0000","last_change":"2024-04-20T10:31:35.650879+0000","last_active":"2024-04-20T10:31:36.604578+0000","last_peered":"2024-04-20T10:31:36.604578+0000","last_clean":"2024-04-20T10:31:36.604578+0000","last_became_active":"2024-04-20T10:31:35.650486+0000","last_became_peered":"2024-04-20T10:31:35.650486+0000","last_unstale":"2024-04-20T10:31:36.604578+0000","last_undegraded":"2024-04-20T10:31:36.604578+0000","last_fullsized":"2024-04-20T10:31:36.604578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:51.893446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131065+0000","last_change":"2024-04-20T10:31:35.619883+0000","last_active":"2024-04-20T10:31:37.131065+0000","last_peered":"2024-04-20T10:31:37.131065+0000","last_clean":"2024-04-20T10:31:37.131065+0000","last_became_active":"2024-04-20T10:31:35.619628+0000","last_became_peered":"2024-04-20T10:31:35.619628+0000","last_unstale":"2024-04-20T10:31:37.131065+0000","last_undegraded":"2024-04-20T10:31:37.131065+0000","last_fullsized":"2024-04-20T10:31:37.131065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:45.018417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.131531+0000","last_change":"2024-04-20T10:31:35.620773+0000","last_active":"2024-04-20T10:31:37.131531+0000","last_peered":"2024-04-20T10:31:37.131531+0000","last_clean":"2024-04-20T10:31:37.131531+0000","last_became_active":"2024-04-20T10:31:35.620467+0000","last_became_peered":"2024-04-20T10:31:35.620467+0000","last_unstale":"2024-04-20T10:31:37.131531+0000","last_undegraded":"2024-04-20T10:31:37.131531+0000","last_fullsized":"2024-04-20T10:31:37.131531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:31:48.009168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.601683+0000","last_change":"2024-04-20T10:31:35.644618+0000","last_active":"2024-04-20T10:31:36.601683+0000","last_peered":"2024-04-20T10:31:36.601683+0000","last_clean":"2024-04-20T10:31:36.601683+0000","last_became_active":"2024-04-20T10:31:35.644068+0000","last_became_peered":"2024-04-20T10:31:35.644068+0000","last_unstale":"2024-04-20T10:31:36.601683+0000","last_undegraded":"2024-04-20T10:31:36.601683+0000","last_fullsized":"2024-04-20T10:31:36.601683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:51.987584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.129630+0000","last_change":"2024-04-20T10:31:35.637588+0000","last_active":"2024-04-20T10:31:37.129630+0000","last_peered":"2024-04-20T10:31:37.129630+0000","last_clean":"2024-04-20T10:31:37.129630+0000","last_became_active":"2024-04-20T10:31:35.637319+0000","last_became_peered":"2024-04-20T10:31:35.637319+0000","last_unstale":"2024-04-20T10:31:37.129630+0000","last_undegraded":"2024-04-20T10:31:37.129630+0000","last_fullsized":"2024-04-20T10:31:37.129630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:54.070894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.600498+0000","last_change":"2024-04-20T10:31:35.629551+0000","last_active":"2024-04-20T10:31:36.600498+0000","last_peered":"2024-04-20T10:31:36.600498+0000","last_clean":"2024-04-20T10:31:36.600498+0000","last_became_active":"2024-04-20T10:31:35.628710+0000","last_became_peered":"2024-04-20T10:31:35.628710+0000","last_unstale":"2024-04-20T10:31:36.600498+0000","last_undegraded":"2024-04-20T10:31:36.600498+0000","last_fullsized":"2024-04-20T10:31:36.600498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:31.883788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.599716+0000","last_change":"2024-04-20T10:31:35.629978+0000","last_active":"2024-04-20T10:31:36.599716+0000","last_peered":"2024-04-20T10:31:36.599716+0000","last_clean":"2024-04-20T10:31:36.599716+0000","last_became_active":"2024-04-20T10:31:35.629596+0000","last_became_peered":"2024-04-20T10:31:35.629596+0000","last_unstale":"2024-04-20T10:31:36.599716+0000","last_undegraded":"2024-04-20T10:31:36.599716+0000","last_fullsized":"2024-04-20T10:31:36.599716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:02.523099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:36.603850+0000","last_change":"2024-04-20T10:31:35.652481+0000","last_active":"2024-04-20T10:31:36.603850+0000","last_peered":"2024-04-20T10:31:36.603850+0000","last_clean":"2024-04-20T10:31:36.603850+0000","last_became_active":"2024-04-20T10:31:35.651837+0000","last_became_peered":"2024-04-20T10:31:35.651837+0000","last_unstale":"2024-04-20T10:31:36.603850+0000","last_undegraded":"2024-04-20T10:31:36.603850+0000","last_fullsized":"2024-04-20T10:31:36.603850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:53.206954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T10:31:37.130588+0000","last_change":"2024-04-20T10:31:35.622228+0000","last_active":"2024-04-20T10:31:37.130588+0000","last_peered":"2024-04-20T10:31:37.130588+0000","last_clean":"2024-04-20T10:31:37.130588+0000","last_became_active":"2024-04-20T10:31:35.621910+0000","last_became_peered":"2024-04-20T10:31:35.621910+0000","last_unstale":"2024-04-20T10:31:37.130588+0000","last_undegraded":"2024-04-20T10:31:37.130588+0000","last_fullsized":"2024-04-20T10:31:37.130588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:47:34.319839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281463,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":21693,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":15,"num_write":52,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":65079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":1037295,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808858,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":439125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":966656,"data_stored":1029471,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":459416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":440167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1029071,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34156,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34156,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34156,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":20929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":21089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:31:44.349 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T10:31:44.350 INFO:tasks.cephadm:Setting up client nodes... 2024-04-20T10:31:44.350 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-20T10:31:44.845 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:31:45.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:44 smithi042 ceph-mon[17078]: from='client.24730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:31:45.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:44 smithi162 ceph-mon[21854]: from='client.24730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:31:45.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:44 smithi046 ceph-mon[21574]: from='client.24730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:31:46.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:45 smithi042 ceph-mon[17078]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:31:46.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:45 smithi042 ceph-mon[17078]: pgmap v322: 97 pgs: 97 active+clean; 598 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-20T10:31:46.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:45 smithi162 ceph-mon[21854]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:31:46.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:45 smithi162 ceph-mon[21854]: pgmap v322: 97 pgs: 97 active+clean; 598 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-20T10:31:46.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:45 smithi046 ceph-mon[21574]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:31:46.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:45 smithi046 ceph-mon[21574]: pgmap v322: 97 pgs: 97 active+clean; 598 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-04-20T10:31:47.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.284+0000 7f62eb600700 1 -- 172.21.15.42:0/1516006588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62e41068a0 msgr2=0x7f62e410d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:47.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.284+0000 7f62eb600700 1 --2- 172.21.15.42:0/1516006588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62e41068a0 0x7f62e410d150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009f90 tx=0x7f62dc00b330 comp rx=0 tx=0).stop 2024-04-20T10:31:47.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.285+0000 7f62eb600700 1 -- 172.21.15.42:0/1516006588 shutdown_connections 2024-04-20T10:31:47.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.285+0000 7f62eb600700 1 --2- 172.21.15.42:0/1516006588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62e4104d00 0x7f62e4105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:47.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.285+0000 7f62eb600700 1 --2- 172.21.15.42:0/1516006588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62e41068a0 0x7f62e410d150 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:47.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.285+0000 7f62eb600700 1 --2- 172.21.15.42:0/1516006588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62e4105f00 0x7f62e4106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:47.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.285+0000 7f62eb600700 1 -- 172.21.15.42:0/1516006588 >> 172.21.15.42:0/1516006588 conn(0x7f62e41002a0 msgr2=0x7f62e41026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:47.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.285+0000 7f62eb600700 1 -- 172.21.15.42:0/1516006588 shutdown_connections 2024-04-20T10:31:47.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.285+0000 7f62eb600700 1 -- 172.21.15.42:0/1516006588 wait complete. 2024-04-20T10:31:47.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.286+0000 7f62eb600700 1 Processor -- start 2024-04-20T10:31:47.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.286+0000 7f62eb600700 1 -- start start 2024-04-20T10:31:47.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.287+0000 7f62eb600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62e4104d00 0x7f62e419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:47.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.287+0000 7f62eb600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62e4105f00 0x7f62e419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.287+0000 7f62eb600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62e41068a0 0x7f62e41a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.287+0000 7f62eb600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41108f0 con 0x7f62e4104d00 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.287+0000 7f62eb600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4110770 con 0x7f62e4105f00 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.287+0000 7f62eb600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4110a70 con 0x7f62e41068a0 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.287+0000 7f62e8c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62e4104d00 0x7f62e419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.287+0000 7f62e8c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62e4104d00 0x7f62e419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51796/0 (socket says 172.21.15.42:51796) 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.287+0000 7f62e8c00700 1 -- 172.21.15.42:0/3955452410 learned_addr learned my addr 172.21.15.42:0/3955452410 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.287+0000 7f62e3e00700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62e4105f00 0x7f62e419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.287+0000 7f62e9600700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62e41068a0 0x7f62e41a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.288+0000 7f62e3e00700 1 -- 172.21.15.42:0/3955452410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62e41068a0 msgr2=0x7f62e41a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.288+0000 7f62e3e00700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62e41068a0 0x7f62e41a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.288+0000 7f62e3e00700 1 -- 172.21.15.42:0/3955452410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62e4104d00 msgr2=0x7f62e419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.288+0000 7f62e3e00700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62e4104d00 0x7f62e419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:47.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.288+0000 7f62e3e00700 1 -- 172.21.15.42:0/3955452410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e41a2a10 con 0x7f62e4105f00 2024-04-20T10:31:47.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.288+0000 7f62e8c00700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62e4104d00 0x7f62e419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:47.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.288+0000 7f62e3e00700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62e4105f00 0x7f62e419b2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62d000eb90 tx=0x7f62d000eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:47.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.288+0000 7f62e1600700 1 -- 172.21.15.42:0/3955452410 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d000cc90 con 0x7f62e4105f00 2024-04-20T10:31:47.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.288+0000 7f62e1600700 1 -- 172.21.15.42:0/3955452410 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d000ce30 con 0x7f62e4105f00 2024-04-20T10:31:47.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.288+0000 7f62eb600700 1 -- 172.21.15.42:0/3955452410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e41a2d00 con 0x7f62e4105f00 2024-04-20T10:31:47.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.288+0000 7f62e1600700 1 -- 172.21.15.42:0/3955452410 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0010430 con 0x7f62e4105f00 2024-04-20T10:31:47.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.288+0000 7f62eb600700 1 -- 172.21.15.42:0/3955452410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e41a3230 con 0x7f62e4105f00 2024-04-20T10:31:47.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.289+0000 7f62eb600700 1 -- 172.21.15.42:0/3955452410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e4050f90 con 0x7f62e4105f00 2024-04-20T10:31:47.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.292+0000 7f62e1600700 1 -- 172.21.15.42:0/3955452410 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f62d0004000 con 0x7f62e4105f00 2024-04-20T10:31:47.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.293+0000 7f62e1600700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f62cc0a3e00 0x7f62cc0a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:47.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.293+0000 7f62e1600700 1 -- 172.21.15.42:0/3955452410 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f62d00d08c0 con 0x7f62e4105f00 2024-04-20T10:31:47.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.293+0000 7f62e8c00700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f62cc0a3e00 0x7f62cc0a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:47.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.294+0000 7f62e8c00700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f62cc0a3e00 0x7f62cc0a62c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f62d8005950 tx=0x7f62d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:47.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.296+0000 7f62e1600700 1 -- 172.21.15.42:0/3955452410 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f62d009d1f0 con 0x7f62e4105f00 2024-04-20T10:31:47.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.619+0000 7f62eb600700 1 -- 172.21.15.42:0/3955452410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f62e402e1e0 con 0x7f62e4105f00 2024-04-20T10:31:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.628+0000 7f62e1600700 1 -- 172.21.15.42:0/3955452410 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f62d009caf0 con 0x7f62e4105f00 2024-04-20T10:31:47.629 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-04-20T10:31:47.629 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQCTmSNmxBsiJRAApRJMnevF+Um6exP+utud0Q== 2024-04-20T10:31:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.631+0000 7f62eb600700 1 -- 172.21.15.42:0/3955452410 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f62cc0a3e00 msgr2=0x7f62cc0a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.631+0000 7f62eb600700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f62cc0a3e00 0x7f62cc0a62c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f62d8005950 tx=0x7f62d8009260 comp rx=0 tx=0).stop 2024-04-20T10:31:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.631+0000 7f62eb600700 1 -- 172.21.15.42:0/3955452410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62e4105f00 msgr2=0x7f62e419b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.631+0000 7f62eb600700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62e4105f00 0x7f62e419b2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62d000eb90 tx=0x7f62d000eea0 comp rx=0 tx=0).stop 2024-04-20T10:31:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.631+0000 7f62eb600700 1 -- 172.21.15.42:0/3955452410 shutdown_connections 2024-04-20T10:31:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.631+0000 7f62eb600700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f62cc0a3e00 0x7f62cc0a62c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.631+0000 7f62eb600700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62e41068a0 0x7f62e41a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:47.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.631+0000 7f62eb600700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62e4104d00 0x7f62e419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.631+0000 7f62eb600700 1 --2- 172.21.15.42:0/3955452410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62e4105f00 0x7f62e419b2f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.631+0000 7f62eb600700 1 -- 172.21.15.42:0/3955452410 >> 172.21.15.42:0/3955452410 conn(0x7f62e41002a0 msgr2=0x7f62e4109130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.631+0000 7f62eb600700 1 -- 172.21.15.42:0/3955452410 shutdown_connections 2024-04-20T10:31:47.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:47.631+0000 7f62eb600700 1 -- 172.21.15.42:0/3955452410 wait complete. 2024-04-20T10:31:48.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:47 smithi042 ceph-mon[17078]: pgmap v323: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-04-20T10:31:48.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:47 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3955452410' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T10:31:48.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:47 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T10:31:48.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:47 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T10:31:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:47 smithi162 ceph-mon[21854]: pgmap v323: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-04-20T10:31:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:47 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3955452410' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T10:31:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:47 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T10:31:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:47 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T10:31:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:47 smithi046 ceph-mon[21574]: pgmap v323: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-04-20T10:31:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:47 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3955452410' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T10:31:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:47 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T10:31:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:47 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T10:31:48.671 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:31:48.671 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-20T10:31:48.671 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T10:31:48.820 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T10:31:48.820 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-20T10:31:48.820 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph mgr dump --format=json 2024-04-20T10:31:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:31:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:49 smithi042 ceph-mon[17078]: pgmap v324: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-04-20T10:31:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:49 smithi162 ceph-mon[21854]: pgmap v324: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-04-20T10:31:50.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:49 smithi046 ceph-mon[21574]: pgmap v324: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-04-20T10:31:51.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.457+0000 7fd593a00700 1 -- 172.21.15.42:0/2603236034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58c0755a0 msgr2=0x7fd58c1136e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:51.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.457+0000 7fd593a00700 1 --2- 172.21.15.42:0/2603236034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58c0755a0 0x7fd58c1136e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd574009990 tx=0x7fd574009ca0 comp rx=0 tx=0).stop 2024-04-20T10:31:51.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.457+0000 7fd593a00700 1 -- 172.21.15.42:0/2603236034 shutdown_connections 2024-04-20T10:31:51.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.457+0000 7fd593a00700 1 --2- 172.21.15.42:0/2603236034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd58c074c80 0x7fd58c075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:51.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.457+0000 7fd593a00700 1 --2- 172.21.15.42:0/2603236034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd58c113c20 0x7fd58c116010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:51.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.457+0000 7fd593a00700 1 --2- 172.21.15.42:0/2603236034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58c0755a0 0x7fd58c1136e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:51.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.457+0000 7fd593a00700 1 -- 172.21.15.42:0/2603236034 >> 172.21.15.42:0/2603236034 conn(0x7fd58c079bb0 msgr2=0x7fd58c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:51.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.457+0000 7fd593a00700 1 -- 172.21.15.42:0/2603236034 shutdown_connections 2024-04-20T10:31:51.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.457+0000 7fd593a00700 1 -- 172.21.15.42:0/2603236034 wait complete. 2024-04-20T10:31:51.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.458+0000 7fd593a00700 1 Processor -- start 2024-04-20T10:31:51.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.458+0000 7fd593a00700 1 -- start start 2024-04-20T10:31:51.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.459+0000 7fd593a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd58c074c80 0x7fd58c1a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:51.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.459+0000 7fd593a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58c0755a0 0x7fd58c1a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.459+0000 7fd593a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd58c113c20 0x7fd58c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.459+0000 7fd593a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1193b0 con 0x7fd58c074c80 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.459+0000 7fd593a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c119230 con 0x7fd58c113c20 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.459+0000 7fd593a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c119530 con 0x7fd58c0755a0 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.459+0000 7fd591000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd58c074c80 0x7fd58c1a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.459+0000 7fd591000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd58c074c80 0x7fd58c1a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52848/0 (socket says 172.21.15.42:52848) 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.459+0000 7fd591000700 1 -- 172.21.15.42:0/2524255140 learned_addr learned my addr 172.21.15.42:0/2524255140 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.459+0000 7fd591a00700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd58c113c20 0x7fd58c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.459+0000 7fd583e00700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58c0755a0 0x7fd58c1a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.460+0000 7fd591a00700 1 -- 172.21.15.42:0/2524255140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58c0755a0 msgr2=0x7fd58c1a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.460+0000 7fd591a00700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58c0755a0 0x7fd58c1a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.460+0000 7fd591a00700 1 -- 172.21.15.42:0/2524255140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd58c074c80 msgr2=0x7fd58c1a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.460+0000 7fd591a00700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd58c074c80 0x7fd58c1a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.460+0000 7fd591a00700 1 -- 172.21.15.42:0/2524255140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c19ff50 con 0x7fd58c113c20 2024-04-20T10:31:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.460+0000 7fd583e00700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58c0755a0 0x7fd58c1a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:31:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.460+0000 7fd591a00700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd58c113c20 0x7fd58c19f670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd58400ef90 tx=0x7fd58400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.460+0000 7fd58a000700 1 -- 172.21.15.42:0/2524255140 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5840092e0 con 0x7fd58c113c20 2024-04-20T10:31:51.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.460+0000 7fd58a000700 1 -- 172.21.15.42:0/2524255140 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd584004bb0 con 0x7fd58c113c20 2024-04-20T10:31:51.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.460+0000 7fd58a000700 1 -- 172.21.15.42:0/2524255140 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584004280 con 0x7fd58c113c20 2024-04-20T10:31:51.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.460+0000 7fd593a00700 1 -- 172.21.15.42:0/2524255140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58c1a01e0 con 0x7fd58c113c20 2024-04-20T10:31:51.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.461+0000 7fd593a00700 1 -- 172.21.15.42:0/2524255140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58c1ac350 con 0x7fd58c113c20 2024-04-20T10:31:51.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.462+0000 7fd593a00700 1 -- 172.21.15.42:0/2524255140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58c076370 con 0x7fd58c113c20 2024-04-20T10:31:51.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.464+0000 7fd58a000700 1 -- 172.21.15.42:0/2524255140 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd58401b030 con 0x7fd58c113c20 2024-04-20T10:31:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.465+0000 7fd58a000700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd5700a3d50 0x7fd5700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.465+0000 7fd58a000700 1 -- 172.21.15.42:0/2524255140 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd5840d02f0 con 0x7fd58c113c20 2024-04-20T10:31:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.465+0000 7fd591000700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd5700a3d50 0x7fd5700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.466+0000 7fd591000700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd5700a3d50 0x7fd5700a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009d20 tx=0x7fd57c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:51.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.468+0000 7fd58a000700 1 -- 172.21.15.42:0/2524255140 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd584022020 con 0x7fd58c113c20 2024-04-20T10:31:51.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.779+0000 7fd593a00700 1 -- 172.21.15.42:0/2524255140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd58c1ac580 con 0x7fd58c113c20 2024-04-20T10:31:51.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.784+0000 7fd58a000700 1 -- 172.21.15.42:0/2524255140 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264995 (secure 0 0 0) 0x7fd584010040 con 0x7fd58c113c20 2024-04-20T10:31:51.786 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.790+0000 7fd593a00700 1 -- 172.21.15.42:0/2524255140 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd5700a3d50 msgr2=0x7fd5700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.790+0000 7fd593a00700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd5700a3d50 0x7fd5700a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009d20 tx=0x7fd57c009260 comp rx=0 tx=0).stop 2024-04-20T10:31:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.790+0000 7fd593a00700 1 -- 172.21.15.42:0/2524255140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd58c113c20 msgr2=0x7fd58c19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.790+0000 7fd593a00700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd58c113c20 0x7fd58c19f670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd58400ef90 tx=0x7fd58400c3a0 comp rx=0 tx=0).stop 2024-04-20T10:31:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.790+0000 7fd593a00700 1 -- 172.21.15.42:0/2524255140 shutdown_connections 2024-04-20T10:31:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.790+0000 7fd593a00700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd5700a3d50 0x7fd5700a6210 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.790+0000 7fd593a00700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd58c113c20 0x7fd58c19f670 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.790+0000 7fd593a00700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd58c074c80 0x7fd58c1a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:51.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.790+0000 7fd593a00700 1 --2- 172.21.15.42:0/2524255140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd58c0755a0 0x7fd58c1a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:51.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.790+0000 7fd593a00700 1 -- 172.21.15.42:0/2524255140 >> 172.21.15.42:0/2524255140 conn(0x7fd58c079bb0 msgr2=0x7fd58c1129e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:51.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.791+0000 7fd593a00700 1 -- 172.21.15.42:0/2524255140 shutdown_connections 2024-04-20T10:31:51.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:51.791+0000 7fd593a00700 1 -- 172.21.15.42:0/2524255140 wait complete. 2024-04-20T10:31:52.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:51 smithi042 ceph-mon[17078]: pgmap v325: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-20T10:31:52.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:51 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2524255140' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T10:31:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:51 smithi162 ceph-mon[21854]: pgmap v325: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-20T10:31:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:51 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2524255140' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T10:31:52.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:51 smithi046 ceph-mon[21574]: pgmap v325: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-20T10:31:52.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:51 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2524255140' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T10:31:52.681 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":3610568749},{"type":"v1","addr":"172.21.15.42:6801","nonce":3610568749}]},"active_addr":"172.21.15.42:6801/3610568749","active_change":"2024-04-20T10:23:30.366415+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.42:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":869471821}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":3804122415}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1321607472}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":230562472}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":483014163}]}]} 2024-04-20T10:31:52.687 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-20T10:31:52.687 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-20T10:31:52.688 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd dump --format=json 2024-04-20T10:31:53.173 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:31:53.945 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:53 smithi042 ceph-mon[17078]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-20T10:31:54.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:53 smithi162 ceph-mon[21854]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-20T10:31:54.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:53 smithi046 ceph-mon[21574]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-20T10:31:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.390+0000 7effada00700 1 -- 172.21.15.42:0/3339851070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa810f500 msgr2=0x7effa81118f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.390+0000 7effada00700 1 --2- 172.21.15.42:0/3339851070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa810f500 0x7effa81118f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7effa000b0a0 tx=0x7effa000b3b0 comp rx=0 tx=0).stop 2024-04-20T10:31:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.390+0000 7effada00700 1 -- 172.21.15.42:0/3339851070 shutdown_connections 2024-04-20T10:31:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.390+0000 7effada00700 1 --2- 172.21.15.42:0/3339851070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa806a760 0x7effa806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:55.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.390+0000 7effada00700 1 --2- 172.21.15.42:0/3339851070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa810f500 0x7effa81118f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:55.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.390+0000 7effada00700 1 --2- 172.21.15.42:0/3339851070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effa806b080 0x7effa810efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:55.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.390+0000 7effada00700 1 -- 172.21.15.42:0/3339851070 >> 172.21.15.42:0/3339851070 conn(0x7effa8078900 msgr2=0x7effa8078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:55.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.390+0000 7effada00700 1 -- 172.21.15.42:0/3339851070 shutdown_connections 2024-04-20T10:31:55.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.390+0000 7effada00700 1 -- 172.21.15.42:0/3339851070 wait complete. 2024-04-20T10:31:55.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.391+0000 7effada00700 1 Processor -- start 2024-04-20T10:31:55.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.391+0000 7effada00700 1 -- start start 2024-04-20T10:31:55.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.392+0000 7effada00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa806a760 0x7effa819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:55.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.392+0000 7effada00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa806b080 0x7effa819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:55.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.392+0000 7effada00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effa810f500 0x7effa819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:55.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.392+0000 7effada00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7effa8114f50 con 0x7effa806a760 2024-04-20T10:31:55.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.392+0000 7effada00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7effa8114dd0 con 0x7effa806b080 2024-04-20T10:31:55.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.392+0000 7effada00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7effa81150d0 con 0x7effa810f500 2024-04-20T10:31:55.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.392+0000 7effa6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa806a760 0x7effa819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:55.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.392+0000 7effa6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa806a760 0x7effa819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52870/0 (socket says 172.21.15.42:52870) 2024-04-20T10:31:55.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.392+0000 7effa6a00700 1 -- 172.21.15.42:0/2301109894 learned_addr learned my addr 172.21.15.42:0/2301109894 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:55.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.392+0000 7effa7400700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effa810f500 0x7effa819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:55.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.392+0000 7eff9e000700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa806b080 0x7effa819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:55.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.393+0000 7eff9e000700 1 -- 172.21.15.42:0/2301109894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effa810f500 msgr2=0x7effa819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:55.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.393+0000 7eff9e000700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effa810f500 0x7effa819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:55.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.393+0000 7eff9e000700 1 -- 172.21.15.42:0/2301109894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa806a760 msgr2=0x7effa819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:55.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.393+0000 7eff9e000700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa806a760 0x7effa819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:55.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.393+0000 7eff9e000700 1 -- 172.21.15.42:0/2301109894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa81a0410 con 0x7effa806b080 2024-04-20T10:31:55.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.393+0000 7eff9e000700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa806b080 0x7effa819b940 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff9000e9f0 tx=0x7eff9000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:55.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.393+0000 7eff9fe00700 1 -- 172.21.15.42:0/2301109894 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90004220 con 0x7effa806b080 2024-04-20T10:31:55.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.393+0000 7effada00700 1 -- 172.21.15.42:0/2301109894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa81a0700 con 0x7effa806b080 2024-04-20T10:31:55.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.393+0000 7eff9fe00700 1 -- 172.21.15.42:0/2301109894 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff90004d10 con 0x7effa806b080 2024-04-20T10:31:55.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.393+0000 7eff9fe00700 1 -- 172.21.15.42:0/2301109894 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90005020 con 0x7effa806b080 2024-04-20T10:31:55.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.394+0000 7effada00700 1 -- 172.21.15.42:0/2301109894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa81a7f80 con 0x7effa806b080 2024-04-20T10:31:55.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.395+0000 7effada00700 1 -- 172.21.15.42:0/2301109894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa8069740 con 0x7effa806b080 2024-04-20T10:31:55.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.397+0000 7eff9fe00700 1 -- 172.21.15.42:0/2301109894 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7eff90004750 con 0x7effa806b080 2024-04-20T10:31:55.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.398+0000 7eff9fe00700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7eff8c0a3cd0 0x7eff8c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:55.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.398+0000 7eff9fe00700 1 -- 172.21.15.42:0/2301109894 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7eff900d0310 con 0x7effa806b080 2024-04-20T10:31:55.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.398+0000 7effa6a00700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7eff8c0a3cd0 0x7eff8c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:55.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.399+0000 7effa6a00700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7eff8c0a3cd0 0x7eff8c0a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7eff98009d20 tx=0x7eff98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:55.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.402+0000 7eff9fe00700 1 -- 172.21.15.42:0/2301109894 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7eff9009cc40 con 0x7effa806b080 2024-04-20T10:31:55.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.633+0000 7effada00700 1 -- 172.21.15.42:0/2301109894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effa819c760 con 0x7effa806b080 2024-04-20T10:31:55.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.635+0000 7eff9fe00700 1 -- 172.21.15.42:0/2301109894 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22197 (secure 0 0 0) 0x7eff9009c540 con 0x7effa806b080 2024-04-20T10:31:55.636 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:55.636 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":85,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:31:36.594768+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:31:55.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.638+0000 7effada00700 1 -- 172.21.15.42:0/2301109894 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7eff8c0a3cd0 msgr2=0x7eff8c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:55.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.638+0000 7effada00700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7eff8c0a3cd0 0x7eff8c0a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7eff98009d20 tx=0x7eff98009260 comp rx=0 tx=0).stop 2024-04-20T10:31:55.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.638+0000 7effada00700 1 -- 172.21.15.42:0/2301109894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa806b080 msgr2=0x7effa819b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:55.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.638+0000 7effada00700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa806b080 0x7effa819b940 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff9000e9f0 tx=0x7eff9000ed00 comp rx=0 tx=0).stop 2024-04-20T10:31:55.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.638+0000 7effada00700 1 -- 172.21.15.42:0/2301109894 shutdown_connections 2024-04-20T10:31:55.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.638+0000 7effada00700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7eff8c0a3cd0 0x7eff8c0a6190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:55.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.638+0000 7effada00700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effa810f500 0x7effa819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:55.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.638+0000 7effada00700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa806a760 0x7effa819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:55.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.638+0000 7effada00700 1 --2- 172.21.15.42:0/2301109894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa806b080 0x7effa819b940 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:55.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.638+0000 7effada00700 1 -- 172.21.15.42:0/2301109894 >> 172.21.15.42:0/2301109894 conn(0x7effa8078900 msgr2=0x7effa8110970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:55.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.638+0000 7effada00700 1 -- 172.21.15.42:0/2301109894 shutdown_connections 2024-04-20T10:31:55.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:55.638+0000 7effada00700 1 -- 172.21.15.42:0/2301109894 wait complete. 2024-04-20T10:31:56.034 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:55 smithi042 ceph-mon[17078]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.9 KiB/s wr, 4 op/s 2024-04-20T10:31:56.034 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:55 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2301109894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:56.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:55 smithi162 ceph-mon[21854]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.9 KiB/s wr, 4 op/s 2024-04-20T10:31:56.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:55 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2301109894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:56.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:55 smithi046 ceph-mon[21574]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.9 KiB/s wr, 4 op/s 2024-04-20T10:31:56.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:55 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2301109894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:31:56.401 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-20T10:31:56.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd dump --format=json 2024-04-20T10:31:56.877 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:31:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:57 smithi042 ceph-mon[17078]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-04-20T10:31:58.105 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:57 smithi046 ceph-mon[21574]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-04-20T10:31:58.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:57 smithi162 ceph-mon[21854]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-04-20T10:31:59.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.179+0000 7fe027200700 1 -- 172.21.15.42:0/510853408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe028101930 msgr2=0x7fe028101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:59.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.179+0000 7fe027200700 1 --2- 172.21.15.42:0/510853408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe028101930 0x7fe028101d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe018009990 tx=0x7fe018009ca0 comp rx=0 tx=0).stop 2024-04-20T10:31:59.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.180+0000 7fe027200700 1 -- 172.21.15.42:0/510853408 shutdown_connections 2024-04-20T10:31:59.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.180+0000 7fe027200700 1 --2- 172.21.15.42:0/510853408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe028101930 0x7fe028101d10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:59.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.180+0000 7fe027200700 1 --2- 172.21.15.42:0/510853408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe028106d40 0x7fe0281071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:59.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.180+0000 7fe027200700 1 --2- 172.21.15.42:0/510853408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe028102250 0x7fe028106720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:59.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.180+0000 7fe027200700 1 -- 172.21.15.42:0/510853408 >> 172.21.15.42:0/510853408 conn(0x7fe0280fd370 msgr2=0x7fe0280ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:59.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.180+0000 7fe027200700 1 -- 172.21.15.42:0/510853408 shutdown_connections 2024-04-20T10:31:59.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.180+0000 7fe027200700 1 -- 172.21.15.42:0/510853408 wait complete. 2024-04-20T10:31:59.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.181+0000 7fe027200700 1 Processor -- start 2024-04-20T10:31:59.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.181+0000 7fe027200700 1 -- start start 2024-04-20T10:31:59.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe027200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe028101930 0x7fe0281a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:59.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe027200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe028102250 0x7fe0281a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:59.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe027200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe028106d40 0x7fe02819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:59.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe027200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe028119320 con 0x7fe028102250 2024-04-20T10:31:59.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe027200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281191a0 con 0x7fe028106d40 2024-04-20T10:31:59.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe027200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281194a0 con 0x7fe028101930 2024-04-20T10:31:59.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe01fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe028102250 0x7fe0281a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:59.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe01fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe028102250 0x7fe0281a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52754/0 (socket says 172.21.15.42:52754) 2024-04-20T10:31:59.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe01fe00700 1 -- 172.21.15.42:0/789836451 learned_addr learned my addr 172.21.15.42:0/789836451 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:31:59.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe025e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe028101930 0x7fe0281a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:59.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe026800700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe028106d40 0x7fe02819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:59.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe01fe00700 1 -- 172.21.15.42:0/789836451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe028101930 msgr2=0x7fe0281a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:59.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe01fe00700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe028101930 0x7fe0281a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:59.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe01fe00700 1 -- 172.21.15.42:0/789836451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe028106d40 msgr2=0x7fe02819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:59.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe01fe00700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe028106d40 0x7fe02819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:59.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.182+0000 7fe01fe00700 1 -- 172.21.15.42:0/789836451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02819fd00 con 0x7fe028102250 2024-04-20T10:31:59.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.183+0000 7fe01fe00700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe028102250 0x7fe0281a5920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe01000e980 tx=0x7fe01000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:59.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.183+0000 7fe01ea00700 1 -- 172.21.15.42:0/789836451 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010004220 con 0x7fe028102250 2024-04-20T10:31:59.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.183+0000 7fe027200700 1 -- 172.21.15.42:0/789836451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe02819fff0 con 0x7fe028102250 2024-04-20T10:31:59.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.183+0000 7fe01ea00700 1 -- 172.21.15.42:0/789836451 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0100043c0 con 0x7fe028102250 2024-04-20T10:31:59.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.183+0000 7fe01ea00700 1 -- 172.21.15.42:0/789836451 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010010430 con 0x7fe028102250 2024-04-20T10:31:59.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.183+0000 7fe027200700 1 -- 172.21.15.42:0/789836451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0281ac2b0 con 0x7fe028102250 2024-04-20T10:31:59.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.184+0000 7fe027200700 1 -- 172.21.15.42:0/789836451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe028069740 con 0x7fe028102250 2024-04-20T10:31:59.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.186+0000 7fe01ea00700 1 -- 172.21.15.42:0/789836451 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe01000c810 con 0x7fe028102250 2024-04-20T10:31:59.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.188+0000 7fe01ea00700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe00c0a3cd0 0x7fe00c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:59.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.188+0000 7fe01ea00700 1 -- 172.21.15.42:0/789836451 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe0100d0580 con 0x7fe028102250 2024-04-20T10:31:59.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.188+0000 7fe025e00700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe00c0a3cd0 0x7fe00c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:59.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.189+0000 7fe025e00700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe00c0a3cd0 0x7fe00c0a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe01800b2e0 tx=0x7fe018005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:59.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.193+0000 7fe01ea00700 1 -- 172.21.15.42:0/789836451 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe01009ceb0 con 0x7fe028102250 2024-04-20T10:31:59.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.425+0000 7fe027200700 1 -- 172.21.15.42:0/789836451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0281ac540 con 0x7fe028102250 2024-04-20T10:31:59.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.428+0000 7fe01ea00700 1 -- 172.21.15.42:0/789836451 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22197 (secure 0 0 0) 0x7fe01009c7b0 con 0x7fe028102250 2024-04-20T10:31:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:31:59.429 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":85,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:31:36.594768+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.431+0000 7fe027200700 1 -- 172.21.15.42:0/789836451 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe00c0a3cd0 msgr2=0x7fe00c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.431+0000 7fe027200700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe00c0a3cd0 0x7fe00c0a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe01800b2e0 tx=0x7fe018005fd0 comp rx=0 tx=0).stop 2024-04-20T10:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.431+0000 7fe027200700 1 -- 172.21.15.42:0/789836451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe028102250 msgr2=0x7fe0281a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.431+0000 7fe027200700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe028102250 0x7fe0281a5920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe01000e980 tx=0x7fe01000ec90 comp rx=0 tx=0).stop 2024-04-20T10:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.431+0000 7fe027200700 1 -- 172.21.15.42:0/789836451 shutdown_connections 2024-04-20T10:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.431+0000 7fe027200700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe00c0a3cd0 0x7fe00c0a6190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.431+0000 7fe027200700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe028106d40 0x7fe02819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.431+0000 7fe027200700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe028101930 0x7fe0281a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.431+0000 7fe027200700 1 --2- 172.21.15.42:0/789836451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe028102250 0x7fe0281a5920 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.431+0000 7fe027200700 1 -- 172.21.15.42:0/789836451 >> 172.21.15.42:0/789836451 conn(0x7fe0280fd370 msgr2=0x7fe0281046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.431+0000 7fe027200700 1 -- 172.21.15.42:0/789836451 shutdown_connections 2024-04-20T10:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:31:59.431+0000 7fe027200700 1 -- 172.21.15.42:0/789836451 wait complete. 2024-04-20T10:32:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:59 smithi042 ceph-mon[17078]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-20T10:32:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:31:59 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/789836451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:32:00.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:59 smithi162 ceph-mon[21854]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-20T10:32:00.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:31:59 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/789836451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:32:00.278 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-20T10:32:00.279 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-20T10:32:00.279 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-20T10:32:00.279 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-20T10:32:00.280 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-20T10:32:00.280 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-20T10:32:00.280 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-20T10:32:00.281 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-20T10:32:00.281 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-20T10:32:00.282 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-20T10:32:00.282 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-20T10:32:00.283 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-20T10:32:00.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:59 smithi046 ceph-mon[21574]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-20T10:32:00.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:31:59 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/789836451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:32:01.254 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:02.102 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:01 smithi042 ceph-mon[17078]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-20T10:32:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:01 smithi162 ceph-mon[21854]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-20T10:32:02.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:01 smithi046 ceph-mon[21574]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-20T10:32:02.553 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:03.045 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:03.046 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:03.315 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:03.318 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:03.319 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:04.008 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:03 smithi042 ceph-mon[17078]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-20T10:32:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:03 smithi162 ceph-mon[21854]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-20T10:32:04.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.192+0000 7ff16aa00700 1 -- 172.21.15.42:0/3749005419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff164105880 msgr2=0x7ff1641098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:04.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.192+0000 7ff16aa00700 1 --2- 172.21.15.42:0/3749005419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff164105880 0x7ff1641098d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff1500099c0 tx=0x7ff150009cd0 comp rx=0 tx=0).stop 2024-04-20T10:32:04.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.192+0000 7ff16aa00700 1 -- 172.21.15.42:0/3749005419 shutdown_connections 2024-04-20T10:32:04.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.192+0000 7ff16aa00700 1 --2- 172.21.15.42:0/3749005419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164104ed0 0x7ff1641052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:04.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.192+0000 7ff16aa00700 1 --2- 172.21.15.42:0/3749005419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff164109fb0 0x7ff164111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:04.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.192+0000 7ff16aa00700 1 --2- 172.21.15.42:0/3749005419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff164105880 0x7ff1641098d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:04.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.192+0000 7ff16aa00700 1 -- 172.21.15.42:0/3749005419 >> 172.21.15.42:0/3749005419 conn(0x7ff164100760 msgr2=0x7ff164102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:04.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.192+0000 7ff16aa00700 1 -- 172.21.15.42:0/3749005419 shutdown_connections 2024-04-20T10:32:04.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.192+0000 7ff16aa00700 1 -- 172.21.15.42:0/3749005419 wait complete. 2024-04-20T10:32:04.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.194+0000 7ff16aa00700 1 Processor -- start 2024-04-20T10:32:04.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.194+0000 7ff16aa00700 1 -- start start 2024-04-20T10:32:04.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.194+0000 7ff16aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff164104ed0 0x7ff16419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:04.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.194+0000 7ff16aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164105880 0x7ff16419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:04.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.194+0000 7ff16aa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff164109fb0 0x7ff16419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:04.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.194+0000 7ff16aa00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff164114f70 con 0x7ff164104ed0 2024-04-20T10:32:04.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.194+0000 7ff16aa00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff164114df0 con 0x7ff164105880 2024-04-20T10:32:04.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.194+0000 7ff16aa00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641150f0 con 0x7ff164109fb0 2024-04-20T10:32:04.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.195+0000 7ff163e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff164104ed0 0x7ff16419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:04.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.195+0000 7ff163e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff164104ed0 0x7ff16419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52770/0 (socket says 172.21.15.42:52770) 2024-04-20T10:32:04.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.195+0000 7ff163e00700 1 -- 172.21.15.42:0/2625861408 learned_addr learned my addr 172.21.15.42:0/2625861408 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:04.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.195+0000 7ff163400700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164105880 0x7ff16419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:04.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.195+0000 7ff168a00700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff164109fb0 0x7ff16419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:04.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.195+0000 7ff163e00700 1 -- 172.21.15.42:0/2625861408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff164109fb0 msgr2=0x7ff16419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:04.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.195+0000 7ff163e00700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff164109fb0 0x7ff16419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:04.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.195+0000 7ff163e00700 1 -- 172.21.15.42:0/2625861408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164105880 msgr2=0x7ff16419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:04.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.195+0000 7ff163e00700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164105880 0x7ff16419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:04.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.195+0000 7ff163e00700 1 -- 172.21.15.42:0/2625861408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1641a03f0 con 0x7ff164104ed0 2024-04-20T10:32:04.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.195+0000 7ff163400700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164105880 0x7ff16419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:04.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.196+0000 7ff163e00700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff164104ed0 0x7ff16419b3e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff15400eb30 tx=0x7ff15400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:04.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.196+0000 7ff160c00700 1 -- 172.21.15.42:0/2625861408 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15400cbd0 con 0x7ff164104ed0 2024-04-20T10:32:04.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.196+0000 7ff160c00700 1 -- 172.21.15.42:0/2625861408 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff154004500 con 0x7ff164104ed0 2024-04-20T10:32:04.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.196+0000 7ff160c00700 1 -- 172.21.15.42:0/2625861408 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154010430 con 0x7ff164104ed0 2024-04-20T10:32:04.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.196+0000 7ff16aa00700 1 -- 172.21.15.42:0/2625861408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1641a06e0 con 0x7ff164104ed0 2024-04-20T10:32:04.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.197+0000 7ff16aa00700 1 -- 172.21.15.42:0/2625861408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1641a7f60 con 0x7ff164104ed0 2024-04-20T10:32:04.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.198+0000 7ff16aa00700 1 -- 172.21.15.42:0/2625861408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff164063160 con 0x7ff164104ed0 2024-04-20T10:32:04.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.200+0000 7ff160c00700 1 -- 172.21.15.42:0/2625861408 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff154003f00 con 0x7ff164104ed0 2024-04-20T10:32:04.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.201+0000 7ff160c00700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff1480a3e20 0x7ff1480a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:04.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.201+0000 7ff160c00700 1 -- 172.21.15.42:0/2625861408 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff1540d05e0 con 0x7ff164104ed0 2024-04-20T10:32:04.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.202+0000 7ff163400700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff1480a3e20 0x7ff1480a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:04.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.202+0000 7ff160c00700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025] conn(0x7ff1480ad600 0x7ff1480afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:04.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.202+0000 7ff160c00700 1 -- 172.21.15.42:0/2625861408 --> [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff15400be50 con 0x7ff1480ad600 2024-04-20T10:32:04.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.202+0000 7ff160c00700 1 -- 172.21.15.42:0/2625861408 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ff1540d7050 con 0x7ff164104ed0 2024-04-20T10:32:04.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.202+0000 7ff168a00700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025] conn(0x7ff1480ad600 0x7ff1480afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:04.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.202+0000 7ff163400700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff1480a3e20 0x7ff1480a62e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff150000c00 tx=0x7ff150005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:04.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.203+0000 7ff168a00700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025] conn(0x7ff1480ad600 0x7ff1480afa20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:04.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.204+0000 7ff160c00700 1 -- 172.21.15.42:0/2625861408 <== osd.1 v2:172.21.15.46:6800/1972645025 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff15400be50 con 0x7ff1480ad600 2024-04-20T10:32:04.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.222+0000 7ff16aa00700 1 -- 172.21.15.42:0/2625861408 --> [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff164063630 con 0x7ff1480ad600 2024-04-20T10:32:04.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.223+0000 7ff160c00700 1 -- 172.21.15.42:0/2625861408 <== osd.1 v2:172.21.15.46:6800/1972645025 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff164063630 con 0x7ff1480ad600 2024-04-20T10:32:04.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.224+0000 7ff16aa00700 1 -- 172.21.15.42:0/2625861408 >> [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025] conn(0x7ff1480ad600 msgr2=0x7ff1480afa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:04.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.224+0000 7ff16aa00700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025] conn(0x7ff1480ad600 0x7ff1480afa20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:04.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.224+0000 7ff16aa00700 1 -- 172.21.15.42:0/2625861408 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff1480a3e20 msgr2=0x7ff1480a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:04.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.224+0000 7ff16aa00700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff1480a3e20 0x7ff1480a62e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff150000c00 tx=0x7ff150005fb0 comp rx=0 tx=0).stop 2024-04-20T10:32:04.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.224+0000 7ff16aa00700 1 -- 172.21.15.42:0/2625861408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff164104ed0 msgr2=0x7ff16419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:04.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.224+0000 7ff16aa00700 1 --2- 172.21.15.42:0/2625861408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff164104ed0 0x7ff16419b3e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff15400eb30 tx=0x7ff15400ee40 comp rx=0 tx=0).stop 2024-04-20T10:32:04.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.224+0000 7ff168a00700 1 -- 172.21.15.42:0/2625861408 reap_dead start 2024-04-20T10:32:04.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.224+0000 7ff16aa00700 1 -- 172.21.15.42:0/2625861408 shutdown_connections 2024-04-20T10:32:04.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.224+0000 7ff16aa00700 1 -- 172.21.15.42:0/2625861408 >> 172.21.15.42:0/2625861408 conn(0x7ff164100760 msgr2=0x7ff164101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:04.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.224+0000 7ff16aa00700 1 -- 172.21.15.42:0/2625861408 shutdown_connections 2024-04-20T10:32:04.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:04.224+0000 7ff16aa00700 1 -- 172.21.15.42:0/2625861408 wait complete. 2024-04-20T10:32:04.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:03 smithi046 ceph-mon[21574]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-20T10:32:05.362 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:05 smithi162 ceph-mon[21854]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T10:32:06.164 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:05 smithi042 ceph-mon[17078]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T10:32:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:05 smithi046 ceph-mon[21574]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T10:32:06.900 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:07 smithi162 ceph-mon[21854]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:32:08.174 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:07 smithi042 ceph-mon[17078]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:32:08.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:07 smithi046 ceph-mon[21574]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:32:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:09 smithi042 ceph-mon[17078]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T10:32:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:09 smithi162 ceph-mon[21854]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T10:32:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:09 smithi046 ceph-mon[21574]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T10:32:11.961 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:11 smithi042 ceph-mon[17078]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T10:32:12.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.061+0000 7f29ad400700 1 -- 172.21.15.42:0/3189375962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29a8104ed0 msgr2=0x7f29a81052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:12.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.061+0000 7f29ad400700 1 --2- 172.21.15.42:0/3189375962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29a8104ed0 0x7f29a81052b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2998009920 tx=0x7f2998009c30 comp rx=0 tx=0).stop 2024-04-20T10:32:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.062+0000 7f29ad400700 1 -- 172.21.15.42:0/3189375962 shutdown_connections 2024-04-20T10:32:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.062+0000 7f29ad400700 1 --2- 172.21.15.42:0/3189375962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29a8109fb0 0x7f29a8111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.062+0000 7f29ad400700 1 --2- 172.21.15.42:0/3189375962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29a8104ed0 0x7f29a81052b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.062+0000 7f29ad400700 1 --2- 172.21.15.42:0/3189375962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a8105880 0x7f29a81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.062+0000 7f29ad400700 1 -- 172.21.15.42:0/3189375962 >> 172.21.15.42:0/3189375962 conn(0x7f29a8100760 msgr2=0x7f29a8102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.062+0000 7f29ad400700 1 -- 172.21.15.42:0/3189375962 shutdown_connections 2024-04-20T10:32:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.062+0000 7f29ad400700 1 -- 172.21.15.42:0/3189375962 wait complete. 2024-04-20T10:32:12.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.063+0000 7f29ad400700 1 Processor -- start 2024-04-20T10:32:12.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.063+0000 7f29ad400700 1 -- start start 2024-04-20T10:32:12.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.064+0000 7f29ad400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29a8104ed0 0x7f29a819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:12.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.064+0000 7f29ad400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a8105880 0x7f29a819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:12.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.064+0000 7f29ad400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29a8109fb0 0x7f29a819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.064+0000 7f29ad400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8114ff0 con 0x7f29a8105880 2024-04-20T10:32:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.064+0000 7f29ad400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8114e70 con 0x7f29a8104ed0 2024-04-20T10:32:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.064+0000 7f29ad400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8115170 con 0x7f29a8109fb0 2024-04-20T10:32:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.064+0000 7f29a5e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a8105880 0x7f29a819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.064+0000 7f29a5e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a8105880 0x7f29a819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54542/0 (socket says 172.21.15.42:54542) 2024-04-20T10:32:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.064+0000 7f29a5e00700 1 -- 172.21.15.42:0/694198742 learned_addr learned my addr 172.21.15.42:0/694198742 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.064+0000 7f29a7200700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29a8109fb0 0x7f29a819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.064+0000 7f29a6800700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29a8104ed0 0x7f29a819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.065+0000 7f29a5e00700 1 -- 172.21.15.42:0/694198742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29a8109fb0 msgr2=0x7f29a819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.065+0000 7f29a5e00700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29a8109fb0 0x7f29a819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.065+0000 7f29a5e00700 1 -- 172.21.15.42:0/694198742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29a8104ed0 msgr2=0x7f29a819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.065+0000 7f29a5e00700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29a8104ed0 0x7f29a819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.065+0000 7f29a5e00700 1 -- 172.21.15.42:0/694198742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a81a04f0 con 0x7f29a8105880 2024-04-20T10:32:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.065+0000 7f29a5e00700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a8105880 0x7f29a819ba20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f299400ec70 tx=0x7f299400ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.066+0000 7f299f400700 1 -- 172.21.15.42:0/694198742 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299400cc90 con 0x7f29a8105880 2024-04-20T10:32:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.066+0000 7f299f400700 1 -- 172.21.15.42:0/694198742 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f299400ce30 con 0x7f29a8105880 2024-04-20T10:32:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.066+0000 7f29ad400700 1 -- 172.21.15.42:0/694198742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a81a07e0 con 0x7f29a8105880 2024-04-20T10:32:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.066+0000 7f299f400700 1 -- 172.21.15.42:0/694198742 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994010470 con 0x7f29a8105880 2024-04-20T10:32:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.066+0000 7f29ad400700 1 -- 172.21.15.42:0/694198742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a81a8060 con 0x7f29a8105880 2024-04-20T10:32:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.067+0000 7f29ad400700 1 -- 172.21.15.42:0/694198742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f29a80631e0 con 0x7f29a8105880 2024-04-20T10:32:12.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.069+0000 7f299f400700 1 -- 172.21.15.42:0/694198742 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2994004000 con 0x7f29a8105880 2024-04-20T10:32:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.070+0000 7f299f400700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f298c0a3d50 0x7f298c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.071+0000 7f299f400700 1 -- 172.21.15.42:0/694198742 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f29940d09f0 con 0x7f29a8105880 2024-04-20T10:32:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.071+0000 7f29a6800700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f298c0a3d50 0x7f298c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.071+0000 7f299f400700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467] conn(0x7f298c0ad530 0x7f298c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.071+0000 7f299f400700 1 -- 172.21.15.42:0/694198742 --> [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f298c0b0000 con 0x7f298c0ad530 2024-04-20T10:32:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.071+0000 7f299f400700 1 -- 172.21.15.42:0/694198742 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f29940d0da0 con 0x7f29a8105880 2024-04-20T10:32:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.072+0000 7f29a7200700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467] conn(0x7f298c0ad530 0x7f298c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.072+0000 7f29a6800700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f298c0a3d50 0x7f298c0a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f29980098f0 tx=0x7f2998019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:12.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.072+0000 7f29a7200700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467] conn(0x7f298c0ad530 0x7f298c0af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:12.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.073+0000 7f299f400700 1 -- 172.21.15.42:0/694198742 <== osd.3 v2:172.21.15.42:6810/2352652467 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f298c0b0000 con 0x7f298c0ad530 2024-04-20T10:32:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.093+0000 7f29ad400700 1 -- 172.21.15.42:0/694198742 --> [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f29a80635e0 con 0x7f298c0ad530 2024-04-20T10:32:12.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.093+0000 7f299f400700 1 -- 172.21.15.42:0/694198742 <== osd.3 v2:172.21.15.42:6810/2352652467 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f29a80635e0 con 0x7f298c0ad530 2024-04-20T10:32:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.094+0000 7f29ad400700 1 -- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467] conn(0x7f298c0ad530 msgr2=0x7f298c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.094+0000 7f29ad400700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467] conn(0x7f298c0ad530 0x7f298c0af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.094+0000 7f29ad400700 1 -- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f298c0a3d50 msgr2=0x7f298c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.094+0000 7f29ad400700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f298c0a3d50 0x7f298c0a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f29980098f0 tx=0x7f2998019040 comp rx=0 tx=0).stop 2024-04-20T10:32:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.094+0000 7f29ad400700 1 -- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a8105880 msgr2=0x7f29a819ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.094+0000 7f29ad400700 1 --2- 172.21.15.42:0/694198742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29a8105880 0x7f29a819ba20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f299400ec70 tx=0x7f299400ef80 comp rx=0 tx=0).stop 2024-04-20T10:32:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.094+0000 7f29a7200700 1 -- 172.21.15.42:0/694198742 reap_dead start 2024-04-20T10:32:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.095+0000 7f29ad400700 1 -- 172.21.15.42:0/694198742 shutdown_connections 2024-04-20T10:32:12.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.095+0000 7f29ad400700 1 -- 172.21.15.42:0/694198742 >> 172.21.15.42:0/694198742 conn(0x7f29a8100760 msgr2=0x7f29a8102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:12.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.095+0000 7f29ad400700 1 -- 172.21.15.42:0/694198742 shutdown_connections 2024-04-20T10:32:12.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.095+0000 7f29ad400700 1 -- 172.21.15.42:0/694198742 wait complete. 2024-04-20T10:32:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:11 smithi162 ceph-mon[21854]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T10:32:12.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.243+0000 7f5130200700 1 -- 172.21.15.42:0/4183819754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5128109c90 msgr2=0x7f512810dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:12.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.243+0000 7f5130200700 1 --2- 172.21.15.42:0/4183819754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5128109c90 0x7f512810dce0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51140099c0 tx=0x7f5114009cd0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.243+0000 7f5130200700 1 -- 172.21.15.42:0/4183819754 shutdown_connections 2024-04-20T10:32:12.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.243+0000 7f5130200700 1 --2- 172.21.15.42:0/4183819754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51281092e0 0x7f51281096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.243+0000 7f5130200700 1 --2- 172.21.15.42:0/4183819754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f512810e3c0 0x7f5128115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.243+0000 7f5130200700 1 --2- 172.21.15.42:0/4183819754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5128109c90 0x7f512810dce0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.243+0000 7f5130200700 1 -- 172.21.15.42:0/4183819754 >> 172.21.15.42:0/4183819754 conn(0x7f5128076220 msgr2=0x7f5128107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:12.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.243+0000 7f5130200700 1 -- 172.21.15.42:0/4183819754 shutdown_connections 2024-04-20T10:32:12.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.244+0000 7f5130200700 1 -- 172.21.15.42:0/4183819754 wait complete. 2024-04-20T10:32:12.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.245+0000 7f5130200700 1 Processor -- start 2024-04-20T10:32:12.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.245+0000 7f5130200700 1 -- start start 2024-04-20T10:32:12.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.246+0000 7f5130200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51281092e0 0x7f51281a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:12.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.246+0000 7f5130200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5128109c90 0x7f51281a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:12.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.246+0000 7f5130200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512810e3c0 0x7f512819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:12.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.246+0000 7f5130200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5128119380 con 0x7f5128109c90 2024-04-20T10:32:12.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.246+0000 7f5130200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5128119200 con 0x7f512810e3c0 2024-04-20T10:32:12.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.246+0000 7f5130200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5128119500 con 0x7f51281092e0 2024-04-20T10:32:12.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.246+0000 7f512d800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51281092e0 0x7f51281a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:12.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.246+0000 7f512ce00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5128109c90 0x7f51281a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:12.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.246+0000 7f512d800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51281092e0 0x7f51281a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:39258/0 (socket says 172.21.15.42:39258) 2024-04-20T10:32:12.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.246+0000 7f512d800700 1 -- 172.21.15.42:0/1917777686 learned_addr learned my addr 172.21.15.42:0/1917777686 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:12.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.246+0000 7f512e200700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512810e3c0 0x7f512819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:12.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.247+0000 7f512d800700 1 -- 172.21.15.42:0/1917777686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512810e3c0 msgr2=0x7f512819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:12.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.247+0000 7f512d800700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512810e3c0 0x7f512819f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.247+0000 7f512d800700 1 -- 172.21.15.42:0/1917777686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5128109c90 msgr2=0x7f51281a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:12.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.247+0000 7f512d800700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5128109c90 0x7f51281a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.247+0000 7f512d800700 1 -- 172.21.15.42:0/1917777686 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512819fd80 con 0x7f51281092e0 2024-04-20T10:32:12.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.247+0000 7f512ce00700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5128109c90 0x7f51281a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:12.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.247+0000 7f512d800700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51281092e0 0x7f51281a5460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f511c00eb30 tx=0x7f511c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:12.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.247+0000 7f5126000700 1 -- 172.21.15.42:0/1917777686 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c00cbd0 con 0x7f51281092e0 2024-04-20T10:32:12.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.247+0000 7f5130200700 1 -- 172.21.15.42:0/1917777686 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51281a0070 con 0x7f51281092e0 2024-04-20T10:32:12.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.247+0000 7f5126000700 1 -- 172.21.15.42:0/1917777686 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f511c004500 con 0x7f51281092e0 2024-04-20T10:32:12.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.247+0000 7f5126000700 1 -- 172.21.15.42:0/1917777686 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c010430 con 0x7f51281092e0 2024-04-20T10:32:12.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.248+0000 7f5130200700 1 -- 172.21.15.42:0/1917777686 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51281ac330 con 0x7f51281092e0 2024-04-20T10:32:12.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.248+0000 7f5130200700 1 -- 172.21.15.42:0/1917777686 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f51280631e0 con 0x7f51281092e0 2024-04-20T10:32:12.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.251+0000 7f5126000700 1 -- 172.21.15.42:0/1917777686 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f511c003f00 con 0x7f51281092e0 2024-04-20T10:32:12.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.252+0000 7f5126000700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f51100a3cd0 0x7f51100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:12.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.252+0000 7f5126000700 1 -- 172.21.15.42:0/1917777686 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f511c0d0ad0 con 0x7f51281092e0 2024-04-20T10:32:12.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.253+0000 7f512ce00700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f51100a3cd0 0x7f51100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:12.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.253+0000 7f5126000700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096] conn(0x7f51100ad3e0 0x7f51100af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:12.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.253+0000 7f5126000700 1 -- 172.21.15.42:0/1917777686 --> [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f511c023e50 con 0x7f51100ad3e0 2024-04-20T10:32:12.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.253+0000 7f5126000700 1 -- 172.21.15.42:0/1917777686 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f511c0d0e80 con 0x7f51281092e0 2024-04-20T10:32:12.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.253+0000 7f512e200700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096] conn(0x7f51100ad3e0 0x7f51100af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:12.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.253+0000 7f512ce00700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f51100a3cd0 0x7f51100a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5114000c00 tx=0x7f5114005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:12.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.254+0000 7f512e200700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096] conn(0x7f51100ad3e0 0x7f51100af800 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:12.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.255+0000 7f5126000700 1 -- 172.21.15.42:0/1917777686 <== osd.5 v2:172.21.15.162:6808/2005776096 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f511c023e50 con 0x7f51100ad3e0 2024-04-20T10:32:12.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.273+0000 7f5130200700 1 -- 172.21.15.42:0/1917777686 --> [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f51280635e0 con 0x7f51100ad3e0 2024-04-20T10:32:12.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.273+0000 7f5126000700 1 -- 172.21.15.42:0/1917777686 <== osd.5 v2:172.21.15.162:6808/2005776096 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f51280635e0 con 0x7f51100ad3e0 2024-04-20T10:32:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.274+0000 7f5130200700 1 -- 172.21.15.42:0/1917777686 >> [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096] conn(0x7f51100ad3e0 msgr2=0x7f51100af800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.274+0000 7f5130200700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096] conn(0x7f51100ad3e0 0x7f51100af800 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.274+0000 7f5130200700 1 -- 172.21.15.42:0/1917777686 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f51100a3cd0 msgr2=0x7f51100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.274+0000 7f5130200700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f51100a3cd0 0x7f51100a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5114000c00 tx=0x7f5114005fb0 comp rx=0 tx=0).stop 2024-04-20T10:32:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.274+0000 7f5130200700 1 -- 172.21.15.42:0/1917777686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51281092e0 msgr2=0x7f51281a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.274+0000 7f5130200700 1 --2- 172.21.15.42:0/1917777686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51281092e0 0x7f51281a5460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f511c00eb30 tx=0x7f511c00ee40 comp rx=0 tx=0).stop 2024-04-20T10:32:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.274+0000 7f512e200700 1 -- 172.21.15.42:0/1917777686 reap_dead start 2024-04-20T10:32:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.274+0000 7f5130200700 1 -- 172.21.15.42:0/1917777686 shutdown_connections 2024-04-20T10:32:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.274+0000 7f5130200700 1 -- 172.21.15.42:0/1917777686 >> 172.21.15.42:0/1917777686 conn(0x7f5128076220 msgr2=0x7f51280767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.274+0000 7f5130200700 1 -- 172.21.15.42:0/1917777686 shutdown_connections 2024-04-20T10:32:12.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:12.274+0000 7f5130200700 1 -- 172.21.15.42:0/1917777686 wait complete. 2024-04-20T10:32:12.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:11 smithi046 ceph-mon[21574]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T10:32:13.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.379+0000 7f0833600700 1 -- 172.21.15.42:0/2137504453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c10f450 msgr2=0x7f082c111840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:13.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.379+0000 7f0833600700 1 --2- 172.21.15.42:0/2137504453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c10f450 0x7f082c111840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0824009990 tx=0x7f0824009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:13.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.379+0000 7f0833600700 1 -- 172.21.15.42:0/2137504453 shutdown_connections 2024-04-20T10:32:13.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.379+0000 7f0833600700 1 --2- 172.21.15.42:0/2137504453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c10f450 0x7f082c111840 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:13.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.379+0000 7f0833600700 1 --2- 172.21.15.42:0/2137504453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f082c074c80 0x7f082c075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:13.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.379+0000 7f0833600700 1 --2- 172.21.15.42:0/2137504453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f082c0755a0 0x7f082c10ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:13.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.379+0000 7f0833600700 1 -- 172.21.15.42:0/2137504453 >> 172.21.15.42:0/2137504453 conn(0x7f082c079bb0 msgr2=0x7f082c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:13.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.379+0000 7f0833600700 1 -- 172.21.15.42:0/2137504453 shutdown_connections 2024-04-20T10:32:13.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.379+0000 7f0833600700 1 -- 172.21.15.42:0/2137504453 wait complete. 2024-04-20T10:32:13.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.381+0000 7f0833600700 1 Processor -- start 2024-04-20T10:32:13.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.381+0000 7f0833600700 1 -- start start 2024-04-20T10:32:13.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0833600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f082c074c80 0x7f082c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:13.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0833600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0755a0 0x7f082c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:13.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0833600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f082c10f450 0x7f082c19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:13.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0833600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f082c114f40 con 0x7f082c074c80 2024-04-20T10:32:13.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0833600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f082c114dc0 con 0x7f082c0755a0 2024-04-20T10:32:13.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0833600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f082c1150c0 con 0x7f082c10f450 2024-04-20T10:32:13.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0830c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f082c074c80 0x7f082c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:13.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0830c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f082c074c80 0x7f082c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54512/0 (socket says 172.21.15.42:54512) 2024-04-20T10:32:13.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0830c00700 1 -- 172.21.15.42:0/1680874522 learned_addr learned my addr 172.21.15.42:0/1680874522 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:13.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0831600700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f082c10f450 0x7f082c19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:13.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f082be00700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0755a0 0x7f082c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:13.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0830c00700 1 -- 172.21.15.42:0/1680874522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f082c10f450 msgr2=0x7f082c19fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:13.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0830c00700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f082c10f450 0x7f082c19fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:13.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0830c00700 1 -- 172.21.15.42:0/1680874522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0755a0 msgr2=0x7f082c19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:13.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0830c00700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f082c0755a0 0x7f082c19b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:13.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.382+0000 7f0830c00700 1 -- 172.21.15.42:0/1680874522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c1a0370 con 0x7f082c074c80 2024-04-20T10:32:13.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.383+0000 7f0830c00700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f082c074c80 0x7f082c19b3b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f082000c370 tx=0x7f082000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:13.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.383+0000 7f0829600700 1 -- 172.21.15.42:0/1680874522 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08200092e0 con 0x7f082c074c80 2024-04-20T10:32:13.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.383+0000 7f0829600700 1 -- 172.21.15.42:0/1680874522 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f082000cbe0 con 0x7f082c074c80 2024-04-20T10:32:13.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.383+0000 7f0833600700 1 -- 172.21.15.42:0/1680874522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082c1a0600 con 0x7f082c074c80 2024-04-20T10:32:13.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.383+0000 7f0829600700 1 -- 172.21.15.42:0/1680874522 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08200047e0 con 0x7f082c074c80 2024-04-20T10:32:13.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.383+0000 7f0833600700 1 -- 172.21.15.42:0/1680874522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082c1a7ee0 con 0x7f082c074c80 2024-04-20T10:32:13.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.384+0000 7f0833600700 1 -- 172.21.15.42:0/1680874522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f082c1005a0 con 0x7f082c074c80 2024-04-20T10:32:13.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.386+0000 7f0829600700 1 -- 172.21.15.42:0/1680874522 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f082001a020 con 0x7f082c074c80 2024-04-20T10:32:13.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.388+0000 7f0829600700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f08140a3cd0 0x7f08140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:13.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.388+0000 7f0829600700 1 -- 172.21.15.42:0/1680874522 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f08200d08e0 con 0x7f082c074c80 2024-04-20T10:32:13.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.388+0000 7f082be00700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f08140a3cd0 0x7f08140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:13.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.388+0000 7f0829600700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758] conn(0x7f08140ad4b0 0x7f08140af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:13.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.388+0000 7f0829600700 1 -- 172.21.15.42:0/1680874522 --> [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0820025ea0 con 0x7f08140ad4b0 2024-04-20T10:32:13.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.388+0000 7f0829600700 1 -- 172.21.15.42:0/1680874522 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f08200d0c90 con 0x7f082c074c80 2024-04-20T10:32:13.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.389+0000 7f0831600700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758] conn(0x7f08140ad4b0 0x7f08140af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:13.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.389+0000 7f082be00700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f08140a3cd0 0x7f08140a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f082c1110e0 tx=0x7f0818009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:13.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.389+0000 7f0831600700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758] conn(0x7f08140ad4b0 0x7f08140af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:13.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.390+0000 7f0829600700 1 -- 172.21.15.42:0/1680874522 <== osd.4 v2:172.21.15.46:6808/4097597758 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0820025ea0 con 0x7f08140ad4b0 2024-04-20T10:32:13.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.408+0000 7f0833600700 1 -- 172.21.15.42:0/1680874522 --> [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f082c076370 con 0x7f08140ad4b0 2024-04-20T10:32:13.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.409+0000 7f0829600700 1 -- 172.21.15.42:0/1680874522 <== osd.4 v2:172.21.15.46:6808/4097597758 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f082c076370 con 0x7f08140ad4b0 2024-04-20T10:32:13.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.409+0000 7f0833600700 1 -- 172.21.15.42:0/1680874522 >> [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758] conn(0x7f08140ad4b0 msgr2=0x7f08140af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:13.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.409+0000 7f0833600700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758] conn(0x7f08140ad4b0 0x7f08140af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:13.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.410+0000 7f0833600700 1 -- 172.21.15.42:0/1680874522 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f08140a3cd0 msgr2=0x7f08140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:13.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.410+0000 7f0833600700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f08140a3cd0 0x7f08140a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f082c1110e0 tx=0x7f0818009260 comp rx=0 tx=0).stop 2024-04-20T10:32:13.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.410+0000 7f0833600700 1 -- 172.21.15.42:0/1680874522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f082c074c80 msgr2=0x7f082c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:13.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.410+0000 7f0833600700 1 --2- 172.21.15.42:0/1680874522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f082c074c80 0x7f082c19b3b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f082000c370 tx=0x7f082000c680 comp rx=0 tx=0).stop 2024-04-20T10:32:13.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.410+0000 7f0831600700 1 -- 172.21.15.42:0/1680874522 reap_dead start 2024-04-20T10:32:13.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.410+0000 7f0833600700 1 -- 172.21.15.42:0/1680874522 shutdown_connections 2024-04-20T10:32:13.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.410+0000 7f0833600700 1 -- 172.21.15.42:0/1680874522 >> 172.21.15.42:0/1680874522 conn(0x7f082c079bb0 msgr2=0x7f082c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:13.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.410+0000 7f0833600700 1 -- 172.21.15.42:0/1680874522 shutdown_connections 2024-04-20T10:32:13.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:11.410+0000 7f0833600700 1 -- 172.21.15.42:0/1680874522 wait complete. 2024-04-20T10:32:13.968 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:13 smithi042 ceph-mon[17078]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:32:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:13 smithi162 ceph-mon[21854]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:32:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:13 smithi046 ceph-mon[21574]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:32:14.716 INFO:teuthology.orchestra.run.smithi042.stdout:55834574926 2024-04-20T10:32:14.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-20T10:32:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:15 smithi162 ceph-mon[21854]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:16.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:15 smithi042 ceph-mon[17078]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:16.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:15 smithi046 ceph-mon[21574]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:16.378 INFO:teuthology.orchestra.run.smithi042.stdout:163208757305 2024-04-20T10:32:16.379 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-20T10:32:16.873 INFO:teuthology.orchestra.run.smithi042.stdout:107374182466 2024-04-20T10:32:16.873 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-20T10:32:18.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:17 smithi162 ceph-mon[21854]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:18.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:17 smithi046 ceph-mon[21574]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:18.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:17 smithi042 ceph-mon[17078]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:18.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.713+0000 7fa4d2200700 1 -- 172.21.15.42:0/3669707653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4cc103460 msgr2=0x7fa4cc10f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:18.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.713+0000 7fa4d2200700 1 --2- 172.21.15.42:0/3669707653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4cc103460 0x7fa4cc10f8c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009f90 tx=0x7fa4c400b330 comp rx=0 tx=0).stop 2024-04-20T10:32:18.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.713+0000 7fa4d2200700 1 -- 172.21.15.42:0/3669707653 shutdown_connections 2024-04-20T10:32:18.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.713+0000 7fa4d2200700 1 --2- 172.21.15.42:0/3669707653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4cc103460 0x7fa4cc10f8c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:18.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.713+0000 7fa4d2200700 1 --2- 172.21.15.42:0/3669707653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc108a60 0x7fa4cc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:18.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.713+0000 7fa4d2200700 1 --2- 172.21.15.42:0/3669707653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4cc102aa0 0x7fa4cc102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:18.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.713+0000 7fa4d2200700 1 -- 172.21.15.42:0/3669707653 >> 172.21.15.42:0/3669707653 conn(0x7fa4cc079bb0 msgr2=0x7fa4cc079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:18.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.713+0000 7fa4d2200700 1 -- 172.21.15.42:0/3669707653 shutdown_connections 2024-04-20T10:32:18.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.713+0000 7fa4d2200700 1 -- 172.21.15.42:0/3669707653 wait complete. 2024-04-20T10:32:18.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.715+0000 7fa4d2200700 1 Processor -- start 2024-04-20T10:32:18.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.715+0000 7fa4d2200700 1 -- start start 2024-04-20T10:32:18.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.715+0000 7fa4d2200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4cc102aa0 0x7fa4cc19b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:18.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.715+0000 7fa4d2200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4cc103460 0x7fa4cc19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.715+0000 7fa4d2200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc108a60 0x7fa4cc19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.715+0000 7fa4d2200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc112db0 con 0x7fa4cc103460 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.715+0000 7fa4d2200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc112c30 con 0x7fa4cc102aa0 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.715+0000 7fa4d2200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc112f30 con 0x7fa4cc108a60 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.716+0000 7fa4cb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4cc102aa0 0x7fa4cc19b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.716+0000 7fa4cb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4cc102aa0 0x7fa4cc19b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:35548/0 (socket says 172.21.15.42:35548) 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.716+0000 7fa4cb400700 1 -- 172.21.15.42:0/418728493 learned_addr learned my addr 172.21.15.42:0/418728493 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.716+0000 7fa4caa00700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4cc103460 0x7fa4cc19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.716+0000 7fa4cbe00700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc108a60 0x7fa4cc19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.716+0000 7fa4cb400700 1 -- 172.21.15.42:0/418728493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc108a60 msgr2=0x7fa4cc19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.716+0000 7fa4cb400700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4cc108a60 0x7fa4cc19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.716+0000 7fa4cb400700 1 -- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4cc103460 msgr2=0x7fa4cc19ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.716+0000 7fa4cb400700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4cc103460 0x7fa4cc19ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:18.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.716+0000 7fa4cb400700 1 -- 172.21.15.42:0/418728493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc1a0560 con 0x7fa4cc102aa0 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.716+0000 7fa4caa00700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4cc103460 0x7fa4cc19ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.716+0000 7fa4cb400700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4cc102aa0 0x7fa4cc19b550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00e9f0 tx=0x7fa4bc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.717+0000 7fa4c3e00700 1 -- 172.21.15.42:0/418728493 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc004a30 con 0x7fa4cc102aa0 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.717+0000 7fa4c3e00700 1 -- 172.21.15.42:0/418728493 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4bc00cca0 con 0x7fa4cc102aa0 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.717+0000 7fa4d2200700 1 -- 172.21.15.42:0/418728493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc1a0850 con 0x7fa4cc102aa0 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.717+0000 7fa4c3e00700 1 -- 172.21.15.42:0/418728493 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0056d0 con 0x7fa4cc102aa0 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.717+0000 7fa4d2200700 1 -- 172.21.15.42:0/418728493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc1a80d0 con 0x7fa4cc102aa0 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.718+0000 7fa4d2200700 1 -- 172.21.15.42:0/418728493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa4cc10f890 con 0x7fa4cc102aa0 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.720+0000 7fa4c3e00700 1 -- 172.21.15.42:0/418728493 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa4bc00c7c0 con 0x7fa4cc102aa0 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.722+0000 7fa4c3e00700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa4b00a3e70 0x7fa4b00a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.722+0000 7fa4c3e00700 1 -- 172.21.15.42:0/418728493 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa4bc0d08e0 con 0x7fa4cc102aa0 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.722+0000 7fa4caa00700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa4b00a3e70 0x7fa4b00a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.722+0000 7fa4c3e00700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076] conn(0x7fa4b00ad650 0x7fa4b00afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.722+0000 7fa4c3e00700 1 -- 172.21.15.42:0/418728493 --> [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4bc0056d0 con 0x7fa4b00ad650 2024-04-20T10:32:18.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.722+0000 7fa4c3e00700 1 -- 172.21.15.42:0/418728493 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fa4bc0d52a0 con 0x7fa4cc102aa0 2024-04-20T10:32:18.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.723+0000 7fa4cbe00700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076] conn(0x7fa4b00ad650 0x7fa4b00afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:18.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.723+0000 7fa4caa00700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa4b00a3e70 0x7fa4b00a6330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc07a580 tx=0x7fa4b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:18.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.723+0000 7fa4cbe00700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076] conn(0x7fa4b00ad650 0x7fa4b00afa70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:18.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.724+0000 7fa4c3e00700 1 -- 172.21.15.42:0/418728493 <== osd.9 v2:172.21.15.42:6826/661029076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4bc0056d0 con 0x7fa4b00ad650 2024-04-20T10:32:18.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.743+0000 7fa4d2200700 1 -- 172.21.15.42:0/418728493 --> [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa4cc104190 con 0x7fa4b00ad650 2024-04-20T10:32:18.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.744+0000 7fa4c3e00700 1 -- 172.21.15.42:0/418728493 <== osd.9 v2:172.21.15.42:6826/661029076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa4cc104190 con 0x7fa4b00ad650 2024-04-20T10:32:18.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.744+0000 7fa4d2200700 1 -- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076] conn(0x7fa4b00ad650 msgr2=0x7fa4b00afa70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:18.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.744+0000 7fa4d2200700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076] conn(0x7fa4b00ad650 0x7fa4b00afa70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:18.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.745+0000 7fa4d2200700 1 -- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa4b00a3e70 msgr2=0x7fa4b00a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:18.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.745+0000 7fa4d2200700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa4b00a3e70 0x7fa4b00a6330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc07a580 tx=0x7fa4b8009260 comp rx=0 tx=0).stop 2024-04-20T10:32:18.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.745+0000 7fa4d2200700 1 -- 172.21.15.42:0/418728493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4cc102aa0 msgr2=0x7fa4cc19b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:18.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.745+0000 7fa4d2200700 1 --2- 172.21.15.42:0/418728493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4cc102aa0 0x7fa4cc19b550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00e9f0 tx=0x7fa4bc00ed00 comp rx=0 tx=0).stop 2024-04-20T10:32:18.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.745+0000 7fa4cbe00700 1 -- 172.21.15.42:0/418728493 reap_dead start 2024-04-20T10:32:18.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.745+0000 7fa4d2200700 1 -- 172.21.15.42:0/418728493 shutdown_connections 2024-04-20T10:32:18.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.745+0000 7fa4d2200700 1 -- 172.21.15.42:0/418728493 >> 172.21.15.42:0/418728493 conn(0x7fa4cc079bb0 msgr2=0x7fa4cc10dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:18.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.745+0000 7fa4d2200700 1 -- 172.21.15.42:0/418728493 shutdown_connections 2024-04-20T10:32:18.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:13.745+0000 7fa4d2200700 1 -- 172.21.15.42:0/418728493 wait complete. 2024-04-20T10:32:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:19 smithi162 ceph-mon[21854]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:20.188 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:19 smithi042 ceph-mon[17078]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:20.194 INFO:teuthology.orchestra.run.smithi042.stdout:287762808865 2024-04-20T10:32:20.194 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-20T10:32:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:19 smithi046 ceph-mon[21574]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:20.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.285+0000 7f1e88a00700 1 -- 172.21.15.42:0/377915767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e84074d70 msgr2=0x7f1e84075150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:20.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.285+0000 7f1e88a00700 1 --2- 172.21.15.42:0/377915767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e84074d70 0x7f1e84075150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009990 tx=0x7f1e74009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:20.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.285+0000 7f1e88a00700 1 -- 172.21.15.42:0/377915767 shutdown_connections 2024-04-20T10:32:20.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.285+0000 7f1e88a00700 1 --2- 172.21.15.42:0/377915767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e8410f240 0x7f1e84111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:20.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.285+0000 7f1e88a00700 1 --2- 172.21.15.42:0/377915767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e84074d70 0x7f1e84075150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:20.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.285+0000 7f1e88a00700 1 --2- 172.21.15.42:0/377915767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e84075690 0x7f1e8410ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:20.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.285+0000 7f1e88a00700 1 -- 172.21.15.42:0/377915767 >> 172.21.15.42:0/377915767 conn(0x7f1e840fe4d0 msgr2=0x7f1e841008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:20.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.285+0000 7f1e88a00700 1 -- 172.21.15.42:0/377915767 shutdown_connections 2024-04-20T10:32:20.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.285+0000 7f1e88a00700 1 -- 172.21.15.42:0/377915767 wait complete. 2024-04-20T10:32:20.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.286+0000 7f1e88a00700 1 Processor -- start 2024-04-20T10:32:20.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.287+0000 7f1e88a00700 1 -- start start 2024-04-20T10:32:20.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.287+0000 7f1e88a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e84074d70 0x7f1e8419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.287+0000 7f1e88a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e84075690 0x7f1e8419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.287+0000 7f1e88a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8410f240 0x7f1e8419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.287+0000 7f1e88a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84114ee0 con 0x7f1e84074d70 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.287+0000 7f1e88a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84114d60 con 0x7f1e8410f240 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.287+0000 7f1e88a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84115060 con 0x7f1e84075690 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.287+0000 7f1e83400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e84074d70 0x7f1e8419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.287+0000 7f1e83400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e84074d70 0x7f1e8419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55520/0 (socket says 172.21.15.42:55520) 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.287+0000 7f1e83400700 1 -- 172.21.15.42:0/2380682112 learned_addr learned my addr 172.21.15.42:0/2380682112 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.287+0000 7f1e83e00700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8410f240 0x7f1e8419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.287+0000 7f1e82a00700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e84075690 0x7f1e8419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.288+0000 7f1e83400700 1 -- 172.21.15.42:0/2380682112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e84075690 msgr2=0x7f1e8419b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.288+0000 7f1e83400700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e84075690 0x7f1e8419b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.288+0000 7f1e83400700 1 -- 172.21.15.42:0/2380682112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8410f240 msgr2=0x7f1e8419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.288+0000 7f1e83400700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8410f240 0x7f1e8419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.288+0000 7f1e83400700 1 -- 172.21.15.42:0/2380682112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e841a03d0 con 0x7f1e84074d70 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.288+0000 7f1e83e00700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e8410f240 0x7f1e8419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.288+0000 7f1e83400700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e84074d70 0x7f1e8419b410 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e74000c00 tx=0x7f1e7400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.288+0000 7f1e7be00700 1 -- 172.21.15.42:0/2380682112 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74027070 con 0x7f1e84074d70 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.289+0000 7f1e7be00700 1 -- 172.21.15.42:0/2380682112 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e74003d70 con 0x7f1e84074d70 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.289+0000 7f1e88a00700 1 -- 172.21.15.42:0/2380682112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e841a0660 con 0x7f1e84074d70 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.289+0000 7f1e7be00700 1 -- 172.21.15.42:0/2380682112 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400f460 con 0x7f1e84074d70 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.289+0000 7f1e88a00700 1 -- 172.21.15.42:0/2380682112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e841a7f40 con 0x7f1e84074d70 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.290+0000 7f1e88a00700 1 -- 172.21.15.42:0/2380682112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e840630e0 con 0x7f1e84074d70 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.292+0000 7f1e7be00700 1 -- 172.21.15.42:0/2380682112 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1e7402a070 con 0x7f1e84074d70 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.293+0000 7f1e7be00700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e680a3d50 0x7f1e680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.293+0000 7f1e7be00700 1 -- 172.21.15.42:0/2380682112 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1e740d0fa0 con 0x7f1e84074d70 2024-04-20T10:32:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.294+0000 7f1e82a00700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e680a3d50 0x7f1e680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.294+0000 7f1e7be00700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339] conn(0x7f1e680ad530 0x7f1e680af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.294+0000 7f1e7be00700 1 -- 172.21.15.42:0/2380682112 --> [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e7402be50 con 0x7f1e680ad530 2024-04-20T10:32:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.294+0000 7f1e7be00700 1 -- 172.21.15.42:0/2380682112 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f1e740d8050 con 0x7f1e84074d70 2024-04-20T10:32:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.294+0000 7f1e83e00700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339] conn(0x7f1e680ad530 0x7f1e680af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.294+0000 7f1e82a00700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e680a3d50 0x7f1e680a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1e84101f00 tx=0x7f1e700073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.295+0000 7f1e83e00700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339] conn(0x7f1e680ad530 0x7f1e680af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.296+0000 7f1e7be00700 1 -- 172.21.15.42:0/2380682112 <== osd.11 v2:172.21.15.162:6824/1519508339 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e7402be50 con 0x7f1e680ad530 2024-04-20T10:32:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.334+0000 7f1e88a00700 1 -- 172.21.15.42:0/2380682112 --> [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1e84063510 con 0x7f1e680ad530 2024-04-20T10:32:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.334+0000 7f1e7be00700 1 -- 172.21.15.42:0/2380682112 <== osd.11 v2:172.21.15.162:6824/1519508339 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1e84063510 con 0x7f1e680ad530 2024-04-20T10:32:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.335+0000 7f1e88a00700 1 -- 172.21.15.42:0/2380682112 >> [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339] conn(0x7f1e680ad530 msgr2=0x7f1e680af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.335+0000 7f1e88a00700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339] conn(0x7f1e680ad530 0x7f1e680af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.335+0000 7f1e88a00700 1 -- 172.21.15.42:0/2380682112 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e680a3d50 msgr2=0x7f1e680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.335+0000 7f1e88a00700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f1e680a3d50 0x7f1e680a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1e84101f00 tx=0x7f1e700073d0 comp rx=0 tx=0).stop 2024-04-20T10:32:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.335+0000 7f1e88a00700 1 -- 172.21.15.42:0/2380682112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e84074d70 msgr2=0x7f1e8419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.335+0000 7f1e88a00700 1 --2- 172.21.15.42:0/2380682112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e84074d70 0x7f1e8419b410 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e74000c00 tx=0x7f1e7400ba90 comp rx=0 tx=0).stop 2024-04-20T10:32:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.335+0000 7f1e83e00700 1 -- 172.21.15.42:0/2380682112 reap_dead start 2024-04-20T10:32:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.335+0000 7f1e88a00700 1 -- 172.21.15.42:0/2380682112 shutdown_connections 2024-04-20T10:32:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.335+0000 7f1e88a00700 1 -- 172.21.15.42:0/2380682112 >> 172.21.15.42:0/2380682112 conn(0x7f1e840fe4d0 msgr2=0x7f1e840ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:20.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.336+0000 7f1e88a00700 1 -- 172.21.15.42:0/2380682112 shutdown_connections 2024-04-20T10:32:20.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:20.336+0000 7f1e88a00700 1 -- 172.21.15.42:0/2380682112 wait complete. 2024-04-20T10:32:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.625+0000 7f3dece00700 1 -- 172.21.15.42:0/2513021681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3de81014e0 msgr2=0x7f3de81018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.625+0000 7f3dece00700 1 --2- 172.21.15.42:0/2513021681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3de81014e0 0x7f3de81018c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009990 tx=0x7f3dd8009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.625+0000 7f3dece00700 1 -- 172.21.15.42:0/2513021681 shutdown_connections 2024-04-20T10:32:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.625+0000 7f3dece00700 1 --2- 172.21.15.42:0/2513021681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3de81014e0 0x7f3de81018c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:21.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.625+0000 7f3dece00700 1 --2- 172.21.15.42:0/2513021681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3de8107d50 0x7f3de810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.625+0000 7f3dece00700 1 --2- 172.21.15.42:0/2513021681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de8101e90 0x7f3de8107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.625+0000 7f3dece00700 1 -- 172.21.15.42:0/2513021681 >> 172.21.15.42:0/2513021681 conn(0x7f3de8076c00 msgr2=0x7f3de8077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.625+0000 7f3dece00700 1 -- 172.21.15.42:0/2513021681 shutdown_connections 2024-04-20T10:32:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.625+0000 7f3dece00700 1 -- 172.21.15.42:0/2513021681 wait complete. 2024-04-20T10:32:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.627+0000 7f3dece00700 1 Processor -- start 2024-04-20T10:32:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.627+0000 7f3dece00700 1 -- start start 2024-04-20T10:32:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.627+0000 7f3dece00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de81014e0 0x7f3de819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.627+0000 7f3dece00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3de8101e90 0x7f3de819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:21.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.627+0000 7f3dece00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3de8107d50 0x7f3de819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.627+0000 7f3dece00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3de8112d50 con 0x7f3de8101e90 2024-04-20T10:32:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.627+0000 7f3dece00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3de8112bd0 con 0x7f3de8107d50 2024-04-20T10:32:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.627+0000 7f3dece00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3de8112ed0 con 0x7f3de81014e0 2024-04-20T10:32:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.628+0000 7f3de6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3de8101e90 0x7f3de819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.628+0000 7f3de7400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de81014e0 0x7f3de819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.628+0000 7f3de7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3de8107d50 0x7f3de819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.628+0000 7f3de7400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de81014e0 0x7f3de819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:39314/0 (socket says 172.21.15.42:39314) 2024-04-20T10:32:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.628+0000 7f3de7400700 1 -- 172.21.15.42:0/3100748144 learned_addr learned my addr 172.21.15.42:0/3100748144 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.628+0000 7f3de7400700 1 -- 172.21.15.42:0/3100748144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3de8107d50 msgr2=0x7f3de819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.628+0000 7f3de7400700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3de8107d50 0x7f3de819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.628+0000 7f3de7400700 1 -- 172.21.15.42:0/3100748144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3de8101e90 msgr2=0x7f3de819b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:21.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.628+0000 7f3de7400700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3de8101e90 0x7f3de819b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:21.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.628+0000 7f3de7400700 1 -- 172.21.15.42:0/3100748144 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de81a0330 con 0x7f3de81014e0 2024-04-20T10:32:21.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.628+0000 7f3de6a00700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3de8101e90 0x7f3de819b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:21.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.628+0000 7f3de7400700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de81014e0 0x7f3de819b320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800bae0 tx=0x7f3dd800bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:21.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.629+0000 7f3ddfe00700 1 -- 172.21.15.42:0/3100748144 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8026070 con 0x7f3de81014e0 2024-04-20T10:32:21.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.629+0000 7f3ddfe00700 1 -- 172.21.15.42:0/3100748144 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd8003d70 con 0x7f3de81014e0 2024-04-20T10:32:21.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.629+0000 7f3ddfe00700 1 -- 172.21.15.42:0/3100748144 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800fe50 con 0x7f3de81014e0 2024-04-20T10:32:21.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.629+0000 7f3dece00700 1 -- 172.21.15.42:0/3100748144 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de81a05c0 con 0x7f3de81014e0 2024-04-20T10:32:21.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.629+0000 7f3dece00700 1 -- 172.21.15.42:0/3100748144 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de81a7ea0 con 0x7f3de81014e0 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.630+0000 7f3dece00700 1 -- 172.21.15.42:0/3100748144 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3de81a0a50 con 0x7f3de81014e0 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.632+0000 7f3ddfe00700 1 -- 172.21.15.42:0/3100748144 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3dd8029070 con 0x7f3de81014e0 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.634+0000 7f3ddfe00700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3dcc0a3ef0 0x7f3dcc0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.634+0000 7f3ddfe00700 1 -- 172.21.15.42:0/3100748144 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f3dd80d1870 con 0x7f3de81014e0 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.634+0000 7f3de6a00700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3dcc0a3ef0 0x7f3dcc0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.634+0000 7f3ddfe00700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643] conn(0x7f3dcc0ad5a0 0x7f3dcc0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.634+0000 7f3ddfe00700 1 -- 172.21.15.42:0/3100748144 --> [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3dd800fe50 con 0x7f3dcc0ad5a0 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.634+0000 7f3ddfe00700 1 -- 172.21.15.42:0/3100748144 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f3dd80d1c20 con 0x7f3de81014e0 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.634+0000 7f3de7e00700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643] conn(0x7f3dcc0ad5a0 0x7f3dcc0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.635+0000 7f3de6a00700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3dcc0a3ef0 0x7f3dcc0a63b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3de81093d0 tx=0x7f3dd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.635+0000 7f3de7e00700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643] conn(0x7f3dcc0ad5a0 0x7f3dcc0af9c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.636+0000 7f3ddfe00700 1 -- 172.21.15.42:0/3100748144 <== osd.7 v2:172.21.15.46:6816/2431095643 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3dd800fe50 con 0x7f3dcc0ad5a0 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.654+0000 7f3dece00700 1 -- 172.21.15.42:0/3100748144 --> [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3de810f190 con 0x7f3dcc0ad5a0 2024-04-20T10:32:21.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.656+0000 7f3ddfe00700 1 -- 172.21.15.42:0/3100748144 <== osd.7 v2:172.21.15.46:6816/2431095643 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3de810f190 con 0x7f3dcc0ad5a0 2024-04-20T10:32:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.657+0000 7f3dece00700 1 -- 172.21.15.42:0/3100748144 >> [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643] conn(0x7f3dcc0ad5a0 msgr2=0x7f3dcc0af9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.657+0000 7f3dece00700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643] conn(0x7f3dcc0ad5a0 0x7f3dcc0af9c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.657+0000 7f3dece00700 1 -- 172.21.15.42:0/3100748144 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3dcc0a3ef0 msgr2=0x7f3dcc0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.657+0000 7f3dece00700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3dcc0a3ef0 0x7f3dcc0a63b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3de81093d0 tx=0x7f3dd4009260 comp rx=0 tx=0).stop 2024-04-20T10:32:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.657+0000 7f3dece00700 1 -- 172.21.15.42:0/3100748144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de81014e0 msgr2=0x7f3de819b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.657+0000 7f3dece00700 1 --2- 172.21.15.42:0/3100748144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3de81014e0 0x7f3de819b320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800bae0 tx=0x7f3dd800bb10 comp rx=0 tx=0).stop 2024-04-20T10:32:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.657+0000 7f3de7e00700 1 -- 172.21.15.42:0/3100748144 reap_dead start 2024-04-20T10:32:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.657+0000 7f3dece00700 1 -- 172.21.15.42:0/3100748144 shutdown_connections 2024-04-20T10:32:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.657+0000 7f3dece00700 1 -- 172.21.15.42:0/3100748144 >> 172.21.15.42:0/3100748144 conn(0x7f3de8076c00 msgr2=0x7f3de80ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.657+0000 7f3dece00700 1 -- 172.21.15.42:0/3100748144 shutdown_connections 2024-04-20T10:32:21.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:18.657+0000 7f3dece00700 1 -- 172.21.15.42:0/3100748144 wait complete. 2024-04-20T10:32:21.969 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:21 smithi042 ceph-mon[17078]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:22.035 INFO:teuthology.orchestra.run.smithi042.stdout:219043332142 2024-04-20T10:32:22.036 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-20T10:32:22.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:21 smithi046 ceph-mon[21574]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:22.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:21 smithi162 ceph-mon[21854]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.727+0000 7faf8aa00700 1 -- 172.21.15.42:0/1813609555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf84108a40 msgr2=0x7faf84108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.727+0000 7faf8aa00700 1 --2- 172.21.15.42:0/1813609555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf84108a40 0x7faf84108e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf74009990 tx=0x7faf74009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.727+0000 7faf8aa00700 1 -- 172.21.15.42:0/1813609555 shutdown_connections 2024-04-20T10:32:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.727+0000 7faf8aa00700 1 --2- 172.21.15.42:0/1813609555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf8406b100 0x7faf8410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.727+0000 7faf8aa00700 1 --2- 172.21.15.42:0/1813609555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf84108a40 0x7faf84108e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.727+0000 7faf8aa00700 1 --2- 172.21.15.42:0/1813609555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf8406a760 0x7faf8406abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.727+0000 7faf8aa00700 1 -- 172.21.15.42:0/1813609555 >> 172.21.15.42:0/1813609555 conn(0x7faf84078900 msgr2=0x7faf84078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.727+0000 7faf8aa00700 1 -- 172.21.15.42:0/1813609555 shutdown_connections 2024-04-20T10:32:22.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.727+0000 7faf8aa00700 1 -- 172.21.15.42:0/1813609555 wait complete. 2024-04-20T10:32:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.728+0000 7faf8aa00700 1 Processor -- start 2024-04-20T10:32:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.728+0000 7faf8aa00700 1 -- start start 2024-04-20T10:32:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.729+0000 7faf8aa00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf8406a760 0x7faf8419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.729+0000 7faf8aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf8406b100 0x7faf8419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.729+0000 7faf8aa00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf84108a40 0x7faf8419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:22.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.729+0000 7faf8aa00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faf84112d80 con 0x7faf8406b100 2024-04-20T10:32:22.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.729+0000 7faf8aa00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf84112c00 con 0x7faf84108a40 2024-04-20T10:32:22.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.729+0000 7faf8aa00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7faf84112f00 con 0x7faf8406a760 2024-04-20T10:32:22.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.729+0000 7faf83400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf8406b100 0x7faf8419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.729+0000 7faf88a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf84108a40 0x7faf8419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.729+0000 7faf88a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf84108a40 0x7faf8419fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:35192/0 (socket says 172.21.15.42:35192) 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.729+0000 7faf83400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf8406b100 0x7faf8419b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55534/0 (socket says 172.21.15.42:55534) 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.729+0000 7faf88a00700 1 -- 172.21.15.42:0/3881976130 learned_addr learned my addr 172.21.15.42:0/3881976130 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.729+0000 7faf83e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf8406a760 0x7faf8419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.730+0000 7faf88a00700 1 -- 172.21.15.42:0/3881976130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf8406a760 msgr2=0x7faf8419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.730+0000 7faf88a00700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf8406a760 0x7faf8419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.730+0000 7faf88a00700 1 -- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf8406b100 msgr2=0x7faf8419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.730+0000 7faf88a00700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf8406b100 0x7faf8419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.730+0000 7faf88a00700 1 -- 172.21.15.42:0/3881976130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf841a04a0 con 0x7faf84108a40 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.730+0000 7faf83400700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf8406b100 0x7faf8419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.730+0000 7faf88a00700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf84108a40 0x7faf8419fd60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf7c007ee0 tx=0x7faf7c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.730+0000 7faf80c00700 1 -- 172.21.15.42:0/3881976130 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00dd20 con 0x7faf84108a40 2024-04-20T10:32:22.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.730+0000 7faf80c00700 1 -- 172.21.15.42:0/3881976130 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7c004020 con 0x7faf84108a40 2024-04-20T10:32:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.730+0000 7faf8aa00700 1 -- 172.21.15.42:0/3881976130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf841a0790 con 0x7faf84108a40 2024-04-20T10:32:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.730+0000 7faf80c00700 1 -- 172.21.15.42:0/3881976130 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c015660 con 0x7faf84108a40 2024-04-20T10:32:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.730+0000 7faf8aa00700 1 -- 172.21.15.42:0/3881976130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf841a8010 con 0x7faf84108a40 2024-04-20T10:32:22.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.731+0000 7faf8aa00700 1 -- 172.21.15.42:0/3881976130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf84100220 con 0x7faf84108a40 2024-04-20T10:32:22.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.734+0000 7faf80c00700 1 -- 172.21.15.42:0/3881976130 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faf7c0041d0 con 0x7faf84108a40 2024-04-20T10:32:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.735+0000 7faf80c00700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faf680a3cd0 0x7faf680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.735+0000 7faf80c00700 1 -- 172.21.15.42:0/3881976130 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7faf7c0d0950 con 0x7faf84108a40 2024-04-20T10:32:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.735+0000 7faf83e00700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faf680a3cd0 0x7faf680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.735+0000 7faf80c00700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493] conn(0x7faf680ad4b0 0x7faf680af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.735+0000 7faf80c00700 1 -- 172.21.15.42:0/3881976130 --> [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf680aff80 con 0x7faf680ad4b0 2024-04-20T10:32:22.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.735+0000 7faf80c00700 1 -- 172.21.15.42:0/3881976130 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7faf7c0d0d00 con 0x7faf84108a40 2024-04-20T10:32:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.736+0000 7faf83400700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493] conn(0x7faf680ad4b0 0x7faf680af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.736+0000 7faf83e00700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faf680a3cd0 0x7faf680a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faf74006010 tx=0x7faf7400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:22.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.736+0000 7faf83400700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493] conn(0x7faf680ad4b0 0x7faf680af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:22.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.737+0000 7faf80c00700 1 -- 172.21.15.42:0/3881976130 <== osd.0 v2:172.21.15.42:6802/1817026493 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faf680aff80 con 0x7faf680ad4b0 2024-04-20T10:32:22.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.755+0000 7faf8aa00700 1 -- 172.21.15.42:0/3881976130 --> [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf84104180 con 0x7faf680ad4b0 2024-04-20T10:32:22.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.756+0000 7faf80c00700 1 -- 172.21.15.42:0/3881976130 <== osd.0 v2:172.21.15.42:6802/1817026493 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faf84104180 con 0x7faf680ad4b0 2024-04-20T10:32:22.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.756+0000 7faf8aa00700 1 -- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493] conn(0x7faf680ad4b0 msgr2=0x7faf680af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:22.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.756+0000 7faf8aa00700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493] conn(0x7faf680ad4b0 0x7faf680af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:22.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.757+0000 7faf8aa00700 1 -- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faf680a3cd0 msgr2=0x7faf680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:22.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.757+0000 7faf8aa00700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7faf680a3cd0 0x7faf680a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faf74006010 tx=0x7faf7400b260 comp rx=0 tx=0).stop 2024-04-20T10:32:22.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.757+0000 7faf8aa00700 1 -- 172.21.15.42:0/3881976130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf84108a40 msgr2=0x7faf8419fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:22.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.757+0000 7faf8aa00700 1 --2- 172.21.15.42:0/3881976130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf84108a40 0x7faf8419fd60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf7c007ee0 tx=0x7faf7c00d3b0 comp rx=0 tx=0).stop 2024-04-20T10:32:22.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.757+0000 7faf88a00700 1 -- 172.21.15.42:0/3881976130 reap_dead start 2024-04-20T10:32:22.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.757+0000 7faf8aa00700 1 -- 172.21.15.42:0/3881976130 shutdown_connections 2024-04-20T10:32:22.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.757+0000 7faf8aa00700 1 -- 172.21.15.42:0/3881976130 >> 172.21.15.42:0/3881976130 conn(0x7faf84078900 msgr2=0x7faf840ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:22.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.757+0000 7faf8aa00700 1 -- 172.21.15.42:0/3881976130 shutdown_connections 2024-04-20T10:32:22.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.757+0000 7faf8aa00700 1 -- 172.21.15.42:0/3881976130 wait complete. 2024-04-20T10:32:23.719 INFO:teuthology.orchestra.run.smithi042.stdout:141733920829 2024-04-20T10:32:23.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-20T10:32:23.721 INFO:teuthology.orchestra.run.smithi042.stdout:34359738454 2024-04-20T10:32:23.725 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-20T10:32:24.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.950+0000 7fea59600700 1 -- 172.21.15.42:0/2242693100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea5410e3c0 msgr2=0x7fea54115f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.950+0000 7fea59600700 1 --2- 172.21.15.42:0/2242693100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea5410e3c0 0x7fea54115f90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00b0d0 tx=0x7fea4c00b3e0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.950+0000 7fea59600700 1 -- 172.21.15.42:0/2242693100 shutdown_connections 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.950+0000 7fea59600700 1 --2- 172.21.15.42:0/2242693100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea5410e3c0 0x7fea54115f90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.950+0000 7fea59600700 1 --2- 172.21.15.42:0/2242693100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea541092e0 0x7fea541096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.950+0000 7fea59600700 1 --2- 172.21.15.42:0/2242693100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea54109c90 0x7fea5410dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.950+0000 7fea59600700 1 -- 172.21.15.42:0/2242693100 >> 172.21.15.42:0/2242693100 conn(0x7fea54076220 msgr2=0x7fea54078640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.951+0000 7fea59600700 1 -- 172.21.15.42:0/2242693100 shutdown_connections 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.951+0000 7fea59600700 1 -- 172.21.15.42:0/2242693100 wait complete. 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.952+0000 7fea59600700 1 Processor -- start 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.952+0000 7fea59600700 1 -- start start 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.952+0000 7fea59600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea541092e0 0x7fea541a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.952+0000 7fea59600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea54109c90 0x7fea541a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:24.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.952+0000 7fea59600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea5410e3c0 0x7fea5419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.952+0000 7fea59600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea54119400 con 0x7fea54109c90 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.952+0000 7fea59600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fea54119280 con 0x7fea541092e0 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.952+0000 7fea59600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fea54119580 con 0x7fea5410e3c0 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.952+0000 7fea52a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea541092e0 0x7fea541a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea52000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea54109c90 0x7fea541a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea53400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea5410e3c0 0x7fea5419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea52a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea541092e0 0x7fea541a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:35218/0 (socket says 172.21.15.42:35218) 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea53400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea5410e3c0 0x7fea5419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:41680/0 (socket says 172.21.15.42:41680) 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea52a00700 1 -- 172.21.15.42:0/1882153337 learned_addr learned my addr 172.21.15.42:0/1882153337 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea52000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea54109c90 0x7fea541a5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55542/0 (socket says 172.21.15.42:55542) 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea52a00700 1 -- 172.21.15.42:0/1882153337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea5410e3c0 msgr2=0x7fea5419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea52a00700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea5410e3c0 0x7fea5419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea52a00700 1 -- 172.21.15.42:0/1882153337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea54109c90 msgr2=0x7fea541a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea52a00700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea54109c90 0x7fea541a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea52a00700 1 -- 172.21.15.42:0/1882153337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5419ff20 con 0x7fea541092e0 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea53400700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea5410e3c0 0x7fea5419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.953+0000 7fea52a00700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea541092e0 0x7fea541a5570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fea4400eb30 tx=0x7fea4400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.954+0000 7fea4b400700 1 -- 172.21.15.42:0/1882153337 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4400cbd0 con 0x7fea541092e0 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.954+0000 7fea4b400700 1 -- 172.21.15.42:0/1882153337 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea44004500 con 0x7fea541092e0 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.954+0000 7fea59600700 1 -- 172.21.15.42:0/1882153337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea541a0210 con 0x7fea541092e0 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.954+0000 7fea4b400700 1 -- 172.21.15.42:0/1882153337 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea44010430 con 0x7fea541092e0 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.954+0000 7fea59600700 1 -- 172.21.15.42:0/1882153337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea541ac4f0 con 0x7fea541092e0 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.955+0000 7fea59600700 1 -- 172.21.15.42:0/1882153337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea540631a0 con 0x7fea541092e0 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.957+0000 7fea4b400700 1 -- 172.21.15.42:0/1882153337 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fea44003f00 con 0x7fea541092e0 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.958+0000 7fea4b400700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fea380a3cd0 0x7fea380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.958+0000 7fea4b400700 1 -- 172.21.15.42:0/1882153337 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fea440d0b00 con 0x7fea541092e0 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.959+0000 7fea52000700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fea380a3cd0 0x7fea380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.959+0000 7fea4b400700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617] conn(0x7fea380ad4b0 0x7fea380af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.959+0000 7fea4b400700 1 -- 172.21.15.42:0/1882153337 --> [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea4401ee50 con 0x7fea380ad4b0 2024-04-20T10:32:24.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.959+0000 7fea4b400700 1 -- 172.21.15.42:0/1882153337 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fea440d0eb0 con 0x7fea541092e0 2024-04-20T10:32:24.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.959+0000 7fea53400700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617] conn(0x7fea380ad4b0 0x7fea380af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:24.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.959+0000 7fea52000700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fea380a3cd0 0x7fea380a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea40009680 tx=0x7fea40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:24.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.960+0000 7fea53400700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617] conn(0x7fea380ad4b0 0x7fea380af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:24.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.961+0000 7fea4b400700 1 -- 172.21.15.42:0/1882153337 <== osd.10 v2:172.21.15.46:6824/1339678617 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea4401ee50 con 0x7fea380ad4b0 2024-04-20T10:32:24.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.979+0000 7fea59600700 1 -- 172.21.15.42:0/1882153337 --> [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea54063620 con 0x7fea380ad4b0 2024-04-20T10:32:24.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.980+0000 7fea4b400700 1 -- 172.21.15.42:0/1882153337 <== osd.10 v2:172.21.15.46:6824/1339678617 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea54063620 con 0x7fea380ad4b0 2024-04-20T10:32:24.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.980+0000 7fea59600700 1 -- 172.21.15.42:0/1882153337 >> [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617] conn(0x7fea380ad4b0 msgr2=0x7fea380af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:24.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.980+0000 7fea59600700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617] conn(0x7fea380ad4b0 0x7fea380af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.981+0000 7fea59600700 1 -- 172.21.15.42:0/1882153337 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fea380a3cd0 msgr2=0x7fea380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:24.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.981+0000 7fea59600700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fea380a3cd0 0x7fea380a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea40009680 tx=0x7fea40009260 comp rx=0 tx=0).stop 2024-04-20T10:32:24.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.981+0000 7fea59600700 1 -- 172.21.15.42:0/1882153337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea541092e0 msgr2=0x7fea541a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:24.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.981+0000 7fea59600700 1 --2- 172.21.15.42:0/1882153337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea541092e0 0x7fea541a5570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fea4400eb30 tx=0x7fea4400ee40 comp rx=0 tx=0).stop 2024-04-20T10:32:24.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.981+0000 7fea53400700 1 -- 172.21.15.42:0/1882153337 reap_dead start 2024-04-20T10:32:24.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.981+0000 7fea59600700 1 -- 172.21.15.42:0/1882153337 shutdown_connections 2024-04-20T10:32:24.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.981+0000 7fea59600700 1 -- 172.21.15.42:0/1882153337 >> 172.21.15.42:0/1882153337 conn(0x7fea54076220 msgr2=0x7fea54078570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:24.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.981+0000 7fea59600700 1 -- 172.21.15.42:0/1882153337 shutdown_connections 2024-04-20T10:32:24.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:22.981+0000 7fea59600700 1 -- 172.21.15.42:0/1882153337 wait complete. 2024-04-20T10:32:24.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:23 smithi046 ceph-mon[21574]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:24.307 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:23 smithi042 ceph-mon[17078]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:24.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:23 smithi162 ceph-mon[21854]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:24.454 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:24.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.491+0000 7f4a5da00700 1 -- 172.21.15.42:0/1306047671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a58103460 msgr2=0x7f4a5810f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:24.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.491+0000 7f4a5da00700 1 --2- 172.21.15.42:0/1306047671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a58103460 0x7f4a5810f8c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000b0a0 tx=0x7f4a5000b3b0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.492+0000 7f4a5da00700 1 -- 172.21.15.42:0/1306047671 shutdown_connections 2024-04-20T10:32:24.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.492+0000 7f4a5da00700 1 --2- 172.21.15.42:0/1306047671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a58108a60 0x7f4a58108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.492+0000 7f4a5da00700 1 --2- 172.21.15.42:0/1306047671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a58103460 0x7f4a5810f8c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.492+0000 7f4a5da00700 1 --2- 172.21.15.42:0/1306047671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a58102aa0 0x7f4a58102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.492+0000 7f4a5da00700 1 -- 172.21.15.42:0/1306047671 >> 172.21.15.42:0/1306047671 conn(0x7f4a58079bb0 msgr2=0x7f4a58079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.492+0000 7f4a5da00700 1 -- 172.21.15.42:0/1306047671 shutdown_connections 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.492+0000 7f4a5da00700 1 -- 172.21.15.42:0/1306047671 wait complete. 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.493+0000 7f4a5da00700 1 Processor -- start 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.493+0000 7f4a5da00700 1 -- start start 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.494+0000 7f4a5da00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a58102aa0 0x7f4a581a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.494+0000 7f4a5da00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a58103460 0x7f4a581a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.494+0000 7f4a5da00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a58108a60 0x7f4a5810ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.494+0000 7f4a5da00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58112db0 con 0x7f4a58102aa0 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.494+0000 7f4a5da00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58112c30 con 0x7f4a58108a60 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.494+0000 7f4a5da00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58112f30 con 0x7f4a58103460 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.494+0000 7f4a56a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a58102aa0 0x7f4a581a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.494+0000 7f4a57400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a58108a60 0x7f4a5810ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:24.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.494+0000 7f4a57400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a58108a60 0x7f4a5810ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:35236/0 (socket says 172.21.15.42:35236) 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.494+0000 7f4a57400700 1 -- 172.21.15.42:0/3633487642 learned_addr learned my addr 172.21.15.42:0/3633487642 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.495+0000 7f4a4fe00700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a58103460 0x7f4a581a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.495+0000 7f4a57400700 1 -- 172.21.15.42:0/3633487642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a58103460 msgr2=0x7f4a581a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.495+0000 7f4a57400700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a58103460 0x7f4a581a49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.495+0000 7f4a57400700 1 -- 172.21.15.42:0/3633487642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a58102aa0 msgr2=0x7f4a581a44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.495+0000 7f4a57400700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a58102aa0 0x7f4a581a44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.495+0000 7f4a57400700 1 -- 172.21.15.42:0/3633487642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5810f1b0 con 0x7f4a58108a60 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.495+0000 7f4a57400700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a58108a60 0x7f4a5810ea20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000b070 tx=0x7f4a50004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.496+0000 7f4a4f400700 1 -- 172.21.15.42:0/3633487642 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50021070 con 0x7f4a58108a60 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.496+0000 7f4a4f400700 1 -- 172.21.15.42:0/3633487642 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a50003c00 con 0x7f4a58108a60 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.496+0000 7f4a4f400700 1 -- 172.21.15.42:0/3633487642 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50004740 con 0x7f4a58108a60 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.496+0000 7f4a5da00700 1 -- 172.21.15.42:0/3633487642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5810f440 con 0x7f4a58108a60 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.496+0000 7f4a5da00700 1 -- 172.21.15.42:0/3633487642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a581ac3d0 con 0x7f4a58108a60 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.497+0000 7f4a5da00700 1 -- 172.21.15.42:0/3633487642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a58102000 con 0x7f4a58108a60 2024-04-20T10:32:24.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.500+0000 7f4a4f400700 1 -- 172.21.15.42:0/3633487642 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4a50007980 con 0x7f4a58108a60 2024-04-20T10:32:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.501+0000 7f4a4f400700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4a3c0a3cd0 0x7f4a3c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.502+0000 7f4a4f400700 1 -- 172.21.15.42:0/3633487642 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f4a5000f020 con 0x7f4a58108a60 2024-04-20T10:32:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.502+0000 7f4a4f400700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810] conn(0x7f4a3c0ad4b0 0x7f4a3c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.502+0000 7f4a56a00700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4a3c0a3cd0 0x7f4a3c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.502+0000 7f4a4f400700 1 -- 172.21.15.42:0/3633487642 --> [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a3c0aff80 con 0x7f4a3c0ad4b0 2024-04-20T10:32:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.502+0000 7f4a4f400700 1 -- 172.21.15.42:0/3633487642 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f4a500d1d80 con 0x7f4a58108a60 2024-04-20T10:32:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.502+0000 7f4a4fe00700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810] conn(0x7f4a3c0ad4b0 0x7f4a3c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.503+0000 7f4a56a00700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4a3c0a3cd0 0x7f4a3c0a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009be0 tx=0x7f4a48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.503+0000 7f4a4fe00700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810] conn(0x7f4a3c0ad4b0 0x7f4a3c0af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:24.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.504+0000 7f4a4f400700 1 -- 172.21.15.42:0/3633487642 <== osd.2 v2:172.21.15.162:6800/1877464810 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a3c0aff80 con 0x7f4a3c0ad4b0 2024-04-20T10:32:24.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.526+0000 7f4a5da00700 1 -- 172.21.15.42:0/3633487642 --> [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a58104190 con 0x7f4a3c0ad4b0 2024-04-20T10:32:24.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.527+0000 7f4a4f400700 1 -- 172.21.15.42:0/3633487642 <== osd.2 v2:172.21.15.162:6800/1877464810 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4a58104190 con 0x7f4a3c0ad4b0 2024-04-20T10:32:24.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.527+0000 7f4a5da00700 1 -- 172.21.15.42:0/3633487642 >> [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810] conn(0x7f4a3c0ad4b0 msgr2=0x7f4a3c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:24.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.527+0000 7f4a5da00700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810] conn(0x7f4a3c0ad4b0 0x7f4a3c0af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:24.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.528+0000 7f4a5da00700 1 -- 172.21.15.42:0/3633487642 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4a3c0a3cd0 msgr2=0x7f4a3c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:24.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.528+0000 7f4a5da00700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4a3c0a3cd0 0x7f4a3c0a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009be0 tx=0x7f4a48009260 comp rx=0 tx=0).stop 2024-04-20T10:32:24.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.528+0000 7f4a5da00700 1 -- 172.21.15.42:0/3633487642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a58108a60 msgr2=0x7f4a5810ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:24.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.528+0000 7f4a5da00700 1 --2- 172.21.15.42:0/3633487642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a58108a60 0x7f4a5810ea20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000b070 tx=0x7f4a50004c60 comp rx=0 tx=0).stop 2024-04-20T10:32:24.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.528+0000 7f4a57400700 1 -- 172.21.15.42:0/3633487642 reap_dead start 2024-04-20T10:32:24.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.528+0000 7f4a5da00700 1 -- 172.21.15.42:0/3633487642 shutdown_connections 2024-04-20T10:32:24.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.528+0000 7f4a5da00700 1 -- 172.21.15.42:0/3633487642 >> 172.21.15.42:0/3633487642 conn(0x7f4a58079bb0 msgr2=0x7f4a58101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:24.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.528+0000 7f4a5da00700 1 -- 172.21.15.42:0/3633487642 shutdown_connections 2024-04-20T10:32:24.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:24.528+0000 7f4a5da00700 1 -- 172.21.15.42:0/3633487642 wait complete. 2024-04-20T10:32:25.646 INFO:teuthology.orchestra.run.smithi042.stdout:309237645340 2024-04-20T10:32:25.646 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-20T10:32:25.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.369+0000 7f2bde800700 1 -- 172.21.15.42:0/4068310117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bd8108a60 msgr2=0x7f2bd8108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:25.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.369+0000 7f2bde800700 1 --2- 172.21.15.42:0/4068310117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bd8108a60 0x7f2bd8108e40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009990 tx=0x7f2bc8009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:25.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.369+0000 7f2bde800700 1 -- 172.21.15.42:0/4068310117 shutdown_connections 2024-04-20T10:32:25.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.369+0000 7f2bde800700 1 --2- 172.21.15.42:0/4068310117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd8103460 0x7f2bd810f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:25.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.369+0000 7f2bde800700 1 --2- 172.21.15.42:0/4068310117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bd8108a60 0x7f2bd8108e40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:25.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.369+0000 7f2bde800700 1 --2- 172.21.15.42:0/4068310117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd8102aa0 0x7f2bd8102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:25.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.369+0000 7f2bde800700 1 -- 172.21.15.42:0/4068310117 >> 172.21.15.42:0/4068310117 conn(0x7f2bd8079bb0 msgr2=0x7f2bd8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:25.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.369+0000 7f2bde800700 1 -- 172.21.15.42:0/4068310117 shutdown_connections 2024-04-20T10:32:25.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.369+0000 7f2bde800700 1 -- 172.21.15.42:0/4068310117 wait complete. 2024-04-20T10:32:25.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.370+0000 7f2bde800700 1 Processor -- start 2024-04-20T10:32:25.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.370+0000 7f2bde800700 1 -- start start 2024-04-20T10:32:25.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.371+0000 7f2bde800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bd8102aa0 0x7f2bd8074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:25.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.371+0000 7f2bde800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd8103460 0x7f2bd8075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:25.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.371+0000 7f2bde800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd8108a60 0x7f2bd80757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:25.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.371+0000 7f2bde800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8112d60 con 0x7f2bd8102aa0 2024-04-20T10:32:25.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.371+0000 7f2bde800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8112be0 con 0x7f2bd8103460 2024-04-20T10:32:25.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.371+0000 7f2bde800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8112ee0 con 0x7f2bd8108a60 2024-04-20T10:32:25.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.371+0000 7f2bd7400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bd8102aa0 0x7f2bd8074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:25.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.371+0000 7f2bd7400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bd8102aa0 0x7f2bd8074d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55586/0 (socket says 172.21.15.42:55586) 2024-04-20T10:32:25.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.371+0000 7f2bd7400700 1 -- 172.21.15.42:0/1342885920 learned_addr learned my addr 172.21.15.42:0/1342885920 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:25.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.371+0000 7f2bd7e00700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd8108a60 0x7f2bd80757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:25.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.371+0000 7f2bd6a00700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd8103460 0x7f2bd8075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.372+0000 7f2bd6a00700 1 -- 172.21.15.42:0/1342885920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd8108a60 msgr2=0x7f2bd80757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.372+0000 7f2bd6a00700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd8108a60 0x7f2bd80757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.372+0000 7f2bd6a00700 1 -- 172.21.15.42:0/1342885920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bd8102aa0 msgr2=0x7f2bd8074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.372+0000 7f2bd6a00700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bd8102aa0 0x7f2bd8074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.372+0000 7f2bd6a00700 1 -- 172.21.15.42:0/1342885920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd81a7760 con 0x7f2bd8103460 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.372+0000 7f2bd7400700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2bd8102aa0 0x7f2bd8074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.372+0000 7f2bd6a00700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd8103460 0x7f2bd8075290 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400e9f0 tx=0x7f2bc400edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.373+0000 7f2bcfe00700 1 -- 172.21.15.42:0/1342885920 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400cb70 con 0x7f2bd8103460 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.373+0000 7f2bcfe00700 1 -- 172.21.15.42:0/1342885920 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc400cd10 con 0x7f2bd8103460 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.373+0000 7f2bde800700 1 -- 172.21.15.42:0/1342885920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd81a7a50 con 0x7f2bd8103460 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.373+0000 7f2bde800700 1 -- 172.21.15.42:0/1342885920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81a7fe0 con 0x7f2bd8103460 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.373+0000 7f2bcfe00700 1 -- 172.21.15.42:0/1342885920 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40188b0 con 0x7f2bd8103460 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.374+0000 7f2bde800700 1 -- 172.21.15.42:0/1342885920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2bd8063120 con 0x7f2bd8103460 2024-04-20T10:32:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.378+0000 7f2bcfe00700 1 -- 172.21.15.42:0/1342885920 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2bc401f020 con 0x7f2bd8103460 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.379+0000 7f2bcfe00700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f2bbc0a3d50 0x7f2bbc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.379+0000 7f2bcfe00700 1 -- 172.21.15.42:0/1342885920 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f2bc40d0800 con 0x7f2bd8103460 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.379+0000 7f2bd7400700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f2bbc0a3d50 0x7f2bbc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.380+0000 7f2bcfe00700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377] conn(0x7f2bbc0ad530 0x7f2bbc0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.380+0000 7f2bcfe00700 1 -- 172.21.15.42:0/1342885920 --> [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2bc40057d0 con 0x7f2bbc0ad530 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.380+0000 7f2bcfe00700 1 -- 172.21.15.42:0/1342885920 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f2bc40d0bb0 con 0x7f2bd8103460 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.380+0000 7f2bd7e00700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377] conn(0x7f2bbc0ad530 0x7f2bbc0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.380+0000 7f2bd7400700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f2bbc0a3d50 0x7f2bbc0a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800b2e0 tx=0x7f2bc801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.381+0000 7f2bd7e00700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377] conn(0x7f2bbc0ad530 0x7f2bbc0af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.382+0000 7f2bcfe00700 1 -- 172.21.15.42:0/1342885920 <== osd.8 v2:172.21.15.162:6816/3373880377 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bc40057d0 con 0x7f2bbc0ad530 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.400+0000 7f2bde800700 1 -- 172.21.15.42:0/1342885920 --> [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2bd8063520 con 0x7f2bbc0ad530 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.401+0000 7f2bcfe00700 1 -- 172.21.15.42:0/1342885920 <== osd.8 v2:172.21.15.162:6816/3373880377 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2bd8063520 con 0x7f2bbc0ad530 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.401+0000 7f2bde800700 1 -- 172.21.15.42:0/1342885920 >> [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377] conn(0x7f2bbc0ad530 msgr2=0x7f2bbc0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.401+0000 7f2bde800700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377] conn(0x7f2bbc0ad530 0x7f2bbc0af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.401+0000 7f2bde800700 1 -- 172.21.15.42:0/1342885920 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f2bbc0a3d50 msgr2=0x7f2bbc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.402+0000 7f2bde800700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f2bbc0a3d50 0x7f2bbc0a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800b2e0 tx=0x7f2bc801a040 comp rx=0 tx=0).stop 2024-04-20T10:32:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.402+0000 7f2bde800700 1 -- 172.21.15.42:0/1342885920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd8103460 msgr2=0x7f2bd8075290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.402+0000 7f2bde800700 1 --2- 172.21.15.42:0/1342885920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bd8103460 0x7f2bd8075290 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400e9f0 tx=0x7f2bc400edb0 comp rx=0 tx=0).stop 2024-04-20T10:32:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.402+0000 7f2bd7e00700 1 -- 172.21.15.42:0/1342885920 reap_dead start 2024-04-20T10:32:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.402+0000 7f2bde800700 1 -- 172.21.15.42:0/1342885920 shutdown_connections 2024-04-20T10:32:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.402+0000 7f2bde800700 1 -- 172.21.15.42:0/1342885920 >> 172.21.15.42:0/1342885920 conn(0x7f2bd8079bb0 msgr2=0x7f2bd81016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.402+0000 7f2bde800700 1 -- 172.21.15.42:0/1342885920 shutdown_connections 2024-04-20T10:32:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:25.402+0000 7f2bde800700 1 -- 172.21.15.42:0/1342885920 wait complete. 2024-04-20T10:32:26.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:25 smithi042 ceph-mon[17078]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T10:32:26.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:25 smithi046 ceph-mon[21574]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T10:32:26.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:25 smithi162 ceph-mon[21854]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T10:32:26.769 INFO:teuthology.orchestra.run.smithi042.stdout:253403070505 2024-04-20T10:32:26.770 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-20T10:32:27.104 INFO:teuthology.orchestra.run.smithi042.stdout:77309411404 2024-04-20T10:32:27.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-20T10:32:27.961 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:27.961 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:27.981 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:27.981 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:28.048 INFO:teuthology.orchestra.run.smithi042.stdout:343597383703 2024-04-20T10:32:28.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-20T10:32:28.187 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:27 smithi042 ceph-mon[17078]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:28.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:27 smithi046 ceph-mon[21574]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:28.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:27 smithi162 ceph-mon[21854]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:28.440 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:30.083 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:29 smithi042 ceph-mon[17078]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T10:32:30.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:29 smithi046 ceph-mon[21574]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T10:32:30.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:29 smithi162 ceph-mon[21854]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T10:32:30.427 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:30.859 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:31.189 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-20T10:32:31.189 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-20T10:32:31.189 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-20T10:32:31.189 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-20T10:32:31.189 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-04-20T10:32:31.189 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-20T10:32:31.189 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]: dispatch 2024-04-20T10:32:31.189 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-20T10:32:31.189 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-20T10:32:31.189 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:30 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-20T10:32:31.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-20T10:32:31.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-20T10:32:31.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-20T10:32:31.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-20T10:32:31.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-04-20T10:32:31.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-20T10:32:31.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]: dispatch 2024-04-20T10:32:31.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-20T10:32:31.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-20T10:32:31.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:30 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-20T10:32:31.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-20T10:32:31.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-20T10:32:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-20T10:32:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-20T10:32:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-04-20T10:32:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-20T10:32:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]: dispatch 2024-04-20T10:32:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-20T10:32:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-20T10:32:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:30 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-20T10:32:31.404 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:32.088 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:31 smithi042 ceph-mon[17078]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T10:32:32.088 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-20T10:32:32.088 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-20T10:32:32.088 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-20T10:32:32.089 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-20T10:32:32.089 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-04-20T10:32:32.089 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-20T10:32:32.089 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]': finished 2024-04-20T10:32:32.089 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-20T10:32:32.089 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-20T10:32:32.089 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:31 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-20T10:32:32.089 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:31 smithi042 ceph-mon[17078]: osdmap e86: 12 total, 12 up, 12 in 2024-04-20T10:32:32.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:31 smithi046 ceph-mon[21574]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T10:32:32.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-20T10:32:32.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-20T10:32:32.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-20T10:32:32.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-20T10:32:32.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-04-20T10:32:32.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-20T10:32:32.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]': finished 2024-04-20T10:32:32.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-20T10:32:32.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-20T10:32:32.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:31 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-20T10:32:32.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:31 smithi046 ceph-mon[21574]: osdmap e86: 12 total, 12 up, 12 in 2024-04-20T10:32:32.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:31 smithi162 ceph-mon[21854]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T10:32:32.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-20T10:32:32.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-20T10:32:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-20T10:32:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-20T10:32:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-04-20T10:32:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-20T10:32:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 7]}]': finished 2024-04-20T10:32:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-20T10:32:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-20T10:32:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:31 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-20T10:32:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:31 smithi162 ceph-mon[21854]: osdmap e86: 12 total, 12 up, 12 in 2024-04-20T10:32:32.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.875+0000 7f69e4000700 1 -- 172.21.15.42:0/2984357944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69dc074c80 msgr2=0x7f69dc075060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:32.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.875+0000 7f69e4000700 1 --2- 172.21.15.42:0/2984357944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69dc074c80 0x7f69dc075060 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f69d00098f0 tx=0x7f69d0009c00 comp rx=0 tx=0).stop 2024-04-20T10:32:32.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.875+0000 7f69e4000700 1 -- 172.21.15.42:0/2984357944 shutdown_connections 2024-04-20T10:32:32.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.875+0000 7f69e4000700 1 --2- 172.21.15.42:0/2984357944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69dc074c80 0x7f69dc075060 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:32.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.875+0000 7f69e4000700 1 --2- 172.21.15.42:0/2984357944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69dc10f450 0x7f69dc111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:32.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.875+0000 7f69e4000700 1 --2- 172.21.15.42:0/2984357944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0755a0 0x7f69dc10ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:32.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.875+0000 7f69e4000700 1 -- 172.21.15.42:0/2984357944 >> 172.21.15.42:0/2984357944 conn(0x7f69dc079bb0 msgr2=0x7f69dc079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:32.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.875+0000 7f69e4000700 1 -- 172.21.15.42:0/2984357944 shutdown_connections 2024-04-20T10:32:32.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.875+0000 7f69e4000700 1 -- 172.21.15.42:0/2984357944 wait complete. 2024-04-20T10:32:32.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.876+0000 7f69e4000700 1 Processor -- start 2024-04-20T10:32:32.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.876+0000 7f69e4000700 1 -- start start 2024-04-20T10:32:32.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.877+0000 7f69e4000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69dc074c80 0x7f69dc19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.877+0000 7f69e4000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0755a0 0x7f69dc19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.877+0000 7f69e4000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69dc10f450 0x7f69dc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.877+0000 7f69e4000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc114f80 con 0x7f69dc0755a0 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.877+0000 7f69e4000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc114e00 con 0x7f69dc074c80 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.877+0000 7f69e4000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc115100 con 0x7f69dc10f450 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.877+0000 7f69e1600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69dc074c80 0x7f69dc19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.877+0000 7f69e0c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0755a0 0x7f69dc19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.877+0000 7f69e1600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69dc074c80 0x7f69dc19b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:37452/0 (socket says 172.21.15.42:37452) 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.877+0000 7f69e1600700 1 -- 172.21.15.42:0/3194053574 learned_addr learned my addr 172.21.15.42:0/3194053574 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.877+0000 7f69e2000700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69dc10f450 0x7f69dc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.878+0000 7f69e1600700 1 -- 172.21.15.42:0/3194053574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69dc10f450 msgr2=0x7f69dc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.878+0000 7f69e1600700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69dc10f450 0x7f69dc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.878+0000 7f69e1600700 1 -- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0755a0 msgr2=0x7f69dc19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.878+0000 7f69e1600700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0755a0 0x7f69dc19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.878+0000 7f69e1600700 1 -- 172.21.15.42:0/3194053574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc1a0420 con 0x7f69dc074c80 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.878+0000 7f69e0c00700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0755a0 0x7f69dc19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.878+0000 7f69e1600700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69dc074c80 0x7f69dc19b410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009230 tx=0x7f69d000fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.878+0000 7f69da000700 1 -- 172.21.15.42:0/3194053574 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0005180 con 0x7f69dc074c80 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.878+0000 7f69e4000700 1 -- 172.21.15.42:0/3194053574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69dc1a06b0 con 0x7f69dc074c80 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.878+0000 7f69da000700 1 -- 172.21.15.42:0/3194053574 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d000b560 con 0x7f69dc074c80 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.878+0000 7f69da000700 1 -- 172.21.15.42:0/3194053574 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0003a70 con 0x7f69dc074c80 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.878+0000 7f69e4000700 1 -- 172.21.15.42:0/3194053574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69dc1a7f90 con 0x7f69dc074c80 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.879+0000 7f69e4000700 1 -- 172.21.15.42:0/3194053574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69dc0631a0 con 0x7f69dc074c80 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.882+0000 7f69da000700 1 -- 172.21.15.42:0/3194053574 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f69d0029060 con 0x7f69dc074c80 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.883+0000 7f69da000700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f69c40a3cd0 0x7f69c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.883+0000 7f69da000700 1 -- 172.21.15.42:0/3194053574 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f69d00d1650 con 0x7f69dc074c80 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.883+0000 7f69e0c00700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f69c40a3cd0 0x7f69c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.884+0000 7f69da000700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714] conn(0x7f69c40ad7e0 0x7f69c40afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.884+0000 7f69da000700 1 -- 172.21.15.42:0/3194053574 --> [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69d0010aa0 con 0x7f69c40ad7e0 2024-04-20T10:32:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.884+0000 7f69da000700 1 -- 172.21.15.42:0/3194053574 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f69d00d1a00 con 0x7f69dc074c80 2024-04-20T10:32:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.884+0000 7f69e2000700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714] conn(0x7f69c40ad7e0 0x7f69c40afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.884+0000 7f69e0c00700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f69c40a3cd0 0x7f69c40a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009750 tx=0x7f69c8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.885+0000 7f69e2000700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714] conn(0x7f69c40ad7e0 0x7f69c40afc80 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.886+0000 7f69da000700 1 -- 172.21.15.42:0/3194053574 <== osd.6 v2:172.21.15.42:6818/2702679714 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f69d0010aa0 con 0x7f69c40ad7e0 2024-04-20T10:32:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.905+0000 7f69e4000700 1 -- 172.21.15.42:0/3194053574 --> [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f69dc0635a0 con 0x7f69c40ad7e0 2024-04-20T10:32:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.905+0000 7f69da000700 1 -- 172.21.15.42:0/3194053574 <== osd.6 v2:172.21.15.42:6818/2702679714 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f69dc0635a0 con 0x7f69c40ad7e0 2024-04-20T10:32:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.906+0000 7f69e4000700 1 -- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714] conn(0x7f69c40ad7e0 msgr2=0x7f69c40afc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.906+0000 7f69e4000700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714] conn(0x7f69c40ad7e0 0x7f69c40afc80 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.906+0000 7f69e4000700 1 -- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f69c40a3cd0 msgr2=0x7f69c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.906+0000 7f69e4000700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f69c40a3cd0 0x7f69c40a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009750 tx=0x7f69c8009310 comp rx=0 tx=0).stop 2024-04-20T10:32:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.906+0000 7f69e4000700 1 -- 172.21.15.42:0/3194053574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69dc074c80 msgr2=0x7f69dc19b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.906+0000 7f69e4000700 1 --2- 172.21.15.42:0/3194053574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69dc074c80 0x7f69dc19b410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009230 tx=0x7f69d000fa70 comp rx=0 tx=0).stop 2024-04-20T10:32:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.906+0000 7f69e2000700 1 -- 172.21.15.42:0/3194053574 reap_dead start 2024-04-20T10:32:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.907+0000 7f69e4000700 1 -- 172.21.15.42:0/3194053574 shutdown_connections 2024-04-20T10:32:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.907+0000 7f69e4000700 1 -- 172.21.15.42:0/3194053574 >> 172.21.15.42:0/3194053574 conn(0x7f69dc079bb0 msgr2=0x7f69dc0ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.907+0000 7f69e4000700 1 -- 172.21.15.42:0/3194053574 shutdown_connections 2024-04-20T10:32:32.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:32.907+0000 7f69e4000700 1 -- 172.21.15.42:0/3194053574 wait complete. 2024-04-20T10:32:33.246 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:32 smithi042 ceph-mon[17078]: osdmap e87: 12 total, 12 up, 12 in 2024-04-20T10:32:33.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:32 smithi046 ceph-mon[21574]: osdmap e87: 12 total, 12 up, 12 in 2024-04-20T10:32:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:33.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:32 smithi162 ceph-mon[21854]: osdmap e87: 12 total, 12 up, 12 in 2024-04-20T10:32:34.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:33 smithi042 ceph-mon[17078]: pgmap v348: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T10:32:34.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:33 smithi046 ceph-mon[21574]: pgmap v348: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T10:32:34.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:33 smithi162 ceph-mon[21854]: pgmap v348: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T10:32:36.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:35 smithi042 ceph-mon[17078]: pgmap v349: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:35 smithi046 ceph-mon[21574]: pgmap v349: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.318+0000 7f6d5f600700 1 -- 172.21.15.42:0/188050190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d58102aa0 msgr2=0x7f6d58102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.318+0000 7f6d5f600700 1 --2- 172.21.15.42:0/188050190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d58102aa0 0x7f6d58102f20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009990 tx=0x7f6d40009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.318+0000 7f6d5f600700 1 -- 172.21.15.42:0/188050190 shutdown_connections 2024-04-20T10:32:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.318+0000 7f6d5f600700 1 --2- 172.21.15.42:0/188050190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d58103460 0x7f6d5810f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.318+0000 7f6d5f600700 1 --2- 172.21.15.42:0/188050190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d58108a60 0x7f6d58108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.318+0000 7f6d5f600700 1 --2- 172.21.15.42:0/188050190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d58102aa0 0x7f6d58102f20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.318+0000 7f6d5f600700 1 -- 172.21.15.42:0/188050190 >> 172.21.15.42:0/188050190 conn(0x7f6d58079bb0 msgr2=0x7f6d58079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.318+0000 7f6d5f600700 1 -- 172.21.15.42:0/188050190 shutdown_connections 2024-04-20T10:32:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.318+0000 7f6d5f600700 1 -- 172.21.15.42:0/188050190 wait complete. 2024-04-20T10:32:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.319+0000 7f6d5f600700 1 Processor -- start 2024-04-20T10:32:36.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.319+0000 7f6d5f600700 1 -- start start 2024-04-20T10:32:36.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.320+0000 7f6d5f600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d58102aa0 0x7f6d5819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:36.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.320+0000 7f6d5f600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d58103460 0x7f6d5819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:36.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.320+0000 7f6d5f600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d58108a60 0x7f6d5819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.320+0000 7f6d5f600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58112db0 con 0x7f6d58108a60 2024-04-20T10:32:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.320+0000 7f6d5f600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58112c30 con 0x7f6d58102aa0 2024-04-20T10:32:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.320+0000 7f6d5f600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58112f30 con 0x7f6d58103460 2024-04-20T10:32:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.320+0000 7f6d4fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d58103460 0x7f6d5819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.320+0000 7f6d5cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d58102aa0 0x7f6d5819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.320+0000 7f6d4fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d58103460 0x7f6d5819b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:34696/0 (socket says 172.21.15.42:34696) 2024-04-20T10:32:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.320+0000 7f6d4fe00700 1 -- 172.21.15.42:0/2781333734 learned_addr learned my addr 172.21.15.42:0/2781333734 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.320+0000 7f6d5d600700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d58108a60 0x7f6d5819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.321+0000 7f6d4fe00700 1 -- 172.21.15.42:0/2781333734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d58102aa0 msgr2=0x7f6d5819b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.321+0000 7f6d4fe00700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d58102aa0 0x7f6d5819b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.321+0000 7f6d4fe00700 1 -- 172.21.15.42:0/2781333734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d58108a60 msgr2=0x7f6d5819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.321+0000 7f6d4fe00700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d58108a60 0x7f6d5819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.321+0000 7f6d4fe00700 1 -- 172.21.15.42:0/2781333734 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d581a0440 con 0x7f6d58103460 2024-04-20T10:32:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.321+0000 7f6d4fe00700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d58103460 0x7f6d5819b970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6d40000c00 tx=0x7f6d4000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.321+0000 7f6d56000700 1 -- 172.21.15.42:0/2781333734 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40027070 con 0x7f6d58103460 2024-04-20T10:32:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.322+0000 7f6d56000700 1 -- 172.21.15.42:0/2781333734 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d400056d0 con 0x7f6d58103460 2024-04-20T10:32:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.322+0000 7f6d56000700 1 -- 172.21.15.42:0/2781333734 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40022e10 con 0x7f6d58103460 2024-04-20T10:32:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.322+0000 7f6d5f600700 1 -- 172.21.15.42:0/2781333734 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d581a06d0 con 0x7f6d58103460 2024-04-20T10:32:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.322+0000 7f6d5f600700 1 -- 172.21.15.42:0/2781333734 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d581a7fb0 con 0x7f6d58103460 2024-04-20T10:32:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.323+0000 7f6d5f600700 1 -- 172.21.15.42:0/2781333734 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d58104190 con 0x7f6d58103460 2024-04-20T10:32:36.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.325+0000 7f6d56000700 1 -- 172.21.15.42:0/2781333734 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6d4002a070 con 0x7f6d58103460 2024-04-20T10:32:36.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.326+0000 7f6d56000700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6d3c0a3cd0 0x7f6d3c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:36.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.327+0000 7f6d56000700 1 -- 172.21.15.42:0/2781333734 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f6d400d1880 con 0x7f6d58103460 2024-04-20T10:32:36.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.327+0000 7f6d5cc00700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6d3c0a3cd0 0x7f6d3c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:36.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.327+0000 7f6d5cc00700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6d3c0a3cd0 0x7f6d3c0a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009680 tx=0x7f6d48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:36.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.329+0000 7f6d56000700 1 -- 172.21.15.42:0/2781333734 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6d40013070 con 0x7f6d58103460 2024-04-20T10:32:36.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:35 smithi162 ceph-mon[21854]: pgmap v349: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:36.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.570+0000 7f6d5f600700 1 -- 172.21.15.42:0/2781333734 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6d581a8180 con 0x7f6d58103460 2024-04-20T10:32:36.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.570+0000 7f6d56000700 1 -- 172.21.15.42:0/2781333734 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6d4009e110 con 0x7f6d58103460 2024-04-20T10:32:36.570 INFO:teuthology.orchestra.run.smithi042.stdout:219043332145 2024-04-20T10:32:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.573+0000 7f6d5f600700 1 -- 172.21.15.42:0/2781333734 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6d3c0a3cd0 msgr2=0x7f6d3c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.573+0000 7f6d5f600700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6d3c0a3cd0 0x7f6d3c0a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009680 tx=0x7f6d48009260 comp rx=0 tx=0).stop 2024-04-20T10:32:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.573+0000 7f6d5f600700 1 -- 172.21.15.42:0/2781333734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d58103460 msgr2=0x7f6d5819b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.574+0000 7f6d5f600700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d58103460 0x7f6d5819b970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6d40000c00 tx=0x7f6d4000bab0 comp rx=0 tx=0).stop 2024-04-20T10:32:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.574+0000 7f6d5f600700 1 -- 172.21.15.42:0/2781333734 shutdown_connections 2024-04-20T10:32:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.574+0000 7f6d5f600700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6d3c0a3cd0 0x7f6d3c0a6190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.574+0000 7f6d5f600700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d58102aa0 0x7f6d5819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.574+0000 7f6d5f600700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d58108a60 0x7f6d5819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.574+0000 7f6d5f600700 1 --2- 172.21.15.42:0/2781333734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d58103460 0x7f6d5819b970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.574+0000 7f6d5f600700 1 -- 172.21.15.42:0/2781333734 >> 172.21.15.42:0/2781333734 conn(0x7f6d58079bb0 msgr2=0x7f6d58101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.574+0000 7f6d5f600700 1 -- 172.21.15.42:0/2781333734 shutdown_connections 2024-04-20T10:32:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:36.574+0000 7f6d5f600700 1 -- 172.21.15.42:0/2781333734 wait complete. 2024-04-20T10:32:37.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:36 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2781333734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-20T10:32:37.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:36 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2781333734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-20T10:32:37.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:36 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2781333734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-20T10:32:38.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:37 smithi162 ceph-mon[21854]: pgmap v350: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:38.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:37 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:32:38.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:37 smithi042 ceph-mon[17078]: pgmap v350: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:38.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:37 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:32:38.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:37 smithi046 ceph-mon[21574]: pgmap v350: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:38.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:37 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:32:39.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:39 smithi042 ceph-mon[17078]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:39.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:39 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:32:39.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:39 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:32:39.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:39 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:32:39.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:39 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:32:39.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:39 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/366675327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T10:32:40.021 INFO:teuthology.orchestra.run.smithi042.stdout:197568495670 2024-04-20T10:32:40.022 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-20T10:32:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:39 smithi046 ceph-mon[21574]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:39 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:32:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:39 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:32:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:39 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:32:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:39 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:32:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:39 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/366675327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T10:32:40.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.396+0000 7f89de200700 1 -- 172.21.15.42:0/2219224114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d806a760 msgr2=0x7f89d806abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:40.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.396+0000 7f89de200700 1 --2- 172.21.15.42:0/2219224114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d806a760 0x7f89d806abc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009990 tx=0x7f89c0009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:40.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.396+0000 7f89de200700 1 -- 172.21.15.42:0/2219224114 shutdown_connections 2024-04-20T10:32:40.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.396+0000 7f89de200700 1 --2- 172.21.15.42:0/2219224114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d806b100 0x7f89d810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:40.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.396+0000 7f89de200700 1 --2- 172.21.15.42:0/2219224114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89d8108a40 0x7f89d8108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.396+0000 7f89de200700 1 --2- 172.21.15.42:0/2219224114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d806a760 0x7f89d806abc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.396+0000 7f89de200700 1 -- 172.21.15.42:0/2219224114 >> 172.21.15.42:0/2219224114 conn(0x7f89d8078900 msgr2=0x7f89d8078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.396+0000 7f89de200700 1 -- 172.21.15.42:0/2219224114 shutdown_connections 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.396+0000 7f89de200700 1 -- 172.21.15.42:0/2219224114 wait complete. 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.398+0000 7f89de200700 1 Processor -- start 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.398+0000 7f89de200700 1 -- start start 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.399+0000 7f89de200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89d806a760 0x7f89d819b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.399+0000 7f89de200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d806b100 0x7f89d819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.399+0000 7f89de200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d8108a40 0x7f89d819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.399+0000 7f89de200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f89d8112da0 con 0x7f89d806a760 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.399+0000 7f89de200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f89d8112c20 con 0x7f89d8108a40 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.399+0000 7f89de200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f89d8112f20 con 0x7f89d806b100 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.399+0000 7f89d7400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89d806a760 0x7f89d819b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:40.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.399+0000 7f89cfe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d806b100 0x7f89d819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.399+0000 7f89d7e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d8108a40 0x7f89d819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.399+0000 7f89cfe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d806b100 0x7f89d819ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:53352/0 (socket says 172.21.15.42:53352) 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.399+0000 7f89cfe00700 1 -- 172.21.15.42:0/366675327 learned_addr learned my addr 172.21.15.42:0/366675327 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.399+0000 7f89d7400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89d806a760 0x7f89d819b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51078/0 (socket says 172.21.15.42:51078) 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.400+0000 7f89cfe00700 1 -- 172.21.15.42:0/366675327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d8108a40 msgr2=0x7f89d819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.400+0000 7f89cfe00700 1 --2- 172.21.15.42:0/366675327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d8108a40 0x7f89d819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.400+0000 7f89cfe00700 1 -- 172.21.15.42:0/366675327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89d806a760 msgr2=0x7f89d819b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.400+0000 7f89cfe00700 1 --2- 172.21.15.42:0/366675327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89d806a760 0x7f89d819b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.400+0000 7f89cfe00700 1 -- 172.21.15.42:0/366675327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d81a04e0 con 0x7f89d806b100 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.400+0000 7f89cfe00700 1 --2- 172.21.15.42:0/366675327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d806b100 0x7f89d819ba10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f89c0000c00 tx=0x7f89c000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.400+0000 7f89d4c00700 1 -- 172.21.15.42:0/366675327 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0027070 con 0x7f89d806b100 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.400+0000 7f89d4c00700 1 -- 172.21.15.42:0/366675327 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c00056d0 con 0x7f89d806b100 2024-04-20T10:32:40.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.400+0000 7f89d4c00700 1 -- 172.21.15.42:0/366675327 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0022e10 con 0x7f89d806b100 2024-04-20T10:32:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.400+0000 7f89de200700 1 -- 172.21.15.42:0/366675327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d81a0770 con 0x7f89d806b100 2024-04-20T10:32:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.400+0000 7f89de200700 1 -- 172.21.15.42:0/366675327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d81a8050 con 0x7f89d806b100 2024-04-20T10:32:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.402+0000 7f89de200700 1 -- 172.21.15.42:0/366675327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d8104180 con 0x7f89d806b100 2024-04-20T10:32:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.403+0000 7f89d4c00700 1 -- 172.21.15.42:0/366675327 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f89c002a070 con 0x7f89d806b100 2024-04-20T10:32:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.405+0000 7f89d4c00700 1 --2- 172.21.15.42:0/366675327 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f89bc0a3cd0 0x7f89bc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.405+0000 7f89d4c00700 1 -- 172.21.15.42:0/366675327 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f89c00d1250 con 0x7f89d806b100 2024-04-20T10:32:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.405+0000 7f89d7400700 1 --2- 172.21.15.42:0/366675327 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f89bc0a3cd0 0x7f89bc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:40.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.406+0000 7f89d7400700 1 --2- 172.21.15.42:0/366675327 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f89bc0a3cd0 0x7f89bc0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009be0 tx=0x7f89c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:40.123 INFO:teuthology.orchestra.run.smithi042.stdout:107374182471 2024-04-20T10:32:40.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.409+0000 7f89d4c00700 1 -- 172.21.15.42:0/366675327 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f89c0013070 con 0x7f89d806b100 2024-04-20T10:32:40.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.648+0000 7f89de200700 1 -- 172.21.15.42:0/366675327 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f89d802df90 con 0x7f89d806b100 2024-04-20T10:32:40.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.649+0000 7f89d4c00700 1 -- 172.21.15.42:0/366675327 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f89c009dc00 con 0x7f89d806b100 2024-04-20T10:32:40.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.653+0000 7f89de200700 1 -- 172.21.15.42:0/366675327 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f89bc0a3cd0 msgr2=0x7f89bc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:40.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.653+0000 7f89de200700 1 --2- 172.21.15.42:0/366675327 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f89bc0a3cd0 0x7f89bc0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009be0 tx=0x7f89c8009260 comp rx=0 tx=0).stop 2024-04-20T10:32:40.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.653+0000 7f89de200700 1 -- 172.21.15.42:0/366675327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d806b100 msgr2=0x7f89d819ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:40.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.653+0000 7f89de200700 1 --2- 172.21.15.42:0/366675327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d806b100 0x7f89d819ba10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f89c0000c00 tx=0x7f89c000bab0 comp rx=0 tx=0).stop 2024-04-20T10:32:40.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.653+0000 7f89de200700 1 -- 172.21.15.42:0/366675327 shutdown_connections 2024-04-20T10:32:40.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.653+0000 7f89de200700 1 --2- 172.21.15.42:0/366675327 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f89bc0a3cd0 0x7f89bc0a6190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:40.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.653+0000 7f89de200700 1 --2- 172.21.15.42:0/366675327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d8108a40 0x7f89d819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.653+0000 7f89de200700 1 --2- 172.21.15.42:0/366675327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89d806a760 0x7f89d819b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.653+0000 7f89de200700 1 --2- 172.21.15.42:0/366675327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89d806b100 0x7f89d819ba10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.653+0000 7f89de200700 1 -- 172.21.15.42:0/366675327 >> 172.21.15.42:0/366675327 conn(0x7f89d8078900 msgr2=0x7f89d810df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.653+0000 7f89de200700 1 -- 172.21.15.42:0/366675327 shutdown_connections 2024-04-20T10:32:40.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.653+0000 7f89de200700 1 -- 172.21.15.42:0/366675327 wait complete. 2024-04-20T10:32:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:39 smithi162 ceph-mon[21854]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:39 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:32:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:39 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:32:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:39 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:32:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:39 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:32:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:39 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/366675327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T10:32:41.011 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:40 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/322750345' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-20T10:32:41.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:40 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/322750345' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-20T10:32:41.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:40 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/322750345' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-20T10:32:41.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.905+0000 7ffaf3200700 1 -- 172.21.15.42:0/2100167709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec105490 msgr2=0x7ffaec105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:41.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.905+0000 7ffaf3200700 1 --2- 172.21.15.42:0/2100167709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec105490 0x7ffaec105920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffae4009f90 tx=0x7ffae400b330 comp rx=0 tx=0).stop 2024-04-20T10:32:41.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.906+0000 7ffaf3200700 1 -- 172.21.15.42:0/2100167709 shutdown_connections 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.906+0000 7ffaf3200700 1 --2- 172.21.15.42:0/2100167709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec105490 0x7ffaec105920 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.906+0000 7ffaf3200700 1 --2- 172.21.15.42:0/2100167709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec107980 0x7ffaec06a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.906+0000 7ffaf3200700 1 --2- 172.21.15.42:0/2100167709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec06aca0 0x7ffaec06b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.906+0000 7ffaf3200700 1 -- 172.21.15.42:0/2100167709 >> 172.21.15.42:0/2100167709 conn(0x7ffaec078900 msgr2=0x7ffaec078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.906+0000 7ffaf3200700 1 -- 172.21.15.42:0/2100167709 shutdown_connections 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.906+0000 7ffaf3200700 1 -- 172.21.15.42:0/2100167709 wait complete. 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.907+0000 7ffaf3200700 1 Processor -- start 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.908+0000 7ffaf3200700 1 -- start start 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.908+0000 7ffaf3200700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec06aca0 0x7ffaec1a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.908+0000 7ffaf3200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec105490 0x7ffaec1a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.908+0000 7ffaf3200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec107980 0x7ffaec19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.908+0000 7ffaf3200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec119390 con 0x7ffaec107980 2024-04-20T10:32:41.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.908+0000 7ffaf3200700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec119210 con 0x7ffaec105490 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.908+0000 7ffaf3200700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec119510 con 0x7ffaec06aca0 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.908+0000 7ffaeb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec105490 0x7ffaec1a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.908+0000 7ffaebe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec06aca0 0x7ffaec1a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.908+0000 7ffaf1200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec107980 0x7ffaec19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.908+0000 7ffaebe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec06aca0 0x7ffaec1a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:53364/0 (socket says 172.21.15.42:53364) 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.909+0000 7ffaebe00700 1 -- 172.21.15.42:0/322750345 learned_addr learned my addr 172.21.15.42:0/322750345 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.909+0000 7ffaeb400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec105490 0x7ffaec1a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:43688/0 (socket says 172.21.15.42:43688) 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.909+0000 7ffaebe00700 1 -- 172.21.15.42:0/322750345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec105490 msgr2=0x7ffaec1a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.909+0000 7ffaebe00700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec105490 0x7ffaec1a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.909+0000 7ffaebe00700 1 -- 172.21.15.42:0/322750345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec107980 msgr2=0x7ffaec19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.909+0000 7ffaebe00700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec107980 0x7ffaec19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.909+0000 7ffaebe00700 1 -- 172.21.15.42:0/322750345 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec19ff50 con 0x7ffaec06aca0 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.909+0000 7ffaf1200700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec107980 0x7ffaec19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.909+0000 7ffaebe00700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec06aca0 0x7ffaec1a55a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00ec10 tx=0x7ffadc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:41.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.910+0000 7ffae8c00700 1 -- 172.21.15.42:0/322750345 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc00cc90 con 0x7ffaec06aca0 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.910+0000 7ffae8c00700 1 -- 172.21.15.42:0/322750345 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffadc00ce30 con 0x7ffaec06aca0 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.910+0000 7ffae8c00700 1 -- 172.21.15.42:0/322750345 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc010690 con 0x7ffaec06aca0 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.910+0000 7ffaf3200700 1 -- 172.21.15.42:0/322750345 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaec1a0240 con 0x7ffaec06aca0 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.910+0000 7ffaf3200700 1 -- 172.21.15.42:0/322750345 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaec1ac520 con 0x7ffaec06aca0 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.911+0000 7ffaf3200700 1 -- 172.21.15.42:0/322750345 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaec101f70 con 0x7ffaec06aca0 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.913+0000 7ffae8c00700 1 -- 172.21.15.42:0/322750345 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffadc003680 con 0x7ffaec06aca0 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.914+0000 7ffae8c00700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ffad00a3d50 0x7ffad00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.914+0000 7ffae8c00700 1 -- 172.21.15.42:0/322750345 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ffadc0d0840 con 0x7ffaec06aca0 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.915+0000 7ffaeb400700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ffad00a3d50 0x7ffad00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.915+0000 7ffaeb400700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ffad00a3d50 0x7ffad00a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffaec1a0e30 tx=0x7ffad8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:39.918+0000 7ffae8c00700 1 -- 172.21.15.42:0/322750345 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffadc09d040 con 0x7ffaec06aca0 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.160+0000 7ffaf3200700 1 -- 172.21.15.42:0/322750345 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ffaec069740 con 0x7ffaec06aca0 2024-04-20T10:32:41.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.161+0000 7ffae8c00700 1 -- 172.21.15.42:0/322750345 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffadc09c940 con 0x7ffaec06aca0 2024-04-20T10:32:41.777 INFO:teuthology.orchestra.run.smithi042.stdout:287762808870 2024-04-20T10:32:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.164+0000 7ffaf3200700 1 -- 172.21.15.42:0/322750345 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ffad00a3d50 msgr2=0x7ffad00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.164+0000 7ffaf3200700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ffad00a3d50 0x7ffad00a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffaec1a0e30 tx=0x7ffad8009260 comp rx=0 tx=0).stop 2024-04-20T10:32:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.164+0000 7ffaf3200700 1 -- 172.21.15.42:0/322750345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec06aca0 msgr2=0x7ffaec1a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.164+0000 7ffaf3200700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec06aca0 0x7ffaec1a55a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00ec10 tx=0x7ffadc00ef20 comp rx=0 tx=0).stop 2024-04-20T10:32:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.164+0000 7ffaf3200700 1 -- 172.21.15.42:0/322750345 shutdown_connections 2024-04-20T10:32:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.164+0000 7ffaf3200700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ffad00a3d50 0x7ffad00a6210 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.164+0000 7ffaf3200700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec107980 0x7ffaec19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.164+0000 7ffaf3200700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec06aca0 0x7ffaec1a55a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.164+0000 7ffaf3200700 1 --2- 172.21.15.42:0/322750345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaec105490 0x7ffaec1a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:41.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.164+0000 7ffaf3200700 1 -- 172.21.15.42:0/322750345 >> 172.21.15.42:0/322750345 conn(0x7ffaec078900 msgr2=0x7ffaec104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:41.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.164+0000 7ffaf3200700 1 -- 172.21.15.42:0/322750345 shutdown_connections 2024-04-20T10:32:41.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.164+0000 7ffaf3200700 1 -- 172.21.15.42:0/322750345 wait complete. 2024-04-20T10:32:42.016 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:41 smithi042 ceph-mon[17078]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 746 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:42.016 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:41 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2672687400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T10:32:42.020 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332142 got 219043332145 for osd.7 2024-04-20T10:32:42.021 DEBUG:teuthology.parallel:result is None 2024-04-20T10:32:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:41 smithi046 ceph-mon[21574]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 746 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:41 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2672687400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T10:32:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:41 smithi162 ceph-mon[21854]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 746 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:41 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2672687400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T10:32:43.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.720+0000 7fe451400700 1 -- 172.21.15.42:0/4290249739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c074c80 msgr2=0x7fe44c075060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:43.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.720+0000 7fe451400700 1 --2- 172.21.15.42:0/4290249739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c074c80 0x7fe44c075060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009990 tx=0x7fe43c009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.720+0000 7fe451400700 1 -- 172.21.15.42:0/4290249739 shutdown_connections 2024-04-20T10:32:43.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.720+0000 7fe451400700 1 --2- 172.21.15.42:0/4290249739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c074c80 0x7fe44c075060 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.720+0000 7fe451400700 1 --2- 172.21.15.42:0/4290249739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe44c10f450 0x7fe44c111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.720+0000 7fe451400700 1 --2- 172.21.15.42:0/4290249739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe44c0755a0 0x7fe44c10ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.720+0000 7fe451400700 1 -- 172.21.15.42:0/4290249739 >> 172.21.15.42:0/4290249739 conn(0x7fe44c079bb0 msgr2=0x7fe44c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:43.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.720+0000 7fe451400700 1 -- 172.21.15.42:0/4290249739 shutdown_connections 2024-04-20T10:32:43.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.720+0000 7fe451400700 1 -- 172.21.15.42:0/4290249739 wait complete. 2024-04-20T10:32:43.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.722+0000 7fe451400700 1 Processor -- start 2024-04-20T10:32:43.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.722+0000 7fe451400700 1 -- start start 2024-04-20T10:32:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe451400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe44c074c80 0x7fe44c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe451400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe44c0755a0 0x7fe44c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe451400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c10f450 0x7fe44c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe451400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c114fa0 con 0x7fe44c074c80 2024-04-20T10:32:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe451400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c114e20 con 0x7fe44c10f450 2024-04-20T10:32:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe451400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c115120 con 0x7fe44c0755a0 2024-04-20T10:32:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe44a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe44c074c80 0x7fe44c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe44a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe44c074c80 0x7fe44c19b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51118/0 (socket says 172.21.15.42:51118) 2024-04-20T10:32:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe44a800700 1 -- 172.21.15.42:0/2672687400 learned_addr learned my addr 172.21.15.42:0/2672687400 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe443e00700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe44c0755a0 0x7fe44c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe44b200700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c10f450 0x7fe44c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe44a800700 1 -- 172.21.15.42:0/2672687400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe44c0755a0 msgr2=0x7fe44c19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe44a800700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe44c0755a0 0x7fe44c19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe44a800700 1 -- 172.21.15.42:0/2672687400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c10f450 msgr2=0x7fe44c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe44a800700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c10f450 0x7fe44c19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.723+0000 7fe44a800700 1 -- 172.21.15.42:0/2672687400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c1a0420 con 0x7fe44c074c80 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.724+0000 7fe44a800700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe44c074c80 0x7fe44c19b410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe43c005910 tx=0x7fe43c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.724+0000 7fe443400700 1 -- 172.21.15.42:0/2672687400 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c027070 con 0x7fe44c074c80 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.724+0000 7fe443400700 1 -- 172.21.15.42:0/2672687400 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe43c003d70 con 0x7fe44c074c80 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.724+0000 7fe451400700 1 -- 172.21.15.42:0/2672687400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c1a06b0 con 0x7fe44c074c80 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.724+0000 7fe443400700 1 -- 172.21.15.42:0/2672687400 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c00f460 con 0x7fe44c074c80 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.725+0000 7fe451400700 1 -- 172.21.15.42:0/2672687400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c1a7f90 con 0x7fe44c074c80 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.726+0000 7fe451400700 1 -- 172.21.15.42:0/2672687400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c069740 con 0x7fe44c074c80 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.728+0000 7fe443400700 1 -- 172.21.15.42:0/2672687400 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe43c02a070 con 0x7fe44c074c80 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.729+0000 7fe443400700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe4300a3d50 0x7fe4300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.729+0000 7fe443400700 1 -- 172.21.15.42:0/2672687400 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe43c0d13c0 con 0x7fe44c074c80 2024-04-20T10:32:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.729+0000 7fe443e00700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe4300a3d50 0x7fe4300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.730+0000 7fe443e00700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe4300a3d50 0x7fe4300a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe434009680 tx=0x7fe434009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:40.734+0000 7fe443400700 1 -- 172.21.15.42:0/2672687400 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe43c05eff0 con 0x7fe44c074c80 2024-04-20T10:32:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:55834574933 2024-04-20T10:32:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.011+0000 7fe451400700 1 -- 172.21.15.42:0/2672687400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe44c1a8160 con 0x7fe44c074c80 2024-04-20T10:32:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.011+0000 7fe443400700 1 -- 172.21.15.42:0/2672687400 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe43c013070 con 0x7fe44c074c80 2024-04-20T10:32:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.015+0000 7fe451400700 1 -- 172.21.15.42:0/2672687400 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe4300a3d50 msgr2=0x7fe4300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.015+0000 7fe451400700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe4300a3d50 0x7fe4300a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe434009680 tx=0x7fe434009260 comp rx=0 tx=0).stop 2024-04-20T10:32:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.015+0000 7fe451400700 1 -- 172.21.15.42:0/2672687400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe44c074c80 msgr2=0x7fe44c19b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.015+0000 7fe451400700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe44c074c80 0x7fe44c19b410 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe43c005910 tx=0x7fe43c00bab0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.015+0000 7fe451400700 1 -- 172.21.15.42:0/2672687400 shutdown_connections 2024-04-20T10:32:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.015+0000 7fe451400700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe4300a3d50 0x7fe4300a6210 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.015+0000 7fe451400700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe44c10f450 0x7fe44c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.015+0000 7fe451400700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe44c074c80 0x7fe44c19b410 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.015+0000 7fe451400700 1 --2- 172.21.15.42:0/2672687400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe44c0755a0 0x7fe44c19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.015+0000 7fe451400700 1 -- 172.21.15.42:0/2672687400 >> 172.21.15.42:0/2672687400 conn(0x7fe44c079bb0 msgr2=0x7fe44c0ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.015+0000 7fe451400700 1 -- 172.21.15.42:0/2672687400 shutdown_connections 2024-04-20T10:32:43.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:41.015+0000 7fe451400700 1 -- 172.21.15.42:0/2672687400 wait complete. 2024-04-20T10:32:43.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.594+0000 7feb71400700 1 -- 172.21.15.42:0/1918095914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb6c1092e0 msgr2=0x7feb6c1096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:43.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.594+0000 7feb71400700 1 --2- 172.21.15.42:0/1918095914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb6c1092e0 0x7feb6c1096c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feb5c0099c0 tx=0x7feb5c009cd0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.595+0000 7feb71400700 1 -- 172.21.15.42:0/1918095914 shutdown_connections 2024-04-20T10:32:43.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.595+0000 7feb71400700 1 --2- 172.21.15.42:0/1918095914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c10e3c0 0x7feb6c115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.595+0000 7feb71400700 1 --2- 172.21.15.42:0/1918095914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb6c1092e0 0x7feb6c1096c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.595+0000 7feb71400700 1 --2- 172.21.15.42:0/1918095914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb6c109c90 0x7feb6c10dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.595+0000 7feb71400700 1 -- 172.21.15.42:0/1918095914 >> 172.21.15.42:0/1918095914 conn(0x7feb6c076220 msgr2=0x7feb6c107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:43.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.595+0000 7feb71400700 1 -- 172.21.15.42:0/1918095914 shutdown_connections 2024-04-20T10:32:43.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.595+0000 7feb71400700 1 -- 172.21.15.42:0/1918095914 wait complete. 2024-04-20T10:32:43.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.596+0000 7feb71400700 1 Processor -- start 2024-04-20T10:32:43.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.596+0000 7feb71400700 1 -- start start 2024-04-20T10:32:43.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.596+0000 7feb71400700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb6c1092e0 0x7feb6c1a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:43.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb71400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb6c109c90 0x7feb6c1a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:43.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb71400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c10e3c0 0x7feb6c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:43.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb71400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c1193c0 con 0x7feb6c109c90 2024-04-20T10:32:43.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb71400700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c119240 con 0x7feb6c10e3c0 2024-04-20T10:32:43.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb71400700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c119540 con 0x7feb6c1092e0 2024-04-20T10:32:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb6a800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb6c1092e0 0x7feb6c1a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb69e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb6c109c90 0x7feb6c1a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb6a800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb6c1092e0 0x7feb6c1a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:53390/0 (socket says 172.21.15.42:53390) 2024-04-20T10:32:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb6a800700 1 -- 172.21.15.42:0/714919016 learned_addr learned my addr 172.21.15.42:0/714919016 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb6b200700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c10e3c0 0x7feb6c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:43.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb69e00700 1 -- 172.21.15.42:0/714919016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb6c1092e0 msgr2=0x7feb6c1a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:43.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb69e00700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb6c1092e0 0x7feb6c1a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb69e00700 1 -- 172.21.15.42:0/714919016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c10e3c0 msgr2=0x7feb6c19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:43.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb69e00700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c10e3c0 0x7feb6c19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.597+0000 7feb69e00700 1 -- 172.21.15.42:0/714919016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c19fd80 con 0x7feb6c109c90 2024-04-20T10:32:43.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.598+0000 7feb6b200700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c10e3c0 0x7feb6c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:43.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.598+0000 7feb69e00700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb6c109c90 0x7feb6c1a59a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb5800e9c0 tx=0x7feb5800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:43.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.598+0000 7feb63400700 1 -- 172.21.15.42:0/714919016 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58004220 con 0x7feb6c109c90 2024-04-20T10:32:43.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.598+0000 7feb63400700 1 -- 172.21.15.42:0/714919016 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb58004d10 con 0x7feb6c109c90 2024-04-20T10:32:43.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.598+0000 7feb71400700 1 -- 172.21.15.42:0/714919016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1a0070 con 0x7feb6c109c90 2024-04-20T10:32:43.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.598+0000 7feb63400700 1 -- 172.21.15.42:0/714919016 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58005020 con 0x7feb6c109c90 2024-04-20T10:32:43.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.598+0000 7feb71400700 1 -- 172.21.15.42:0/714919016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c1ac330 con 0x7feb6c109c90 2024-04-20T10:32:43.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.600+0000 7feb71400700 1 -- 172.21.15.42:0/714919016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c069740 con 0x7feb6c109c90 2024-04-20T10:32:43.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.602+0000 7feb63400700 1 -- 172.21.15.42:0/714919016 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feb58004750 con 0x7feb6c109c90 2024-04-20T10:32:43.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.603+0000 7feb63400700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7feb500a3cd0 0x7feb500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:43.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.603+0000 7feb63400700 1 -- 172.21.15.42:0/714919016 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7feb580d04d0 con 0x7feb6c109c90 2024-04-20T10:32:43.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.604+0000 7feb6a800700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7feb500a3cd0 0x7feb500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:43.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.604+0000 7feb6a800700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7feb500a3cd0 0x7feb500a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009750 tx=0x7feb5c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:43.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.612+0000 7feb63400700 1 -- 172.21.15.42:0/714919016 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7feb5809ccd0 con 0x7feb6c109c90 2024-04-20T10:32:43.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.856+0000 7feb71400700 1 -- 172.21.15.42:0/714919016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7feb6c1a0b50 con 0x7feb6c109c90 2024-04-20T10:32:43.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.857+0000 7feb63400700 1 -- 172.21.15.42:0/714919016 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feb5809c5d0 con 0x7feb6c109c90 2024-04-20T10:32:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:253403070508 2024-04-20T10:32:43.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.860+0000 7feb71400700 1 -- 172.21.15.42:0/714919016 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7feb500a3cd0 msgr2=0x7feb500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.860+0000 7feb71400700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7feb500a3cd0 0x7feb500a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009750 tx=0x7feb5c005960 comp rx=0 tx=0).stop 2024-04-20T10:32:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.860+0000 7feb71400700 1 -- 172.21.15.42:0/714919016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb6c109c90 msgr2=0x7feb6c1a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.860+0000 7feb71400700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb6c109c90 0x7feb6c1a59a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb5800e9c0 tx=0x7feb5800ecd0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.860+0000 7feb71400700 1 -- 172.21.15.42:0/714919016 shutdown_connections 2024-04-20T10:32:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.860+0000 7feb71400700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7feb500a3cd0 0x7feb500a6190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.860+0000 7feb71400700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c10e3c0 0x7feb6c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.860+0000 7feb71400700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb6c1092e0 0x7feb6c1a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.860+0000 7feb71400700 1 --2- 172.21.15.42:0/714919016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb6c109c90 0x7feb6c1a59a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:43.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.860+0000 7feb71400700 1 -- 172.21.15.42:0/714919016 >> 172.21.15.42:0/714919016 conn(0x7feb6c076220 msgr2=0x7feb6c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:43.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.860+0000 7feb71400700 1 -- 172.21.15.42:0/714919016 shutdown_connections 2024-04-20T10:32:43.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:43.860+0000 7feb71400700 1 -- 172.21.15.42:0/714919016 wait complete. 2024-04-20T10:32:44.032 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:43 smithi042 ceph-mon[17078]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 675 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:44.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:43 smithi046 ceph-mon[21574]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 675 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:43 smithi162 ceph-mon[21854]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 675 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:44.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.713+0000 7f2660a00700 1 -- 172.21.15.42:0/837468563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f265c0778a0 msgr2=0x7f265c1132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:44.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.713+0000 7f2660a00700 1 --2- 172.21.15.42:0/837468563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f265c0778a0 0x7f265c1132c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2648009990 tx=0x7f2648009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:44.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.713+0000 7f2660a00700 1 -- 172.21.15.42:0/837468563 shutdown_connections 2024-04-20T10:32:44.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.713+0000 7f2660a00700 1 --2- 172.21.15.42:0/837468563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f265c113800 0x7f265c115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:44.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.713+0000 7f2660a00700 1 --2- 172.21.15.42:0/837468563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f265c076f80 0x7f265c077360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:44.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.713+0000 7f2660a00700 1 --2- 172.21.15.42:0/837468563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f265c0778a0 0x7f265c1132c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:44.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.713+0000 7f2660a00700 1 -- 172.21.15.42:0/837468563 >> 172.21.15.42:0/837468563 conn(0x7f265c100630 msgr2=0x7f265c102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:44.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.713+0000 7f2660a00700 1 -- 172.21.15.42:0/837468563 shutdown_connections 2024-04-20T10:32:44.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.713+0000 7f2660a00700 1 -- 172.21.15.42:0/837468563 wait complete. 2024-04-20T10:32:44.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.715+0000 7f2660a00700 1 Processor -- start 2024-04-20T10:32:44.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.715+0000 7f2660a00700 1 -- start start 2024-04-20T10:32:44.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.715+0000 7f2660a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f265c076f80 0x7f265c1a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:44.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.715+0000 7f2660a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f265c0778a0 0x7f265c1a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:44.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.715+0000 7f2660a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f265c113800 0x7f265c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:44.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.715+0000 7f2660a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1192f0 con 0x7f265c0778a0 2024-04-20T10:32:44.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.715+0000 7f2660a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f265c119170 con 0x7f265c076f80 2024-04-20T10:32:44.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.715+0000 7f2660a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f265c119470 con 0x7f265c113800 2024-04-20T10:32:44.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.715+0000 7f265a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f265c076f80 0x7f265c1a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:44.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.715+0000 7f265a800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f265c076f80 0x7f265c1a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:43756/0 (socket says 172.21.15.42:43756) 2024-04-20T10:32:44.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.715+0000 7f2659e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f265c0778a0 0x7f265c1a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:44.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.715+0000 7f265a800700 1 -- 172.21.15.42:0/2280988285 learned_addr learned my addr 172.21.15.42:0/2280988285 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.716+0000 7f265b200700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f265c113800 0x7f265c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.716+0000 7f265a800700 1 -- 172.21.15.42:0/2280988285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f265c113800 msgr2=0x7f265c19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.716+0000 7f265a800700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f265c113800 0x7f265c19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.716+0000 7f265a800700 1 -- 172.21.15.42:0/2280988285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f265c0778a0 msgr2=0x7f265c1a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.716+0000 7f265a800700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f265c0778a0 0x7f265c1a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.716+0000 7f265a800700 1 -- 172.21.15.42:0/2280988285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c19fd10 con 0x7f265c076f80 2024-04-20T10:32:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.716+0000 7f265a800700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f265c076f80 0x7f265c1a53f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f264c00e9c0 tx=0x7f264c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.716+0000 7f2653400700 1 -- 172.21.15.42:0/2280988285 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c00cbd0 con 0x7f265c076f80 2024-04-20T10:32:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.717+0000 7f2653400700 1 -- 172.21.15.42:0/2280988285 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f264c00ee60 con 0x7f265c076f80 2024-04-20T10:32:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.717+0000 7f2653400700 1 -- 172.21.15.42:0/2280988285 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c005630 con 0x7f265c076f80 2024-04-20T10:32:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.717+0000 7f2660a00700 1 -- 172.21.15.42:0/2280988285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c1a0000 con 0x7f265c076f80 2024-04-20T10:32:44.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.717+0000 7f2660a00700 1 -- 172.21.15.42:0/2280988285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c1ac2c0 con 0x7f265c076f80 2024-04-20T10:32:44.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.718+0000 7f2660a00700 1 -- 172.21.15.42:0/2280988285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265c069740 con 0x7f265c076f80 2024-04-20T10:32:44.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.720+0000 7f2653400700 1 -- 172.21.15.42:0/2280988285 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f264c004750 con 0x7f265c076f80 2024-04-20T10:32:44.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.721+0000 7f2653400700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f26400a3d50 0x7f26400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:44.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.722+0000 7f2653400700 1 -- 172.21.15.42:0/2280988285 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f264c0d0a20 con 0x7f265c076f80 2024-04-20T10:32:44.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.722+0000 7f2659e00700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f26400a3d50 0x7f26400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:44.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.722+0000 7f2659e00700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f26400a3d50 0x7f26400a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f264800b2e0 tx=0x7f2648005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:44.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.724+0000 7f2653400700 1 -- 172.21.15.42:0/2280988285 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f264c053ff0 con 0x7f265c076f80 2024-04-20T10:32:44.803 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:44 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/714919016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-20T10:32:44.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.977+0000 7f2660a00700 1 -- 172.21.15.42:0/2280988285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f265c1a0ae0 con 0x7f265c076f80 2024-04-20T10:32:44.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.978+0000 7f2653400700 1 -- 172.21.15.42:0/2280988285 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f264c09d220 con 0x7f265c076f80 2024-04-20T10:32:44.978 INFO:teuthology.orchestra.run.smithi042.stdout:34359738458 2024-04-20T10:32:44.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.981+0000 7f2660a00700 1 -- 172.21.15.42:0/2280988285 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f26400a3d50 msgr2=0x7f26400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:44.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.981+0000 7f2660a00700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f26400a3d50 0x7f26400a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f264800b2e0 tx=0x7f2648005960 comp rx=0 tx=0).stop 2024-04-20T10:32:44.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.981+0000 7f2660a00700 1 -- 172.21.15.42:0/2280988285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f265c076f80 msgr2=0x7f265c1a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:44.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.981+0000 7f2660a00700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f265c076f80 0x7f265c1a53f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f264c00e9c0 tx=0x7f264c00ecd0 comp rx=0 tx=0).stop 2024-04-20T10:32:44.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.981+0000 7f2660a00700 1 -- 172.21.15.42:0/2280988285 shutdown_connections 2024-04-20T10:32:44.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.981+0000 7f2660a00700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f26400a3d50 0x7f26400a6210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:44.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.981+0000 7f2660a00700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f265c076f80 0x7f265c1a53f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:44.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.981+0000 7f2660a00700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f265c113800 0x7f265c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:44.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.981+0000 7f2660a00700 1 --2- 172.21.15.42:0/2280988285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f265c0778a0 0x7f265c1a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:44.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.981+0000 7f2660a00700 1 -- 172.21.15.42:0/2280988285 >> 172.21.15.42:0/2280988285 conn(0x7f265c100630 msgr2=0x7f265c114470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:44.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.982+0000 7f2660a00700 1 -- 172.21.15.42:0/2280988285 shutdown_connections 2024-04-20T10:32:44.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:44.982+0000 7f2660a00700 1 -- 172.21.15.42:0/2280988285 wait complete. 2024-04-20T10:32:45.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:44 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/714919016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-20T10:32:45.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:44 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/714919016' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-20T10:32:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:45 smithi046 ceph-mon[21574]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:46.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:45 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2280988285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T10:32:46.056 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:45 smithi042 ceph-mon[17078]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:46.056 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:45 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2280988285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T10:32:46.060 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808865 got 287762808870 for osd.9 2024-04-20T10:32:46.060 DEBUG:teuthology.parallel:result is None 2024-04-20T10:32:46.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:45 smithi162 ceph-mon[21854]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-20T10:32:46.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:45 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2280988285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T10:32:46.501 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738454 got 34359738458 for osd.0 2024-04-20T10:32:46.501 DEBUG:teuthology.parallel:result is None 2024-04-20T10:32:46.621 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574933 for osd.1 2024-04-20T10:32:46.621 DEBUG:teuthology.parallel:result is None 2024-04-20T10:32:47.335 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:47.340 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:47.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:47 smithi042 ceph-mon[17078]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:47 smithi046 ceph-mon[21574]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:48.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.054+0000 7f025cc00700 1 -- 172.21.15.42:0/2126482955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0258107b30 msgr2=0x7f0258107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:48.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.054+0000 7f025cc00700 1 --2- 172.21.15.42:0/2126482955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0258107b30 0x7f0258107f10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f024800b0a0 tx=0x7f024800b3b0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.054+0000 7f025cc00700 1 -- 172.21.15.42:0/2126482955 shutdown_connections 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.054+0000 7f025cc00700 1 --2- 172.21.15.42:0/2126482955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0258101dd0 0x7f025810f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.054+0000 7f025cc00700 1 --2- 172.21.15.42:0/2126482955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0258107b30 0x7f0258107f10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.054+0000 7f025cc00700 1 --2- 172.21.15.42:0/2126482955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0258101410 0x7f0258101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.054+0000 7f025cc00700 1 -- 172.21.15.42:0/2126482955 >> 172.21.15.42:0/2126482955 conn(0x7f0258076c70 msgr2=0x7f0258077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.054+0000 7f025cc00700 1 -- 172.21.15.42:0/2126482955 shutdown_connections 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.054+0000 7f025cc00700 1 -- 172.21.15.42:0/2126482955 wait complete. 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.055+0000 7f025cc00700 1 Processor -- start 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.056+0000 7f025cc00700 1 -- start start 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.056+0000 7f025cc00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0258101410 0x7f025819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.056+0000 7f025cc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0258101dd0 0x7f025819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.056+0000 7f025cc00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0258107b30 0x7f025819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.056+0000 7f025cc00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0258112e10 con 0x7f0258101dd0 2024-04-20T10:32:48.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.056+0000 7f025cc00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0258112c90 con 0x7f0258101410 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.056+0000 7f025cc00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0258112f90 con 0x7f0258107b30 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.056+0000 7f0255400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0258101dd0 0x7f025819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.056+0000 7f0255400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0258101dd0 0x7f025819b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51180/0 (socket says 172.21.15.42:51180) 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.056+0000 7f0255e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0258101410 0x7f025819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.056+0000 7f0255400700 1 -- 172.21.15.42:0/2288753906 learned_addr learned my addr 172.21.15.42:0/2288753906 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.056+0000 7f0256800700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0258107b30 0x7f025819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.057+0000 7f0255400700 1 -- 172.21.15.42:0/2288753906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0258107b30 msgr2=0x7f025819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.057+0000 7f0255400700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0258107b30 0x7f025819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.057+0000 7f0255400700 1 -- 172.21.15.42:0/2288753906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0258101410 msgr2=0x7f025819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.057+0000 7f0255400700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0258101410 0x7f025819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.057+0000 7f0255400700 1 -- 172.21.15.42:0/2288753906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02581a0410 con 0x7f0258101dd0 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.057+0000 7f0255e00700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0258101410 0x7f025819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.057+0000 7f0255400700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0258101dd0 0x7f025819b940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f024400d980 tx=0x7f024400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.057+0000 7f024ea00700 1 -- 172.21.15.42:0/2288753906 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244016070 con 0x7f0258101dd0 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.058+0000 7f024ea00700 1 -- 172.21.15.42:0/2288753906 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0244004d10 con 0x7f0258101dd0 2024-04-20T10:32:48.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.058+0000 7f024ea00700 1 -- 172.21.15.42:0/2288753906 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244005670 con 0x7f0258101dd0 2024-04-20T10:32:48.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.058+0000 7f025cc00700 1 -- 172.21.15.42:0/2288753906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02581a0700 con 0x7f0258101dd0 2024-04-20T10:32:48.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.058+0000 7f025cc00700 1 -- 172.21.15.42:0/2288753906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02581a7f80 con 0x7f0258101dd0 2024-04-20T10:32:48.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.059+0000 7f025cc00700 1 -- 172.21.15.42:0/2288753906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0258069740 con 0x7f0258101dd0 2024-04-20T10:32:48.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.061+0000 7f024ea00700 1 -- 172.21.15.42:0/2288753906 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0244004750 con 0x7f0258101dd0 2024-04-20T10:32:48.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.062+0000 7f024ea00700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f023c0a3d50 0x7f023c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:48.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.063+0000 7f024ea00700 1 -- 172.21.15.42:0/2288753906 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f02440d03c0 con 0x7f0258101dd0 2024-04-20T10:32:48.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.063+0000 7f0255e00700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f023c0a3d50 0x7f023c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:48.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.064+0000 7f0255e00700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f023c0a3d50 0x7f023c0a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f02480062a0 tx=0x7f0248006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:48.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.068+0000 7f024ea00700 1 -- 172.21.15.42:0/2288753906 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f024401f020 con 0x7f0258101dd0 2024-04-20T10:32:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:47 smithi162 ceph-mon[21854]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.163+0000 7f944ca00700 1 -- 172.21.15.42:0/1193759519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9448074e80 msgr2=0x7f9448075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.163+0000 7f944ca00700 1 --2- 172.21.15.42:0/1193759519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9448074e80 0x7f9448075260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9438009990 tx=0x7f9438009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.163+0000 7f944ca00700 1 -- 172.21.15.42:0/1193759519 shutdown_connections 2024-04-20T10:32:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.163+0000 7f944ca00700 1 --2- 172.21.15.42:0/1193759519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944810f1c0 0x7f9448111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.163+0000 7f944ca00700 1 --2- 172.21.15.42:0/1193759519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9448074e80 0x7f9448075260 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.163+0000 7f944ca00700 1 --2- 172.21.15.42:0/1193759519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94480757a0 0x7f944810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.163+0000 7f944ca00700 1 -- 172.21.15.42:0/1193759519 >> 172.21.15.42:0/1193759519 conn(0x7f94480fe4d0 msgr2=0x7f94481008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.163+0000 7f944ca00700 1 -- 172.21.15.42:0/1193759519 shutdown_connections 2024-04-20T10:32:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.163+0000 7f944ca00700 1 -- 172.21.15.42:0/1193759519 wait complete. 2024-04-20T10:32:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.164+0000 7f944ca00700 1 Processor -- start 2024-04-20T10:32:48.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.164+0000 7f944ca00700 1 -- start start 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.165+0000 7f944ca00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9448074e80 0x7f944819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.165+0000 7f944ca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94480757a0 0x7f944819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.165+0000 7f944ca00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944810f1c0 0x7f944819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.165+0000 7f944ca00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9448114ee0 con 0x7f94480757a0 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.165+0000 7f944ca00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9448114d60 con 0x7f944810f1c0 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.165+0000 7f944ca00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9448115060 con 0x7f9448074e80 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.165+0000 7f9446800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9448074e80 0x7f944819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.165+0000 7f9446800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9448074e80 0x7f944819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:53418/0 (socket says 172.21.15.42:53418) 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.165+0000 7f9446800700 1 -- 172.21.15.42:0/3148580923 learned_addr learned my addr 172.21.15.42:0/3148580923 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.166+0000 7f9447200700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944810f1c0 0x7f944819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.166+0000 7f9445e00700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94480757a0 0x7f944819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.166+0000 7f9446800700 1 -- 172.21.15.42:0/3148580923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944810f1c0 msgr2=0x7f944819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:48.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.166+0000 7f9446800700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944810f1c0 0x7f944819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.166+0000 7f9446800700 1 -- 172.21.15.42:0/3148580923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94480757a0 msgr2=0x7f944819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.166+0000 7f9446800700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94480757a0 0x7f944819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.166+0000 7f9446800700 1 -- 172.21.15.42:0/3148580923 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94481a0350 con 0x7f9448074e80 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.166+0000 7f9445e00700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94480757a0 0x7f944819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.166+0000 7f9447200700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944810f1c0 0x7f944819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.166+0000 7f9446800700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9448074e80 0x7f944819b390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9438000c00 tx=0x7f9438003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.167+0000 7f943f400700 1 -- 172.21.15.42:0/3148580923 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438027070 con 0x7f9448074e80 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.167+0000 7f943f400700 1 -- 172.21.15.42:0/3148580923 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9438005690 con 0x7f9448074e80 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.167+0000 7f944ca00700 1 -- 172.21.15.42:0/3148580923 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94481a05e0 con 0x7f9448074e80 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.167+0000 7f943f400700 1 -- 172.21.15.42:0/3148580923 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438022d40 con 0x7f9448074e80 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.167+0000 7f944ca00700 1 -- 172.21.15.42:0/3148580923 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94481a7ec0 con 0x7f9448074e80 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.168+0000 7f944ca00700 1 -- 172.21.15.42:0/3148580923 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9448069740 con 0x7f9448074e80 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.170+0000 7f943f400700 1 -- 172.21.15.42:0/3148580923 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f943802a070 con 0x7f9448074e80 2024-04-20T10:32:48.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.171+0000 7f943f400700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f942c0a3d50 0x7f942c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:48.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.172+0000 7f943f400700 1 -- 172.21.15.42:0/3148580923 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f94380d15e0 con 0x7f9448074e80 2024-04-20T10:32:48.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.172+0000 7f9445e00700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f942c0a3d50 0x7f942c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:48.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.172+0000 7f9445e00700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f942c0a3d50 0x7f942c0a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f94481104f0 tx=0x7f943400b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:48.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.174+0000 7f943f400700 1 -- 172.21.15.42:0/3148580923 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9438013070 con 0x7f9448074e80 2024-04-20T10:32:48.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.306+0000 7f025cc00700 1 -- 172.21.15.42:0/2288753906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f02581a8150 con 0x7f0258101dd0 2024-04-20T10:32:48.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.306+0000 7f024ea00700 1 -- 172.21.15.42:0/2288753906 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f024409cbc0 con 0x7f0258101dd0 2024-04-20T10:32:48.306 INFO:teuthology.orchestra.run.smithi042.stdout:163208757312 2024-04-20T10:32:48.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.309+0000 7f025cc00700 1 -- 172.21.15.42:0/2288753906 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f023c0a3d50 msgr2=0x7f023c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:48.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.309+0000 7f025cc00700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f023c0a3d50 0x7f023c0a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f02480062a0 tx=0x7f0248006210 comp rx=0 tx=0).stop 2024-04-20T10:32:48.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.309+0000 7f025cc00700 1 -- 172.21.15.42:0/2288753906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0258101dd0 msgr2=0x7f025819b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:48.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.309+0000 7f025cc00700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0258101dd0 0x7f025819b940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f024400d980 tx=0x7f024400dc90 comp rx=0 tx=0).stop 2024-04-20T10:32:48.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.309+0000 7f025cc00700 1 -- 172.21.15.42:0/2288753906 shutdown_connections 2024-04-20T10:32:48.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.309+0000 7f025cc00700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f023c0a3d50 0x7f023c0a6210 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.309+0000 7f025cc00700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0258101410 0x7f025819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.309+0000 7f025cc00700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0258107b30 0x7f025819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.309+0000 7f025cc00700 1 --2- 172.21.15.42:0/2288753906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0258101dd0 0x7f025819b940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.310+0000 7f025cc00700 1 -- 172.21.15.42:0/2288753906 >> 172.21.15.42:0/2288753906 conn(0x7f0258076c70 msgr2=0x7f02580ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:48.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.310+0000 7f025cc00700 1 -- 172.21.15.42:0/2288753906 shutdown_connections 2024-04-20T10:32:48.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.310+0000 7f025cc00700 1 -- 172.21.15.42:0/2288753906 wait complete. 2024-04-20T10:32:48.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.420+0000 7f944ca00700 1 -- 172.21.15.42:0/3148580923 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f944802df90 con 0x7f9448074e80 2024-04-20T10:32:48.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.420+0000 7f943f400700 1 -- 172.21.15.42:0/3148580923 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f943809de70 con 0x7f9448074e80 2024-04-20T10:32:48.420 INFO:teuthology.orchestra.run.smithi042.stdout:141733920837 2024-04-20T10:32:48.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.423+0000 7f944ca00700 1 -- 172.21.15.42:0/3148580923 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f942c0a3d50 msgr2=0x7f942c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:48.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.423+0000 7f944ca00700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f942c0a3d50 0x7f942c0a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f94481104f0 tx=0x7f943400b5f0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.423+0000 7f944ca00700 1 -- 172.21.15.42:0/3148580923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9448074e80 msgr2=0x7f944819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:48.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.423+0000 7f944ca00700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9448074e80 0x7f944819b390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9438000c00 tx=0x7f9438003890 comp rx=0 tx=0).stop 2024-04-20T10:32:48.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.423+0000 7f944ca00700 1 -- 172.21.15.42:0/3148580923 shutdown_connections 2024-04-20T10:32:48.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.423+0000 7f944ca00700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f942c0a3d50 0x7f942c0a6210 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.424+0000 7f944ca00700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f944810f1c0 0x7f944819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.424+0000 7f944ca00700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9448074e80 0x7f944819b390 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.424+0000 7f944ca00700 1 --2- 172.21.15.42:0/3148580923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94480757a0 0x7f944819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:48.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.424+0000 7f944ca00700 1 -- 172.21.15.42:0/3148580923 >> 172.21.15.42:0/3148580923 conn(0x7f94480fe4d0 msgr2=0x7f94480ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:48.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.424+0000 7f944ca00700 1 -- 172.21.15.42:0/3148580923 shutdown_connections 2024-04-20T10:32:48.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.424+0000 7f944ca00700 1 -- 172.21.15.42:0/3148580923 wait complete. 2024-04-20T10:32:49.026 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:48 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2288753906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T10:32:49.026 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:48 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3148580923' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T10:32:49.026 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:48 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4221863422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-20T10:32:49.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:48 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2288753906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T10:32:49.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:48 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3148580923' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T10:32:49.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:48 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/4221863422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-20T10:32:49.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:48 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2288753906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T10:32:49.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:48 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3148580923' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T10:32:49.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:48 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/4221863422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-20T10:32:49.426 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070505 got 253403070508 for osd.8 2024-04-20T10:32:49.426 DEBUG:teuthology.parallel:result is None 2024-04-20T10:32:49.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.499+0000 7f56d1000700 1 -- 172.21.15.42:0/3722318826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc108a60 msgr2=0x7f56cc108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:49.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.499+0000 7f56d1000700 1 --2- 172.21.15.42:0/3722318826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc108a60 0x7f56cc108e40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009990 tx=0x7f56bc009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:49.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.499+0000 7f56d1000700 1 -- 172.21.15.42:0/3722318826 shutdown_connections 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.499+0000 7f56d1000700 1 --2- 172.21.15.42:0/3722318826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc108a60 0x7f56cc108e40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.499+0000 7f56d1000700 1 --2- 172.21.15.42:0/3722318826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56cc103460 0x7f56cc10f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.499+0000 7f56d1000700 1 --2- 172.21.15.42:0/3722318826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56cc102aa0 0x7f56cc102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.499+0000 7f56d1000700 1 -- 172.21.15.42:0/3722318826 >> 172.21.15.42:0/3722318826 conn(0x7f56cc079bb0 msgr2=0x7f56cc079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.499+0000 7f56d1000700 1 -- 172.21.15.42:0/3722318826 shutdown_connections 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.499+0000 7f56d1000700 1 -- 172.21.15.42:0/3722318826 wait complete. 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.500+0000 7f56d1000700 1 Processor -- start 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.500+0000 7f56d1000700 1 -- start start 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.501+0000 7f56d1000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56cc102aa0 0x7f56cc199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.501+0000 7f56d1000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56cc103460 0x7f56cc199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.501+0000 7f56d1000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc108a60 0x7f56cc19db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.501+0000 7f56d1000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc112d60 con 0x7f56cc103460 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.501+0000 7f56d1000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc112be0 con 0x7f56cc108a60 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.501+0000 7f56d1000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc112ee0 con 0x7f56cc102aa0 2024-04-20T10:32:49.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.501+0000 7f56c3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56cc103460 0x7f56cc199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.501+0000 7f56c3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56cc103460 0x7f56cc199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51218/0 (socket says 172.21.15.42:51218) 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.501+0000 7f56c3e00700 1 -- 172.21.15.42:0/4221863422 learned_addr learned my addr 172.21.15.42:0/4221863422 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.501+0000 7f56ca800700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc108a60 0x7f56cc19db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.501+0000 7f56c9e00700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56cc102aa0 0x7f56cc199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.502+0000 7f56c3e00700 1 -- 172.21.15.42:0/4221863422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56cc102aa0 msgr2=0x7f56cc199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.502+0000 7f56c3e00700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56cc102aa0 0x7f56cc199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.502+0000 7f56c3e00700 1 -- 172.21.15.42:0/4221863422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc108a60 msgr2=0x7f56cc19db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.502+0000 7f56c3e00700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc108a60 0x7f56cc19db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.502+0000 7f56c3e00700 1 -- 172.21.15.42:0/4221863422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc19e260 con 0x7f56cc103460 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.502+0000 7f56c9e00700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56cc102aa0 0x7f56cc199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.502+0000 7f56c3e00700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56cc103460 0x7f56cc199790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f56b400e9a0 tx=0x7f56b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.502+0000 7f56c2a00700 1 -- 172.21.15.42:0/4221863422 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b400ca10 con 0x7f56cc103460 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.502+0000 7f56c2a00700 1 -- 172.21.15.42:0/4221863422 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b400cbb0 con 0x7f56cc103460 2024-04-20T10:32:49.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.502+0000 7f56d1000700 1 -- 172.21.15.42:0/4221863422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc19e550 con 0x7f56cc103460 2024-04-20T10:32:49.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.502+0000 7f56c2a00700 1 -- 172.21.15.42:0/4221863422 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4018680 con 0x7f56cc103460 2024-04-20T10:32:49.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.502+0000 7f56d1000700 1 -- 172.21.15.42:0/4221863422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc1a5dd0 con 0x7f56cc103460 2024-04-20T10:32:49.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.504+0000 7f56d1000700 1 -- 172.21.15.42:0/4221863422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56cc069740 con 0x7f56cc103460 2024-04-20T10:32:49.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.506+0000 7f56c2a00700 1 -- 172.21.15.42:0/4221863422 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f56b4010430 con 0x7f56cc103460 2024-04-20T10:32:49.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.507+0000 7f56c2a00700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f56b00a3cd0 0x7f56b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:49.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.507+0000 7f56c2a00700 1 -- 172.21.15.42:0/4221863422 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f56b40d0c40 con 0x7f56cc103460 2024-04-20T10:32:49.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.507+0000 7f56c9e00700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f56b00a3cd0 0x7f56b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:49.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.508+0000 7f56c9e00700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f56b00a3cd0 0x7f56b00a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0098e0 tx=0x7f56bc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:49.462 INFO:teuthology.orchestra.run.smithi042.stdout:309237645345 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.510+0000 7f56c2a00700 1 -- 172.21.15.42:0/4221863422 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f56b409d4a0 con 0x7f56cc103460 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.744+0000 7f56d1000700 1 -- 172.21.15.42:0/4221863422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f56cc02df90 con 0x7f56cc103460 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.744+0000 7f56c2a00700 1 -- 172.21.15.42:0/4221863422 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f56b409cda0 con 0x7f56cc103460 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.747+0000 7f56d1000700 1 -- 172.21.15.42:0/4221863422 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f56b00a3cd0 msgr2=0x7f56b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.747+0000 7f56d1000700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f56b00a3cd0 0x7f56b00a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0098e0 tx=0x7f56bc005960 comp rx=0 tx=0).stop 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.747+0000 7f56d1000700 1 -- 172.21.15.42:0/4221863422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56cc103460 msgr2=0x7f56cc199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.747+0000 7f56d1000700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56cc103460 0x7f56cc199790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f56b400e9a0 tx=0x7f56b400ecb0 comp rx=0 tx=0).stop 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.747+0000 7f56d1000700 1 -- 172.21.15.42:0/4221863422 shutdown_connections 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.747+0000 7f56d1000700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f56b00a3cd0 0x7f56b00a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.748+0000 7f56d1000700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc108a60 0x7f56cc19db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.748+0000 7f56d1000700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56cc102aa0 0x7f56cc199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.748+0000 7f56d1000700 1 --2- 172.21.15.42:0/4221863422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56cc103460 0x7f56cc199790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.748+0000 7f56d1000700 1 -- 172.21.15.42:0/4221863422 >> 172.21.15.42:0/4221863422 conn(0x7f56cc079bb0 msgr2=0x7f56cc1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:49.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.748+0000 7f56d1000700 1 -- 172.21.15.42:0/4221863422 shutdown_connections 2024-04-20T10:32:49.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:48.748+0000 7f56d1000700 1 -- 172.21.15.42:0/4221863422 wait complete. 2024-04-20T10:32:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:49 smithi046 ceph-mon[21574]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:49 smithi042 ceph-mon[17078]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:49 smithi162 ceph-mon[21854]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:50.367 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645340 got 309237645345 for osd.10 2024-04-20T10:32:50.368 DEBUG:teuthology.parallel:result is None 2024-04-20T10:32:50.670 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920829 got 141733920837 for osd.4 2024-04-20T10:32:50.671 DEBUG:teuthology.parallel:result is None 2024-04-20T10:32:51.183 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757305 got 163208757312 for osd.5 2024-04-20T10:32:51.183 DEBUG:teuthology.parallel:result is None 2024-04-20T10:32:51.246 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182466 got 107374182471 for osd.3 2024-04-20T10:32:51.246 DEBUG:teuthology.parallel:result is None 2024-04-20T10:32:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:51 smithi046 ceph-mon[21574]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:52.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:51 smithi042 ceph-mon[17078]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:51 smithi162 ceph-mon[21854]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:53.110 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:32:53.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:53 smithi042 ceph-mon[17078]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:54.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:53 smithi046 ceph-mon[21574]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:54.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:53 smithi162 ceph-mon[21854]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:54.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.468+0000 7fd436600700 1 -- 172.21.15.42:0/824423957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd430104500 msgr2=0x7fd43010c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:54.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.468+0000 7fd436600700 1 --2- 172.21.15.42:0/824423957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd430104500 0x7fd43010c9f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009990 tx=0x7fd41c009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:54.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.468+0000 7fd436600700 1 -- 172.21.15.42:0/824423957 shutdown_connections 2024-04-20T10:32:54.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.468+0000 7fd436600700 1 --2- 172.21.15.42:0/824423957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd43010cfc0 0x7fd43010f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:54.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.468+0000 7fd436600700 1 --2- 172.21.15.42:0/824423957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430103be0 0x7fd430103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:54.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.468+0000 7fd436600700 1 --2- 172.21.15.42:0/824423957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd430104500 0x7fd43010c9f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:54.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.468+0000 7fd436600700 1 -- 172.21.15.42:0/824423957 >> 172.21.15.42:0/824423957 conn(0x7fd4300fd370 msgr2=0x7fd4300ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:54.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.469+0000 7fd436600700 1 -- 172.21.15.42:0/824423957 shutdown_connections 2024-04-20T10:32:54.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.469+0000 7fd436600700 1 -- 172.21.15.42:0/824423957 wait complete. 2024-04-20T10:32:54.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.470+0000 7fd436600700 1 Processor -- start 2024-04-20T10:32:54.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.470+0000 7fd436600700 1 -- start start 2024-04-20T10:32:54.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.471+0000 7fd436600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd430103be0 0x7fd4301991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:54.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.471+0000 7fd436600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430104500 0x7fd430199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:54.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.471+0000 7fd436600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43010cfc0 0x7fd43019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:54.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.471+0000 7fd436600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd430112ce0 con 0x7fd43010cfc0 2024-04-20T10:32:54.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.471+0000 7fd436600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd430112b60 con 0x7fd430103be0 2024-04-20T10:32:54.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.471+0000 7fd436600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd430112e60 con 0x7fd430104500 2024-04-20T10:32:54.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.471+0000 7fd42fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430104500 0x7fd430199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:54.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.471+0000 7fd435200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd430103be0 0x7fd4301991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:54.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.471+0000 7fd435200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd430103be0 0x7fd4301991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:51100/0 (socket says 172.21.15.42:51100) 2024-04-20T10:32:54.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.471+0000 7fd42fe00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430104500 0x7fd430199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:60698/0 (socket says 172.21.15.42:60698) 2024-04-20T10:32:54.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.471+0000 7fd435200700 1 -- 172.21.15.42:0/3068748293 learned_addr learned my addr 172.21.15.42:0/3068748293 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:54.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.471+0000 7fd435c00700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43010cfc0 0x7fd43019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:54.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.472+0000 7fd435200700 1 -- 172.21.15.42:0/3068748293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430104500 msgr2=0x7fd430199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:54.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.472+0000 7fd435200700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430104500 0x7fd430199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:54.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.472+0000 7fd435200700 1 -- 172.21.15.42:0/3068748293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43010cfc0 msgr2=0x7fd43019da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:54.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.472+0000 7fd435200700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43010cfc0 0x7fd43019da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:54.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.472+0000 7fd435200700 1 -- 172.21.15.42:0/3068748293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43019e190 con 0x7fd430103be0 2024-04-20T10:32:54.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.472+0000 7fd435c00700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43010cfc0 0x7fd43019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:32:54.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.472+0000 7fd42fe00700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430104500 0x7fd430199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:32:54.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.472+0000 7fd435200700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd430103be0 0x7fd4301991d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd42400e9a0 tx=0x7fd42400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:54.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.472+0000 7fd42d600700 1 -- 172.21.15.42:0/3068748293 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42400cbd0 con 0x7fd430103be0 2024-04-20T10:32:54.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.472+0000 7fd42d600700 1 -- 172.21.15.42:0/3068748293 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd42400cd70 con 0x7fd430103be0 2024-04-20T10:32:54.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.472+0000 7fd436600700 1 -- 172.21.15.42:0/3068748293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43019e480 con 0x7fd430103be0 2024-04-20T10:32:54.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.472+0000 7fd42d600700 1 -- 172.21.15.42:0/3068748293 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424018750 con 0x7fd430103be0 2024-04-20T10:32:54.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.473+0000 7fd436600700 1 -- 172.21.15.42:0/3068748293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4301a5d00 con 0x7fd430103be0 2024-04-20T10:32:54.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.474+0000 7fd436600700 1 -- 172.21.15.42:0/3068748293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd430069740 con 0x7fd430103be0 2024-04-20T10:32:54.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.476+0000 7fd42d600700 1 -- 172.21.15.42:0/3068748293 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd424010430 con 0x7fd430103be0 2024-04-20T10:32:54.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.478+0000 7fd42d600700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd4180a3d50 0x7fd4180a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:54.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.478+0000 7fd42d600700 1 -- 172.21.15.42:0/3068748293 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd4240d0850 con 0x7fd430103be0 2024-04-20T10:32:54.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.478+0000 7fd42fe00700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd4180a3d50 0x7fd4180a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:54.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.479+0000 7fd42fe00700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd4180a3d50 0x7fd4180a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd430100f40 tx=0x7fd41c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:54.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.482+0000 7fd42d600700 1 -- 172.21.15.42:0/3068748293 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd42409d050 con 0x7fd430103be0 2024-04-20T10:32:54.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.725+0000 7fd436600700 1 -- 172.21.15.42:0/3068748293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd43002df90 con 0x7fd430103be0 2024-04-20T10:32:54.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.726+0000 7fd42d600700 1 -- 172.21.15.42:0/3068748293 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd42409c950 con 0x7fd430103be0 2024-04-20T10:32:54.726 INFO:teuthology.orchestra.run.smithi042.stdout:343597383709 2024-04-20T10:32:54.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.729+0000 7fd436600700 1 -- 172.21.15.42:0/3068748293 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd4180a3d50 msgr2=0x7fd4180a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:54.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.729+0000 7fd436600700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd4180a3d50 0x7fd4180a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd430100f40 tx=0x7fd41c005fd0 comp rx=0 tx=0).stop 2024-04-20T10:32:54.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.729+0000 7fd436600700 1 -- 172.21.15.42:0/3068748293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd430103be0 msgr2=0x7fd4301991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:54.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.729+0000 7fd436600700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd430103be0 0x7fd4301991d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd42400e9a0 tx=0x7fd42400ecb0 comp rx=0 tx=0).stop 2024-04-20T10:32:54.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.729+0000 7fd436600700 1 -- 172.21.15.42:0/3068748293 shutdown_connections 2024-04-20T10:32:54.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.729+0000 7fd436600700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd4180a3d50 0x7fd4180a6210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:54.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.729+0000 7fd436600700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd430103be0 0x7fd4301991d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:54.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.729+0000 7fd436600700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd43010cfc0 0x7fd43019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:54.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.729+0000 7fd436600700 1 --2- 172.21.15.42:0/3068748293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430104500 0x7fd430199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:54.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.729+0000 7fd436600700 1 -- 172.21.15.42:0/3068748293 >> 172.21.15.42:0/3068748293 conn(0x7fd4300fd370 msgr2=0x7fd4300fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:54.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.729+0000 7fd436600700 1 -- 172.21.15.42:0/3068748293 shutdown_connections 2024-04-20T10:32:54.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:54.729+0000 7fd436600700 1 -- 172.21.15.42:0/3068748293 wait complete. 2024-04-20T10:32:55.100 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:54 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3068748293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-20T10:32:55.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:54 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3068748293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-20T10:32:55.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:54 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3068748293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-20T10:32:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:55 smithi042 ceph-mon[17078]: pgmap v359: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:56.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:55 smithi162 ceph-mon[21854]: pgmap v359: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:56.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:55 smithi046 ceph-mon[21574]: pgmap v359: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:32:56.588 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383703 got 343597383709 for osd.11 2024-04-20T10:32:56.588 DEBUG:teuthology.parallel:result is None 2024-04-20T10:32:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.102+0000 7f6a38e00700 1 -- 172.21.15.42:0/1042122152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a3406b080 msgr2=0x7f6a3410efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.102+0000 7f6a38e00700 1 --2- 172.21.15.42:0/1042122152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a3406b080 0x7f6a3410efc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009990 tx=0x7f6a20009ca0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.102+0000 7f6a38e00700 1 -- 172.21.15.42:0/1042122152 shutdown_connections 2024-04-20T10:32:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.102+0000 7f6a38e00700 1 --2- 172.21.15.42:0/1042122152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3410f500 0x7f6a341118f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.102+0000 7f6a38e00700 1 --2- 172.21.15.42:0/1042122152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3406a760 0x7f6a3406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.102+0000 7f6a38e00700 1 --2- 172.21.15.42:0/1042122152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a3406b080 0x7f6a3410efc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.102+0000 7f6a38e00700 1 -- 172.21.15.42:0/1042122152 >> 172.21.15.42:0/1042122152 conn(0x7f6a34078900 msgr2=0x7f6a34078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.102+0000 7f6a38e00700 1 -- 172.21.15.42:0/1042122152 shutdown_connections 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.102+0000 7f6a38e00700 1 -- 172.21.15.42:0/1042122152 wait complete. 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.104+0000 7f6a38e00700 1 Processor -- start 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.104+0000 7f6a38e00700 1 -- start start 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.104+0000 7f6a38e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3406a760 0x7f6a3419b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.104+0000 7f6a38e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a3406b080 0x7f6a3419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.104+0000 7f6a38e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3410f500 0x7f6a3419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.104+0000 7f6a38e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34114f60 con 0x7f6a3410f500 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.104+0000 7f6a38e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34114de0 con 0x7f6a3406b080 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.104+0000 7f6a38e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341150e0 con 0x7f6a3406a760 2024-04-20T10:32:57.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.105+0000 7f6a31400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a3406b080 0x7f6a3419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.105+0000 7f6a31e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3406a760 0x7f6a3419b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.105+0000 7f6a31400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a3406b080 0x7f6a3419ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:51126/0 (socket says 172.21.15.42:51126) 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.105+0000 7f6a31e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3406a760 0x7f6a3419b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:60726/0 (socket says 172.21.15.42:60726) 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.105+0000 7f6a31400700 1 -- 172.21.15.42:0/4117436427 learned_addr learned my addr 172.21.15.42:0/4117436427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.105+0000 7f6a32800700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3410f500 0x7f6a3419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.105+0000 7f6a31400700 1 -- 172.21.15.42:0/4117436427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3406a760 msgr2=0x7f6a3419b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.105+0000 7f6a31400700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3406a760 0x7f6a3419b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.105+0000 7f6a31400700 1 -- 172.21.15.42:0/4117436427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3410f500 msgr2=0x7f6a3419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.105+0000 7f6a31400700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3410f500 0x7f6a3419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.105+0000 7f6a31400700 1 -- 172.21.15.42:0/4117436427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a341a0500 con 0x7f6a3406b080 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.105+0000 7f6a31400700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a3406b080 0x7f6a3419ba30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6a20000c00 tx=0x7f6a2000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.106+0000 7f6a2aa00700 1 -- 172.21.15.42:0/4117436427 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20027070 con 0x7f6a3406b080 2024-04-20T10:32:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.106+0000 7f6a2aa00700 1 -- 172.21.15.42:0/4117436427 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a20003d70 con 0x7f6a3406b080 2024-04-20T10:32:57.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.106+0000 7f6a2aa00700 1 -- 172.21.15.42:0/4117436427 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2000f460 con 0x7f6a3406b080 2024-04-20T10:32:57.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.106+0000 7f6a38e00700 1 -- 172.21.15.42:0/4117436427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a341a0790 con 0x7f6a3406b080 2024-04-20T10:32:57.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.106+0000 7f6a38e00700 1 -- 172.21.15.42:0/4117436427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a341a8070 con 0x7f6a3406b080 2024-04-20T10:32:57.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.107+0000 7f6a38e00700 1 -- 172.21.15.42:0/4117436427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a34101f70 con 0x7f6a3406b080 2024-04-20T10:32:57.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.109+0000 7f6a2aa00700 1 -- 172.21.15.42:0/4117436427 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6a2002a070 con 0x7f6a3406b080 2024-04-20T10:32:57.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.111+0000 7f6a2aa00700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6a180a3cd0 0x7f6a180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:57.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.111+0000 7f6a2aa00700 1 -- 172.21.15.42:0/4117436427 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f6a200d13a0 con 0x7f6a3406b080 2024-04-20T10:32:57.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.111+0000 7f6a31e00700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6a180a3cd0 0x7f6a180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:57.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.112+0000 7f6a31e00700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6a180a3cd0 0x7f6a180a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009c60 tx=0x7f6a24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:57.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.115+0000 7f6a2aa00700 1 -- 172.21.15.42:0/4117436427 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6a20013070 con 0x7f6a3406b080 2024-04-20T10:32:57.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.353+0000 7f6a38e00700 1 -- 172.21.15.42:0/4117436427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6a3402df90 con 0x7f6a3406b080 2024-04-20T10:32:57.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.354+0000 7f6a2aa00700 1 -- 172.21.15.42:0/4117436427 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6a2009dc30 con 0x7f6a3406b080 2024-04-20T10:32:57.354 INFO:teuthology.orchestra.run.smithi042.stdout:77309411411 2024-04-20T10:32:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.358+0000 7f6a38e00700 1 -- 172.21.15.42:0/4117436427 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6a180a3cd0 msgr2=0x7f6a180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.358+0000 7f6a38e00700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6a180a3cd0 0x7f6a180a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009c60 tx=0x7f6a24009260 comp rx=0 tx=0).stop 2024-04-20T10:32:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.358+0000 7f6a38e00700 1 -- 172.21.15.42:0/4117436427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a3406b080 msgr2=0x7f6a3419ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.358+0000 7f6a38e00700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a3406b080 0x7f6a3419ba30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6a20000c00 tx=0x7f6a2000bab0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.358+0000 7f6a38e00700 1 -- 172.21.15.42:0/4117436427 shutdown_connections 2024-04-20T10:32:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.358+0000 7f6a38e00700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f6a180a3cd0 0x7f6a180a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.358+0000 7f6a38e00700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a3410f500 0x7f6a3419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.358+0000 7f6a38e00700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3406a760 0x7f6a3419b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.358+0000 7f6a38e00700 1 --2- 172.21.15.42:0/4117436427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a3406b080 0x7f6a3419ba30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.358+0000 7f6a38e00700 1 -- 172.21.15.42:0/4117436427 >> 172.21.15.42:0/4117436427 conn(0x7f6a34078900 msgr2=0x7f6a34110a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:57.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.358+0000 7f6a38e00700 1 -- 172.21.15.42:0/4117436427 shutdown_connections 2024-04-20T10:32:57.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.358+0000 7f6a38e00700 1 -- 172.21.15.42:0/4117436427 wait complete. 2024-04-20T10:32:57.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.413+0000 7f156d000700 1 -- 172.21.15.42:0/920900702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1568104ec0 msgr2=0x7f15681052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:57.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.413+0000 7f156d000700 1 --2- 172.21.15.42:0/920900702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1568104ec0 0x7f15681052a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1558009920 tx=0x7f1558009c30 comp rx=0 tx=0).stop 2024-04-20T10:32:57.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.413+0000 7f156d000700 1 -- 172.21.15.42:0/920900702 shutdown_connections 2024-04-20T10:32:57.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.413+0000 7f156d000700 1 --2- 172.21.15.42:0/920900702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1568109fa0 0x7f1568111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.413+0000 7f156d000700 1 --2- 172.21.15.42:0/920900702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1568104ec0 0x7f15681052a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.413+0000 7f156d000700 1 --2- 172.21.15.42:0/920900702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1568105870 0x7f15681098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.413+0000 7f156d000700 1 -- 172.21.15.42:0/920900702 >> 172.21.15.42:0/920900702 conn(0x7f1568100730 msgr2=0x7f1568102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:57.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.413+0000 7f156d000700 1 -- 172.21.15.42:0/920900702 shutdown_connections 2024-04-20T10:32:57.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.414+0000 7f156d000700 1 -- 172.21.15.42:0/920900702 wait complete. 2024-04-20T10:32:57.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.415+0000 7f156d000700 1 Processor -- start 2024-04-20T10:32:57.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.415+0000 7f156d000700 1 -- start start 2024-04-20T10:32:57.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.416+0000 7f156d000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1568104ec0 0x7f156819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:57.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.416+0000 7f156d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1568105870 0x7f156819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:57.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.416+0000 7f156d000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1568109fa0 0x7f156819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:57.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.416+0000 7f156d000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1568114fb0 con 0x7f1568105870 2024-04-20T10:32:57.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.416+0000 7f156d000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1568114e30 con 0x7f1568104ec0 2024-04-20T10:32:57.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.416+0000 7f156d000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1568115130 con 0x7f1568109fa0 2024-04-20T10:32:57.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.416+0000 7f1565e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1568104ec0 0x7f156819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:57.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.416+0000 7f1565e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1568104ec0 0x7f156819b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:51142/0 (socket says 172.21.15.42:51142) 2024-04-20T10:32:57.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.416+0000 7f1565e00700 1 -- 172.21.15.42:0/3171244884 learned_addr learned my addr 172.21.15.42:0/3171244884 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:32:57.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.417+0000 7f1565e00700 1 -- 172.21.15.42:0/3171244884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1568109fa0 msgr2=0x7f156819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:57.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.417+0000 7f1566800700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1568109fa0 0x7f156819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.417+0000 7f1565400700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1568105870 0x7f156819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.417+0000 7f1565e00700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1568109fa0 0x7f156819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.417+0000 7f1565e00700 1 -- 172.21.15.42:0/3171244884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1568105870 msgr2=0x7f156819b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.417+0000 7f1565e00700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1568105870 0x7f156819b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.417+0000 7f1565e00700 1 -- 172.21.15.42:0/3171244884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15681a04b0 con 0x7f1568104ec0 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.417+0000 7f1566800700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1568109fa0 0x7f156819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.417+0000 7f1565400700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1568105870 0x7f156819b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.417+0000 7f1565e00700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1568104ec0 0x7f156819b4a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1558005e90 tx=0x7f155800fb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.418+0000 7f155ea00700 1 -- 172.21.15.42:0/3171244884 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558005270 con 0x7f1568104ec0 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.418+0000 7f155ea00700 1 -- 172.21.15.42:0/3171244884 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f155800ba40 con 0x7f1568104ec0 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.418+0000 7f155ea00700 1 -- 172.21.15.42:0/3171244884 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558003c80 con 0x7f1568104ec0 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.418+0000 7f156d000700 1 -- 172.21.15.42:0/3171244884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15681a0740 con 0x7f1568104ec0 2024-04-20T10:32:57.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.418+0000 7f156d000700 1 -- 172.21.15.42:0/3171244884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15681a8020 con 0x7f1568104ec0 2024-04-20T10:32:57.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.419+0000 7f156d000700 1 -- 172.21.15.42:0/3171244884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15681063a0 con 0x7f1568104ec0 2024-04-20T10:32:57.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.421+0000 7f155ea00700 1 -- 172.21.15.42:0/3171244884 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1558005530 con 0x7f1568104ec0 2024-04-20T10:32:57.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.423+0000 7f155ea00700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f154c0a3d50 0x7f154c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:32:57.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.424+0000 7f155ea00700 1 -- 172.21.15.42:0/3171244884 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f15580d1780 con 0x7f1568104ec0 2024-04-20T10:32:57.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.424+0000 7f1565400700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f154c0a3d50 0x7f154c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:32:57.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.424+0000 7f1565400700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f154c0a3d50 0x7f154c0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f156810b550 tx=0x7f1554009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:32:57.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.425+0000 7f155ea00700 1 -- 172.21.15.42:0/3171244884 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1558013030 con 0x7f1568104ec0 2024-04-20T10:32:57.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.659+0000 7f156d000700 1 -- 172.21.15.42:0/3171244884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f156802df90 con 0x7f1568104ec0 2024-04-20T10:32:57.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.660+0000 7f155ea00700 1 -- 172.21.15.42:0/3171244884 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f155809e010 con 0x7f1568104ec0 2024-04-20T10:32:57.660 INFO:teuthology.orchestra.run.smithi042.stdout:197568495675 2024-04-20T10:32:57.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.664+0000 7f156d000700 1 -- 172.21.15.42:0/3171244884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f154c0a3d50 msgr2=0x7f154c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:57.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.664+0000 7f156d000700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f154c0a3d50 0x7f154c0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f156810b550 tx=0x7f1554009310 comp rx=0 tx=0).stop 2024-04-20T10:32:57.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.664+0000 7f156d000700 1 -- 172.21.15.42:0/3171244884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1568104ec0 msgr2=0x7f156819b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:32:57.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.664+0000 7f156d000700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1568104ec0 0x7f156819b4a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1558005e90 tx=0x7f155800fb40 comp rx=0 tx=0).stop 2024-04-20T10:32:57.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.664+0000 7f156d000700 1 -- 172.21.15.42:0/3171244884 shutdown_connections 2024-04-20T10:32:57.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.664+0000 7f156d000700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f154c0a3d50 0x7f154c0a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.664+0000 7f156d000700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1568104ec0 0x7f156819b4a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.664+0000 7f156d000700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1568109fa0 0x7f156819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.664+0000 7f156d000700 1 --2- 172.21.15.42:0/3171244884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1568105870 0x7f156819b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:32:57.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.664+0000 7f156d000700 1 -- 172.21.15.42:0/3171244884 >> 172.21.15.42:0/3171244884 conn(0x7f1568100730 msgr2=0x7f1568102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:32:57.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.665+0000 7f156d000700 1 -- 172.21.15.42:0/3171244884 shutdown_connections 2024-04-20T10:32:57.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:32:57.665+0000 7f156d000700 1 -- 172.21.15.42:0/3171244884 wait complete. 2024-04-20T10:32:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:57 smithi042 ceph-mon[17078]: pgmap v360: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:57 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4117436427' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T10:32:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:57 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3171244884' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-20T10:32:58.105 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:57 smithi046 ceph-mon[21574]: pgmap v360: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:58.105 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:57 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/4117436427' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T10:32:58.105 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:57 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3171244884' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-20T10:32:58.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:57 smithi162 ceph-mon[21854]: pgmap v360: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:32:58.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:57 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/4117436427' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T10:32:58.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:57 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3171244884' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-20T10:32:58.474 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411404 got 77309411411 for osd.2 2024-04-20T10:32:58.474 DEBUG:teuthology.parallel:result is None 2024-04-20T10:32:59.484 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495670 got 197568495675 for osd.6 2024-04-20T10:32:59.484 DEBUG:teuthology.parallel:result is None 2024-04-20T10:32:59.484 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-20T10:32:59.485 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph pg dump --format=json 2024-04-20T10:32:59.955 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:33:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:32:59 smithi042 ceph-mon[17078]: pgmap v361: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:00.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:32:59 smithi162 ceph-mon[21854]: pgmap v361: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:00.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:32:59 smithi046 ceph-mon[21574]: pgmap v361: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:02.080 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:01 smithi042 ceph-mon[17078]: pgmap v362: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:02.080 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:01 smithi042 ceph-mon[17078]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-20T10:33:02.080 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:01 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:33:02.080 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:01 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:33:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:01 smithi162 ceph-mon[21854]: pgmap v362: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:01 smithi162 ceph-mon[21854]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-20T10:33:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:01 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:33:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:01 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:33:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.253+0000 7f1795a00700 1 -- 172.21.15.42:0/2209330981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790103460 msgr2=0x7f179010f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.253+0000 7f1795a00700 1 --2- 172.21.15.42:0/2209330981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790103460 0x7f179010f8f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f178800b0a0 tx=0x7f178800b3b0 comp rx=0 tx=0).stop 2024-04-20T10:33:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.253+0000 7f1795a00700 1 -- 172.21.15.42:0/2209330981 shutdown_connections 2024-04-20T10:33:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.253+0000 7f1795a00700 1 --2- 172.21.15.42:0/2209330981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790103460 0x7f179010f8f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.253+0000 7f1795a00700 1 --2- 172.21.15.42:0/2209330981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1790108a90 0x7f1790108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.253+0000 7f1795a00700 1 --2- 172.21.15.42:0/2209330981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790102aa0 0x7f1790102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.253+0000 7f1795a00700 1 -- 172.21.15.42:0/2209330981 >> 172.21.15.42:0/2209330981 conn(0x7f1790079c00 msgr2=0x7f179007a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.253+0000 7f1795a00700 1 -- 172.21.15.42:0/2209330981 shutdown_connections 2024-04-20T10:33:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.253+0000 7f1795a00700 1 -- 172.21.15.42:0/2209330981 wait complete. 2024-04-20T10:33:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.254+0000 7f1795a00700 1 Processor -- start 2024-04-20T10:33:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.254+0000 7f1795a00700 1 -- start start 2024-04-20T10:33:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f1795a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790102aa0 0x7f179019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f1795a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790103460 0x7f179019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f1795a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1790108a90 0x7f179019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f1795a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1790112d70 con 0x7f1790103460 2024-04-20T10:33:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f1795a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1790112bf0 con 0x7f1790102aa0 2024-04-20T10:33:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f1795a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1790112ef0 con 0x7f1790108a90 2024-04-20T10:33:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f178e000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790103460 0x7f179019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f178ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790102aa0 0x7f179019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f178ea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790102aa0 0x7f179019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:57442/0 (socket says 172.21.15.42:57442) 2024-04-20T10:33:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f178ea00700 1 -- 172.21.15.42:0/1334603065 learned_addr learned my addr 172.21.15.42:0/1334603065 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f178f400700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1790108a90 0x7f179019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f178ea00700 1 -- 172.21.15.42:0/1334603065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1790108a90 msgr2=0x7f179019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f178ea00700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1790108a90 0x7f179019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f178ea00700 1 -- 172.21.15.42:0/1334603065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790103460 msgr2=0x7f179019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.255+0000 7f178ea00700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790103460 0x7f179019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.256+0000 7f178ea00700 1 -- 172.21.15.42:0/1334603065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17901a03a0 con 0x7f1790102aa0 2024-04-20T10:33:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.256+0000 7f178f400700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1790108a90 0x7f179019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:33:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.256+0000 7f178ea00700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790102aa0 0x7f179019b390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f178000e9a0 tx=0x7f178000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.256+0000 7f1787400700 1 -- 172.21.15.42:0/1334603065 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178000cbd0 con 0x7f1790102aa0 2024-04-20T10:33:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.256+0000 7f1787400700 1 -- 172.21.15.42:0/1334603065 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178000cd70 con 0x7f1790102aa0 2024-04-20T10:33:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.256+0000 7f1795a00700 1 -- 172.21.15.42:0/1334603065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17901a0690 con 0x7f1790102aa0 2024-04-20T10:33:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.256+0000 7f1787400700 1 -- 172.21.15.42:0/1334603065 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780018750 con 0x7f1790102aa0 2024-04-20T10:33:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.256+0000 7f1795a00700 1 -- 172.21.15.42:0/1334603065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901a7f10 con 0x7f1790102aa0 2024-04-20T10:33:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.258+0000 7f1795a00700 1 -- 172.21.15.42:0/1334603065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1790069740 con 0x7f1790102aa0 2024-04-20T10:33:02.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.259+0000 7f1787400700 1 -- 172.21.15.42:0/1334603065 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1780010430 con 0x7f1790102aa0 2024-04-20T10:33:02.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.261+0000 7f1787400700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f17740a3c40 0x7f17740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:02.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.261+0000 7f1787400700 1 -- 172.21.15.42:0/1334603065 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f17800d06b0 con 0x7f1790102aa0 2024-04-20T10:33:02.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.261+0000 7f178e000700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f17740a3c40 0x7f17740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:02.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.262+0000 7f178e000700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f17740a3c40 0x7f17740a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f179007a590 tx=0x7f177c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.265+0000 7f1787400700 1 -- 172.21.15.42:0/1334603065 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f178009d010 con 0x7f1790102aa0 2024-04-20T10:33:02.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:01 smithi046 ceph-mon[21574]: pgmap v362: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:02.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:01 smithi046 ceph-mon[21574]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-20T10:33:02.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:01 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:33:02.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:01 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:33:02.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.495+0000 7f1795a00700 1 -- 172.21.15.42:0/1334603065 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17900763a0 con 0x7f17740a3c40 2024-04-20T10:33:02.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.520+0000 7f1787400700 1 -- 172.21.15.42:0/1334603065 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258931 (secure 0 0 0) 0x7f17900763a0 con 0x7f17740a3c40 2024-04-20T10:33:02.526 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:02.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.526+0000 7f1795a00700 1 -- 172.21.15.42:0/1334603065 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f17740a3c40 msgr2=0x7f17740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:02.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.526+0000 7f1795a00700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f17740a3c40 0x7f17740a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f179007a590 tx=0x7f177c009260 comp rx=0 tx=0).stop 2024-04-20T10:33:02.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.526+0000 7f1795a00700 1 -- 172.21.15.42:0/1334603065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790102aa0 msgr2=0x7f179019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:02.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.526+0000 7f1795a00700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790102aa0 0x7f179019b390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f178000e9a0 tx=0x7f178000ecb0 comp rx=0 tx=0).stop 2024-04-20T10:33:02.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.526+0000 7f1795a00700 1 -- 172.21.15.42:0/1334603065 shutdown_connections 2024-04-20T10:33:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.526+0000 7f1795a00700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f17740a3c40 0x7f17740a6100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.526+0000 7f1795a00700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1790102aa0 0x7f179019b390 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.526+0000 7f1795a00700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1790108a90 0x7f179019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.526+0000 7f1795a00700 1 --2- 172.21.15.42:0/1334603065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1790103460 0x7f179019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.526+0000 7f1795a00700 1 -- 172.21.15.42:0/1334603065 >> 172.21.15.42:0/1334603065 conn(0x7f1790079c00 msgr2=0x7f1790074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.526+0000 7f1795a00700 1 -- 172.21.15.42:0/1334603065 shutdown_connections 2024-04-20T10:33:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:02.527+0000 7f1795a00700 1 -- 172.21.15.42:0/1334603065 wait complete. 2024-04-20T10:33:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-04-20T10:33:03.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:02 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:33:03.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:02 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:33:03.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:02 smithi042 ceph-mon[17078]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:33:03.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:02 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:33:03.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:02 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:33:03.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:02 smithi162 ceph-mon[21854]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:33:03.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:02 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:33:03.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:02 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:33:03.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:02 smithi046 ceph-mon[21574]: from='mgr.14152 172.21.15.42:0/1098121660' entity='mgr.x' 2024-04-20T10:33:03.439 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-20T10:33:00.545857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652446,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":302,"num_read_kb":263,"num_write":262,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364808,"kb_used_data":11412,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577016,"statfs":{"total":1151940427776,"available":1151566864384,"internally_reserved":0,"allocated":11685888,"data_stored":7427190,"data_compressed":182622,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005602"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.936640+0000","last_change":"2024-04-20T10:31:35.649585+0000","last_active":"2024-04-20T10:32:31.936640+0000","last_peered":"2024-04-20T10:32:31.936640+0000","last_clean":"2024-04-20T10:32:31.936640+0000","last_became_active":"2024-04-20T10:31:35.649282+0000","last_became_peered":"2024-04-20T10:31:35.649282+0000","last_unstale":"2024-04-20T10:32:31.936640+0000","last_undegraded":"2024-04-20T10:32:31.936640+0000","last_fullsized":"2024-04-20T10:32:31.936640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:01:00.517383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.947849+0000","last_change":"2024-04-20T10:31:35.648299+0000","last_active":"2024-04-20T10:32:31.947849+0000","last_peered":"2024-04-20T10:32:31.947849+0000","last_clean":"2024-04-20T10:32:31.947849+0000","last_became_active":"2024-04-20T10:31:35.647283+0000","last_became_peered":"2024-04-20T10:31:35.647283+0000","last_unstale":"2024-04-20T10:32:31.947849+0000","last_undegraded":"2024-04-20T10:32:31.947849+0000","last_fullsized":"2024-04-20T10:32:31.947849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:59.006234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.954284+0000","last_change":"2024-04-20T10:31:35.647029+0000","last_active":"2024-04-20T10:32:31.954284+0000","last_peered":"2024-04-20T10:32:31.954284+0000","last_clean":"2024-04-20T10:32:31.954284+0000","last_became_active":"2024-04-20T10:31:35.646649+0000","last_became_peered":"2024-04-20T10:31:35.646649+0000","last_unstale":"2024-04-20T10:32:31.954284+0000","last_undegraded":"2024-04-20T10:32:31.954284+0000","last_fullsized":"2024-04-20T10:32:31.954284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:03:23.420078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:32.041724+0000","last_change":"2024-04-20T10:31:35.624628+0000","last_active":"2024-04-20T10:32:32.041724+0000","last_peered":"2024-04-20T10:32:32.041724+0000","last_clean":"2024-04-20T10:32:32.041724+0000","last_became_active":"2024-04-20T10:31:35.623607+0000","last_became_peered":"2024-04-20T10:31:35.623607+0000","last_unstale":"2024-04-20T10:32:32.041724+0000","last_undegraded":"2024-04-20T10:32:32.041724+0000","last_fullsized":"2024-04-20T10:32:32.041724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:44:46.768584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934181+0000","last_change":"2024-04-20T10:31:35.638293+0000","last_active":"2024-04-20T10:32:31.934181+0000","last_peered":"2024-04-20T10:32:31.934181+0000","last_clean":"2024-04-20T10:32:31.934181+0000","last_became_active":"2024-04-20T10:31:35.637967+0000","last_became_peered":"2024-04-20T10:31:35.637967+0000","last_unstale":"2024-04-20T10:32:31.934181+0000","last_undegraded":"2024-04-20T10:32:31.934181+0000","last_fullsized":"2024-04-20T10:32:31.934181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:58:00.765015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.942279+0000","last_change":"2024-04-20T10:31:35.634565+0000","last_active":"2024-04-20T10:32:31.942279+0000","last_peered":"2024-04-20T10:32:31.942279+0000","last_clean":"2024-04-20T10:32:31.942279+0000","last_became_active":"2024-04-20T10:31:35.634200+0000","last_became_peered":"2024-04-20T10:31:35.634200+0000","last_unstale":"2024-04-20T10:32:31.942279+0000","last_undegraded":"2024-04-20T10:32:31.942279+0000","last_fullsized":"2024-04-20T10:32:31.942279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:14.298732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.949166+0000","last_change":"2024-04-20T10:32:31.949166+0000","last_active":"2024-04-20T10:32:31.949166+0000","last_peered":"2024-04-20T10:32:31.949166+0000","last_clean":"2024-04-20T10:32:31.949166+0000","last_became_active":"2024-04-20T10:32:31.948825+0000","last_became_peered":"2024-04-20T10:32:31.948825+0000","last_unstale":"2024-04-20T10:32:31.949166+0000","last_undegraded":"2024-04-20T10:32:31.949166+0000","last_fullsized":"2024-04-20T10:32:31.949166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:05.337941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.948416+0000","last_change":"2024-04-20T10:31:35.631996+0000","last_active":"2024-04-20T10:32:31.948416+0000","last_peered":"2024-04-20T10:32:31.948416+0000","last_clean":"2024-04-20T10:32:31.948416+0000","last_became_active":"2024-04-20T10:31:35.631576+0000","last_became_peered":"2024-04-20T10:31:35.631576+0000","last_unstale":"2024-04-20T10:32:31.948416+0000","last_undegraded":"2024-04-20T10:32:31.948416+0000","last_fullsized":"2024-04-20T10:32:31.948416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:51.448025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.941489+0000","last_change":"2024-04-20T10:31:35.624430+0000","last_active":"2024-04-20T10:32:31.941489+0000","last_peered":"2024-04-20T10:32:31.941489+0000","last_clean":"2024-04-20T10:32:31.941489+0000","last_became_active":"2024-04-20T10:31:35.623018+0000","last_became_peered":"2024-04-20T10:31:35.623018+0000","last_unstale":"2024-04-20T10:32:31.941489+0000","last_undegraded":"2024-04-20T10:32:31.941489+0000","last_fullsized":"2024-04-20T10:32:31.941489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:43.100925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.942776+0000","last_change":"2024-04-20T10:31:35.619038+0000","last_active":"2024-04-20T10:32:31.942776+0000","last_peered":"2024-04-20T10:32:31.942776+0000","last_clean":"2024-04-20T10:32:31.942776+0000","last_became_active":"2024-04-20T10:31:35.618699+0000","last_became_peered":"2024-04-20T10:31:35.618699+0000","last_unstale":"2024-04-20T10:32:31.942776+0000","last_undegraded":"2024-04-20T10:32:31.942776+0000","last_fullsized":"2024-04-20T10:32:31.942776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:27.205634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934491+0000","last_change":"2024-04-20T10:31:35.619908+0000","last_active":"2024-04-20T10:32:31.934491+0000","last_peered":"2024-04-20T10:32:31.934491+0000","last_clean":"2024-04-20T10:32:31.934491+0000","last_became_active":"2024-04-20T10:31:35.619580+0000","last_became_peered":"2024-04-20T10:31:35.619580+0000","last_unstale":"2024-04-20T10:32:31.934491+0000","last_undegraded":"2024-04-20T10:32:31.934491+0000","last_fullsized":"2024-04-20T10:32:31.934491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:54:48.999925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.956078+0000","last_change":"2024-04-20T10:32:31.952701+0000","last_active":"2024-04-20T10:32:31.956078+0000","last_peered":"2024-04-20T10:32:31.956078+0000","last_clean":"2024-04-20T10:32:31.956078+0000","last_became_active":"2024-04-20T10:32:31.951329+0000","last_became_peered":"2024-04-20T10:32:31.951329+0000","last_unstale":"2024-04-20T10:32:31.956078+0000","last_undegraded":"2024-04-20T10:32:31.956078+0000","last_fullsized":"2024-04-20T10:32:31.956078+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:29.442878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.941146+0000","last_change":"2024-04-20T10:31:35.619594+0000","last_active":"2024-04-20T10:32:31.941146+0000","last_peered":"2024-04-20T10:32:31.941146+0000","last_clean":"2024-04-20T10:32:31.941146+0000","last_became_active":"2024-04-20T10:31:35.618831+0000","last_became_peered":"2024-04-20T10:31:35.618831+0000","last_unstale":"2024-04-20T10:32:31.941146+0000","last_undegraded":"2024-04-20T10:32:31.941146+0000","last_fullsized":"2024-04-20T10:32:31.941146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:07.625225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.953763+0000","last_change":"2024-04-20T10:31:35.647476+0000","last_active":"2024-04-20T10:32:31.953763+0000","last_peered":"2024-04-20T10:32:31.953763+0000","last_clean":"2024-04-20T10:32:31.953763+0000","last_became_active":"2024-04-20T10:31:35.647009+0000","last_became_peered":"2024-04-20T10:31:35.647009+0000","last_unstale":"2024-04-20T10:32:31.953763+0000","last_undegraded":"2024-04-20T10:32:31.953763+0000","last_fullsized":"2024-04-20T10:32:31.953763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:11:37.526115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.958321+0000","last_change":"2024-04-20T10:32:31.958321+0000","last_active":"2024-04-20T10:32:31.958321+0000","last_peered":"2024-04-20T10:32:31.958321+0000","last_clean":"2024-04-20T10:32:31.958321+0000","last_became_active":"2024-04-20T10:32:31.957681+0000","last_became_peered":"2024-04-20T10:32:31.957681+0000","last_unstale":"2024-04-20T10:32:31.958321+0000","last_undegraded":"2024-04-20T10:32:31.958321+0000","last_fullsized":"2024-04-20T10:32:31.958321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:49:31.154909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.940632+0000","last_change":"2024-04-20T10:31:35.643142+0000","last_active":"2024-04-20T10:32:31.940632+0000","last_peered":"2024-04-20T10:32:31.940632+0000","last_clean":"2024-04-20T10:32:31.940632+0000","last_became_active":"2024-04-20T10:31:35.642742+0000","last_became_peered":"2024-04-20T10:31:35.642742+0000","last_unstale":"2024-04-20T10:32:31.940632+0000","last_undegraded":"2024-04-20T10:32:31.940632+0000","last_fullsized":"2024-04-20T10:32:31.940632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:40.818808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.936645+0000","last_change":"2024-04-20T10:31:35.650333+0000","last_active":"2024-04-20T10:32:31.936645+0000","last_peered":"2024-04-20T10:32:31.936645+0000","last_clean":"2024-04-20T10:32:31.936645+0000","last_became_active":"2024-04-20T10:31:35.649933+0000","last_became_peered":"2024-04-20T10:31:35.649933+0000","last_unstale":"2024-04-20T10:32:31.936645+0000","last_undegraded":"2024-04-20T10:32:31.936645+0000","last_fullsized":"2024-04-20T10:32:31.936645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:34.513990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.946171+0000","last_change":"2024-04-20T10:31:35.637315+0000","last_active":"2024-04-20T10:32:31.946171+0000","last_peered":"2024-04-20T10:32:31.946171+0000","last_clean":"2024-04-20T10:32:31.946171+0000","last_became_active":"2024-04-20T10:31:35.635909+0000","last_became_peered":"2024-04-20T10:31:35.635909+0000","last_unstale":"2024-04-20T10:32:31.946171+0000","last_undegraded":"2024-04-20T10:32:31.946171+0000","last_fullsized":"2024-04-20T10:32:31.946171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:24:07.400032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.938983+0000","last_change":"2024-04-20T10:31:35.638888+0000","last_active":"2024-04-20T10:32:31.938983+0000","last_peered":"2024-04-20T10:32:31.938983+0000","last_clean":"2024-04-20T10:32:31.938983+0000","last_became_active":"2024-04-20T10:31:35.638568+0000","last_became_peered":"2024-04-20T10:31:35.638568+0000","last_unstale":"2024-04-20T10:32:31.938983+0000","last_undegraded":"2024-04-20T10:32:31.938983+0000","last_fullsized":"2024-04-20T10:32:31.938983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:48.363874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.945343+0000","last_change":"2024-04-20T10:31:34.597814+0000","last_active":"2024-04-20T10:32:31.945343+0000","last_peered":"2024-04-20T10:32:31.945343+0000","last_clean":"2024-04-20T10:32:31.945343+0000","last_became_active":"2024-04-20T10:31:34.597480+0000","last_became_peered":"2024-04-20T10:31:34.597480+0000","last_unstale":"2024-04-20T10:32:31.945343+0000","last_undegraded":"2024-04-20T10:32:31.945343+0000","last_fullsized":"2024-04-20T10:32:31.945343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:48:26.102787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.958870+0000","last_change":"2024-04-20T10:32:31.956771+0000","last_active":"2024-04-20T10:32:31.958870+0000","last_peered":"2024-04-20T10:32:31.958870+0000","last_clean":"2024-04-20T10:32:31.958870+0000","last_became_active":"2024-04-20T10:32:31.955413+0000","last_became_peered":"2024-04-20T10:32:31.955413+0000","last_unstale":"2024-04-20T10:32:31.958870+0000","last_undegraded":"2024-04-20T10:32:31.958870+0000","last_fullsized":"2024-04-20T10:32:31.958870+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:37.199764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.952148+0000","last_change":"2024-04-20T10:31:34.609942+0000","last_active":"2024-04-20T10:32:31.952148+0000","last_peered":"2024-04-20T10:32:31.952148+0000","last_clean":"2024-04-20T10:32:31.952148+0000","last_became_active":"2024-04-20T10:31:34.608994+0000","last_became_peered":"2024-04-20T10:31:34.608994+0000","last_unstale":"2024-04-20T10:32:31.952148+0000","last_undegraded":"2024-04-20T10:32:31.952148+0000","last_fullsized":"2024-04-20T10:32:31.952148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:26:53.488091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.947010+0000","last_change":"2024-04-20T10:31:35.647710+0000","last_active":"2024-04-20T10:32:31.947010+0000","last_peered":"2024-04-20T10:32:31.947010+0000","last_clean":"2024-04-20T10:32:31.947010+0000","last_became_active":"2024-04-20T10:31:35.646037+0000","last_became_peered":"2024-04-20T10:31:35.646037+0000","last_unstale":"2024-04-20T10:32:31.947010+0000","last_undegraded":"2024-04-20T10:32:31.947010+0000","last_fullsized":"2024-04-20T10:32:31.947010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:50:21.092892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934609+0000","last_change":"2024-04-20T10:31:34.593529+0000","last_active":"2024-04-20T10:32:31.934609+0000","last_peered":"2024-04-20T10:32:31.934609+0000","last_clean":"2024-04-20T10:32:31.934609+0000","last_became_active":"2024-04-20T10:31:34.592583+0000","last_became_peered":"2024-04-20T10:31:34.592583+0000","last_unstale":"2024-04-20T10:32:31.934609+0000","last_undegraded":"2024-04-20T10:32:31.934609+0000","last_fullsized":"2024-04-20T10:32:31.934609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:51:20.570499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.940811+0000","last_change":"2024-04-20T10:31:35.630739+0000","last_active":"2024-04-20T10:32:31.940811+0000","last_peered":"2024-04-20T10:32:31.940811+0000","last_clean":"2024-04-20T10:32:31.940811+0000","last_became_active":"2024-04-20T10:31:35.630424+0000","last_became_peered":"2024-04-20T10:31:35.630424+0000","last_unstale":"2024-04-20T10:32:31.940811+0000","last_undegraded":"2024-04-20T10:32:31.940811+0000","last_fullsized":"2024-04-20T10:32:31.940811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:48:17.485677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934692+0000","last_change":"2024-04-20T10:31:34.591652+0000","last_active":"2024-04-20T10:32:31.934692+0000","last_peered":"2024-04-20T10:32:31.934692+0000","last_clean":"2024-04-20T10:32:31.934692+0000","last_became_active":"2024-04-20T10:31:34.588945+0000","last_became_peered":"2024-04-20T10:31:34.588945+0000","last_unstale":"2024-04-20T10:32:31.934692+0000","last_undegraded":"2024-04-20T10:32:31.934692+0000","last_fullsized":"2024-04-20T10:32:31.934692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:15.137046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937865+0000","last_change":"2024-04-20T10:31:35.622462+0000","last_active":"2024-04-20T10:32:31.937865+0000","last_peered":"2024-04-20T10:32:31.937865+0000","last_clean":"2024-04-20T10:32:31.937865+0000","last_became_active":"2024-04-20T10:31:35.622125+0000","last_became_peered":"2024-04-20T10:31:35.622125+0000","last_unstale":"2024-04-20T10:32:31.937865+0000","last_undegraded":"2024-04-20T10:32:31.937865+0000","last_fullsized":"2024-04-20T10:32:31.937865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:27.894564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:32.043629+0000","last_change":"2024-04-20T10:32:32.043629+0000","last_active":"2024-04-20T10:32:32.043629+0000","last_peered":"2024-04-20T10:32:32.043629+0000","last_clean":"2024-04-20T10:32:32.043629+0000","last_became_active":"2024-04-20T10:32:32.042849+0000","last_became_peered":"2024-04-20T10:32:32.042849+0000","last_unstale":"2024-04-20T10:32:32.043629+0000","last_undegraded":"2024-04-20T10:32:32.043629+0000","last_fullsized":"2024-04-20T10:32:32.043629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:52.646201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.951658+0000","last_change":"2024-04-20T10:32:31.951658+0000","last_active":"2024-04-20T10:32:31.951658+0000","last_peered":"2024-04-20T10:32:31.951658+0000","last_clean":"2024-04-20T10:32:31.951658+0000","last_became_active":"2024-04-20T10:32:31.951307+0000","last_became_peered":"2024-04-20T10:32:31.951307+0000","last_unstale":"2024-04-20T10:32:31.951658+0000","last_undegraded":"2024-04-20T10:32:31.951658+0000","last_fullsized":"2024-04-20T10:32:31.951658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:42.743969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.942138+0000","last_change":"2024-04-20T10:31:34.612038+0000","last_active":"2024-04-20T10:32:31.942138+0000","last_peered":"2024-04-20T10:32:31.942138+0000","last_clean":"2024-04-20T10:32:31.942138+0000","last_became_active":"2024-04-20T10:31:34.611671+0000","last_became_peered":"2024-04-20T10:31:34.611671+0000","last_unstale":"2024-04-20T10:32:31.942138+0000","last_undegraded":"2024-04-20T10:32:31.942138+0000","last_fullsized":"2024-04-20T10:32:31.942138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:47.862702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937551+0000","last_change":"2024-04-20T10:31:35.643930+0000","last_active":"2024-04-20T10:32:31.937551+0000","last_peered":"2024-04-20T10:32:31.937551+0000","last_clean":"2024-04-20T10:32:31.937551+0000","last_became_active":"2024-04-20T10:31:35.643470+0000","last_became_peered":"2024-04-20T10:31:35.643470+0000","last_unstale":"2024-04-20T10:32:31.937551+0000","last_undegraded":"2024-04-20T10:32:31.937551+0000","last_fullsized":"2024-04-20T10:32:31.937551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:55.570176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937828+0000","last_change":"2024-04-20T10:31:34.613224+0000","last_active":"2024-04-20T10:32:31.937828+0000","last_peered":"2024-04-20T10:32:31.937828+0000","last_clean":"2024-04-20T10:32:31.937828+0000","last_became_active":"2024-04-20T10:31:34.612871+0000","last_became_peered":"2024-04-20T10:31:34.612871+0000","last_unstale":"2024-04-20T10:32:31.937828+0000","last_undegraded":"2024-04-20T10:32:31.937828+0000","last_fullsized":"2024-04-20T10:32:31.937828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:05:57.284870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.948025+0000","last_change":"2024-04-20T10:31:35.643824+0000","last_active":"2024-04-20T10:32:31.948025+0000","last_peered":"2024-04-20T10:32:31.948025+0000","last_clean":"2024-04-20T10:32:31.948025+0000","last_became_active":"2024-04-20T10:31:35.643425+0000","last_became_peered":"2024-04-20T10:31:35.643425+0000","last_unstale":"2024-04-20T10:32:31.948025+0000","last_undegraded":"2024-04-20T10:32:31.948025+0000","last_fullsized":"2024-04-20T10:32:31.948025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:34.008374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.936942+0000","last_change":"2024-04-20T10:31:34.593071+0000","last_active":"2024-04-20T10:32:31.936942+0000","last_peered":"2024-04-20T10:32:31.936942+0000","last_clean":"2024-04-20T10:32:31.936942+0000","last_became_active":"2024-04-20T10:31:34.592721+0000","last_became_peered":"2024-04-20T10:31:34.592721+0000","last_unstale":"2024-04-20T10:32:31.936942+0000","last_undegraded":"2024-04-20T10:32:31.936942+0000","last_fullsized":"2024-04-20T10:32:31.936942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:23:43.749586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.944702+0000","last_change":"2024-04-20T10:31:35.647524+0000","last_active":"2024-04-20T10:32:31.944702+0000","last_peered":"2024-04-20T10:32:31.944702+0000","last_clean":"2024-04-20T10:32:31.944702+0000","last_became_active":"2024-04-20T10:31:35.647122+0000","last_became_peered":"2024-04-20T10:31:35.647122+0000","last_unstale":"2024-04-20T10:32:31.944702+0000","last_undegraded":"2024-04-20T10:32:31.944702+0000","last_fullsized":"2024-04-20T10:32:31.944702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.942771+0000","last_change":"2024-04-20T10:31:34.604500+0000","last_active":"2024-04-20T10:32:31.942771+0000","last_peered":"2024-04-20T10:32:31.942771+0000","last_clean":"2024-04-20T10:32:31.942771+0000","last_became_active":"2024-04-20T10:31:34.604148+0000","last_became_peered":"2024-04-20T10:31:34.604148+0000","last_unstale":"2024-04-20T10:32:31.942771+0000","last_undegraded":"2024-04-20T10:32:31.942771+0000","last_fullsized":"2024-04-20T10:32:31.942771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:49:15.174414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.941002+0000","last_change":"2024-04-20T10:31:35.636751+0000","last_active":"2024-04-20T10:32:31.941002+0000","last_peered":"2024-04-20T10:32:31.941002+0000","last_clean":"2024-04-20T10:32:31.941002+0000","last_became_active":"2024-04-20T10:31:35.636103+0000","last_became_peered":"2024-04-20T10:31:35.636103+0000","last_unstale":"2024-04-20T10:32:31.941002+0000","last_undegraded":"2024-04-20T10:32:31.941002+0000","last_fullsized":"2024-04-20T10:32:31.941002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:22:40.580512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934374+0000","last_change":"2024-04-20T10:31:34.595207+0000","last_active":"2024-04-20T10:32:31.934374+0000","last_peered":"2024-04-20T10:32:31.934374+0000","last_clean":"2024-04-20T10:32:31.934374+0000","last_became_active":"2024-04-20T10:31:34.594860+0000","last_became_peered":"2024-04-20T10:31:34.594860+0000","last_unstale":"2024-04-20T10:32:31.934374+0000","last_undegraded":"2024-04-20T10:32:31.934374+0000","last_fullsized":"2024-04-20T10:32:31.934374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:50.054454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.946702+0000","last_change":"2024-04-20T10:31:35.648086+0000","last_active":"2024-04-20T10:32:31.946702+0000","last_peered":"2024-04-20T10:32:31.946702+0000","last_clean":"2024-04-20T10:32:31.946702+0000","last_became_active":"2024-04-20T10:31:35.647275+0000","last_became_peered":"2024-04-20T10:31:35.647275+0000","last_unstale":"2024-04-20T10:32:31.946702+0000","last_undegraded":"2024-04-20T10:32:31.946702+0000","last_fullsized":"2024-04-20T10:32:31.946702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:27.669008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937923+0000","last_change":"2024-04-20T10:31:34.601674+0000","last_active":"2024-04-20T10:32:31.937923+0000","last_peered":"2024-04-20T10:32:31.937923+0000","last_clean":"2024-04-20T10:32:31.937923+0000","last_became_active":"2024-04-20T10:31:34.601255+0000","last_became_peered":"2024-04-20T10:31:34.601255+0000","last_unstale":"2024-04-20T10:32:31.937923+0000","last_undegraded":"2024-04-20T10:32:31.937923+0000","last_fullsized":"2024-04-20T10:32:31.937923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:57.821635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.941064+0000","last_change":"2024-04-20T10:31:35.621888+0000","last_active":"2024-04-20T10:32:31.941064+0000","last_peered":"2024-04-20T10:32:31.941064+0000","last_clean":"2024-04-20T10:32:31.941064+0000","last_became_active":"2024-04-20T10:31:35.621503+0000","last_became_peered":"2024-04-20T10:31:35.621503+0000","last_unstale":"2024-04-20T10:32:31.941064+0000","last_undegraded":"2024-04-20T10:32:31.941064+0000","last_fullsized":"2024-04-20T10:32:31.941064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:15:21.118638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'26","reported_seq":207,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:55.079065+0000","last_change":"2024-04-20T10:31:34.590558+0000","last_active":"2024-04-20T10:32:55.079065+0000","last_peered":"2024-04-20T10:32:55.079065+0000","last_clean":"2024-04-20T10:32:55.079065+0000","last_became_active":"2024-04-20T10:31:34.590140+0000","last_became_peered":"2024-04-20T10:31:34.590140+0000","last_unstale":"2024-04-20T10:32:55.079065+0000","last_undegraded":"2024-04-20T10:32:55.079065+0000","last_fullsized":"2024-04-20T10:32:55.079065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:15:18.591748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":140,"num_write":26,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934345+0000","last_change":"2024-04-20T10:31:35.620780+0000","last_active":"2024-04-20T10:32:31.934345+0000","last_peered":"2024-04-20T10:32:31.934345+0000","last_clean":"2024-04-20T10:32:31.934345+0000","last_became_active":"2024-04-20T10:31:35.620429+0000","last_became_peered":"2024-04-20T10:31:35.620429+0000","last_unstale":"2024-04-20T10:32:31.934345+0000","last_undegraded":"2024-04-20T10:32:31.934345+0000","last_fullsized":"2024-04-20T10:32:31.934345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:47:45.587920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.938798+0000","last_change":"2024-04-20T10:31:34.602135+0000","last_active":"2024-04-20T10:32:31.938798+0000","last_peered":"2024-04-20T10:32:31.938798+0000","last_clean":"2024-04-20T10:32:31.938798+0000","last_became_active":"2024-04-20T10:31:34.601460+0000","last_became_peered":"2024-04-20T10:31:34.601460+0000","last_unstale":"2024-04-20T10:32:31.938798+0000","last_undegraded":"2024-04-20T10:32:31.938798+0000","last_fullsized":"2024-04-20T10:32:31.938798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:00:00.832702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.939919+0000","last_change":"2024-04-20T10:31:35.635526+0000","last_active":"2024-04-20T10:32:31.939919+0000","last_peered":"2024-04-20T10:32:31.939919+0000","last_clean":"2024-04-20T10:32:31.939919+0000","last_became_active":"2024-04-20T10:31:35.635143+0000","last_became_peered":"2024-04-20T10:31:35.635143+0000","last_unstale":"2024-04-20T10:32:31.939919+0000","last_undegraded":"2024-04-20T10:32:31.939919+0000","last_fullsized":"2024-04-20T10:32:31.939919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:57:24.509318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937383+0000","last_change":"2024-04-20T10:31:34.604561+0000","last_active":"2024-04-20T10:32:31.937383+0000","last_peered":"2024-04-20T10:32:31.937383+0000","last_clean":"2024-04-20T10:32:31.937383+0000","last_became_active":"2024-04-20T10:31:34.604232+0000","last_became_peered":"2024-04-20T10:31:34.604232+0000","last_unstale":"2024-04-20T10:32:31.937383+0000","last_undegraded":"2024-04-20T10:32:31.937383+0000","last_fullsized":"2024-04-20T10:32:31.937383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:39.317473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934492+0000","last_change":"2024-04-20T10:31:35.632840+0000","last_active":"2024-04-20T10:32:31.934492+0000","last_peered":"2024-04-20T10:32:31.934492+0000","last_clean":"2024-04-20T10:32:31.934492+0000","last_became_active":"2024-04-20T10:31:35.632471+0000","last_became_peered":"2024-04-20T10:31:35.632471+0000","last_unstale":"2024-04-20T10:32:31.934492+0000","last_undegraded":"2024-04-20T10:32:31.934492+0000","last_fullsized":"2024-04-20T10:32:31.934492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:06.679967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:32.041323+0000","last_change":"2024-04-20T10:31:34.591696+0000","last_active":"2024-04-20T10:32:32.041323+0000","last_peered":"2024-04-20T10:32:32.041323+0000","last_clean":"2024-04-20T10:32:32.041323+0000","last_became_active":"2024-04-20T10:31:34.591330+0000","last_became_peered":"2024-04-20T10:31:34.591330+0000","last_unstale":"2024-04-20T10:32:32.041323+0000","last_undegraded":"2024-04-20T10:32:32.041323+0000","last_fullsized":"2024-04-20T10:32:32.041323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:44:36.385963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.933760+0000","last_change":"2024-04-20T10:31:35.627201+0000","last_active":"2024-04-20T10:32:31.933760+0000","last_peered":"2024-04-20T10:32:31.933760+0000","last_clean":"2024-04-20T10:32:31.933760+0000","last_became_active":"2024-04-20T10:31:35.626880+0000","last_became_peered":"2024-04-20T10:31:35.626880+0000","last_unstale":"2024-04-20T10:32:31.933760+0000","last_undegraded":"2024-04-20T10:32:31.933760+0000","last_fullsized":"2024-04-20T10:32:31.933760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:44.562723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934142+0000","last_change":"2024-04-20T10:31:34.600683+0000","last_active":"2024-04-20T10:32:31.934142+0000","last_peered":"2024-04-20T10:32:31.934142+0000","last_clean":"2024-04-20T10:32:31.934142+0000","last_became_active":"2024-04-20T10:31:34.600305+0000","last_became_peered":"2024-04-20T10:31:34.600305+0000","last_unstale":"2024-04-20T10:32:31.934142+0000","last_undegraded":"2024-04-20T10:32:31.934142+0000","last_fullsized":"2024-04-20T10:32:31.934142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:05.441158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937296+0000","last_change":"2024-04-20T10:31:35.619835+0000","last_active":"2024-04-20T10:32:31.937296+0000","last_peered":"2024-04-20T10:32:31.937296+0000","last_clean":"2024-04-20T10:32:31.937296+0000","last_became_active":"2024-04-20T10:31:35.619554+0000","last_became_peered":"2024-04-20T10:31:35.619554+0000","last_unstale":"2024-04-20T10:32:31.937296+0000","last_undegraded":"2024-04-20T10:32:31.937296+0000","last_fullsized":"2024-04-20T10:32:31.937296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:33.161563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934465+0000","last_change":"2024-04-20T10:31:34.591128+0000","last_active":"2024-04-20T10:32:31.934465+0000","last_peered":"2024-04-20T10:32:31.934465+0000","last_clean":"2024-04-20T10:32:31.934465+0000","last_became_active":"2024-04-20T10:31:34.587772+0000","last_became_peered":"2024-04-20T10:31:34.587772+0000","last_unstale":"2024-04-20T10:32:31.934465+0000","last_undegraded":"2024-04-20T10:32:31.934465+0000","last_fullsized":"2024-04-20T10:32:31.934465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:12:42.541111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.936502+0000","last_change":"2024-04-20T10:31:35.616743+0000","last_active":"2024-04-20T10:32:31.936502+0000","last_peered":"2024-04-20T10:32:31.936502+0000","last_clean":"2024-04-20T10:32:31.936502+0000","last_became_active":"2024-04-20T10:31:35.616419+0000","last_became_peered":"2024-04-20T10:31:35.616419+0000","last_unstale":"2024-04-20T10:32:31.936502+0000","last_undegraded":"2024-04-20T10:32:31.936502+0000","last_fullsized":"2024-04-20T10:32:31.936502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:37.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.945445+0000","last_change":"2024-04-20T10:31:34.603734+0000","last_active":"2024-04-20T10:32:31.945445+0000","last_peered":"2024-04-20T10:32:31.945445+0000","last_clean":"2024-04-20T10:32:31.945445+0000","last_became_active":"2024-04-20T10:31:34.603388+0000","last_became_peered":"2024-04-20T10:31:34.603388+0000","last_unstale":"2024-04-20T10:32:31.945445+0000","last_undegraded":"2024-04-20T10:32:31.945445+0000","last_fullsized":"2024-04-20T10:32:31.945445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:58:57.199173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.942650+0000","last_change":"2024-04-20T10:31:35.638996+0000","last_active":"2024-04-20T10:32:31.942650+0000","last_peered":"2024-04-20T10:32:31.942650+0000","last_clean":"2024-04-20T10:32:31.942650+0000","last_became_active":"2024-04-20T10:31:35.638571+0000","last_became_peered":"2024-04-20T10:31:35.638571+0000","last_unstale":"2024-04-20T10:32:31.942650+0000","last_undegraded":"2024-04-20T10:32:31.942650+0000","last_fullsized":"2024-04-20T10:32:31.942650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:59.541738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934002+0000","last_change":"2024-04-20T10:31:34.612501+0000","last_active":"2024-04-20T10:32:31.934002+0000","last_peered":"2024-04-20T10:32:31.934002+0000","last_clean":"2024-04-20T10:32:31.934002+0000","last_became_active":"2024-04-20T10:31:34.612150+0000","last_became_peered":"2024-04-20T10:31:34.612150+0000","last_unstale":"2024-04-20T10:32:31.934002+0000","last_undegraded":"2024-04-20T10:32:31.934002+0000","last_fullsized":"2024-04-20T10:32:31.934002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:28.172851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.957767+0000","last_change":"2024-04-20T10:32:31.957767+0000","last_active":"2024-04-20T10:32:31.957767+0000","last_peered":"2024-04-20T10:32:31.957767+0000","last_clean":"2024-04-20T10:32:31.957767+0000","last_became_active":"2024-04-20T10:32:31.957319+0000","last_became_peered":"2024-04-20T10:32:31.957319+0000","last_unstale":"2024-04-20T10:32:31.957767+0000","last_undegraded":"2024-04-20T10:32:31.957767+0000","last_fullsized":"2024-04-20T10:32:31.957767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:56.375705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937052+0000","last_change":"2024-04-20T10:31:35.617775+0000","last_active":"2024-04-20T10:32:31.937052+0000","last_peered":"2024-04-20T10:32:31.937052+0000","last_clean":"2024-04-20T10:32:31.937052+0000","last_became_active":"2024-04-20T10:31:35.617465+0000","last_became_peered":"2024-04-20T10:31:35.617465+0000","last_unstale":"2024-04-20T10:32:31.937052+0000","last_undegraded":"2024-04-20T10:32:31.937052+0000","last_fullsized":"2024-04-20T10:32:31.937052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:37.830953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937866+0000","last_change":"2024-04-20T10:31:34.615044+0000","last_active":"2024-04-20T10:32:31.937866+0000","last_peered":"2024-04-20T10:32:31.937866+0000","last_clean":"2024-04-20T10:32:31.937866+0000","last_became_active":"2024-04-20T10:31:34.612875+0000","last_became_peered":"2024-04-20T10:31:34.612875+0000","last_unstale":"2024-04-20T10:32:31.937866+0000","last_undegraded":"2024-04-20T10:32:31.937866+0000","last_fullsized":"2024-04-20T10:32:31.937866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:59.867501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.949863+0000","last_change":"2024-04-20T10:31:35.630909+0000","last_active":"2024-04-20T10:32:31.949863+0000","last_peered":"2024-04-20T10:32:31.949863+0000","last_clean":"2024-04-20T10:32:31.949863+0000","last_became_active":"2024-04-20T10:31:35.630455+0000","last_became_peered":"2024-04-20T10:31:35.630455+0000","last_unstale":"2024-04-20T10:32:31.949863+0000","last_undegraded":"2024-04-20T10:32:31.949863+0000","last_fullsized":"2024-04-20T10:32:31.949863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:02.522485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:32.041897+0000","last_change":"2024-04-20T10:31:34.602466+0000","last_active":"2024-04-20T10:32:32.041897+0000","last_peered":"2024-04-20T10:32:32.041897+0000","last_clean":"2024-04-20T10:32:32.041897+0000","last_became_active":"2024-04-20T10:31:34.602089+0000","last_became_peered":"2024-04-20T10:31:34.602089+0000","last_unstale":"2024-04-20T10:32:32.041897+0000","last_undegraded":"2024-04-20T10:32:32.041897+0000","last_fullsized":"2024-04-20T10:32:32.041897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:19:09.849840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.933929+0000","last_change":"2024-04-20T10:30:14.724347+0000","last_active":"2024-04-20T10:32:31.933929+0000","last_peered":"2024-04-20T10:32:31.933929+0000","last_clean":"2024-04-20T10:32:31.933929+0000","last_became_active":"2024-04-20T10:30:14.723877+0000","last_became_peered":"2024-04-20T10:30:14.723877+0000","last_unstale":"2024-04-20T10:32:31.933929+0000","last_undegraded":"2024-04-20T10:32:31.933929+0000","last_fullsized":"2024-04-20T10:32:31.933929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_clean_scrub_stamp":"2024-04-20T10:26:16.803986+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:30.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937510+0000","last_change":"2024-04-20T10:31:35.626505+0000","last_active":"2024-04-20T10:32:31.937510+0000","last_peered":"2024-04-20T10:32:31.937510+0000","last_clean":"2024-04-20T10:32:31.937510+0000","last_became_active":"2024-04-20T10:31:35.626141+0000","last_became_peered":"2024-04-20T10:31:35.626141+0000","last_unstale":"2024-04-20T10:32:31.937510+0000","last_undegraded":"2024-04-20T10:32:31.937510+0000","last_fullsized":"2024-04-20T10:32:31.937510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:29.827225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.933993+0000","last_change":"2024-04-20T10:31:34.596936+0000","last_active":"2024-04-20T10:32:31.933993+0000","last_peered":"2024-04-20T10:32:31.933993+0000","last_clean":"2024-04-20T10:32:31.933993+0000","last_became_active":"2024-04-20T10:31:34.596692+0000","last_became_peered":"2024-04-20T10:31:34.596692+0000","last_unstale":"2024-04-20T10:32:31.933993+0000","last_undegraded":"2024-04-20T10:32:31.933993+0000","last_fullsized":"2024-04-20T10:32:31.933993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:02:39.819096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934417+0000","last_change":"2024-04-20T10:31:34.601407+0000","last_active":"2024-04-20T10:32:31.934417+0000","last_peered":"2024-04-20T10:32:31.934417+0000","last_clean":"2024-04-20T10:32:31.934417+0000","last_became_active":"2024-04-20T10:31:34.601007+0000","last_became_peered":"2024-04-20T10:31:34.601007+0000","last_unstale":"2024-04-20T10:32:31.934417+0000","last_undegraded":"2024-04-20T10:32:31.934417+0000","last_fullsized":"2024-04-20T10:32:31.934417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:05:03.037841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.939916+0000","last_change":"2024-04-20T10:31:35.641046+0000","last_active":"2024-04-20T10:32:31.939916+0000","last_peered":"2024-04-20T10:32:31.939916+0000","last_clean":"2024-04-20T10:32:31.939916+0000","last_became_active":"2024-04-20T10:31:35.640666+0000","last_became_peered":"2024-04-20T10:31:35.640666+0000","last_unstale":"2024-04-20T10:32:31.939916+0000","last_undegraded":"2024-04-20T10:32:31.939916+0000","last_fullsized":"2024-04-20T10:32:31.939916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:42.496037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.946031+0000","last_change":"2024-04-20T10:31:34.604643+0000","last_active":"2024-04-20T10:32:31.946031+0000","last_peered":"2024-04-20T10:32:31.946031+0000","last_clean":"2024-04-20T10:32:31.946031+0000","last_became_active":"2024-04-20T10:31:34.604210+0000","last_became_peered":"2024-04-20T10:31:34.604210+0000","last_unstale":"2024-04-20T10:32:31.946031+0000","last_undegraded":"2024-04-20T10:32:31.946031+0000","last_fullsized":"2024-04-20T10:32:31.946031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:24:09.748012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.952607+0000","last_change":"2024-04-20T10:31:35.641981+0000","last_active":"2024-04-20T10:32:31.952607+0000","last_peered":"2024-04-20T10:32:31.952607+0000","last_clean":"2024-04-20T10:32:31.952607+0000","last_became_active":"2024-04-20T10:31:35.641607+0000","last_became_peered":"2024-04-20T10:31:35.641607+0000","last_unstale":"2024-04-20T10:32:31.952607+0000","last_undegraded":"2024-04-20T10:32:31.952607+0000","last_fullsized":"2024-04-20T10:32:31.952607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:48.520813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:32.041725+0000","last_change":"2024-04-20T10:31:34.611218+0000","last_active":"2024-04-20T10:32:32.041725+0000","last_peered":"2024-04-20T10:32:32.041725+0000","last_clean":"2024-04-20T10:32:32.041725+0000","last_became_active":"2024-04-20T10:31:34.610950+0000","last_became_peered":"2024-04-20T10:31:34.610950+0000","last_unstale":"2024-04-20T10:32:32.041725+0000","last_undegraded":"2024-04-20T10:32:32.041725+0000","last_fullsized":"2024-04-20T10:32:32.041725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:14:11.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.948170+0000","last_change":"2024-04-20T10:31:35.647531+0000","last_active":"2024-04-20T10:32:31.948170+0000","last_peered":"2024-04-20T10:32:31.948170+0000","last_clean":"2024-04-20T10:32:31.948170+0000","last_became_active":"2024-04-20T10:31:35.646389+0000","last_became_peered":"2024-04-20T10:31:35.646389+0000","last_unstale":"2024-04-20T10:32:31.948170+0000","last_undegraded":"2024-04-20T10:32:31.948170+0000","last_fullsized":"2024-04-20T10:32:31.948170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:50.141385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.957675+0000","last_change":"2024-04-20T10:32:31.957675+0000","last_active":"2024-04-20T10:32:31.957675+0000","last_peered":"2024-04-20T10:32:31.957675+0000","last_clean":"2024-04-20T10:32:31.957675+0000","last_became_active":"2024-04-20T10:32:31.957202+0000","last_became_peered":"2024-04-20T10:32:31.957202+0000","last_unstale":"2024-04-20T10:32:31.957675+0000","last_undegraded":"2024-04-20T10:32:31.957675+0000","last_fullsized":"2024-04-20T10:32:31.957675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:28.001088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934134+0000","last_change":"2024-04-20T10:31:35.616751+0000","last_active":"2024-04-20T10:32:31.934134+0000","last_peered":"2024-04-20T10:32:31.934134+0000","last_clean":"2024-04-20T10:32:31.934134+0000","last_became_active":"2024-04-20T10:31:35.616432+0000","last_became_peered":"2024-04-20T10:31:35.616432+0000","last_unstale":"2024-04-20T10:32:31.934134+0000","last_undegraded":"2024-04-20T10:32:31.934134+0000","last_fullsized":"2024-04-20T10:32:31.934134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:25.052140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.948999+0000","last_change":"2024-04-20T10:31:34.609139+0000","last_active":"2024-04-20T10:32:31.948999+0000","last_peered":"2024-04-20T10:32:31.948999+0000","last_clean":"2024-04-20T10:32:31.948999+0000","last_became_active":"2024-04-20T10:31:34.608818+0000","last_became_peered":"2024-04-20T10:31:34.608818+0000","last_unstale":"2024-04-20T10:32:31.948999+0000","last_undegraded":"2024-04-20T10:32:31.948999+0000","last_fullsized":"2024-04-20T10:32:31.948999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:42:39.581059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.936334+0000","last_change":"2024-04-20T10:31:35.643791+0000","last_active":"2024-04-20T10:32:31.936334+0000","last_peered":"2024-04-20T10:32:31.936334+0000","last_clean":"2024-04-20T10:32:31.936334+0000","last_became_active":"2024-04-20T10:31:35.643390+0000","last_became_peered":"2024-04-20T10:31:35.643390+0000","last_unstale":"2024-04-20T10:32:31.936334+0000","last_undegraded":"2024-04-20T10:32:31.936334+0000","last_fullsized":"2024-04-20T10:32:31.936334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:10.149979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.939109+0000","last_change":"2024-04-20T10:31:34.606178+0000","last_active":"2024-04-20T10:32:31.939109+0000","last_peered":"2024-04-20T10:32:31.939109+0000","last_clean":"2024-04-20T10:32:31.939109+0000","last_became_active":"2024-04-20T10:31:34.605817+0000","last_became_peered":"2024-04-20T10:31:34.605817+0000","last_unstale":"2024-04-20T10:32:31.939109+0000","last_undegraded":"2024-04-20T10:32:31.939109+0000","last_fullsized":"2024-04-20T10:32:31.939109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:15.929905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.951003+0000","last_change":"2024-04-20T10:31:35.649487+0000","last_active":"2024-04-20T10:32:31.951003+0000","last_peered":"2024-04-20T10:32:31.951003+0000","last_clean":"2024-04-20T10:32:31.951003+0000","last_became_active":"2024-04-20T10:31:35.648884+0000","last_became_peered":"2024-04-20T10:31:35.648884+0000","last_unstale":"2024-04-20T10:32:31.951003+0000","last_undegraded":"2024-04-20T10:32:31.951003+0000","last_fullsized":"2024-04-20T10:32:31.951003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:07.389100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.943441+0000","last_change":"2024-04-20T10:32:31.943005+0000","last_active":"2024-04-20T10:32:31.943441+0000","last_peered":"2024-04-20T10:32:31.943441+0000","last_clean":"2024-04-20T10:32:31.943441+0000","last_became_active":"2024-04-20T10:32:31.940063+0000","last_became_peered":"2024-04-20T10:32:31.940063+0000","last_unstale":"2024-04-20T10:32:31.943441+0000","last_undegraded":"2024-04-20T10:32:31.943441+0000","last_fullsized":"2024-04-20T10:32:31.943441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:11:06.308028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934133+0000","last_change":"2024-04-20T10:31:35.630256+0000","last_active":"2024-04-20T10:32:31.934133+0000","last_peered":"2024-04-20T10:32:31.934133+0000","last_clean":"2024-04-20T10:32:31.934133+0000","last_became_active":"2024-04-20T10:31:35.629879+0000","last_became_peered":"2024-04-20T10:31:35.629879+0000","last_unstale":"2024-04-20T10:32:31.934133+0000","last_undegraded":"2024-04-20T10:32:31.934133+0000","last_fullsized":"2024-04-20T10:32:31.934133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:51.601923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.939172+0000","last_change":"2024-04-20T10:31:34.595149+0000","last_active":"2024-04-20T10:32:31.939172+0000","last_peered":"2024-04-20T10:32:31.939172+0000","last_clean":"2024-04-20T10:32:31.939172+0000","last_became_active":"2024-04-20T10:31:34.594808+0000","last_became_peered":"2024-04-20T10:31:34.594808+0000","last_unstale":"2024-04-20T10:32:31.939172+0000","last_undegraded":"2024-04-20T10:32:31.939172+0000","last_fullsized":"2024-04-20T10:32:31.939172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:51.707710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934496+0000","last_change":"2024-04-20T10:31:35.620612+0000","last_active":"2024-04-20T10:32:31.934496+0000","last_peered":"2024-04-20T10:32:31.934496+0000","last_clean":"2024-04-20T10:32:31.934496+0000","last_became_active":"2024-04-20T10:31:35.620312+0000","last_became_peered":"2024-04-20T10:31:35.620312+0000","last_unstale":"2024-04-20T10:32:31.934496+0000","last_undegraded":"2024-04-20T10:32:31.934496+0000","last_fullsized":"2024-04-20T10:32:31.934496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:37:50.756950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937750+0000","last_change":"2024-04-20T10:31:34.604613+0000","last_active":"2024-04-20T10:32:31.937750+0000","last_peered":"2024-04-20T10:32:31.937750+0000","last_clean":"2024-04-20T10:32:31.937750+0000","last_became_active":"2024-04-20T10:31:34.604246+0000","last_became_peered":"2024-04-20T10:31:34.604246+0000","last_unstale":"2024-04-20T10:32:31.937750+0000","last_undegraded":"2024-04-20T10:32:31.937750+0000","last_fullsized":"2024-04-20T10:32:31.937750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:26.521769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934197+0000","last_change":"2024-04-20T10:31:35.639396+0000","last_active":"2024-04-20T10:32:31.934197+0000","last_peered":"2024-04-20T10:32:31.934197+0000","last_clean":"2024-04-20T10:32:31.934197+0000","last_became_active":"2024-04-20T10:31:35.638979+0000","last_became_peered":"2024-04-20T10:31:35.638979+0000","last_unstale":"2024-04-20T10:32:31.934197+0000","last_undegraded":"2024-04-20T10:32:31.934197+0000","last_fullsized":"2024-04-20T10:32:31.934197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:47.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.941634+0000","last_change":"2024-04-20T10:31:34.607420+0000","last_active":"2024-04-20T10:32:31.941634+0000","last_peered":"2024-04-20T10:32:31.941634+0000","last_clean":"2024-04-20T10:32:31.941634+0000","last_became_active":"2024-04-20T10:31:34.602587+0000","last_became_peered":"2024-04-20T10:31:34.602587+0000","last_unstale":"2024-04-20T10:32:31.941634+0000","last_undegraded":"2024-04-20T10:32:31.941634+0000","last_fullsized":"2024-04-20T10:32:31.941634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:56.447944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934549+0000","last_change":"2024-04-20T10:31:35.624903+0000","last_active":"2024-04-20T10:32:31.934549+0000","last_peered":"2024-04-20T10:32:31.934549+0000","last_clean":"2024-04-20T10:32:31.934549+0000","last_became_active":"2024-04-20T10:31:35.619479+0000","last_became_peered":"2024-04-20T10:31:35.619479+0000","last_unstale":"2024-04-20T10:32:31.934549+0000","last_undegraded":"2024-04-20T10:32:31.934549+0000","last_fullsized":"2024-04-20T10:32:31.934549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:25:40.601983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.936383+0000","last_change":"2024-04-20T10:31:35.620351+0000","last_active":"2024-04-20T10:32:31.936383+0000","last_peered":"2024-04-20T10:32:31.936383+0000","last_clean":"2024-04-20T10:32:31.936383+0000","last_became_active":"2024-04-20T10:31:35.620028+0000","last_became_peered":"2024-04-20T10:31:35.620028+0000","last_unstale":"2024-04-20T10:32:31.936383+0000","last_undegraded":"2024-04-20T10:32:31.936383+0000","last_fullsized":"2024-04-20T10:32:31.936383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:51:53.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.938835+0000","last_change":"2024-04-20T10:31:35.615917+0000","last_active":"2024-04-20T10:32:31.938835+0000","last_peered":"2024-04-20T10:32:31.938835+0000","last_clean":"2024-04-20T10:32:31.938835+0000","last_became_active":"2024-04-20T10:31:35.615582+0000","last_became_peered":"2024-04-20T10:31:35.615582+0000","last_unstale":"2024-04-20T10:32:31.938835+0000","last_undegraded":"2024-04-20T10:32:31.938835+0000","last_fullsized":"2024-04-20T10:32:31.938835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:54:59.754819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.938547+0000","last_change":"2024-04-20T10:31:35.650006+0000","last_active":"2024-04-20T10:32:31.938547+0000","last_peered":"2024-04-20T10:32:31.938547+0000","last_clean":"2024-04-20T10:32:31.938547+0000","last_became_active":"2024-04-20T10:31:35.649683+0000","last_became_peered":"2024-04-20T10:31:35.649683+0000","last_unstale":"2024-04-20T10:32:31.938547+0000","last_undegraded":"2024-04-20T10:32:31.938547+0000","last_fullsized":"2024-04-20T10:32:31.938547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:10:01.484809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.952062+0000","last_change":"2024-04-20T10:31:35.646591+0000","last_active":"2024-04-20T10:32:31.952062+0000","last_peered":"2024-04-20T10:32:31.952062+0000","last_clean":"2024-04-20T10:32:31.952062+0000","last_became_active":"2024-04-20T10:31:35.646188+0000","last_became_peered":"2024-04-20T10:31:35.646188+0000","last_unstale":"2024-04-20T10:32:31.952062+0000","last_undegraded":"2024-04-20T10:32:31.952062+0000","last_fullsized":"2024-04-20T10:32:31.952062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:23.412022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.939639+0000","last_change":"2024-04-20T10:31:35.650879+0000","last_active":"2024-04-20T10:32:31.939639+0000","last_peered":"2024-04-20T10:32:31.939639+0000","last_clean":"2024-04-20T10:32:31.939639+0000","last_became_active":"2024-04-20T10:31:35.650486+0000","last_became_peered":"2024-04-20T10:31:35.650486+0000","last_unstale":"2024-04-20T10:32:31.939639+0000","last_undegraded":"2024-04-20T10:32:31.939639+0000","last_fullsized":"2024-04-20T10:32:31.939639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:51.893446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.936276+0000","last_change":"2024-04-20T10:31:35.619883+0000","last_active":"2024-04-20T10:32:31.936276+0000","last_peered":"2024-04-20T10:32:31.936276+0000","last_clean":"2024-04-20T10:32:31.936276+0000","last_became_active":"2024-04-20T10:31:35.619628+0000","last_became_peered":"2024-04-20T10:31:35.619628+0000","last_unstale":"2024-04-20T10:32:31.936276+0000","last_undegraded":"2024-04-20T10:32:31.936276+0000","last_fullsized":"2024-04-20T10:32:31.936276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:45.018417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.939330+0000","last_change":"2024-04-20T10:31:35.620773+0000","last_active":"2024-04-20T10:32:31.939330+0000","last_peered":"2024-04-20T10:32:31.939330+0000","last_clean":"2024-04-20T10:32:31.939330+0000","last_became_active":"2024-04-20T10:31:35.620467+0000","last_became_peered":"2024-04-20T10:31:35.620467+0000","last_unstale":"2024-04-20T10:32:31.939330+0000","last_undegraded":"2024-04-20T10:32:31.939330+0000","last_fullsized":"2024-04-20T10:32:31.939330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:31:48.009168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.953580+0000","last_change":"2024-04-20T10:32:31.953580+0000","last_active":"2024-04-20T10:32:31.953580+0000","last_peered":"2024-04-20T10:32:31.953580+0000","last_clean":"2024-04-20T10:32:31.953580+0000","last_became_active":"2024-04-20T10:32:31.952985+0000","last_became_peered":"2024-04-20T10:32:31.952985+0000","last_unstale":"2024-04-20T10:32:31.953580+0000","last_undegraded":"2024-04-20T10:32:31.953580+0000","last_fullsized":"2024-04-20T10:32:31.953580+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:13:03.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.936977+0000","last_change":"2024-04-20T10:31:35.637588+0000","last_active":"2024-04-20T10:32:31.936977+0000","last_peered":"2024-04-20T10:32:31.936977+0000","last_clean":"2024-04-20T10:32:31.936977+0000","last_became_active":"2024-04-20T10:31:35.637319+0000","last_became_peered":"2024-04-20T10:31:35.637319+0000","last_unstale":"2024-04-20T10:32:31.936977+0000","last_undegraded":"2024-04-20T10:32:31.936977+0000","last_fullsized":"2024-04-20T10:32:31.936977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:54.070894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.948912+0000","last_change":"2024-04-20T10:31:35.629551+0000","last_active":"2024-04-20T10:32:31.948912+0000","last_peered":"2024-04-20T10:32:31.948912+0000","last_clean":"2024-04-20T10:32:31.948912+0000","last_became_active":"2024-04-20T10:31:35.628710+0000","last_became_peered":"2024-04-20T10:31:35.628710+0000","last_unstale":"2024-04-20T10:32:31.948912+0000","last_undegraded":"2024-04-20T10:32:31.948912+0000","last_fullsized":"2024-04-20T10:32:31.948912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:31.883788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.940948+0000","last_change":"2024-04-20T10:31:35.629978+0000","last_active":"2024-04-20T10:32:31.940948+0000","last_peered":"2024-04-20T10:32:31.940948+0000","last_clean":"2024-04-20T10:32:31.940948+0000","last_became_active":"2024-04-20T10:31:35.629596+0000","last_became_peered":"2024-04-20T10:31:35.629596+0000","last_unstale":"2024-04-20T10:32:31.940948+0000","last_undegraded":"2024-04-20T10:32:31.940948+0000","last_fullsized":"2024-04-20T10:32:31.940948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:02.523099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.938749+0000","last_change":"2024-04-20T10:31:35.652481+0000","last_active":"2024-04-20T10:32:31.938749+0000","last_peered":"2024-04-20T10:32:31.938749+0000","last_clean":"2024-04-20T10:32:31.938749+0000","last_became_active":"2024-04-20T10:31:35.651837+0000","last_became_peered":"2024-04-20T10:31:35.651837+0000","last_unstale":"2024-04-20T10:32:31.938749+0000","last_undegraded":"2024-04-20T10:32:31.938749+0000","last_fullsized":"2024-04-20T10:32:31.938749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:53.206954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.936161+0000","last_change":"2024-04-20T10:31:35.622228+0000","last_active":"2024-04-20T10:32:31.936161+0000","last_peered":"2024-04-20T10:32:31.936161+0000","last_clean":"2024-04-20T10:32:31.936161+0000","last_became_active":"2024-04-20T10:31:35.621910+0000","last_became_peered":"2024-04-20T10:31:35.621910+0000","last_unstale":"2024-04-20T10:32:31.936161+0000","last_undegraded":"2024-04-20T10:32:31.936161+0000","last_fullsized":"2024-04-20T10:32:31.936161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:47:34.319839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281463,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61844,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":166,"num_read_kb":144,"num_write":79,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":185532,"data_compressed":5541,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383711,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":455821,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645347,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":1107546,"data_compressed":43453,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":456403,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":456425,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":995328,"data_stored":1046888,"data_compressed":41606,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":455938,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":516867,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":455911,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":516901,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":456159,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":455843,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1046488,"data_compressed":41606,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34156,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34156,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34156,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61046,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61080,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61240,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:33:03.445 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph pg dump --format=json 2024-04-20T10:33:03.957 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:33:04.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:04 smithi042 ceph-mon[17078]: from='client.24890 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:33:04.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:04 smithi042 ceph-mon[17078]: pgmap v363: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:04.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:04 smithi046 ceph-mon[21574]: from='client.24890 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:33:04.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:04 smithi046 ceph-mon[21574]: pgmap v363: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:04.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:04 smithi162 ceph-mon[21854]: from='client.24890 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:33:04.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:04 smithi162 ceph-mon[21854]: pgmap v363: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:06.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.340+0000 7f18aea00700 1 -- 172.21.15.42:0/508691942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8105790 msgr2=0x7f18a81097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:06.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.340+0000 7f18aea00700 1 --2- 172.21.15.42:0/508691942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8105790 0x7f18a81097e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1894009990 tx=0x7f1894009ca0 comp rx=0 tx=0).stop 2024-04-20T10:33:06.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.340+0000 7f18aea00700 1 -- 172.21.15.42:0/508691942 shutdown_connections 2024-04-20T10:33:06.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.340+0000 7f18aea00700 1 --2- 172.21.15.42:0/508691942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a8109ec0 0x7f18a8111a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:06.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.340+0000 7f18aea00700 1 --2- 172.21.15.42:0/508691942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a8104de0 0x7f18a81051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:06.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.340+0000 7f18aea00700 1 --2- 172.21.15.42:0/508691942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8105790 0x7f18a81097e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:06.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.340+0000 7f18aea00700 1 -- 172.21.15.42:0/508691942 >> 172.21.15.42:0/508691942 conn(0x7f18a8100670 msgr2=0x7f18a8102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:06.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.340+0000 7f18aea00700 1 -- 172.21.15.42:0/508691942 shutdown_connections 2024-04-20T10:33:06.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.340+0000 7f18aea00700 1 -- 172.21.15.42:0/508691942 wait complete. 2024-04-20T10:33:06.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.342+0000 7f18aea00700 1 Processor -- start 2024-04-20T10:33:06.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.342+0000 7f18aea00700 1 -- start start 2024-04-20T10:33:06.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.342+0000 7f18aea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a8104de0 0x7f18a819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:06.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.342+0000 7f18aea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a8105790 0x7f18a819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:06.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.342+0000 7f18aea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8109ec0 0x7f18a819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:06.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18aea00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f18a8114e90 con 0x7f18a8104de0 2024-04-20T10:33:06.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18aea00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f18a8114d10 con 0x7f18a8109ec0 2024-04-20T10:33:06.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18aea00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f18a8115010 con 0x7f18a8105790 2024-04-20T10:33:06.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18ad600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a8104de0 0x7f18a819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:06.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18ad600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a8104de0 0x7f18a819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58776/0 (socket says 172.21.15.42:58776) 2024-04-20T10:33:06.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18ad600700 1 -- 172.21.15.42:0/2575466879 learned_addr learned my addr 172.21.15.42:0/2575466879 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:06.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18acc00700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a8105790 0x7f18a819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:06.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18ae000700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8109ec0 0x7f18a819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:06.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18ad600700 1 -- 172.21.15.42:0/2575466879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a8105790 msgr2=0x7f18a819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:06.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18ad600700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a8105790 0x7f18a819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:06.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18ad600700 1 -- 172.21.15.42:0/2575466879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8109ec0 msgr2=0x7f18a819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:06.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18ad600700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8109ec0 0x7f18a819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:06.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.343+0000 7f18ad600700 1 -- 172.21.15.42:0/2575466879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a81a0340 con 0x7f18a8104de0 2024-04-20T10:33:06.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.344+0000 7f18ad600700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a8104de0 0x7f18a819b380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f189c00e9a0 tx=0x7f189c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:06.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.344+0000 7f18a6000700 1 -- 172.21.15.42:0/2575466879 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c00ca10 con 0x7f18a8104de0 2024-04-20T10:33:06.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.344+0000 7f18aea00700 1 -- 172.21.15.42:0/2575466879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a81a0630 con 0x7f18a8104de0 2024-04-20T10:33:06.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.344+0000 7f18a6000700 1 -- 172.21.15.42:0/2575466879 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f189c00cbb0 con 0x7f18a8104de0 2024-04-20T10:33:06.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.344+0000 7f18a6000700 1 -- 172.21.15.42:0/2575466879 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c018620 con 0x7f18a8104de0 2024-04-20T10:33:06.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.344+0000 7f18aea00700 1 -- 172.21.15.42:0/2575466879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a81a7eb0 con 0x7f18a8104de0 2024-04-20T10:33:06.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.346+0000 7f18aea00700 1 -- 172.21.15.42:0/2575466879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a8069740 con 0x7f18a8104de0 2024-04-20T10:33:06.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.347+0000 7f18a6000700 1 -- 172.21.15.42:0/2575466879 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f189c010430 con 0x7f18a8104de0 2024-04-20T10:33:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.349+0000 7f18a6000700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f18900a3c40 0x7f18900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:06.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.349+0000 7f18a6000700 1 -- 172.21.15.42:0/2575466879 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f189c0d0630 con 0x7f18a8104de0 2024-04-20T10:33:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.349+0000 7f18acc00700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f18900a3c40 0x7f18900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:06.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.350+0000 7f18acc00700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f18900a3c40 0x7f18900a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f189400b2e0 tx=0x7f1894005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:06.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.354+0000 7f18a6000700 1 -- 172.21.15.42:0/2575466879 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f189c09cf90 con 0x7f18a8104de0 2024-04-20T10:33:06.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:06 smithi046 ceph-mon[21574]: pgmap v364: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T10:33:06.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.578+0000 7f18aea00700 1 -- 172.21.15.42:0/2575466879 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18a819c690 con 0x7f18900a3c40 2024-04-20T10:33:06.580 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:06 smithi042 ceph-mon[17078]: pgmap v364: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T10:33:06.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.603+0000 7f18a6000700 1 -- 172.21.15.42:0/2575466879 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259118 (secure 0 0 0) 0x7f1894065440 con 0x7f18900a3c40 2024-04-20T10:33:06.610 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.610+0000 7f18aea00700 1 -- 172.21.15.42:0/2575466879 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f18900a3c40 msgr2=0x7f18900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.610+0000 7f18aea00700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f18900a3c40 0x7f18900a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f189400b2e0 tx=0x7f1894005960 comp rx=0 tx=0).stop 2024-04-20T10:33:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.610+0000 7f18aea00700 1 -- 172.21.15.42:0/2575466879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a8104de0 msgr2=0x7f18a819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.610+0000 7f18aea00700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a8104de0 0x7f18a819b380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f189c00e9a0 tx=0x7f189c00ecb0 comp rx=0 tx=0).stop 2024-04-20T10:33:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.610+0000 7f18aea00700 1 -- 172.21.15.42:0/2575466879 shutdown_connections 2024-04-20T10:33:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.610+0000 7f18aea00700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f18900a3c40 0x7f18900a6100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.610+0000 7f18aea00700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8109ec0 0x7f18a819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:06.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.610+0000 7f18aea00700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18a8104de0 0x7f18a819b380 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:06.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.610+0000 7f18aea00700 1 --2- 172.21.15.42:0/2575466879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18a8105790 0x7f18a819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:06.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.610+0000 7f18aea00700 1 -- 172.21.15.42:0/2575466879 >> 172.21.15.42:0/2575466879 conn(0x7f18a8100670 msgr2=0x7f18a81019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:06.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.611+0000 7f18aea00700 1 -- 172.21.15.42:0/2575466879 shutdown_connections 2024-04-20T10:33:06.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:06.611+0000 7f18aea00700 1 -- 172.21.15.42:0/2575466879 wait complete. 2024-04-20T10:33:06.613 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-04-20T10:33:06.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:06 smithi162 ceph-mon[21854]: pgmap v364: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 255 B/s wr, 1 op/s 2024-04-20T10:33:07.534 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-20T10:33:06.548231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652695,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":323,"num_read_kb":274,"num_write":267,"num_write_kb":3027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":709,"ondisk_log_size":709,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368884,"kb_used_data":11412,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572940,"statfs":{"total":1151940427776,"available":1151562690560,"internally_reserved":0,"allocated":11685888,"data_stored":7427937,"data_compressed":182622,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":16,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005528"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.526783+0000","last_change":"2024-04-20T10:31:35.649585+0000","last_active":"2024-04-20T10:33:00.526783+0000","last_peered":"2024-04-20T10:33:00.526783+0000","last_clean":"2024-04-20T10:33:00.526783+0000","last_became_active":"2024-04-20T10:31:35.649282+0000","last_became_peered":"2024-04-20T10:31:35.649282+0000","last_unstale":"2024-04-20T10:33:00.526783+0000","last_undegraded":"2024-04-20T10:33:00.526783+0000","last_fullsized":"2024-04-20T10:33:00.526783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:01:00.517383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.509924+0000","last_change":"2024-04-20T10:31:35.648299+0000","last_active":"2024-04-20T10:33:00.509924+0000","last_peered":"2024-04-20T10:33:00.509924+0000","last_clean":"2024-04-20T10:33:00.509924+0000","last_became_active":"2024-04-20T10:31:35.647283+0000","last_became_peered":"2024-04-20T10:31:35.647283+0000","last_unstale":"2024-04-20T10:33:00.509924+0000","last_undegraded":"2024-04-20T10:33:00.509924+0000","last_fullsized":"2024-04-20T10:33:00.509924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:59.006234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.521907+0000","last_change":"2024-04-20T10:31:35.647029+0000","last_active":"2024-04-20T10:33:00.521907+0000","last_peered":"2024-04-20T10:33:00.521907+0000","last_clean":"2024-04-20T10:33:00.521907+0000","last_became_active":"2024-04-20T10:31:35.646649+0000","last_became_peered":"2024-04-20T10:31:35.646649+0000","last_unstale":"2024-04-20T10:33:00.521907+0000","last_undegraded":"2024-04-20T10:33:00.521907+0000","last_fullsized":"2024-04-20T10:33:00.521907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:03:23.420078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.516113+0000","last_change":"2024-04-20T10:31:35.624628+0000","last_active":"2024-04-20T10:33:00.516113+0000","last_peered":"2024-04-20T10:33:00.516113+0000","last_clean":"2024-04-20T10:33:00.516113+0000","last_became_active":"2024-04-20T10:31:35.623607+0000","last_became_peered":"2024-04-20T10:31:35.623607+0000","last_unstale":"2024-04-20T10:33:00.516113+0000","last_undegraded":"2024-04-20T10:33:00.516113+0000","last_fullsized":"2024-04-20T10:33:00.516113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:44:46.768584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.536035+0000","last_change":"2024-04-20T10:31:35.638293+0000","last_active":"2024-04-20T10:33:00.536035+0000","last_peered":"2024-04-20T10:33:00.536035+0000","last_clean":"2024-04-20T10:33:00.536035+0000","last_became_active":"2024-04-20T10:31:35.637967+0000","last_became_peered":"2024-04-20T10:31:35.637967+0000","last_unstale":"2024-04-20T10:33:00.536035+0000","last_undegraded":"2024-04-20T10:33:00.536035+0000","last_fullsized":"2024-04-20T10:33:00.536035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:58:00.765015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.513998+0000","last_change":"2024-04-20T10:31:35.634565+0000","last_active":"2024-04-20T10:33:00.513998+0000","last_peered":"2024-04-20T10:33:00.513998+0000","last_clean":"2024-04-20T10:33:00.513998+0000","last_became_active":"2024-04-20T10:31:35.634200+0000","last_became_peered":"2024-04-20T10:31:35.634200+0000","last_unstale":"2024-04-20T10:33:00.513998+0000","last_undegraded":"2024-04-20T10:33:00.513998+0000","last_fullsized":"2024-04-20T10:33:00.513998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:14.298732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.515622+0000","last_change":"2024-04-20T10:32:31.949166+0000","last_active":"2024-04-20T10:33:00.515622+0000","last_peered":"2024-04-20T10:33:00.515622+0000","last_clean":"2024-04-20T10:33:00.515622+0000","last_became_active":"2024-04-20T10:32:31.948825+0000","last_became_peered":"2024-04-20T10:32:31.948825+0000","last_unstale":"2024-04-20T10:33:00.515622+0000","last_undegraded":"2024-04-20T10:33:00.515622+0000","last_fullsized":"2024-04-20T10:33:00.515622+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:05.337941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.540161+0000","last_change":"2024-04-20T10:31:35.631996+0000","last_active":"2024-04-20T10:33:00.540161+0000","last_peered":"2024-04-20T10:33:00.540161+0000","last_clean":"2024-04-20T10:33:00.540161+0000","last_became_active":"2024-04-20T10:31:35.631576+0000","last_became_peered":"2024-04-20T10:31:35.631576+0000","last_unstale":"2024-04-20T10:33:00.540161+0000","last_undegraded":"2024-04-20T10:33:00.540161+0000","last_fullsized":"2024-04-20T10:33:00.540161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:51.448025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.523152+0000","last_change":"2024-04-20T10:31:35.624430+0000","last_active":"2024-04-20T10:33:00.523152+0000","last_peered":"2024-04-20T10:33:00.523152+0000","last_clean":"2024-04-20T10:33:00.523152+0000","last_became_active":"2024-04-20T10:31:35.623018+0000","last_became_peered":"2024-04-20T10:31:35.623018+0000","last_unstale":"2024-04-20T10:33:00.523152+0000","last_undegraded":"2024-04-20T10:33:00.523152+0000","last_fullsized":"2024-04-20T10:33:00.523152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:43.100925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.512120+0000","last_change":"2024-04-20T10:31:35.619038+0000","last_active":"2024-04-20T10:33:00.512120+0000","last_peered":"2024-04-20T10:33:00.512120+0000","last_clean":"2024-04-20T10:33:00.512120+0000","last_became_active":"2024-04-20T10:31:35.618699+0000","last_became_peered":"2024-04-20T10:31:35.618699+0000","last_unstale":"2024-04-20T10:33:00.512120+0000","last_undegraded":"2024-04-20T10:33:00.512120+0000","last_fullsized":"2024-04-20T10:33:00.512120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:27.205634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.519073+0000","last_change":"2024-04-20T10:31:35.619908+0000","last_active":"2024-04-20T10:33:00.519073+0000","last_peered":"2024-04-20T10:33:00.519073+0000","last_clean":"2024-04-20T10:33:00.519073+0000","last_became_active":"2024-04-20T10:31:35.619580+0000","last_became_peered":"2024-04-20T10:31:35.619580+0000","last_unstale":"2024-04-20T10:33:00.519073+0000","last_undegraded":"2024-04-20T10:33:00.519073+0000","last_fullsized":"2024-04-20T10:33:00.519073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:54:48.999925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.512158+0000","last_change":"2024-04-20T10:32:31.952701+0000","last_active":"2024-04-20T10:33:00.512158+0000","last_peered":"2024-04-20T10:33:00.512158+0000","last_clean":"2024-04-20T10:33:00.512158+0000","last_became_active":"2024-04-20T10:32:31.951329+0000","last_became_peered":"2024-04-20T10:32:31.951329+0000","last_unstale":"2024-04-20T10:33:00.512158+0000","last_undegraded":"2024-04-20T10:33:00.512158+0000","last_fullsized":"2024-04-20T10:33:00.512158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:29.442878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.537855+0000","last_change":"2024-04-20T10:31:35.619594+0000","last_active":"2024-04-20T10:33:00.537855+0000","last_peered":"2024-04-20T10:33:00.537855+0000","last_clean":"2024-04-20T10:33:00.537855+0000","last_became_active":"2024-04-20T10:31:35.618831+0000","last_became_peered":"2024-04-20T10:31:35.618831+0000","last_unstale":"2024-04-20T10:33:00.537855+0000","last_undegraded":"2024-04-20T10:33:00.537855+0000","last_fullsized":"2024-04-20T10:33:00.537855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:07.625225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.513806+0000","last_change":"2024-04-20T10:31:35.647476+0000","last_active":"2024-04-20T10:33:00.513806+0000","last_peered":"2024-04-20T10:33:00.513806+0000","last_clean":"2024-04-20T10:33:00.513806+0000","last_became_active":"2024-04-20T10:31:35.647009+0000","last_became_peered":"2024-04-20T10:31:35.647009+0000","last_unstale":"2024-04-20T10:33:00.513806+0000","last_undegraded":"2024-04-20T10:33:00.513806+0000","last_fullsized":"2024-04-20T10:33:00.513806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:11:37.526115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.516728+0000","last_change":"2024-04-20T10:32:31.958321+0000","last_active":"2024-04-20T10:33:00.516728+0000","last_peered":"2024-04-20T10:33:00.516728+0000","last_clean":"2024-04-20T10:33:00.516728+0000","last_became_active":"2024-04-20T10:32:31.957681+0000","last_became_peered":"2024-04-20T10:32:31.957681+0000","last_unstale":"2024-04-20T10:33:00.516728+0000","last_undegraded":"2024-04-20T10:33:00.516728+0000","last_fullsized":"2024-04-20T10:33:00.516728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:49:31.154909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.523606+0000","last_change":"2024-04-20T10:31:35.643142+0000","last_active":"2024-04-20T10:33:00.523606+0000","last_peered":"2024-04-20T10:33:00.523606+0000","last_clean":"2024-04-20T10:33:00.523606+0000","last_became_active":"2024-04-20T10:31:35.642742+0000","last_became_peered":"2024-04-20T10:31:35.642742+0000","last_unstale":"2024-04-20T10:33:00.523606+0000","last_undegraded":"2024-04-20T10:33:00.523606+0000","last_fullsized":"2024-04-20T10:33:00.523606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:40.818808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.526916+0000","last_change":"2024-04-20T10:31:35.650333+0000","last_active":"2024-04-20T10:33:00.526916+0000","last_peered":"2024-04-20T10:33:00.526916+0000","last_clean":"2024-04-20T10:33:00.526916+0000","last_became_active":"2024-04-20T10:31:35.649933+0000","last_became_peered":"2024-04-20T10:31:35.649933+0000","last_unstale":"2024-04-20T10:33:00.526916+0000","last_undegraded":"2024-04-20T10:33:00.526916+0000","last_fullsized":"2024-04-20T10:33:00.526916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:34.513990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.511918+0000","last_change":"2024-04-20T10:31:35.637315+0000","last_active":"2024-04-20T10:33:00.511918+0000","last_peered":"2024-04-20T10:33:00.511918+0000","last_clean":"2024-04-20T10:33:00.511918+0000","last_became_active":"2024-04-20T10:31:35.635909+0000","last_became_peered":"2024-04-20T10:31:35.635909+0000","last_unstale":"2024-04-20T10:33:00.511918+0000","last_undegraded":"2024-04-20T10:33:00.511918+0000","last_fullsized":"2024-04-20T10:33:00.511918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:24:07.400032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.541356+0000","last_change":"2024-04-20T10:31:35.638888+0000","last_active":"2024-04-20T10:33:00.541356+0000","last_peered":"2024-04-20T10:33:00.541356+0000","last_clean":"2024-04-20T10:33:00.541356+0000","last_became_active":"2024-04-20T10:31:35.638568+0000","last_became_peered":"2024-04-20T10:31:35.638568+0000","last_unstale":"2024-04-20T10:33:00.541356+0000","last_undegraded":"2024-04-20T10:33:00.541356+0000","last_fullsized":"2024-04-20T10:33:00.541356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:48.363874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.945343+0000","last_change":"2024-04-20T10:31:34.597814+0000","last_active":"2024-04-20T10:32:31.945343+0000","last_peered":"2024-04-20T10:32:31.945343+0000","last_clean":"2024-04-20T10:32:31.945343+0000","last_became_active":"2024-04-20T10:31:34.597480+0000","last_became_peered":"2024-04-20T10:31:34.597480+0000","last_unstale":"2024-04-20T10:32:31.945343+0000","last_undegraded":"2024-04-20T10:32:31.945343+0000","last_fullsized":"2024-04-20T10:32:31.945343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:48:26.102787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.506411+0000","last_change":"2024-04-20T10:32:31.956771+0000","last_active":"2024-04-20T10:33:00.506411+0000","last_peered":"2024-04-20T10:33:00.506411+0000","last_clean":"2024-04-20T10:33:00.506411+0000","last_became_active":"2024-04-20T10:32:31.955413+0000","last_became_peered":"2024-04-20T10:32:31.955413+0000","last_unstale":"2024-04-20T10:33:00.506411+0000","last_undegraded":"2024-04-20T10:33:00.506411+0000","last_fullsized":"2024-04-20T10:33:00.506411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:37.199764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.952148+0000","last_change":"2024-04-20T10:31:34.609942+0000","last_active":"2024-04-20T10:32:31.952148+0000","last_peered":"2024-04-20T10:32:31.952148+0000","last_clean":"2024-04-20T10:32:31.952148+0000","last_became_active":"2024-04-20T10:31:34.608994+0000","last_became_peered":"2024-04-20T10:31:34.608994+0000","last_unstale":"2024-04-20T10:32:31.952148+0000","last_undegraded":"2024-04-20T10:32:31.952148+0000","last_fullsized":"2024-04-20T10:32:31.952148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:26:53.488091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.520983+0000","last_change":"2024-04-20T10:31:35.647710+0000","last_active":"2024-04-20T10:33:00.520983+0000","last_peered":"2024-04-20T10:33:00.520983+0000","last_clean":"2024-04-20T10:33:00.520983+0000","last_became_active":"2024-04-20T10:31:35.646037+0000","last_became_peered":"2024-04-20T10:31:35.646037+0000","last_unstale":"2024-04-20T10:33:00.520983+0000","last_undegraded":"2024-04-20T10:33:00.520983+0000","last_fullsized":"2024-04-20T10:33:00.520983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:50:21.092892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934609+0000","last_change":"2024-04-20T10:31:34.593529+0000","last_active":"2024-04-20T10:32:31.934609+0000","last_peered":"2024-04-20T10:32:31.934609+0000","last_clean":"2024-04-20T10:32:31.934609+0000","last_became_active":"2024-04-20T10:31:34.592583+0000","last_became_peered":"2024-04-20T10:31:34.592583+0000","last_unstale":"2024-04-20T10:32:31.934609+0000","last_undegraded":"2024-04-20T10:32:31.934609+0000","last_fullsized":"2024-04-20T10:32:31.934609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:51:20.570499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.525755+0000","last_change":"2024-04-20T10:31:35.630739+0000","last_active":"2024-04-20T10:33:00.525755+0000","last_peered":"2024-04-20T10:33:00.525755+0000","last_clean":"2024-04-20T10:33:00.525755+0000","last_became_active":"2024-04-20T10:31:35.630424+0000","last_became_peered":"2024-04-20T10:31:35.630424+0000","last_unstale":"2024-04-20T10:33:00.525755+0000","last_undegraded":"2024-04-20T10:33:00.525755+0000","last_fullsized":"2024-04-20T10:33:00.525755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:48:17.485677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934692+0000","last_change":"2024-04-20T10:31:34.591652+0000","last_active":"2024-04-20T10:32:31.934692+0000","last_peered":"2024-04-20T10:32:31.934692+0000","last_clean":"2024-04-20T10:32:31.934692+0000","last_became_active":"2024-04-20T10:31:34.588945+0000","last_became_peered":"2024-04-20T10:31:34.588945+0000","last_unstale":"2024-04-20T10:32:31.934692+0000","last_undegraded":"2024-04-20T10:32:31.934692+0000","last_fullsized":"2024-04-20T10:32:31.934692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:15.137046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.521270+0000","last_change":"2024-04-20T10:31:35.622462+0000","last_active":"2024-04-20T10:33:00.521270+0000","last_peered":"2024-04-20T10:33:00.521270+0000","last_clean":"2024-04-20T10:33:00.521270+0000","last_became_active":"2024-04-20T10:31:35.622125+0000","last_became_peered":"2024-04-20T10:31:35.622125+0000","last_unstale":"2024-04-20T10:33:00.521270+0000","last_undegraded":"2024-04-20T10:33:00.521270+0000","last_fullsized":"2024-04-20T10:33:00.521270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:27.894564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:32.043629+0000","last_change":"2024-04-20T10:32:32.043629+0000","last_active":"2024-04-20T10:32:32.043629+0000","last_peered":"2024-04-20T10:32:32.043629+0000","last_clean":"2024-04-20T10:32:32.043629+0000","last_became_active":"2024-04-20T10:32:32.042849+0000","last_became_peered":"2024-04-20T10:32:32.042849+0000","last_unstale":"2024-04-20T10:32:32.043629+0000","last_undegraded":"2024-04-20T10:32:32.043629+0000","last_fullsized":"2024-04-20T10:32:32.043629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:52.646201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.520423+0000","last_change":"2024-04-20T10:32:31.951658+0000","last_active":"2024-04-20T10:33:00.520423+0000","last_peered":"2024-04-20T10:33:00.520423+0000","last_clean":"2024-04-20T10:33:00.520423+0000","last_became_active":"2024-04-20T10:32:31.951307+0000","last_became_peered":"2024-04-20T10:32:31.951307+0000","last_unstale":"2024-04-20T10:33:00.520423+0000","last_undegraded":"2024-04-20T10:33:00.520423+0000","last_fullsized":"2024-04-20T10:33:00.520423+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:42.743969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.942138+0000","last_change":"2024-04-20T10:31:34.612038+0000","last_active":"2024-04-20T10:32:31.942138+0000","last_peered":"2024-04-20T10:32:31.942138+0000","last_clean":"2024-04-20T10:32:31.942138+0000","last_became_active":"2024-04-20T10:31:34.611671+0000","last_became_peered":"2024-04-20T10:31:34.611671+0000","last_unstale":"2024-04-20T10:32:31.942138+0000","last_undegraded":"2024-04-20T10:32:31.942138+0000","last_fullsized":"2024-04-20T10:32:31.942138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:47.862702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.540398+0000","last_change":"2024-04-20T10:31:35.643930+0000","last_active":"2024-04-20T10:33:00.540398+0000","last_peered":"2024-04-20T10:33:00.540398+0000","last_clean":"2024-04-20T10:33:00.540398+0000","last_became_active":"2024-04-20T10:31:35.643470+0000","last_became_peered":"2024-04-20T10:31:35.643470+0000","last_unstale":"2024-04-20T10:33:00.540398+0000","last_undegraded":"2024-04-20T10:33:00.540398+0000","last_fullsized":"2024-04-20T10:33:00.540398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:55.570176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937828+0000","last_change":"2024-04-20T10:31:34.613224+0000","last_active":"2024-04-20T10:32:31.937828+0000","last_peered":"2024-04-20T10:32:31.937828+0000","last_clean":"2024-04-20T10:32:31.937828+0000","last_became_active":"2024-04-20T10:31:34.612871+0000","last_became_peered":"2024-04-20T10:31:34.612871+0000","last_unstale":"2024-04-20T10:32:31.937828+0000","last_undegraded":"2024-04-20T10:32:31.937828+0000","last_fullsized":"2024-04-20T10:32:31.937828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:05:57.284870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.540045+0000","last_change":"2024-04-20T10:31:35.643824+0000","last_active":"2024-04-20T10:33:00.540045+0000","last_peered":"2024-04-20T10:33:00.540045+0000","last_clean":"2024-04-20T10:33:00.540045+0000","last_became_active":"2024-04-20T10:31:35.643425+0000","last_became_peered":"2024-04-20T10:31:35.643425+0000","last_unstale":"2024-04-20T10:33:00.540045+0000","last_undegraded":"2024-04-20T10:33:00.540045+0000","last_fullsized":"2024-04-20T10:33:00.540045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:34.008374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.936942+0000","last_change":"2024-04-20T10:31:34.593071+0000","last_active":"2024-04-20T10:32:31.936942+0000","last_peered":"2024-04-20T10:32:31.936942+0000","last_clean":"2024-04-20T10:32:31.936942+0000","last_became_active":"2024-04-20T10:31:34.592721+0000","last_became_peered":"2024-04-20T10:31:34.592721+0000","last_unstale":"2024-04-20T10:32:31.936942+0000","last_undegraded":"2024-04-20T10:32:31.936942+0000","last_fullsized":"2024-04-20T10:32:31.936942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:23:43.749586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.544605+0000","last_change":"2024-04-20T10:31:35.647524+0000","last_active":"2024-04-20T10:33:00.544605+0000","last_peered":"2024-04-20T10:33:00.544605+0000","last_clean":"2024-04-20T10:33:00.544605+0000","last_became_active":"2024-04-20T10:31:35.647122+0000","last_became_peered":"2024-04-20T10:31:35.647122+0000","last_unstale":"2024-04-20T10:33:00.544605+0000","last_undegraded":"2024-04-20T10:33:00.544605+0000","last_fullsized":"2024-04-20T10:33:00.544605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.942771+0000","last_change":"2024-04-20T10:31:34.604500+0000","last_active":"2024-04-20T10:32:31.942771+0000","last_peered":"2024-04-20T10:32:31.942771+0000","last_clean":"2024-04-20T10:32:31.942771+0000","last_became_active":"2024-04-20T10:31:34.604148+0000","last_became_peered":"2024-04-20T10:31:34.604148+0000","last_unstale":"2024-04-20T10:32:31.942771+0000","last_undegraded":"2024-04-20T10:32:31.942771+0000","last_fullsized":"2024-04-20T10:32:31.942771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:49:15.174414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.517026+0000","last_change":"2024-04-20T10:31:35.636751+0000","last_active":"2024-04-20T10:33:00.517026+0000","last_peered":"2024-04-20T10:33:00.517026+0000","last_clean":"2024-04-20T10:33:00.517026+0000","last_became_active":"2024-04-20T10:31:35.636103+0000","last_became_peered":"2024-04-20T10:31:35.636103+0000","last_unstale":"2024-04-20T10:33:00.517026+0000","last_undegraded":"2024-04-20T10:33:00.517026+0000","last_fullsized":"2024-04-20T10:33:00.517026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:22:40.580512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934374+0000","last_change":"2024-04-20T10:31:34.595207+0000","last_active":"2024-04-20T10:32:31.934374+0000","last_peered":"2024-04-20T10:32:31.934374+0000","last_clean":"2024-04-20T10:32:31.934374+0000","last_became_active":"2024-04-20T10:31:34.594860+0000","last_became_peered":"2024-04-20T10:31:34.594860+0000","last_unstale":"2024-04-20T10:32:31.934374+0000","last_undegraded":"2024-04-20T10:32:31.934374+0000","last_fullsized":"2024-04-20T10:32:31.934374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:50.054454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.513570+0000","last_change":"2024-04-20T10:31:35.648086+0000","last_active":"2024-04-20T10:33:00.513570+0000","last_peered":"2024-04-20T10:33:00.513570+0000","last_clean":"2024-04-20T10:33:00.513570+0000","last_became_active":"2024-04-20T10:31:35.647275+0000","last_became_peered":"2024-04-20T10:31:35.647275+0000","last_unstale":"2024-04-20T10:33:00.513570+0000","last_undegraded":"2024-04-20T10:33:00.513570+0000","last_fullsized":"2024-04-20T10:33:00.513570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:27.669008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937923+0000","last_change":"2024-04-20T10:31:34.601674+0000","last_active":"2024-04-20T10:32:31.937923+0000","last_peered":"2024-04-20T10:32:31.937923+0000","last_clean":"2024-04-20T10:32:31.937923+0000","last_became_active":"2024-04-20T10:31:34.601255+0000","last_became_peered":"2024-04-20T10:31:34.601255+0000","last_unstale":"2024-04-20T10:32:31.937923+0000","last_undegraded":"2024-04-20T10:32:31.937923+0000","last_fullsized":"2024-04-20T10:32:31.937923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:57.821635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.520486+0000","last_change":"2024-04-20T10:31:35.621888+0000","last_active":"2024-04-20T10:33:00.520486+0000","last_peered":"2024-04-20T10:33:00.520486+0000","last_clean":"2024-04-20T10:33:00.520486+0000","last_became_active":"2024-04-20T10:31:35.621503+0000","last_became_peered":"2024-04-20T10:31:35.621503+0000","last_unstale":"2024-04-20T10:33:00.520486+0000","last_undegraded":"2024-04-20T10:33:00.520486+0000","last_fullsized":"2024-04-20T10:33:00.520486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:15:21.118638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'29","reported_seq":231,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:05.097033+0000","last_change":"2024-04-20T10:31:34.590558+0000","last_active":"2024-04-20T10:33:05.097033+0000","last_peered":"2024-04-20T10:33:05.097033+0000","last_clean":"2024-04-20T10:33:05.097033+0000","last_became_active":"2024-04-20T10:31:34.590140+0000","last_became_peered":"2024-04-20T10:31:34.590140+0000","last_unstale":"2024-04-20T10:33:05.097033+0000","last_undegraded":"2024-04-20T10:33:05.097033+0000","last_fullsized":"2024-04-20T10:33:05.097033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:15:18.591748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":151,"num_write":29,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.520681+0000","last_change":"2024-04-20T10:31:35.620780+0000","last_active":"2024-04-20T10:33:00.520681+0000","last_peered":"2024-04-20T10:33:00.520681+0000","last_clean":"2024-04-20T10:33:00.520681+0000","last_became_active":"2024-04-20T10:31:35.620429+0000","last_became_peered":"2024-04-20T10:31:35.620429+0000","last_unstale":"2024-04-20T10:33:00.520681+0000","last_undegraded":"2024-04-20T10:33:00.520681+0000","last_fullsized":"2024-04-20T10:33:00.520681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:47:45.587920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.938798+0000","last_change":"2024-04-20T10:31:34.602135+0000","last_active":"2024-04-20T10:32:31.938798+0000","last_peered":"2024-04-20T10:32:31.938798+0000","last_clean":"2024-04-20T10:32:31.938798+0000","last_became_active":"2024-04-20T10:31:34.601460+0000","last_became_peered":"2024-04-20T10:31:34.601460+0000","last_unstale":"2024-04-20T10:32:31.938798+0000","last_undegraded":"2024-04-20T10:32:31.938798+0000","last_fullsized":"2024-04-20T10:32:31.938798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:00:00.832702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.541577+0000","last_change":"2024-04-20T10:31:35.635526+0000","last_active":"2024-04-20T10:33:00.541577+0000","last_peered":"2024-04-20T10:33:00.541577+0000","last_clean":"2024-04-20T10:33:00.541577+0000","last_became_active":"2024-04-20T10:31:35.635143+0000","last_became_peered":"2024-04-20T10:31:35.635143+0000","last_unstale":"2024-04-20T10:33:00.541577+0000","last_undegraded":"2024-04-20T10:33:00.541577+0000","last_fullsized":"2024-04-20T10:33:00.541577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:57:24.509318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937383+0000","last_change":"2024-04-20T10:31:34.604561+0000","last_active":"2024-04-20T10:32:31.937383+0000","last_peered":"2024-04-20T10:32:31.937383+0000","last_clean":"2024-04-20T10:32:31.937383+0000","last_became_active":"2024-04-20T10:31:34.604232+0000","last_became_peered":"2024-04-20T10:31:34.604232+0000","last_unstale":"2024-04-20T10:32:31.937383+0000","last_undegraded":"2024-04-20T10:32:31.937383+0000","last_fullsized":"2024-04-20T10:32:31.937383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:39.317473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.539256+0000","last_change":"2024-04-20T10:31:35.632840+0000","last_active":"2024-04-20T10:33:00.539256+0000","last_peered":"2024-04-20T10:33:00.539256+0000","last_clean":"2024-04-20T10:33:00.539256+0000","last_became_active":"2024-04-20T10:31:35.632471+0000","last_became_peered":"2024-04-20T10:31:35.632471+0000","last_unstale":"2024-04-20T10:33:00.539256+0000","last_undegraded":"2024-04-20T10:33:00.539256+0000","last_fullsized":"2024-04-20T10:33:00.539256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:06.679967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:32.041323+0000","last_change":"2024-04-20T10:31:34.591696+0000","last_active":"2024-04-20T10:32:32.041323+0000","last_peered":"2024-04-20T10:32:32.041323+0000","last_clean":"2024-04-20T10:32:32.041323+0000","last_became_active":"2024-04-20T10:31:34.591330+0000","last_became_peered":"2024-04-20T10:31:34.591330+0000","last_unstale":"2024-04-20T10:32:32.041323+0000","last_undegraded":"2024-04-20T10:32:32.041323+0000","last_fullsized":"2024-04-20T10:32:32.041323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:44:36.385963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.516715+0000","last_change":"2024-04-20T10:31:35.627201+0000","last_active":"2024-04-20T10:33:00.516715+0000","last_peered":"2024-04-20T10:33:00.516715+0000","last_clean":"2024-04-20T10:33:00.516715+0000","last_became_active":"2024-04-20T10:31:35.626880+0000","last_became_peered":"2024-04-20T10:31:35.626880+0000","last_unstale":"2024-04-20T10:33:00.516715+0000","last_undegraded":"2024-04-20T10:33:00.516715+0000","last_fullsized":"2024-04-20T10:33:00.516715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:44.562723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934142+0000","last_change":"2024-04-20T10:31:34.600683+0000","last_active":"2024-04-20T10:32:31.934142+0000","last_peered":"2024-04-20T10:32:31.934142+0000","last_clean":"2024-04-20T10:32:31.934142+0000","last_became_active":"2024-04-20T10:31:34.600305+0000","last_became_peered":"2024-04-20T10:31:34.600305+0000","last_unstale":"2024-04-20T10:32:31.934142+0000","last_undegraded":"2024-04-20T10:32:31.934142+0000","last_fullsized":"2024-04-20T10:32:31.934142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:05.441158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.517910+0000","last_change":"2024-04-20T10:31:35.619835+0000","last_active":"2024-04-20T10:33:00.517910+0000","last_peered":"2024-04-20T10:33:00.517910+0000","last_clean":"2024-04-20T10:33:00.517910+0000","last_became_active":"2024-04-20T10:31:35.619554+0000","last_became_peered":"2024-04-20T10:31:35.619554+0000","last_unstale":"2024-04-20T10:33:00.517910+0000","last_undegraded":"2024-04-20T10:33:00.517910+0000","last_fullsized":"2024-04-20T10:33:00.517910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:33.161563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934465+0000","last_change":"2024-04-20T10:31:34.591128+0000","last_active":"2024-04-20T10:32:31.934465+0000","last_peered":"2024-04-20T10:32:31.934465+0000","last_clean":"2024-04-20T10:32:31.934465+0000","last_became_active":"2024-04-20T10:31:34.587772+0000","last_became_peered":"2024-04-20T10:31:34.587772+0000","last_unstale":"2024-04-20T10:32:31.934465+0000","last_undegraded":"2024-04-20T10:32:31.934465+0000","last_fullsized":"2024-04-20T10:32:31.934465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:12:42.541111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.519172+0000","last_change":"2024-04-20T10:31:35.616743+0000","last_active":"2024-04-20T10:33:00.519172+0000","last_peered":"2024-04-20T10:33:00.519172+0000","last_clean":"2024-04-20T10:33:00.519172+0000","last_became_active":"2024-04-20T10:31:35.616419+0000","last_became_peered":"2024-04-20T10:31:35.616419+0000","last_unstale":"2024-04-20T10:33:00.519172+0000","last_undegraded":"2024-04-20T10:33:00.519172+0000","last_fullsized":"2024-04-20T10:33:00.519172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:37.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.945445+0000","last_change":"2024-04-20T10:31:34.603734+0000","last_active":"2024-04-20T10:32:31.945445+0000","last_peered":"2024-04-20T10:32:31.945445+0000","last_clean":"2024-04-20T10:32:31.945445+0000","last_became_active":"2024-04-20T10:31:34.603388+0000","last_became_peered":"2024-04-20T10:31:34.603388+0000","last_unstale":"2024-04-20T10:32:31.945445+0000","last_undegraded":"2024-04-20T10:32:31.945445+0000","last_fullsized":"2024-04-20T10:32:31.945445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:58:57.199173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.541251+0000","last_change":"2024-04-20T10:31:35.638996+0000","last_active":"2024-04-20T10:33:00.541251+0000","last_peered":"2024-04-20T10:33:00.541251+0000","last_clean":"2024-04-20T10:33:00.541251+0000","last_became_active":"2024-04-20T10:31:35.638571+0000","last_became_peered":"2024-04-20T10:31:35.638571+0000","last_unstale":"2024-04-20T10:33:00.541251+0000","last_undegraded":"2024-04-20T10:33:00.541251+0000","last_fullsized":"2024-04-20T10:33:00.541251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:59.541738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934002+0000","last_change":"2024-04-20T10:31:34.612501+0000","last_active":"2024-04-20T10:32:31.934002+0000","last_peered":"2024-04-20T10:32:31.934002+0000","last_clean":"2024-04-20T10:32:31.934002+0000","last_became_active":"2024-04-20T10:31:34.612150+0000","last_became_peered":"2024-04-20T10:31:34.612150+0000","last_unstale":"2024-04-20T10:32:31.934002+0000","last_undegraded":"2024-04-20T10:32:31.934002+0000","last_fullsized":"2024-04-20T10:32:31.934002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:28.172851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.519265+0000","last_change":"2024-04-20T10:32:31.957767+0000","last_active":"2024-04-20T10:33:00.519265+0000","last_peered":"2024-04-20T10:33:00.519265+0000","last_clean":"2024-04-20T10:33:00.519265+0000","last_became_active":"2024-04-20T10:32:31.957319+0000","last_became_peered":"2024-04-20T10:32:31.957319+0000","last_unstale":"2024-04-20T10:33:00.519265+0000","last_undegraded":"2024-04-20T10:33:00.519265+0000","last_fullsized":"2024-04-20T10:33:00.519265+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:56.375705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.508678+0000","last_change":"2024-04-20T10:31:35.617775+0000","last_active":"2024-04-20T10:33:00.508678+0000","last_peered":"2024-04-20T10:33:00.508678+0000","last_clean":"2024-04-20T10:33:00.508678+0000","last_became_active":"2024-04-20T10:31:35.617465+0000","last_became_peered":"2024-04-20T10:31:35.617465+0000","last_unstale":"2024-04-20T10:33:00.508678+0000","last_undegraded":"2024-04-20T10:33:00.508678+0000","last_fullsized":"2024-04-20T10:33:00.508678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:37.830953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937866+0000","last_change":"2024-04-20T10:31:34.615044+0000","last_active":"2024-04-20T10:32:31.937866+0000","last_peered":"2024-04-20T10:32:31.937866+0000","last_clean":"2024-04-20T10:32:31.937866+0000","last_became_active":"2024-04-20T10:31:34.612875+0000","last_became_peered":"2024-04-20T10:31:34.612875+0000","last_unstale":"2024-04-20T10:32:31.937866+0000","last_undegraded":"2024-04-20T10:32:31.937866+0000","last_fullsized":"2024-04-20T10:32:31.937866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:59.867501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.529502+0000","last_change":"2024-04-20T10:31:35.630909+0000","last_active":"2024-04-20T10:33:00.529502+0000","last_peered":"2024-04-20T10:33:00.529502+0000","last_clean":"2024-04-20T10:33:00.529502+0000","last_became_active":"2024-04-20T10:31:35.630455+0000","last_became_peered":"2024-04-20T10:31:35.630455+0000","last_unstale":"2024-04-20T10:33:00.529502+0000","last_undegraded":"2024-04-20T10:33:00.529502+0000","last_fullsized":"2024-04-20T10:33:00.529502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:02.522485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:32.041897+0000","last_change":"2024-04-20T10:31:34.602466+0000","last_active":"2024-04-20T10:32:32.041897+0000","last_peered":"2024-04-20T10:32:32.041897+0000","last_clean":"2024-04-20T10:32:32.041897+0000","last_became_active":"2024-04-20T10:31:34.602089+0000","last_became_peered":"2024-04-20T10:31:34.602089+0000","last_unstale":"2024-04-20T10:32:32.041897+0000","last_undegraded":"2024-04-20T10:32:32.041897+0000","last_fullsized":"2024-04-20T10:32:32.041897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:19:09.849840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.933929+0000","last_change":"2024-04-20T10:30:14.724347+0000","last_active":"2024-04-20T10:32:31.933929+0000","last_peered":"2024-04-20T10:32:31.933929+0000","last_clean":"2024-04-20T10:32:31.933929+0000","last_became_active":"2024-04-20T10:30:14.723877+0000","last_became_peered":"2024-04-20T10:30:14.723877+0000","last_unstale":"2024-04-20T10:32:31.933929+0000","last_undegraded":"2024-04-20T10:32:31.933929+0000","last_fullsized":"2024-04-20T10:32:31.933929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_clean_scrub_stamp":"2024-04-20T10:26:16.803986+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:30.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.530137+0000","last_change":"2024-04-20T10:31:35.626505+0000","last_active":"2024-04-20T10:33:00.530137+0000","last_peered":"2024-04-20T10:33:00.530137+0000","last_clean":"2024-04-20T10:33:00.530137+0000","last_became_active":"2024-04-20T10:31:35.626141+0000","last_became_peered":"2024-04-20T10:31:35.626141+0000","last_unstale":"2024-04-20T10:33:00.530137+0000","last_undegraded":"2024-04-20T10:33:00.530137+0000","last_fullsized":"2024-04-20T10:33:00.530137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:29.827225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.933993+0000","last_change":"2024-04-20T10:31:34.596936+0000","last_active":"2024-04-20T10:32:31.933993+0000","last_peered":"2024-04-20T10:32:31.933993+0000","last_clean":"2024-04-20T10:32:31.933993+0000","last_became_active":"2024-04-20T10:31:34.596692+0000","last_became_peered":"2024-04-20T10:31:34.596692+0000","last_unstale":"2024-04-20T10:32:31.933993+0000","last_undegraded":"2024-04-20T10:32:31.933993+0000","last_fullsized":"2024-04-20T10:32:31.933993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:02:39.819096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.934417+0000","last_change":"2024-04-20T10:31:34.601407+0000","last_active":"2024-04-20T10:32:31.934417+0000","last_peered":"2024-04-20T10:32:31.934417+0000","last_clean":"2024-04-20T10:32:31.934417+0000","last_became_active":"2024-04-20T10:31:34.601007+0000","last_became_peered":"2024-04-20T10:31:34.601007+0000","last_unstale":"2024-04-20T10:32:31.934417+0000","last_undegraded":"2024-04-20T10:32:31.934417+0000","last_fullsized":"2024-04-20T10:32:31.934417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:05:03.037841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.526136+0000","last_change":"2024-04-20T10:31:35.641046+0000","last_active":"2024-04-20T10:33:00.526136+0000","last_peered":"2024-04-20T10:33:00.526136+0000","last_clean":"2024-04-20T10:33:00.526136+0000","last_became_active":"2024-04-20T10:31:35.640666+0000","last_became_peered":"2024-04-20T10:31:35.640666+0000","last_unstale":"2024-04-20T10:33:00.526136+0000","last_undegraded":"2024-04-20T10:33:00.526136+0000","last_fullsized":"2024-04-20T10:33:00.526136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:42.496037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.946154+0000","last_change":"2024-04-20T10:31:34.604643+0000","last_active":"2024-04-20T10:32:31.946154+0000","last_peered":"2024-04-20T10:32:31.946154+0000","last_clean":"2024-04-20T10:32:31.946154+0000","last_became_active":"2024-04-20T10:31:34.604210+0000","last_became_peered":"2024-04-20T10:31:34.604210+0000","last_unstale":"2024-04-20T10:32:31.946154+0000","last_undegraded":"2024-04-20T10:32:31.946154+0000","last_fullsized":"2024-04-20T10:32:31.946154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:24:09.748012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.542666+0000","last_change":"2024-04-20T10:31:35.641981+0000","last_active":"2024-04-20T10:33:00.542666+0000","last_peered":"2024-04-20T10:33:00.542666+0000","last_clean":"2024-04-20T10:33:00.542666+0000","last_became_active":"2024-04-20T10:31:35.641607+0000","last_became_peered":"2024-04-20T10:31:35.641607+0000","last_unstale":"2024-04-20T10:33:00.542666+0000","last_undegraded":"2024-04-20T10:33:00.542666+0000","last_fullsized":"2024-04-20T10:33:00.542666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:48.520813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:32.041725+0000","last_change":"2024-04-20T10:31:34.611218+0000","last_active":"2024-04-20T10:32:32.041725+0000","last_peered":"2024-04-20T10:32:32.041725+0000","last_clean":"2024-04-20T10:32:32.041725+0000","last_became_active":"2024-04-20T10:31:34.610950+0000","last_became_peered":"2024-04-20T10:31:34.610950+0000","last_unstale":"2024-04-20T10:32:32.041725+0000","last_undegraded":"2024-04-20T10:32:32.041725+0000","last_fullsized":"2024-04-20T10:32:32.041725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:14:11.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.521310+0000","last_change":"2024-04-20T10:31:35.647531+0000","last_active":"2024-04-20T10:33:00.521310+0000","last_peered":"2024-04-20T10:33:00.521310+0000","last_clean":"2024-04-20T10:33:00.521310+0000","last_became_active":"2024-04-20T10:31:35.646389+0000","last_became_peered":"2024-04-20T10:31:35.646389+0000","last_unstale":"2024-04-20T10:33:00.521310+0000","last_undegraded":"2024-04-20T10:33:00.521310+0000","last_fullsized":"2024-04-20T10:33:00.521310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:50.141385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.957675+0000","last_change":"2024-04-20T10:32:31.957675+0000","last_active":"2024-04-20T10:32:31.957675+0000","last_peered":"2024-04-20T10:32:31.957675+0000","last_clean":"2024-04-20T10:32:31.957675+0000","last_became_active":"2024-04-20T10:32:31.957202+0000","last_became_peered":"2024-04-20T10:32:31.957202+0000","last_unstale":"2024-04-20T10:32:31.957675+0000","last_undegraded":"2024-04-20T10:32:31.957675+0000","last_fullsized":"2024-04-20T10:32:31.957675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:28.001088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.482567+0000","last_change":"2024-04-20T10:31:35.616751+0000","last_active":"2024-04-20T10:33:00.482567+0000","last_peered":"2024-04-20T10:33:00.482567+0000","last_clean":"2024-04-20T10:33:00.482567+0000","last_became_active":"2024-04-20T10:31:35.616432+0000","last_became_peered":"2024-04-20T10:31:35.616432+0000","last_unstale":"2024-04-20T10:33:00.482567+0000","last_undegraded":"2024-04-20T10:33:00.482567+0000","last_fullsized":"2024-04-20T10:33:00.482567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:25.052140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.948999+0000","last_change":"2024-04-20T10:31:34.609139+0000","last_active":"2024-04-20T10:32:31.948999+0000","last_peered":"2024-04-20T10:32:31.948999+0000","last_clean":"2024-04-20T10:32:31.948999+0000","last_became_active":"2024-04-20T10:31:34.608818+0000","last_became_peered":"2024-04-20T10:31:34.608818+0000","last_unstale":"2024-04-20T10:32:31.948999+0000","last_undegraded":"2024-04-20T10:32:31.948999+0000","last_fullsized":"2024-04-20T10:32:31.948999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:42:39.581059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.521495+0000","last_change":"2024-04-20T10:31:35.643791+0000","last_active":"2024-04-20T10:33:00.521495+0000","last_peered":"2024-04-20T10:33:00.521495+0000","last_clean":"2024-04-20T10:33:00.521495+0000","last_became_active":"2024-04-20T10:31:35.643390+0000","last_became_peered":"2024-04-20T10:31:35.643390+0000","last_unstale":"2024-04-20T10:33:00.521495+0000","last_undegraded":"2024-04-20T10:33:00.521495+0000","last_fullsized":"2024-04-20T10:33:00.521495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:10.149979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.939109+0000","last_change":"2024-04-20T10:31:34.606178+0000","last_active":"2024-04-20T10:32:31.939109+0000","last_peered":"2024-04-20T10:32:31.939109+0000","last_clean":"2024-04-20T10:32:31.939109+0000","last_became_active":"2024-04-20T10:31:34.605817+0000","last_became_peered":"2024-04-20T10:31:34.605817+0000","last_unstale":"2024-04-20T10:32:31.939109+0000","last_undegraded":"2024-04-20T10:32:31.939109+0000","last_fullsized":"2024-04-20T10:32:31.939109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:15.929905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.536128+0000","last_change":"2024-04-20T10:31:35.649487+0000","last_active":"2024-04-20T10:33:00.536128+0000","last_peered":"2024-04-20T10:33:00.536128+0000","last_clean":"2024-04-20T10:33:00.536128+0000","last_became_active":"2024-04-20T10:31:35.648884+0000","last_became_peered":"2024-04-20T10:31:35.648884+0000","last_unstale":"2024-04-20T10:33:00.536128+0000","last_undegraded":"2024-04-20T10:33:00.536128+0000","last_fullsized":"2024-04-20T10:33:00.536128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:07.389100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.943441+0000","last_change":"2024-04-20T10:32:31.943005+0000","last_active":"2024-04-20T10:32:31.943441+0000","last_peered":"2024-04-20T10:32:31.943441+0000","last_clean":"2024-04-20T10:32:31.943441+0000","last_became_active":"2024-04-20T10:32:31.940063+0000","last_became_peered":"2024-04-20T10:32:31.940063+0000","last_unstale":"2024-04-20T10:32:31.943441+0000","last_undegraded":"2024-04-20T10:32:31.943441+0000","last_fullsized":"2024-04-20T10:32:31.943441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:11:06.308028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.524270+0000","last_change":"2024-04-20T10:31:35.630256+0000","last_active":"2024-04-20T10:33:00.524270+0000","last_peered":"2024-04-20T10:33:00.524270+0000","last_clean":"2024-04-20T10:33:00.524270+0000","last_became_active":"2024-04-20T10:31:35.629879+0000","last_became_peered":"2024-04-20T10:31:35.629879+0000","last_unstale":"2024-04-20T10:33:00.524270+0000","last_undegraded":"2024-04-20T10:33:00.524270+0000","last_fullsized":"2024-04-20T10:33:00.524270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:51.601923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.939172+0000","last_change":"2024-04-20T10:31:34.595149+0000","last_active":"2024-04-20T10:32:31.939172+0000","last_peered":"2024-04-20T10:32:31.939172+0000","last_clean":"2024-04-20T10:32:31.939172+0000","last_became_active":"2024-04-20T10:31:34.594808+0000","last_became_peered":"2024-04-20T10:31:34.594808+0000","last_unstale":"2024-04-20T10:32:31.939172+0000","last_undegraded":"2024-04-20T10:32:31.939172+0000","last_fullsized":"2024-04-20T10:32:31.939172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:51.707710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.522399+0000","last_change":"2024-04-20T10:31:35.620612+0000","last_active":"2024-04-20T10:33:00.522399+0000","last_peered":"2024-04-20T10:33:00.522399+0000","last_clean":"2024-04-20T10:33:00.522399+0000","last_became_active":"2024-04-20T10:31:35.620312+0000","last_became_peered":"2024-04-20T10:31:35.620312+0000","last_unstale":"2024-04-20T10:33:00.522399+0000","last_undegraded":"2024-04-20T10:33:00.522399+0000","last_fullsized":"2024-04-20T10:33:00.522399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:37:50.756950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.937750+0000","last_change":"2024-04-20T10:31:34.604613+0000","last_active":"2024-04-20T10:32:31.937750+0000","last_peered":"2024-04-20T10:32:31.937750+0000","last_clean":"2024-04-20T10:32:31.937750+0000","last_became_active":"2024-04-20T10:31:34.604246+0000","last_became_peered":"2024-04-20T10:31:34.604246+0000","last_unstale":"2024-04-20T10:32:31.937750+0000","last_undegraded":"2024-04-20T10:32:31.937750+0000","last_fullsized":"2024-04-20T10:32:31.937750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:26.521769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.541399+0000","last_change":"2024-04-20T10:31:35.639396+0000","last_active":"2024-04-20T10:33:00.541399+0000","last_peered":"2024-04-20T10:33:00.541399+0000","last_clean":"2024-04-20T10:33:00.541399+0000","last_became_active":"2024-04-20T10:31:35.638979+0000","last_became_peered":"2024-04-20T10:31:35.638979+0000","last_unstale":"2024-04-20T10:33:00.541399+0000","last_undegraded":"2024-04-20T10:33:00.541399+0000","last_fullsized":"2024-04-20T10:33:00.541399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:47.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:32:31.941634+0000","last_change":"2024-04-20T10:31:34.607420+0000","last_active":"2024-04-20T10:32:31.941634+0000","last_peered":"2024-04-20T10:32:31.941634+0000","last_clean":"2024-04-20T10:32:31.941634+0000","last_became_active":"2024-04-20T10:31:34.602587+0000","last_became_peered":"2024-04-20T10:31:34.602587+0000","last_unstale":"2024-04-20T10:32:31.941634+0000","last_undegraded":"2024-04-20T10:32:31.941634+0000","last_fullsized":"2024-04-20T10:32:31.941634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:56.447944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.521472+0000","last_change":"2024-04-20T10:31:35.624903+0000","last_active":"2024-04-20T10:33:00.521472+0000","last_peered":"2024-04-20T10:33:00.521472+0000","last_clean":"2024-04-20T10:33:00.521472+0000","last_became_active":"2024-04-20T10:31:35.619479+0000","last_became_peered":"2024-04-20T10:31:35.619479+0000","last_unstale":"2024-04-20T10:33:00.521472+0000","last_undegraded":"2024-04-20T10:33:00.521472+0000","last_fullsized":"2024-04-20T10:33:00.521472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:25:40.601983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.519812+0000","last_change":"2024-04-20T10:31:35.620351+0000","last_active":"2024-04-20T10:33:00.519812+0000","last_peered":"2024-04-20T10:33:00.519812+0000","last_clean":"2024-04-20T10:33:00.519812+0000","last_became_active":"2024-04-20T10:31:35.620028+0000","last_became_peered":"2024-04-20T10:31:35.620028+0000","last_unstale":"2024-04-20T10:33:00.519812+0000","last_undegraded":"2024-04-20T10:33:00.519812+0000","last_fullsized":"2024-04-20T10:33:00.519812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:51:53.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.515398+0000","last_change":"2024-04-20T10:31:35.615917+0000","last_active":"2024-04-20T10:33:00.515398+0000","last_peered":"2024-04-20T10:33:00.515398+0000","last_clean":"2024-04-20T10:33:00.515398+0000","last_became_active":"2024-04-20T10:31:35.615582+0000","last_became_peered":"2024-04-20T10:31:35.615582+0000","last_unstale":"2024-04-20T10:33:00.515398+0000","last_undegraded":"2024-04-20T10:33:00.515398+0000","last_fullsized":"2024-04-20T10:33:00.515398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:54:59.754819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.541383+0000","last_change":"2024-04-20T10:31:35.650006+0000","last_active":"2024-04-20T10:33:00.541383+0000","last_peered":"2024-04-20T10:33:00.541383+0000","last_clean":"2024-04-20T10:33:00.541383+0000","last_became_active":"2024-04-20T10:31:35.649683+0000","last_became_peered":"2024-04-20T10:31:35.649683+0000","last_unstale":"2024-04-20T10:33:00.541383+0000","last_undegraded":"2024-04-20T10:33:00.541383+0000","last_fullsized":"2024-04-20T10:33:00.541383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:10:01.484809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.536499+0000","last_change":"2024-04-20T10:31:35.646591+0000","last_active":"2024-04-20T10:33:00.536499+0000","last_peered":"2024-04-20T10:33:00.536499+0000","last_clean":"2024-04-20T10:33:00.536499+0000","last_became_active":"2024-04-20T10:31:35.646188+0000","last_became_peered":"2024-04-20T10:31:35.646188+0000","last_unstale":"2024-04-20T10:33:00.536499+0000","last_undegraded":"2024-04-20T10:33:00.536499+0000","last_fullsized":"2024-04-20T10:33:00.536499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:23.412022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.528581+0000","last_change":"2024-04-20T10:31:35.650879+0000","last_active":"2024-04-20T10:33:00.528581+0000","last_peered":"2024-04-20T10:33:00.528581+0000","last_clean":"2024-04-20T10:33:00.528581+0000","last_became_active":"2024-04-20T10:31:35.650486+0000","last_became_peered":"2024-04-20T10:31:35.650486+0000","last_unstale":"2024-04-20T10:33:00.528581+0000","last_undegraded":"2024-04-20T10:33:00.528581+0000","last_fullsized":"2024-04-20T10:33:00.528581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:51.893446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.519146+0000","last_change":"2024-04-20T10:31:35.619883+0000","last_active":"2024-04-20T10:33:00.519146+0000","last_peered":"2024-04-20T10:33:00.519146+0000","last_clean":"2024-04-20T10:33:00.519146+0000","last_became_active":"2024-04-20T10:31:35.619628+0000","last_became_peered":"2024-04-20T10:31:35.619628+0000","last_unstale":"2024-04-20T10:33:00.519146+0000","last_undegraded":"2024-04-20T10:33:00.519146+0000","last_fullsized":"2024-04-20T10:33:00.519146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:45.018417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.508312+0000","last_change":"2024-04-20T10:31:35.620773+0000","last_active":"2024-04-20T10:33:00.508312+0000","last_peered":"2024-04-20T10:33:00.508312+0000","last_clean":"2024-04-20T10:33:00.508312+0000","last_became_active":"2024-04-20T10:31:35.620467+0000","last_became_peered":"2024-04-20T10:31:35.620467+0000","last_unstale":"2024-04-20T10:33:00.508312+0000","last_undegraded":"2024-04-20T10:33:00.508312+0000","last_fullsized":"2024-04-20T10:33:00.508312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:31:48.009168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.538807+0000","last_change":"2024-04-20T10:32:31.953580+0000","last_active":"2024-04-20T10:33:00.538807+0000","last_peered":"2024-04-20T10:33:00.538807+0000","last_clean":"2024-04-20T10:33:00.538807+0000","last_became_active":"2024-04-20T10:32:31.952985+0000","last_became_peered":"2024-04-20T10:32:31.952985+0000","last_unstale":"2024-04-20T10:33:00.538807+0000","last_undegraded":"2024-04-20T10:33:00.538807+0000","last_fullsized":"2024-04-20T10:33:00.538807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:13:03.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.534975+0000","last_change":"2024-04-20T10:31:35.637588+0000","last_active":"2024-04-20T10:33:00.534975+0000","last_peered":"2024-04-20T10:33:00.534975+0000","last_clean":"2024-04-20T10:33:00.534975+0000","last_became_active":"2024-04-20T10:31:35.637319+0000","last_became_peered":"2024-04-20T10:31:35.637319+0000","last_unstale":"2024-04-20T10:33:00.534975+0000","last_undegraded":"2024-04-20T10:33:00.534975+0000","last_fullsized":"2024-04-20T10:33:00.534975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:54.070894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.542309+0000","last_change":"2024-04-20T10:31:35.629551+0000","last_active":"2024-04-20T10:33:00.542309+0000","last_peered":"2024-04-20T10:33:00.542309+0000","last_clean":"2024-04-20T10:33:00.542309+0000","last_became_active":"2024-04-20T10:31:35.628710+0000","last_became_peered":"2024-04-20T10:31:35.628710+0000","last_unstale":"2024-04-20T10:33:00.542309+0000","last_undegraded":"2024-04-20T10:33:00.542309+0000","last_fullsized":"2024-04-20T10:33:00.542309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:31.883788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.540571+0000","last_change":"2024-04-20T10:31:35.629978+0000","last_active":"2024-04-20T10:33:00.540571+0000","last_peered":"2024-04-20T10:33:00.540571+0000","last_clean":"2024-04-20T10:33:00.540571+0000","last_became_active":"2024-04-20T10:31:35.629596+0000","last_became_peered":"2024-04-20T10:31:35.629596+0000","last_unstale":"2024-04-20T10:33:00.540571+0000","last_undegraded":"2024-04-20T10:33:00.540571+0000","last_fullsized":"2024-04-20T10:33:00.540571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:02.523099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.536845+0000","last_change":"2024-04-20T10:31:35.652481+0000","last_active":"2024-04-20T10:33:00.536845+0000","last_peered":"2024-04-20T10:33:00.536845+0000","last_clean":"2024-04-20T10:33:00.536845+0000","last_became_active":"2024-04-20T10:31:35.651837+0000","last_became_peered":"2024-04-20T10:31:35.651837+0000","last_unstale":"2024-04-20T10:33:00.536845+0000","last_undegraded":"2024-04-20T10:33:00.536845+0000","last_fullsized":"2024-04-20T10:33:00.536845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:53.206954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-20T10:33:00.520310+0000","last_change":"2024-04-20T10:31:35.622228+0000","last_active":"2024-04-20T10:33:00.520310+0000","last_peered":"2024-04-20T10:33:00.520310+0000","last_clean":"2024-04-20T10:33:00.520310+0000","last_became_active":"2024-04-20T10:31:35.621910+0000","last_became_peered":"2024-04-20T10:31:35.621910+0000","last_unstale":"2024-04-20T10:33:00.520310+0000","last_undegraded":"2024-04-20T10:33:00.520310+0000","last_fullsized":"2024-04-20T10:33:00.520310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:47:34.319839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281463,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62093,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":187,"num_read_kb":155,"num_write":84,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":186279,"data_compressed":5541,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383712,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":455821,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1069056,"data_stored":1107795,"data_compressed":43453,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":456403,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":456425,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":1046888,"data_compressed":41606,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":455938,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":517116,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":455911,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":517150,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":456159,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":455843,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":1046488,"data_compressed":41606,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34156,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34156,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34156,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61295,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61329,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61489,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:33:07.538 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-20T10:33:07.538 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T10:33:07.538 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-20T10:33:07.539 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph health --format=json 2024-04-20T10:33:08.046 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:33:08.516 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:08 smithi042 ceph-mon[17078]: pgmap v365: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:08.516 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:08 smithi042 ceph-mon[17078]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:33:08.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:08 smithi046 ceph-mon[21574]: pgmap v365: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:08.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:08 smithi046 ceph-mon[21574]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:33:08.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:08 smithi162 ceph-mon[21854]: pgmap v365: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:08.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:08 smithi162 ceph-mon[21854]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:33:10.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.212+0000 7fb8d1800700 1 -- 172.21.15.42:0/3036633733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8cc10f040 msgr2=0x7fb8cc10f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.212+0000 7fb8d1800700 1 --2- 172.21.15.42:0/3036633733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8cc10f040 0x7fb8cc10f420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009990 tx=0x7fb8bc009ca0 comp rx=0 tx=0).stop 2024-04-20T10:33:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.212+0000 7fb8d1800700 1 -- 172.21.15.42:0/3036633733 shutdown_connections 2024-04-20T10:33:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.212+0000 7fb8d1800700 1 --2- 172.21.15.42:0/3036633733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8cc109950 0x7fb8cc115df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.212+0000 7fb8d1800700 1 --2- 172.21.15.42:0/3036633733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8cc10f040 0x7fb8cc10f420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.212+0000 7fb8d1800700 1 --2- 172.21.15.42:0/3036633733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8cc108f90 0x7fb8cc109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.212+0000 7fb8d1800700 1 -- 172.21.15.42:0/3036633733 >> 172.21.15.42:0/3036633733 conn(0x7fb8cc0787e0 msgr2=0x7fb8cc107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.212+0000 7fb8d1800700 1 -- 172.21.15.42:0/3036633733 shutdown_connections 2024-04-20T10:33:10.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.212+0000 7fb8d1800700 1 -- 172.21.15.42:0/3036633733 wait complete. 2024-04-20T10:33:10.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.214+0000 7fb8d1800700 1 Processor -- start 2024-04-20T10:33:10.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.214+0000 7fb8d1800700 1 -- start start 2024-04-20T10:33:10.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.214+0000 7fb8d1800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8cc108f90 0x7fb8cc19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.214+0000 7fb8d1800700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8cc109950 0x7fb8cc19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.214+0000 7fb8d1800700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8cc10f040 0x7fb8cc1a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.214+0000 7fb8d1800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc119280 con 0x7fb8cc108f90 2024-04-20T10:33:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.214+0000 7fb8d1800700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc119100 con 0x7fb8cc10f040 2024-04-20T10:33:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.214+0000 7fb8d1800700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc119400 con 0x7fb8cc109950 2024-04-20T10:33:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.215+0000 7fb8cbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8cc108f90 0x7fb8cc19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.215+0000 7fb8cbe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8cc108f90 0x7fb8cc19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57068/0 (socket says 172.21.15.42:57068) 2024-04-20T10:33:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.215+0000 7fb8cbe00700 1 -- 172.21.15.42:0/4080461509 learned_addr learned my addr 172.21.15.42:0/4080461509 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.215+0000 7fb8cb400700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8cc109950 0x7fb8cc19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.215+0000 7fb8d0e00700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8cc10f040 0x7fb8cc1a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:10.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.215+0000 7fb8cb400700 1 -- 172.21.15.42:0/4080461509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8cc10f040 msgr2=0x7fb8cc1a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:10.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.215+0000 7fb8cb400700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8cc10f040 0x7fb8cc1a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:10.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.215+0000 7fb8cb400700 1 -- 172.21.15.42:0/4080461509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8cc108f90 msgr2=0x7fb8cc19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:10.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.215+0000 7fb8cb400700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8cc108f90 0x7fb8cc19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.215+0000 7fb8cb400700 1 -- 172.21.15.42:0/4080461509 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc1a4720 con 0x7fb8cc109950 2024-04-20T10:33:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.215+0000 7fb8cbe00700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8cc108f90 0x7fb8cc19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.216+0000 7fb8cb400700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8cc109950 0x7fb8cc19fc50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e9f0 tx=0x7fb8b800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.216+0000 7fb8c8c00700 1 -- 172.21.15.42:0/4080461509 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800c9c0 con 0x7fb8cc109950 2024-04-20T10:33:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.216+0000 7fb8c8c00700 1 -- 172.21.15.42:0/4080461509 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b800cb60 con 0x7fb8cc109950 2024-04-20T10:33:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.216+0000 7fb8c8c00700 1 -- 172.21.15.42:0/4080461509 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8018680 con 0x7fb8cc109950 2024-04-20T10:33:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.216+0000 7fb8d1800700 1 -- 172.21.15.42:0/4080461509 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc1a4a10 con 0x7fb8cc109950 2024-04-20T10:33:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.216+0000 7fb8d1800700 1 -- 172.21.15.42:0/4080461509 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc1ac290 con 0x7fb8cc109950 2024-04-20T10:33:10.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.217+0000 7fb8d1800700 1 -- 172.21.15.42:0/4080461509 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8cc069740 con 0x7fb8cc109950 2024-04-20T10:33:10.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.219+0000 7fb8c8c00700 1 -- 172.21.15.42:0/4080461509 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb8b8010430 con 0x7fb8cc109950 2024-04-20T10:33:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.221+0000 7fb8c8c00700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb8b00a3cc0 0x7fb8b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.221+0000 7fb8c8c00700 1 -- 172.21.15.42:0/4080461509 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fb8b80d04c0 con 0x7fb8cc109950 2024-04-20T10:33:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.221+0000 7fb8cbe00700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb8b00a3cc0 0x7fb8b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:10.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.222+0000 7fb8cbe00700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb8b00a3cc0 0x7fb8b00a6180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc006010 tx=0x7fb8bc009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:10.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.223+0000 7fb8c8c00700 1 -- 172.21.15.42:0/4080461509 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb8b809ce20 con 0x7fb8cc109950 2024-04-20T10:33:10.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:10 smithi042 ceph-mon[17078]: pgmap v366: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:10.438 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:10 smithi046 ceph-mon[21574]: pgmap v366: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:10.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.506+0000 7fb8d1800700 1 -- 172.21.15.42:0/4080461509 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb8cc02df90 con 0x7fb8cc109950 2024-04-20T10:33:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.507+0000 7fb8c8c00700 1 -- 172.21.15.42:0/4080461509 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb8b809c720 con 0x7fb8cc109950 2024-04-20T10:33:10.508 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:10.508 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T10:33:10.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.510+0000 7fb8d1800700 1 -- 172.21.15.42:0/4080461509 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb8b00a3cc0 msgr2=0x7fb8b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:10.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.510+0000 7fb8d1800700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb8b00a3cc0 0x7fb8b00a6180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc006010 tx=0x7fb8bc009570 comp rx=0 tx=0).stop 2024-04-20T10:33:10.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.510+0000 7fb8d1800700 1 -- 172.21.15.42:0/4080461509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8cc109950 msgr2=0x7fb8cc19fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:10.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.510+0000 7fb8d1800700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8cc109950 0x7fb8cc19fc50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e9f0 tx=0x7fb8b800ed00 comp rx=0 tx=0).stop 2024-04-20T10:33:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.510+0000 7fb8d1800700 1 -- 172.21.15.42:0/4080461509 shutdown_connections 2024-04-20T10:33:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.510+0000 7fb8d1800700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb8b00a3cc0 0x7fb8b00a6180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.510+0000 7fb8d1800700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8cc10f040 0x7fb8cc1a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.510+0000 7fb8d1800700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8cc108f90 0x7fb8cc19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.510+0000 7fb8d1800700 1 --2- 172.21.15.42:0/4080461509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8cc109950 0x7fb8cc19fc50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.510+0000 7fb8d1800700 1 -- 172.21.15.42:0/4080461509 >> 172.21.15.42:0/4080461509 conn(0x7fb8cc0787e0 msgr2=0x7fb8cc079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.510+0000 7fb8d1800700 1 -- 172.21.15.42:0/4080461509 shutdown_connections 2024-04-20T10:33:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:10.510+0000 7fb8d1800700 1 -- 172.21.15.42:0/4080461509 wait complete. 2024-04-20T10:33:10.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:10 smithi162 ceph-mon[21854]: pgmap v366: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:11.426 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-20T10:33:11.427 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:33:11.505 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:11 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4080461509' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T10:33:11.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:11 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/4080461509' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T10:33:11.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:11 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/4080461509' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T10:33:11.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.651+0000 7f373fe00640 1 -- 172.21.15.42:0/1434812687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37401019d0 msgr2=0x7f3740101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:11.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.651+0000 7f373fe00640 1 --2- 172.21.15.42:0/1434812687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37401019d0 0x7f3740101e50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3738009be0 tx=0x7f373802bf80 comp rx=0 tx=0).stop 2024-04-20T10:33:11.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.651+0000 7f373fe00640 1 -- 172.21.15.42:0/1434812687 shutdown_connections 2024-04-20T10:33:11.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.651+0000 7f373fe00640 1 --2- 172.21.15.42:0/1434812687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37401019d0 0x7f3740101e50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:11.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.651+0000 7f373fe00640 1 --2- 172.21.15.42:0/1434812687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3740101030 0x7f3740101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:11.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.651+0000 7f373fe00640 1 --2- 172.21.15.42:0/1434812687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37401006d0 0x7f3740100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:11.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.651+0000 7f373fe00640 1 -- 172.21.15.42:0/1434812687 >> 172.21.15.42:0/1434812687 conn(0x7f37400fc230 msgr2=0x7f37400fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:11.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.652+0000 7f373fe00640 1 -- 172.21.15.42:0/1434812687 shutdown_connections 2024-04-20T10:33:11.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.652+0000 7f373fe00640 1 -- 172.21.15.42:0/1434812687 wait complete. 2024-04-20T10:33:11.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.653+0000 7f373fe00640 1 Processor -- start 2024-04-20T10:33:11.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.653+0000 7f373fe00640 1 -- start start 2024-04-20T10:33:11.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.653+0000 7f373fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37401006d0 0x7f374019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:11.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.653+0000 7f373fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3740101030 0x7f374019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:11.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.653+0000 7f373fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37401019d0 0x7f37401a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:11.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.653+0000 7f373fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f374010fc70 con 0x7f37401006d0 2024-04-20T10:33:11.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.653+0000 7f373fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f374010faf0 con 0x7f3740101030 2024-04-20T10:33:11.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.653+0000 7f373fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f374010fdf0 con 0x7f37401019d0 2024-04-20T10:33:11.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.654+0000 7f373ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37401006d0 0x7f374019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:11.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.654+0000 7f373ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37401006d0 0x7f374019a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57090/0 (socket says 172.21.15.42:57090) 2024-04-20T10:33:11.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.654+0000 7f373f400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37401019d0 0x7f37401a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:11.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.654+0000 7f373ea00640 1 -- 172.21.15.42:0/1776585684 learned_addr learned my addr 172.21.15.42:0/1776585684 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:11.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.654+0000 7f373e000640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3740101030 0x7f374019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:11.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.654+0000 7f373ea00640 1 -- 172.21.15.42:0/1776585684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37401019d0 msgr2=0x7f37401a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:11.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.654+0000 7f373ea00640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37401019d0 0x7f37401a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:11.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.654+0000 7f373ea00640 1 -- 172.21.15.42:0/1776585684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3740101030 msgr2=0x7f374019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:11.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.654+0000 7f373ea00640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3740101030 0x7f374019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:11.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.654+0000 7f373ea00640 1 -- 172.21.15.42:0/1776585684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37401a1ec0 con 0x7f37401006d0 2024-04-20T10:33:11.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.654+0000 7f373e000640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3740101030 0x7f374019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:11.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.655+0000 7f373ea00640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37401006d0 0x7f374019a2a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f373000e9f0 tx=0x7f373000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:11.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.655+0000 7f3737400640 1 -- 172.21.15.42:0/1776585684 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373000cc40 con 0x7f37401006d0 2024-04-20T10:33:11.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.655+0000 7f3737400640 1 -- 172.21.15.42:0/1776585684 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f373000cde0 con 0x7f37401006d0 2024-04-20T10:33:11.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.655+0000 7f373fe00640 1 -- 172.21.15.42:0/1776585684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37401a21b0 con 0x7f37401006d0 2024-04-20T10:33:11.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.655+0000 7f3737400640 1 -- 172.21.15.42:0/1776585684 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730018860 con 0x7f37401006d0 2024-04-20T10:33:11.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.655+0000 7f373fe00640 1 -- 172.21.15.42:0/1776585684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37401a2700 con 0x7f37401006d0 2024-04-20T10:33:11.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.656+0000 7f373fe00640 1 -- 172.21.15.42:0/1776585684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374010b420 con 0x7f37401006d0 2024-04-20T10:33:11.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.658+0000 7f3737400640 1 -- 172.21.15.42:0/1776585684 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3730018a00 con 0x7f37401006d0 2024-04-20T10:33:11.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.660+0000 7f3737400640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f37240a3c00 0x7f37240a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:11.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.660+0000 7f3737400640 1 -- 172.21.15.42:0/1776585684 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f37300d0760 con 0x7f37401006d0 2024-04-20T10:33:11.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.660+0000 7f373e000640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f37240a3c00 0x7f37240a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:11.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.661+0000 7f373e000640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f37240a3c00 0x7f37240a60c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f374019b7b0 tx=0x7f372c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:11.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.665+0000 7f3737400640 1 -- 172.21.15.42:0/1776585684 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f373009d0c0 con 0x7f37401006d0 2024-04-20T10:33:11.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.880+0000 7f373fe00640 1 -- 172.21.15.42:0/1776585684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3740062b90 con 0x7f37401006d0 2024-04-20T10:33:11.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.883+0000 7f3737400640 1 -- 172.21.15.42:0/1776585684 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7f373009c9c0 con 0x7f37401006d0 2024-04-20T10:33:11.884 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:11.884 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:32:31.928088+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:33:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.885+0000 7f373fe00640 1 -- 172.21.15.42:0/1776585684 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f37240a3c00 msgr2=0x7f37240a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.885+0000 7f373fe00640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f37240a3c00 0x7f37240a60c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f374019b7b0 tx=0x7f372c009260 comp rx=0 tx=0).stop 2024-04-20T10:33:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.885+0000 7f373fe00640 1 -- 172.21.15.42:0/1776585684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37401006d0 msgr2=0x7f374019a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.885+0000 7f373fe00640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37401006d0 0x7f374019a2a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f373000e9f0 tx=0x7f373000edf0 comp rx=0 tx=0).stop 2024-04-20T10:33:11.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.886+0000 7f373fe00640 1 -- 172.21.15.42:0/1776585684 shutdown_connections 2024-04-20T10:33:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.886+0000 7f373fe00640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f37240a3c00 0x7f37240a60c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.886+0000 7f373fe00640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37401019d0 0x7f37401a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.886+0000 7f373fe00640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3740101030 0x7f374019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.886+0000 7f373fe00640 1 --2- 172.21.15.42:0/1776585684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37401006d0 0x7f374019a2a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.886+0000 7f373fe00640 1 -- 172.21.15.42:0/1776585684 >> 172.21.15.42:0/1776585684 conn(0x7f37400fc230 msgr2=0x7f37400fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.886+0000 7f373fe00640 1 -- 172.21.15.42:0/1776585684 shutdown_connections 2024-04-20T10:33:11.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:11.886+0000 7f373fe00640 1 -- 172.21.15.42:0/1776585684 wait complete. 2024-04-20T10:33:11.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:26:16.537072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:31:33.523692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:31:34.019519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:33:11.907 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:33:12.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.123+0000 7f4381c00640 1 -- 172.21.15.42:0/1050932721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f437c1041c0 msgr2=0x7f437c1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:12.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.123+0000 7f4381c00640 1 --2- 172.21.15.42:0/1050932721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f437c1041c0 0x7f437c1045e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f436c009900 tx=0x7f436c02bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:12.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.124+0000 7f4381c00640 1 -- 172.21.15.42:0/1050932721 shutdown_connections 2024-04-20T10:33:12.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.124+0000 7f4381c00640 1 --2- 172.21.15.42:0/1050932721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f437c105d70 0x7f437c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.124+0000 7f4381c00640 1 --2- 172.21.15.42:0/1050932721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f437c1053b0 0x7f437c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.124+0000 7f4381c00640 1 --2- 172.21.15.42:0/1050932721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f437c1041c0 0x7f437c1045e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.124+0000 7f4381c00640 1 -- 172.21.15.42:0/1050932721 >> 172.21.15.42:0/1050932721 conn(0x7f437c0ff750 msgr2=0x7f437c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:12.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.124+0000 7f4381c00640 1 -- 172.21.15.42:0/1050932721 shutdown_connections 2024-04-20T10:33:12.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.124+0000 7f4381c00640 1 -- 172.21.15.42:0/1050932721 wait complete. 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.125+0000 7f4381c00640 1 Processor -- start 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.125+0000 7f4381c00640 1 -- start start 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.125+0000 7f4381c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f437c1041c0 0x7f437c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.125+0000 7f4381c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f437c1053b0 0x7f437c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.125+0000 7f4381c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f437c105d70 0x7f437c1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.125+0000 7f4381c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f437c10fe50 con 0x7f437c105d70 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.125+0000 7f4381c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f437c10fcd0 con 0x7f437c1041c0 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.125+0000 7f4381c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f437c10ffd0 con 0x7f437c1053b0 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.126+0000 7f437aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f437c1041c0 0x7f437c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.126+0000 7f437a000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f437c1053b0 0x7f437c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.126+0000 7f437aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f437c1041c0 0x7f437c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46522/0 (socket says 172.21.15.42:46522) 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.126+0000 7f437aa00640 1 -- 172.21.15.42:0/1600513804 learned_addr learned my addr 172.21.15.42:0/1600513804 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:12.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.126+0000 7f437b400640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f437c105d70 0x7f437c1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.126+0000 7f437aa00640 1 -- 172.21.15.42:0/1600513804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f437c1053b0 msgr2=0x7f437c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.126+0000 7f437aa00640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f437c1053b0 0x7f437c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.126+0000 7f437aa00640 1 -- 172.21.15.42:0/1600513804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f437c105d70 msgr2=0x7f437c1a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.126+0000 7f437aa00640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f437c105d70 0x7f437c1a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.126+0000 7f437aa00640 1 -- 172.21.15.42:0/1600513804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c1a20b0 con 0x7f437c1041c0 2024-04-20T10:33:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.126+0000 7f437b400640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f437c105d70 0x7f437c1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:12.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.127+0000 7f437aa00640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f437c1041c0 0x7f437c19a490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f436c000c00 tx=0x7f436c02c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.127+0000 7f4373400640 1 -- 172.21.15.42:0/1600513804 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c041070 con 0x7f437c1041c0 2024-04-20T10:33:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.127+0000 7f4373400640 1 -- 172.21.15.42:0/1600513804 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c004450 con 0x7f437c1041c0 2024-04-20T10:33:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.127+0000 7f4381c00640 1 -- 172.21.15.42:0/1600513804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c1a2340 con 0x7f437c1041c0 2024-04-20T10:33:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.127+0000 7f4373400640 1 -- 172.21.15.42:0/1600513804 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c034440 con 0x7f437c1041c0 2024-04-20T10:33:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.127+0000 7f4381c00640 1 -- 172.21.15.42:0/1600513804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c1a2830 con 0x7f437c1041c0 2024-04-20T10:33:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.128+0000 7f4381c00640 1 -- 172.21.15.42:0/1600513804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437c10b4e0 con 0x7f437c1041c0 2024-04-20T10:33:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.130+0000 7f4373400640 1 -- 172.21.15.42:0/1600513804 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f436c044070 con 0x7f437c1041c0 2024-04-20T10:33:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.132+0000 7f4373400640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f43600a3bb0 0x7f43600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.132+0000 7f4373400640 1 -- 172.21.15.42:0/1600513804 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f436c0f2180 con 0x7f437c1041c0 2024-04-20T10:33:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.132+0000 7f437a000640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f43600a3bb0 0x7f43600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.133+0000 7f437a000640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f43600a3bb0 0x7f43600a6070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f43680095f0 tx=0x7f4368005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:12.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.134+0000 7f4373400640 1 -- 172.21.15.42:0/1600513804 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f436c0be520 con 0x7f437c1041c0 2024-04-20T10:33:12.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.341+0000 7f4381c00640 1 -- 172.21.15.42:0/1600513804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f437c02d760 con 0x7f437c1041c0 2024-04-20T10:33:12.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.342+0000 7f4373400640 1 -- 172.21.15.42:0/1600513804 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f436c0c2820 con 0x7f437c1041c0 2024-04-20T10:33:12.342 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-20T10:33:12.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.344+0000 7f4381c00640 1 -- 172.21.15.42:0/1600513804 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f43600a3bb0 msgr2=0x7f43600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:12.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.344+0000 7f4381c00640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f43600a3bb0 0x7f43600a6070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f43680095f0 tx=0x7f4368005f90 comp rx=0 tx=0).stop 2024-04-20T10:33:12.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.344+0000 7f4381c00640 1 -- 172.21.15.42:0/1600513804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f437c1041c0 msgr2=0x7f437c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:12.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.344+0000 7f4381c00640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f437c1041c0 0x7f437c19a490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f436c000c00 tx=0x7f436c02c640 comp rx=0 tx=0).stop 2024-04-20T10:33:12.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.344+0000 7f4381c00640 1 -- 172.21.15.42:0/1600513804 shutdown_connections 2024-04-20T10:33:12.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.344+0000 7f4381c00640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f437c105d70 0x7f437c1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.344+0000 7f4381c00640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f43600a3bb0 0x7f43600a6070 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.345+0000 7f4381c00640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f437c1053b0 0x7f437c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.345+0000 7f4381c00640 1 --2- 172.21.15.42:0/1600513804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f437c1041c0 0x7f437c19a490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.345+0000 7f4381c00640 1 -- 172.21.15.42:0/1600513804 >> 172.21.15.42:0/1600513804 conn(0x7f437c0ff750 msgr2=0x7f437c1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:12.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.345+0000 7f4381c00640 1 -- 172.21.15.42:0/1600513804 shutdown_connections 2024-04-20T10:33:12.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.345+0000 7f4381c00640 1 -- 172.21.15.42:0/1600513804 wait complete. 2024-04-20T10:33:12.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:33:12.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:12 smithi046 ceph-mon[21574]: pgmap v367: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:12.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:12 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1776585684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:12.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.581+0000 7fb7ea000640 1 -- 172.21.15.42:0/4038589364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7e4102fc0 msgr2=0x7fb7e4103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:12.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.581+0000 7fb7ea000640 1 --2- 172.21.15.42:0/4038589364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7e4102fc0 0x7fb7e4103440 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009900 tx=0x7fb7d802bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:12.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.581+0000 7fb7ea000640 1 -- 172.21.15.42:0/4038589364 shutdown_connections 2024-04-20T10:33:12.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.581+0000 7fb7ea000640 1 --2- 172.21.15.42:0/4038589364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7e4103980 0x7fb7e410a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.581+0000 7fb7ea000640 1 --2- 172.21.15.42:0/4038589364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7e4102fc0 0x7fb7e4103440 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.581+0000 7fb7ea000640 1 --2- 172.21.15.42:0/4038589364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e4101e70 0x7fb7e4102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.581+0000 7fb7ea000640 1 -- 172.21.15.42:0/4038589364 >> 172.21.15.42:0/4038589364 conn(0x7fb7e40fd400 msgr2=0x7fb7e40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:12.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.581+0000 7fb7ea000640 1 -- 172.21.15.42:0/4038589364 shutdown_connections 2024-04-20T10:33:12.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.581+0000 7fb7ea000640 1 -- 172.21.15.42:0/4038589364 wait complete. 2024-04-20T10:33:12.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.582+0000 7fb7ea000640 1 Processor -- start 2024-04-20T10:33:12.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.583+0000 7fb7ea000640 1 -- start start 2024-04-20T10:33:12.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.583+0000 7fb7ea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e4101e70 0x7fb7e419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.583+0000 7fb7ea000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7e4102fc0 0x7fb7e419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.583+0000 7fb7ea000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7e4103980 0x7fb7e41a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.583+0000 7fb7ea000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e410db20 con 0x7fb7e4103980 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.583+0000 7fb7ea000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e410d9a0 con 0x7fb7e4101e70 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.583+0000 7fb7ea000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e410dca0 con 0x7fb7e4102fc0 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.583+0000 7fb7e8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e4101e70 0x7fb7e419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.583+0000 7fb7e8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e4101e70 0x7fb7e419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46536/0 (socket says 172.21.15.42:46536) 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.583+0000 7fb7e8c00640 1 -- 172.21.15.42:0/1189301672 learned_addr learned my addr 172.21.15.42:0/1189301672 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.583+0000 7fb7e9600640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7e4103980 0x7fb7e41a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.583+0000 7fb7e3e00640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7e4102fc0 0x7fb7e419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.584+0000 7fb7e8c00640 1 -- 172.21.15.42:0/1189301672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7e4102fc0 msgr2=0x7fb7e419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.584+0000 7fb7e8c00640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7e4102fc0 0x7fb7e419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.584+0000 7fb7e8c00640 1 -- 172.21.15.42:0/1189301672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7e4103980 msgr2=0x7fb7e41a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:12.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.584+0000 7fb7e8c00640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7e4103980 0x7fb7e41a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.584+0000 7fb7e8c00640 1 -- 172.21.15.42:0/1189301672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e41a2000 con 0x7fb7e4101e70 2024-04-20T10:33:12.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.584+0000 7fb7e9600640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7e4103980 0x7fb7e41a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:12.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.584+0000 7fb7e8c00640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e4101e70 0x7fb7e419a3e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400e9f0 tx=0x7fb7d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:12.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.584+0000 7fb7e1600640 1 -- 172.21.15.42:0/1189301672 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d400cd80 con 0x7fb7e4101e70 2024-04-20T10:33:12.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.585+0000 7fb7e1600640 1 -- 172.21.15.42:0/1189301672 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7d4010910 con 0x7fb7e4101e70 2024-04-20T10:33:12.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.585+0000 7fb7ea000640 1 -- 172.21.15.42:0/1189301672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e41a22f0 con 0x7fb7e4101e70 2024-04-20T10:33:12.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.585+0000 7fb7e1600640 1 -- 172.21.15.42:0/1189301672 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4018990 con 0x7fb7e4101e70 2024-04-20T10:33:12.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.585+0000 7fb7ea000640 1 -- 172.21.15.42:0/1189301672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e41a2810 con 0x7fb7e4101e70 2024-04-20T10:33:12.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.586+0000 7fb7ea000640 1 -- 172.21.15.42:0/1189301672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e41a2a70 con 0x7fb7e4101e70 2024-04-20T10:33:12.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.588+0000 7fb7e1600640 1 -- 172.21.15.42:0/1189301672 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb7d4010ac0 con 0x7fb7e4101e70 2024-04-20T10:33:12.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.589+0000 7fb7e1600640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb7cc0a3bb0 0x7fb7cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:12.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.589+0000 7fb7e1600640 1 -- 172.21.15.42:0/1189301672 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fb7d40d07a0 con 0x7fb7e4101e70 2024-04-20T10:33:12.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.590+0000 7fb7e3e00640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb7cc0a3bb0 0x7fb7cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:12.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.590+0000 7fb7e3e00640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb7cc0a3bb0 0x7fb7cc0a6070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8000c00 tx=0x7fb7d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:12.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.594+0000 7fb7e1600640 1 -- 172.21.15.42:0/1189301672 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb7d409cab0 con 0x7fb7e4101e70 2024-04-20T10:33:12.602 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:12 smithi042 ceph-mon[17078]: pgmap v367: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:12.602 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:12 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1776585684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:12.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:12 smithi162 ceph-mon[21854]: pgmap v367: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:12.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:12 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1776585684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:12.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.799+0000 7fb7ea000640 1 -- 172.21.15.42:0/1189301672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb7e41a2e00 con 0x7fb7e4101e70 2024-04-20T10:33:12.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.800+0000 7fb7e1600640 1 -- 172.21.15.42:0/1189301672 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fb7d40a0db0 con 0x7fb7e4101e70 2024-04-20T10:33:12.801 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-20T10:33:12.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.803+0000 7fb7ea000640 1 -- 172.21.15.42:0/1189301672 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb7cc0a3bb0 msgr2=0x7fb7cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:12.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.803+0000 7fb7ea000640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb7cc0a3bb0 0x7fb7cc0a6070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8000c00 tx=0x7fb7d8005fd0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.803+0000 7fb7ea000640 1 -- 172.21.15.42:0/1189301672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e4101e70 msgr2=0x7fb7e419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:12.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.803+0000 7fb7ea000640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e4101e70 0x7fb7e419a3e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400e9f0 tx=0x7fb7d400eeb0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.803+0000 7fb7ea000640 1 -- 172.21.15.42:0/1189301672 shutdown_connections 2024-04-20T10:33:12.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.803+0000 7fb7ea000640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7e4103980 0x7fb7e41a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.803+0000 7fb7ea000640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb7cc0a3bb0 0x7fb7cc0a6070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.803+0000 7fb7ea000640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7e4102fc0 0x7fb7e419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.803+0000 7fb7ea000640 1 --2- 172.21.15.42:0/1189301672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e4101e70 0x7fb7e419a3e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:12.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.803+0000 7fb7ea000640 1 -- 172.21.15.42:0/1189301672 >> 172.21.15.42:0/1189301672 conn(0x7fb7e40fd400 msgr2=0x7fb7e4074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:12.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.803+0000 7fb7ea000640 1 -- 172.21.15.42:0/1189301672 shutdown_connections 2024-04-20T10:33:12.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:12.803+0000 7fb7ea000640 1 -- 172.21.15.42:0/1189301672 wait complete. 2024-04-20T10:33:12.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:33:13.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.048+0000 7f3570c00640 1 -- 172.21.15.42:0/3084133878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c1051b0 msgr2=0x7f356c105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:13.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.048+0000 7f3570c00640 1 --2- 172.21.15.42:0/3084133878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c1051b0 0x7f356c105630 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3558009900 tx=0x7f355802bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:13.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.048+0000 7f3570c00640 1 -- 172.21.15.42:0/3084133878 shutdown_connections 2024-04-20T10:33:13.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.048+0000 7f3570c00640 1 --2- 172.21.15.42:0/3084133878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f356c105b70 0x7f356c10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.048+0000 7f3570c00640 1 --2- 172.21.15.42:0/3084133878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c1051b0 0x7f356c105630 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.048+0000 7f3570c00640 1 --2- 172.21.15.42:0/3084133878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c103fc0 0x7f356c1043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.048+0000 7f3570c00640 1 -- 172.21.15.42:0/3084133878 >> 172.21.15.42:0/3084133878 conn(0x7f356c0ff530 msgr2=0x7f356c101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:13.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.049+0000 7f3570c00640 1 -- 172.21.15.42:0/3084133878 shutdown_connections 2024-04-20T10:33:13.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.049+0000 7f3570c00640 1 -- 172.21.15.42:0/3084133878 wait complete. 2024-04-20T10:33:13.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.050+0000 7f3570c00640 1 Processor -- start 2024-04-20T10:33:13.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.050+0000 7f3570c00640 1 -- start start 2024-04-20T10:33:13.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.050+0000 7f3570c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c103fc0 0x7f356c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:13.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.050+0000 7f3570c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f356c1051b0 0x7f356c19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:13.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.050+0000 7f3570c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c105b70 0x7f356c1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:13.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.050+0000 7f3570c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f356c10fc50 con 0x7f356c1051b0 2024-04-20T10:33:13.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.050+0000 7f3570c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f356c10fad0 con 0x7f356c105b70 2024-04-20T10:33:13.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.050+0000 7f3570c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f356c10fdd0 con 0x7f356c103fc0 2024-04-20T10:33:13.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.051+0000 7f356aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f356c1051b0 0x7f356c19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:13.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.051+0000 7f356aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f356c1051b0 0x7f356c19a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57160/0 (socket says 172.21.15.42:57160) 2024-04-20T10:33:13.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.051+0000 7f356aa00640 1 -- 172.21.15.42:0/3342313521 learned_addr learned my addr 172.21.15.42:0/3342313521 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:13.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.051+0000 7f356b400640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c103fc0 0x7f356c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:13.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.051+0000 7f356be00640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c105b70 0x7f356c1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:13.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.051+0000 7f356aa00640 1 -- 172.21.15.42:0/3342313521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c103fc0 msgr2=0x7f356c19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:13.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.051+0000 7f356aa00640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c103fc0 0x7f356c19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.051+0000 7f356aa00640 1 -- 172.21.15.42:0/3342313521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c105b70 msgr2=0x7f356c1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:13.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.051+0000 7f356aa00640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c105b70 0x7f356c1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.051+0000 7f356aa00640 1 -- 172.21.15.42:0/3342313521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356c1a1ed0 con 0x7f356c1051b0 2024-04-20T10:33:13.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.052+0000 7f356aa00640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f356c1051b0 0x7f356c19a7f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3558009230 tx=0x7f355802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:13.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.052+0000 7f3563e00640 1 -- 172.21.15.42:0/3342313521 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558041070 con 0x7f356c1051b0 2024-04-20T10:33:13.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.052+0000 7f3570c00640 1 -- 172.21.15.42:0/3342313521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356c1a2160 con 0x7f356c1051b0 2024-04-20T10:33:13.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.052+0000 7f3563e00640 1 -- 172.21.15.42:0/3342313521 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3558004450 con 0x7f356c1051b0 2024-04-20T10:33:13.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.052+0000 7f3563e00640 1 -- 172.21.15.42:0/3342313521 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558003e20 con 0x7f356c1051b0 2024-04-20T10:33:13.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.052+0000 7f3570c00640 1 -- 172.21.15.42:0/3342313521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356c1a26a0 con 0x7f356c1051b0 2024-04-20T10:33:13.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.053+0000 7f3570c00640 1 -- 172.21.15.42:0/3342313521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f356c10b330 con 0x7f356c1051b0 2024-04-20T10:33:13.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.056+0000 7f3563e00640 1 -- 172.21.15.42:0/3342313521 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3558044070 con 0x7f356c1051b0 2024-04-20T10:33:13.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.057+0000 7f3563e00640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f35500a3c30 0x7f35500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:13.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.058+0000 7f3563e00640 1 -- 172.21.15.42:0/3342313521 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f35580f2330 con 0x7f356c1051b0 2024-04-20T10:33:13.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.058+0000 7f356b400640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f35500a3c30 0x7f35500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:13.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.059+0000 7f356b400640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f35500a3c30 0x7f35500a60f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f355c0095f0 tx=0x7f355c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:13.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.059+0000 7f3563e00640 1 -- 172.21.15.42:0/3342313521 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f35580be640 con 0x7f356c1051b0 2024-04-20T10:33:13.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.266+0000 7f3570c00640 1 -- 172.21.15.42:0/3342313521 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f356c02d5d0 con 0x7f356c1051b0 2024-04-20T10:33:13.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.267+0000 7f3563e00640 1 -- 172.21.15.42:0/3342313521 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f35580ba020 con 0x7f356c1051b0 2024-04-20T10:33:13.268 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-20T10:33:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.270+0000 7f3570c00640 1 -- 172.21.15.42:0/3342313521 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f35500a3c30 msgr2=0x7f35500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.270+0000 7f3570c00640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f35500a3c30 0x7f35500a60f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f355c0095f0 tx=0x7f355c005f90 comp rx=0 tx=0).stop 2024-04-20T10:33:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.270+0000 7f3570c00640 1 -- 172.21.15.42:0/3342313521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f356c1051b0 msgr2=0x7f356c19a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.270+0000 7f3570c00640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f356c1051b0 0x7f356c19a7f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3558009230 tx=0x7f355802c650 comp rx=0 tx=0).stop 2024-04-20T10:33:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.270+0000 7f3570c00640 1 -- 172.21.15.42:0/3342313521 shutdown_connections 2024-04-20T10:33:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.270+0000 7f3570c00640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f356c105b70 0x7f356c1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.270+0000 7f3570c00640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f356c1051b0 0x7f356c19a7f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.270+0000 7f3570c00640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f35500a3c30 0x7f35500a60f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.270+0000 7f3570c00640 1 --2- 172.21.15.42:0/3342313521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c103fc0 0x7f356c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.270+0000 7f3570c00640 1 -- 172.21.15.42:0/3342313521 >> 172.21.15.42:0/3342313521 conn(0x7f356c0ff530 msgr2=0x7f356c108500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.270+0000 7f3570c00640 1 -- 172.21.15.42:0/3342313521 shutdown_connections 2024-04-20T10:33:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.270+0000 7f3570c00640 1 -- 172.21.15.42:0/3342313521 wait complete. 2024-04-20T10:33:13.289 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:33:13.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.498+0000 7fe8f7e00640 1 -- 172.21.15.42:0/648106198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8f8105200 msgr2=0x7fe8f81075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:13.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.498+0000 7fe8f7e00640 1 --2- 172.21.15.42:0/648106198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8f8105200 0x7fe8f81075f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009900 tx=0x7fe8e002bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:13.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.498+0000 7fe8f7e00640 1 -- 172.21.15.42:0/648106198 shutdown_connections 2024-04-20T10:33:13.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.498+0000 7fe8f7e00640 1 --2- 172.21.15.42:0/648106198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8f8107b30 0x7fe8f8109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.498+0000 7fe8f7e00640 1 --2- 172.21.15.42:0/648106198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8f8105200 0x7fe8f81075f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.498+0000 7fe8f7e00640 1 --2- 172.21.15.42:0/648106198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f81028d0 0x7fe8f8104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.498+0000 7fe8f7e00640 1 -- 172.21.15.42:0/648106198 >> 172.21.15.42:0/648106198 conn(0x7fe8f80fc300 msgr2=0x7fe8f80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:13.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.498+0000 7fe8f7e00640 1 -- 172.21.15.42:0/648106198 shutdown_connections 2024-04-20T10:33:13.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.498+0000 7fe8f7e00640 1 -- 172.21.15.42:0/648106198 wait complete. 2024-04-20T10:33:13.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.499+0000 7fe8f7e00640 1 Processor -- start 2024-04-20T10:33:13.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.499+0000 7fe8f7e00640 1 -- start start 2024-04-20T10:33:13.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.500+0000 7fe8f7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8f81028d0 0x7fe8f819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:13.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.500+0000 7fe8f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f8105200 0x7fe8f819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:13.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.500+0000 7fe8f7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8f8107b30 0x7fe8f81a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.500+0000 7fe8f7e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f810dab0 con 0x7fe8f8107b30 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.500+0000 7fe8f7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f810d930 con 0x7fe8f8105200 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.500+0000 7fe8f7e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f810dc30 con 0x7fe8f81028d0 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.500+0000 7fe8f7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8f8107b30 0x7fe8f81a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.500+0000 7fe8efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f8105200 0x7fe8f819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.500+0000 7fe8efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f8105200 0x7fe8f819a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46568/0 (socket says 172.21.15.42:46568) 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.500+0000 7fe8efe00640 1 -- 172.21.15.42:0/4247536884 learned_addr learned my addr 172.21.15.42:0/4247536884 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.500+0000 7fe8f6a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8f81028d0 0x7fe8f819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.501+0000 7fe8efe00640 1 -- 172.21.15.42:0/4247536884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8f81028d0 msgr2=0x7fe8f819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.501+0000 7fe8efe00640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8f81028d0 0x7fe8f819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.501+0000 7fe8efe00640 1 -- 172.21.15.42:0/4247536884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8f8107b30 msgr2=0x7fe8f81a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.501+0000 7fe8efe00640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8f8107b30 0x7fe8f81a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.501+0000 7fe8efe00640 1 -- 172.21.15.42:0/4247536884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f81a1f60 con 0x7fe8f8105200 2024-04-20T10:33:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.501+0000 7fe8f6a00640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8f81028d0 0x7fe8f819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.501+0000 7fe8efe00640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f8105200 0x7fe8f819a880 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0000c00 tx=0x7fe8e002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.501+0000 7fe8ef400640 1 -- 172.21.15.42:0/4247536884 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0043020 con 0x7fe8f8105200 2024-04-20T10:33:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.501+0000 7fe8f7e00640 1 -- 172.21.15.42:0/4247536884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f81a21f0 con 0x7fe8f8105200 2024-04-20T10:33:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.501+0000 7fe8ef400640 1 -- 172.21.15.42:0/4247536884 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e00054e0 con 0x7fe8f8105200 2024-04-20T10:33:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.501+0000 7fe8ef400640 1 -- 172.21.15.42:0/4247536884 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0003c40 con 0x7fe8f8105200 2024-04-20T10:33:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.502+0000 7fe8f7e00640 1 -- 172.21.15.42:0/4247536884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f81a2790 con 0x7fe8f8105200 2024-04-20T10:33:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.503+0000 7fe8f7e00640 1 -- 172.21.15.42:0/4247536884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f8109680 con 0x7fe8f8105200 2024-04-20T10:33:13.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.505+0000 7fe8ef400640 1 -- 172.21.15.42:0/4247536884 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe8e0046020 con 0x7fe8f8105200 2024-04-20T10:33:13.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.506+0000 7fe8ef400640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe8dc0a3bb0 0x7fe8dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.506+0000 7fe8ef400640 1 -- 172.21.15.42:0/4247536884 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe8e00f2b30 con 0x7fe8f8105200 2024-04-20T10:33:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.506+0000 7fe8f6a00640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe8dc0a3bb0 0x7fe8dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.507+0000 7fe8f6a00640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe8dc0a3bb0 0x7fe8dc0a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009fc0 tx=0x7fe8e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.509+0000 7fe8ef400640 1 -- 172.21.15.42:0/4247536884 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe8e004d020 con 0x7fe8f8105200 2024-04-20T10:33:13.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:13 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1600513804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:33:13.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:13 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1189301672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:33:13.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:13 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1600513804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:33:13.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:13 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1189301672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:33:13.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:13 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1600513804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:33:13.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:13 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1189301672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:33:13.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.757+0000 7fe8f7e00640 1 -- 172.21.15.42:0/4247536884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8f802d5d0 con 0x7fe8f8105200 2024-04-20T10:33:13.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.759+0000 7fe8ef400640 1 -- 172.21.15.42:0/4247536884 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7fe8e00bf490 con 0x7fe8f8105200 2024-04-20T10:33:13.760 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:13.760 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:33:13.760 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:33:13.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.761+0000 7fe8f7e00640 1 -- 172.21.15.42:0/4247536884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe8dc0a3bb0 msgr2=0x7fe8dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:13.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.761+0000 7fe8f7e00640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe8dc0a3bb0 0x7fe8dc0a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009fc0 tx=0x7fe8e8005f90 comp rx=0 tx=0).stop 2024-04-20T10:33:13.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.762+0000 7fe8f7e00640 1 -- 172.21.15.42:0/4247536884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f8105200 msgr2=0x7fe8f819a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.762+0000 7fe8f7e00640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f8105200 0x7fe8f819a880 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0000c00 tx=0x7fe8e002c650 comp rx=0 tx=0).stop 2024-04-20T10:33:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.762+0000 7fe8f7e00640 1 -- 172.21.15.42:0/4247536884 shutdown_connections 2024-04-20T10:33:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.762+0000 7fe8f7e00640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8f8107b30 0x7fe8f81a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.762+0000 7fe8f7e00640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8f8105200 0x7fe8f819a880 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.762+0000 7fe8f7e00640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe8dc0a3bb0 0x7fe8dc0a6070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.762+0000 7fe8f7e00640 1 --2- 172.21.15.42:0/4247536884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8f81028d0 0x7fe8f819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.762+0000 7fe8f7e00640 1 -- 172.21.15.42:0/4247536884 >> 172.21.15.42:0/4247536884 conn(0x7fe8f80fc300 msgr2=0x7fe8f8102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.762+0000 7fe8f7e00640 1 -- 172.21.15.42:0/4247536884 shutdown_connections 2024-04-20T10:33:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:13.762+0000 7fe8f7e00640 1 -- 172.21.15.42:0/4247536884 wait complete. 2024-04-20T10:33:13.783 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:33:14.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.001+0000 7f3919e00640 1 -- 172.21.15.42:0/1310706818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39141051d0 msgr2=0x7f3914105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.001+0000 7f3919e00640 1 --2- 172.21.15.42:0/1310706818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39141051d0 0x7f3914105650 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009900 tx=0x7f38fc02bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:14.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.001+0000 7f3919e00640 1 -- 172.21.15.42:0/1310706818 shutdown_connections 2024-04-20T10:33:14.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.001+0000 7f3919e00640 1 --2- 172.21.15.42:0/1310706818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3914105b90 0x7f391410c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.001+0000 7f3919e00640 1 --2- 172.21.15.42:0/1310706818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39141051d0 0x7f3914105650 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.001+0000 7f3919e00640 1 --2- 172.21.15.42:0/1310706818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3914103fe0 0x7f3914104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.001+0000 7f3919e00640 1 -- 172.21.15.42:0/1310706818 >> 172.21.15.42:0/1310706818 conn(0x7f39140ff570 msgr2=0x7f3914101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:14.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.001+0000 7f3919e00640 1 -- 172.21.15.42:0/1310706818 shutdown_connections 2024-04-20T10:33:14.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.001+0000 7f3919e00640 1 -- 172.21.15.42:0/1310706818 wait complete. 2024-04-20T10:33:14.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.002+0000 7f3919e00640 1 Processor -- start 2024-04-20T10:33:14.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.002+0000 7f3919e00640 1 -- start start 2024-04-20T10:33:14.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.002+0000 7f3919e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3914103fe0 0x7f391419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:14.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.002+0000 7f3919e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39141051d0 0x7f391419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.002+0000 7f3919e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3914105b90 0x7f39141a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.002+0000 7f3919e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f391410fc70 con 0x7f3914103fe0 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.002+0000 7f3919e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f391410faf0 con 0x7f39141051d0 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.002+0000 7f3919e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f391410fdf0 con 0x7f3914105b90 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.003+0000 7f3918a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3914103fe0 0x7f391419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.003+0000 7f390be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39141051d0 0x7f391419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.003+0000 7f390be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39141051d0 0x7f391419a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46588/0 (socket says 172.21.15.42:46588) 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.003+0000 7f390be00640 1 -- 172.21.15.42:0/2114165113 learned_addr learned my addr 172.21.15.42:0/2114165113 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.003+0000 7f3919400640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3914105b90 0x7f39141a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.003+0000 7f390be00640 1 -- 172.21.15.42:0/2114165113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3914105b90 msgr2=0x7f39141a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.003+0000 7f390be00640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3914105b90 0x7f39141a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.003+0000 7f390be00640 1 -- 172.21.15.42:0/2114165113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3914103fe0 msgr2=0x7f391419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.003+0000 7f390be00640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3914103fe0 0x7f391419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.003+0000 7f390be00640 1 -- 172.21.15.42:0/2114165113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39141a1ea0 con 0x7f39141051d0 2024-04-20T10:33:14.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.003+0000 7f3918a00640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3914103fe0 0x7f391419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:14.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.004+0000 7f390be00640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39141051d0 0x7f391419a7c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009230 tx=0x7f38fc02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:14.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.004+0000 7f3912000640 1 -- 172.21.15.42:0/2114165113 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc041070 con 0x7f39141051d0 2024-04-20T10:33:14.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.004+0000 7f3919e00640 1 -- 172.21.15.42:0/2114165113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39141a2130 con 0x7f39141051d0 2024-04-20T10:33:14.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.004+0000 7f3912000640 1 -- 172.21.15.42:0/2114165113 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38fc004450 con 0x7f39141051d0 2024-04-20T10:33:14.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.004+0000 7f3912000640 1 -- 172.21.15.42:0/2114165113 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc003e20 con 0x7f39141051d0 2024-04-20T10:33:14.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.004+0000 7f3919e00640 1 -- 172.21.15.42:0/2114165113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39141a2620 con 0x7f39141051d0 2024-04-20T10:33:14.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.005+0000 7f3919e00640 1 -- 172.21.15.42:0/2114165113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391410b300 con 0x7f39141051d0 2024-04-20T10:33:14.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.007+0000 7f3912000640 1 -- 172.21.15.42:0/2114165113 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f38fc044070 con 0x7f39141051d0 2024-04-20T10:33:14.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.009+0000 7f3912000640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f38f80a3bb0 0x7f38f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:14.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.009+0000 7f3912000640 1 -- 172.21.15.42:0/2114165113 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f38fc0f1f70 con 0x7f39141051d0 2024-04-20T10:33:14.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.009+0000 7f3918a00640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f38f80a3bb0 0x7f38f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:14.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.009+0000 7f3918a00640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f38f80a3bb0 0x7f38f80a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3914105030 tx=0x7f3904005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:14.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.013+0000 7f3912000640 1 -- 172.21.15.42:0/2114165113 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f38fc0be2e0 con 0x7f39141051d0 2024-04-20T10:33:14.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.212+0000 7f3919e00640 1 -- 172.21.15.42:0/2114165113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3914062b60 con 0x7f39141051d0 2024-04-20T10:33:14.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.218+0000 7f3912000640 1 -- 172.21.15.42:0/2114165113 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7f38fc0c25e0 con 0x7f39141051d0 2024-04-20T10:33:14.219 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:14.219 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:32:31.928088+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:33:14.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.221+0000 7f3919e00640 1 -- 172.21.15.42:0/2114165113 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f38f80a3bb0 msgr2=0x7f38f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.221+0000 7f3919e00640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f38f80a3bb0 0x7f38f80a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3914105030 tx=0x7f3904005f90 comp rx=0 tx=0).stop 2024-04-20T10:33:14.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.221+0000 7f3919e00640 1 -- 172.21.15.42:0/2114165113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39141051d0 msgr2=0x7f391419a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.221+0000 7f3919e00640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39141051d0 0x7f391419a7c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009230 tx=0x7f38fc02c650 comp rx=0 tx=0).stop 2024-04-20T10:33:14.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.221+0000 7f3919e00640 1 -- 172.21.15.42:0/2114165113 shutdown_connections 2024-04-20T10:33:14.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.221+0000 7f3919e00640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f38f80a3bb0 0x7f38f80a6070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.221+0000 7f3919e00640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3914105b90 0x7f39141a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.221+0000 7f3919e00640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39141051d0 0x7f391419a7c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.221+0000 7f3919e00640 1 --2- 172.21.15.42:0/2114165113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3914103fe0 0x7f391419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.221+0000 7f3919e00640 1 -- 172.21.15.42:0/2114165113 >> 172.21.15.42:0/2114165113 conn(0x7f39140ff570 msgr2=0x7f39141084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:14.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.221+0000 7f3919e00640 1 -- 172.21.15.42:0/2114165113 shutdown_connections 2024-04-20T10:33:14.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.221+0000 7f3919e00640 1 -- 172.21.15.42:0/2114165113 wait complete. 2024-04-20T10:33:14.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:26:16.537072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:31:33.523692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:31:34.019519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:33:14.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:33:14.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.454+0000 7f585f600640 1 -- 172.21.15.42:0/3229620824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58581041a0 msgr2=0x7f58581045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.454+0000 7f585f600640 1 --2- 172.21.15.42:0/3229620824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58581041a0 0x7f58581045c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f584c009900 tx=0x7f584c02bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:14.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.455+0000 7f585f600640 1 -- 172.21.15.42:0/3229620824 shutdown_connections 2024-04-20T10:33:14.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.455+0000 7f585f600640 1 --2- 172.21.15.42:0/3229620824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5858105d50 0x7f585810c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.455+0000 7f585f600640 1 --2- 172.21.15.42:0/3229620824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5858105390 0x7f5858105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.455+0000 7f585f600640 1 --2- 172.21.15.42:0/3229620824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58581041a0 0x7f58581045c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.455+0000 7f585f600640 1 -- 172.21.15.42:0/3229620824 >> 172.21.15.42:0/3229620824 conn(0x7f58580ff750 msgr2=0x7f5858101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:14.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.455+0000 7f585f600640 1 -- 172.21.15.42:0/3229620824 shutdown_connections 2024-04-20T10:33:14.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.455+0000 7f585f600640 1 -- 172.21.15.42:0/3229620824 wait complete. 2024-04-20T10:33:14.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.456+0000 7f585f600640 1 Processor -- start 2024-04-20T10:33:14.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.456+0000 7f585f600640 1 -- start start 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.456+0000 7f585f600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58581041a0 0x7f585819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.456+0000 7f585f600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5858105390 0x7f585819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.456+0000 7f585f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5858105d50 0x7f58581a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.456+0000 7f585f600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f585810fe80 con 0x7f58581041a0 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.456+0000 7f585f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f585810fd00 con 0x7f5858105d50 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.456+0000 7f585f600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5858110000 con 0x7f5858105390 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.457+0000 7f585cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58581041a0 0x7f585819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.457+0000 7f585cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58581041a0 0x7f585819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57212/0 (socket says 172.21.15.42:57212) 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.457+0000 7f585cc00640 1 -- 172.21.15.42:0/3173386420 learned_addr learned my addr 172.21.15.42:0/3173386420 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.457+0000 7f585d600640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5858105d50 0x7f58581a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.457+0000 7f5857e00640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5858105390 0x7f585819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.457+0000 7f585cc00640 1 -- 172.21.15.42:0/3173386420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5858105390 msgr2=0x7f585819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.457+0000 7f585cc00640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5858105390 0x7f585819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.457+0000 7f585cc00640 1 -- 172.21.15.42:0/3173386420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5858105d50 msgr2=0x7f58581a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.457+0000 7f585cc00640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5858105d50 0x7f58581a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.457+0000 7f585cc00640 1 -- 172.21.15.42:0/3173386420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58581a2130 con 0x7f58581041a0 2024-04-20T10:33:14.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.458+0000 7f585cc00640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58581041a0 0x7f585819a510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f584c000c00 tx=0x7f584c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:14.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.458+0000 7f5855600640 1 -- 172.21.15.42:0/3173386420 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c043020 con 0x7f58581041a0 2024-04-20T10:33:14.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.458+0000 7f585f600640 1 -- 172.21.15.42:0/3173386420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58581a23c0 con 0x7f58581041a0 2024-04-20T10:33:14.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.458+0000 7f5855600640 1 -- 172.21.15.42:0/3173386420 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f584c0054e0 con 0x7f58581041a0 2024-04-20T10:33:14.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.458+0000 7f5855600640 1 -- 172.21.15.42:0/3173386420 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c003c40 con 0x7f58581041a0 2024-04-20T10:33:14.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.458+0000 7f585f600640 1 -- 172.21.15.42:0/3173386420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58581a2960 con 0x7f58581041a0 2024-04-20T10:33:14.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.459+0000 7f585f600640 1 -- 172.21.15.42:0/3173386420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585810b460 con 0x7f58581041a0 2024-04-20T10:33:14.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.461+0000 7f5855600640 1 -- 172.21.15.42:0/3173386420 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f584c046020 con 0x7f58581041a0 2024-04-20T10:33:14.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.462+0000 7f5855600640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f58400a3bb0 0x7f58400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:14.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.463+0000 7f5855600640 1 -- 172.21.15.42:0/3173386420 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f584c0f2410 con 0x7f58581041a0 2024-04-20T10:33:14.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.463+0000 7f5857e00640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f58400a3bb0 0x7f58400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:14.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.464+0000 7f5857e00640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f58400a3bb0 0x7f58400a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f58440095f0 tx=0x7f5844005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:14.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.468+0000 7f5855600640 1 -- 172.21.15.42:0/3173386420 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f584c04d020 con 0x7f58581041a0 2024-04-20T10:33:14.515 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:14 smithi042 ceph-mon[17078]: pgmap v368: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:14.515 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:14 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3342313521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:33:14.516 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:14 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4247536884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:14.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:14 smithi046 ceph-mon[21574]: pgmap v368: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:14.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:14 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3342313521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:33:14.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:14 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/4247536884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:14.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:14 smithi162 ceph-mon[21854]: pgmap v368: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-20T10:33:14.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:14 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3342313521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:33:14.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:14 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/4247536884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:14.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.674+0000 7f585f600640 1 -- 172.21.15.42:0/3173386420 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5858062b60 con 0x7f58581041a0 2024-04-20T10:33:14.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.675+0000 7f5855600640 1 -- 172.21.15.42:0/3173386420 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f584c0bedb0 con 0x7f58581041a0 2024-04-20T10:33:14.676 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-20T10:33:14.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.678+0000 7f585f600640 1 -- 172.21.15.42:0/3173386420 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f58400a3bb0 msgr2=0x7f58400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.678+0000 7f585f600640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f58400a3bb0 0x7f58400a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f58440095f0 tx=0x7f5844005f90 comp rx=0 tx=0).stop 2024-04-20T10:33:14.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.678+0000 7f585f600640 1 -- 172.21.15.42:0/3173386420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58581041a0 msgr2=0x7f585819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.678+0000 7f585f600640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58581041a0 0x7f585819a510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f584c000c00 tx=0x7f584c02c650 comp rx=0 tx=0).stop 2024-04-20T10:33:14.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.679+0000 7f585f600640 1 -- 172.21.15.42:0/3173386420 shutdown_connections 2024-04-20T10:33:14.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.679+0000 7f585f600640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5858105d50 0x7f58581a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.679+0000 7f585f600640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f58400a3bb0 0x7f58400a6070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.679+0000 7f585f600640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5858105390 0x7f585819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.679+0000 7f585f600640 1 --2- 172.21.15.42:0/3173386420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58581041a0 0x7f585819a510 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.679+0000 7f585f600640 1 -- 172.21.15.42:0/3173386420 >> 172.21.15.42:0/3173386420 conn(0x7f58580ff750 msgr2=0x7f5858108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:14.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.679+0000 7f585f600640 1 -- 172.21.15.42:0/3173386420 shutdown_connections 2024-04-20T10:33:14.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.679+0000 7f585f600640 1 -- 172.21.15.42:0/3173386420 wait complete. 2024-04-20T10:33:14.699 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:33:14.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.917+0000 7fd9a1c00640 1 -- 172.21.15.42:0/475205893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd99c105290 msgr2=0x7fd99c105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.917+0000 7fd9a1c00640 1 --2- 172.21.15.42:0/475205893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd99c105290 0x7fd99c105710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd984009900 tx=0x7fd98402bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:14.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.917+0000 7fd9a1c00640 1 -- 172.21.15.42:0/475205893 shutdown_connections 2024-04-20T10:33:14.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.917+0000 7fd9a1c00640 1 --2- 172.21.15.42:0/475205893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd99c105c50 0x7fd99c10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.917+0000 7fd9a1c00640 1 --2- 172.21.15.42:0/475205893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd99c105290 0x7fd99c105710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.917+0000 7fd9a1c00640 1 --2- 172.21.15.42:0/475205893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd99c1040a0 0x7fd99c1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.917+0000 7fd9a1c00640 1 -- 172.21.15.42:0/475205893 >> 172.21.15.42:0/475205893 conn(0x7fd99c0ff610 msgr2=0x7fd99c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:14.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.917+0000 7fd9a1c00640 1 -- 172.21.15.42:0/475205893 shutdown_connections 2024-04-20T10:33:14.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.917+0000 7fd9a1c00640 1 -- 172.21.15.42:0/475205893 wait complete. 2024-04-20T10:33:14.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.918+0000 7fd9a1c00640 1 Processor -- start 2024-04-20T10:33:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.918+0000 7fd9a1c00640 1 -- start start 2024-04-20T10:33:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.918+0000 7fd9a1c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd99c1040a0 0x7fd99c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd9a1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd99c105290 0x7fd99c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd9a1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd99c105c50 0x7fd99c1a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd9a1c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c10fde0 con 0x7fd99c105290 2024-04-20T10:33:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd9a1c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c10fc60 con 0x7fd99c105c50 2024-04-20T10:33:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd9a1c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c10ff60 con 0x7fd99c1040a0 2024-04-20T10:33:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd99b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd99c105290 0x7fd99c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd99be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd99c1040a0 0x7fd99c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd99b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd99c105290 0x7fd99c19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57236/0 (socket says 172.21.15.42:57236) 2024-04-20T10:33:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd99b400640 1 -- 172.21.15.42:0/2959302994 learned_addr learned my addr 172.21.15.42:0/2959302994 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd9a1200640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd99c105c50 0x7fd99c1a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd99be00640 1 -- 172.21.15.42:0/2959302994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd99c105c50 msgr2=0x7fd99c1a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd99be00640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd99c105c50 0x7fd99c1a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd99be00640 1 -- 172.21.15.42:0/2959302994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd99c105290 msgr2=0x7fd99c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd99be00640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd99c105290 0x7fd99c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd99be00640 1 -- 172.21.15.42:0/2959302994 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c1a2010 con 0x7fd99c1040a0 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.919+0000 7fd9a1200640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd99c105c50 0x7fd99c1a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.920+0000 7fd99be00640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd99c1040a0 0x7fd99c19a3f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00e9c0 tx=0x7fd98c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.920+0000 7fd998c00640 1 -- 172.21.15.42:0/2959302994 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c00ccc0 con 0x7fd99c1040a0 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.920+0000 7fd998c00640 1 -- 172.21.15.42:0/2959302994 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd98c0035f0 con 0x7fd99c1040a0 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.920+0000 7fd9a1c00640 1 -- 172.21.15.42:0/2959302994 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99c1a2300 con 0x7fd99c1040a0 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.920+0000 7fd998c00640 1 -- 172.21.15.42:0/2959302994 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c00ce60 con 0x7fd99c1040a0 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.920+0000 7fd9a1c00640 1 -- 172.21.15.42:0/2959302994 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99c1a27d0 con 0x7fd99c1040a0 2024-04-20T10:33:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.921+0000 7fd9a1c00640 1 -- 172.21.15.42:0/2959302994 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99c10b360 con 0x7fd99c1040a0 2024-04-20T10:33:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.923+0000 7fd998c00640 1 -- 172.21.15.42:0/2959302994 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd98c0037a0 con 0x7fd99c1040a0 2024-04-20T10:33:14.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.925+0000 7fd998c00640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd9800a3bb0 0x7fd9800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:14.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.925+0000 7fd998c00640 1 -- 172.21.15.42:0/2959302994 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd98c0d09f0 con 0x7fd99c1040a0 2024-04-20T10:33:14.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.925+0000 7fd99b400640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd9800a3bb0 0x7fd9800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:14.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.926+0000 7fd99b400640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd9800a3bb0 0x7fd9800a6070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd98402c8e0 tx=0x7fd98402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:14.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:14.929+0000 7fd998c00640 1 -- 172.21.15.42:0/2959302994 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd98c09d350 con 0x7fd99c1040a0 2024-04-20T10:33:15.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.135+0000 7fd9a1c00640 1 -- 172.21.15.42:0/2959302994 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd99c062b60 con 0x7fd99c1040a0 2024-04-20T10:33:15.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.136+0000 7fd998c00640 1 -- 172.21.15.42:0/2959302994 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fd98c09cc50 con 0x7fd99c1040a0 2024-04-20T10:33:15.136 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-20T10:33:15.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.138+0000 7fd9a1c00640 1 -- 172.21.15.42:0/2959302994 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd9800a3bb0 msgr2=0x7fd9800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:15.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.138+0000 7fd9a1c00640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd9800a3bb0 0x7fd9800a6070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd98402c8e0 tx=0x7fd98402df90 comp rx=0 tx=0).stop 2024-04-20T10:33:15.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.138+0000 7fd9a1c00640 1 -- 172.21.15.42:0/2959302994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd99c1040a0 msgr2=0x7fd99c19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:15.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.138+0000 7fd9a1c00640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd99c1040a0 0x7fd99c19a3f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00e9c0 tx=0x7fd98c00ee80 comp rx=0 tx=0).stop 2024-04-20T10:33:15.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.139+0000 7fd9a1c00640 1 -- 172.21.15.42:0/2959302994 shutdown_connections 2024-04-20T10:33:15.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.139+0000 7fd9a1c00640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd99c105c50 0x7fd99c1a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.139+0000 7fd9a1c00640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd99c105290 0x7fd99c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.139+0000 7fd9a1c00640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd9800a3bb0 0x7fd9800a6070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.139+0000 7fd9a1c00640 1 --2- 172.21.15.42:0/2959302994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd99c1040a0 0x7fd99c19a3f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.139+0000 7fd9a1c00640 1 -- 172.21.15.42:0/2959302994 >> 172.21.15.42:0/2959302994 conn(0x7fd99c0ff610 msgr2=0x7fd99c108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:15.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.139+0000 7fd9a1c00640 1 -- 172.21.15.42:0/2959302994 shutdown_connections 2024-04-20T10:33:15.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.139+0000 7fd9a1c00640 1 -- 172.21.15.42:0/2959302994 wait complete. 2024-04-20T10:33:15.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:33:15.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.375+0000 7f63caa00640 1 -- 172.21.15.42:0/761836337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c4075f40 msgr2=0x7f63c4074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:15.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.375+0000 7f63caa00640 1 --2- 172.21.15.42:0/761836337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c4075f40 0x7f63c4074390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009900 tx=0x7f63b802bcd0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.376+0000 7f63caa00640 1 -- 172.21.15.42:0/761836337 shutdown_connections 2024-04-20T10:33:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.376+0000 7f63caa00640 1 --2- 172.21.15.42:0/761836337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c4109c10 0x7f63c410c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.376+0000 7f63caa00640 1 --2- 172.21.15.42:0/761836337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c4074960 0x7f63c4074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.376+0000 7f63caa00640 1 --2- 172.21.15.42:0/761836337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c4075f40 0x7f63c4074390 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.376+0000 7f63caa00640 1 -- 172.21.15.42:0/761836337 >> 172.21.15.42:0/761836337 conn(0x7f63c40fd2f0 msgr2=0x7f63c40ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.376+0000 7f63caa00640 1 -- 172.21.15.42:0/761836337 shutdown_connections 2024-04-20T10:33:15.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.376+0000 7f63caa00640 1 -- 172.21.15.42:0/761836337 wait complete. 2024-04-20T10:33:15.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.377+0000 7f63caa00640 1 Processor -- start 2024-04-20T10:33:15.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.377+0000 7f63caa00640 1 -- start start 2024-04-20T10:33:15.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.377+0000 7f63caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c4074960 0x7f63c419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:15.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.377+0000 7f63caa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c4075f40 0x7f63c419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:15.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.377+0000 7f63caa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c4109c10 0x7f63c41a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:15.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.377+0000 7f63caa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f63c410fc50 con 0x7f63c4109c10 2024-04-20T10:33:15.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.377+0000 7f63caa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63c410fad0 con 0x7f63c4074960 2024-04-20T10:33:15.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.377+0000 7f63caa00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f63c410fdd0 con 0x7f63c4075f40 2024-04-20T10:33:15.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.378+0000 7f63c9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c4074960 0x7f63c419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:15.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.378+0000 7f63c8c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c4075f40 0x7f63c419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:15.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.378+0000 7f63c9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c4074960 0x7f63c419a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46644/0 (socket says 172.21.15.42:46644) 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.378+0000 7f63c9600640 1 -- 172.21.15.42:0/1217860258 learned_addr learned my addr 172.21.15.42:0/1217860258 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.378+0000 7f63ca000640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c4109c10 0x7f63c41a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.378+0000 7f63c9600640 1 -- 172.21.15.42:0/1217860258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c4075f40 msgr2=0x7f63c419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.378+0000 7f63c9600640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c4075f40 0x7f63c419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.378+0000 7f63c9600640 1 -- 172.21.15.42:0/1217860258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c4109c10 msgr2=0x7f63c41a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.378+0000 7f63c9600640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c4109c10 0x7f63c41a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.378+0000 7f63c9600640 1 -- 172.21.15.42:0/1217860258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c41a1ec0 con 0x7f63c4074960 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.378+0000 7f63c8c00640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c4075f40 0x7f63c419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.378+0000 7f63c9600640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c4074960 0x7f63c419a2a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f63b8000c00 tx=0x7f63b802c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.379+0000 7f63c2000640 1 -- 172.21.15.42:0/1217860258 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8043020 con 0x7f63c4074960 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.379+0000 7f63caa00640 1 -- 172.21.15.42:0/1217860258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c41a2150 con 0x7f63c4074960 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.379+0000 7f63c2000640 1 -- 172.21.15.42:0/1217860258 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b80054e0 con 0x7f63c4074960 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.379+0000 7f63c2000640 1 -- 172.21.15.42:0/1217860258 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8003c20 con 0x7f63c4074960 2024-04-20T10:33:15.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.379+0000 7f63caa00640 1 -- 172.21.15.42:0/1217860258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c41a26f0 con 0x7f63c4074960 2024-04-20T10:33:15.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.380+0000 7f63caa00640 1 -- 172.21.15.42:0/1217860258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c41089d0 con 0x7f63c4074960 2024-04-20T10:33:15.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.382+0000 7f63c2000640 1 -- 172.21.15.42:0/1217860258 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f63b8046020 con 0x7f63c4074960 2024-04-20T10:33:15.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.383+0000 7f63c2000640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f63ac0a3bb0 0x7f63ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:15.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.384+0000 7f63c2000640 1 -- 172.21.15.42:0/1217860258 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f63b80f24b0 con 0x7f63c4074960 2024-04-20T10:33:15.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.384+0000 7f63c8c00640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f63ac0a3bb0 0x7f63ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.384+0000 7f63c8c00640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f63ac0a3bb0 0x7f63ac0a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f63c419b7b0 tx=0x7f63b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:15.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.388+0000 7f63c2000640 1 -- 172.21.15.42:0/1217860258 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f63b804d020 con 0x7f63c4074960 2024-04-20T10:33:15.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:15 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2114165113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:15.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:15 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3173386420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:33:15.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:15 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2959302994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:33:15.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.597+0000 7f63caa00640 1 -- 172.21.15.42:0/1217860258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f63c4062b60 con 0x7f63c4074960 2024-04-20T10:33:15.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.598+0000 7f63c2000640 1 -- 172.21.15.42:0/1217860258 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f63b80bee10 con 0x7f63c4074960 2024-04-20T10:33:15.598 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-20T10:33:15.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.601+0000 7f63caa00640 1 -- 172.21.15.42:0/1217860258 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f63ac0a3bb0 msgr2=0x7f63ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:15.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.601+0000 7f63caa00640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f63ac0a3bb0 0x7f63ac0a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f63c419b7b0 tx=0x7f63b0005f90 comp rx=0 tx=0).stop 2024-04-20T10:33:15.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.601+0000 7f63caa00640 1 -- 172.21.15.42:0/1217860258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c4074960 msgr2=0x7f63c419a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:15.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.601+0000 7f63caa00640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c4074960 0x7f63c419a2a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f63b8000c00 tx=0x7f63b802c6f0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.601+0000 7f63caa00640 1 -- 172.21.15.42:0/1217860258 shutdown_connections 2024-04-20T10:33:15.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.601+0000 7f63caa00640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c4109c10 0x7f63c41a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.601+0000 7f63caa00640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f63ac0a3bb0 0x7f63ac0a6070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.601+0000 7f63caa00640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c4075f40 0x7f63c419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.601+0000 7f63caa00640 1 --2- 172.21.15.42:0/1217860258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c4074960 0x7f63c419a2a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.601+0000 7f63caa00640 1 -- 172.21.15.42:0/1217860258 >> 172.21.15.42:0/1217860258 conn(0x7f63c40fd2f0 msgr2=0x7f63c4105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:15.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.601+0000 7f63caa00640 1 -- 172.21.15.42:0/1217860258 shutdown_connections 2024-04-20T10:33:15.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.601+0000 7f63caa00640 1 -- 172.21.15.42:0/1217860258 wait complete. 2024-04-20T10:33:15.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:15 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2114165113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:15.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:15 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3173386420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:33:15.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:15 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2959302994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:33:15.622 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:33:15.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:15 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2114165113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:15.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:15 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3173386420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:33:15.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:15 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2959302994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:33:15.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.839+0000 7f3d17200640 1 -- 172.21.15.42:0/1791461516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d10074960 msgr2=0x7f3d10074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:15.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.839+0000 7f3d17200640 1 --2- 172.21.15.42:0/1791461516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d10074960 0x7f3d10074dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009900 tx=0x7f3cf802bcc0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.839+0000 7f3d17200640 1 -- 172.21.15.42:0/1791461516 shutdown_connections 2024-04-20T10:33:15.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.839+0000 7f3d17200640 1 --2- 172.21.15.42:0/1791461516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d10109e40 0x7f3d1010c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.839+0000 7f3d17200640 1 --2- 172.21.15.42:0/1791461516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d10074960 0x7f3d10074dc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.839+0000 7f3d17200640 1 --2- 172.21.15.42:0/1791461516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d10075f40 0x7f3d10074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.839+0000 7f3d17200640 1 -- 172.21.15.42:0/1791461516 >> 172.21.15.42:0/1791461516 conn(0x7f3d100fd540 msgr2=0x7f3d100ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:15.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.839+0000 7f3d17200640 1 -- 172.21.15.42:0/1791461516 shutdown_connections 2024-04-20T10:33:15.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.839+0000 7f3d17200640 1 -- 172.21.15.42:0/1791461516 wait complete. 2024-04-20T10:33:15.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.841+0000 7f3d17200640 1 Processor -- start 2024-04-20T10:33:15.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.841+0000 7f3d17200640 1 -- start start 2024-04-20T10:33:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.841+0000 7f3d17200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d10074960 0x7f3d1019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.841+0000 7f3d17200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d10075f40 0x7f3d1019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.841+0000 7f3d17200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d10109e40 0x7f3d101a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.841+0000 7f3d17200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010fe40 con 0x7f3d10074960 2024-04-20T10:33:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.841+0000 7f3d17200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010fcc0 con 0x7f3d10109e40 2024-04-20T10:33:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.841+0000 7f3d17200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010ffc0 con 0x7f3d10075f40 2024-04-20T10:33:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.842+0000 7f3d0fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d10074960 0x7f3d1019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.842+0000 7f3d0fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d10074960 0x7f3d1019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57280/0 (socket says 172.21.15.42:57280) 2024-04-20T10:33:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.842+0000 7f3d0fe00640 1 -- 172.21.15.42:0/3094801944 learned_addr learned my addr 172.21.15.42:0/3094801944 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.842+0000 7f3d15200640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d10109e40 0x7f3d101a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.842+0000 7f3d07e00640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d10075f40 0x7f3d1019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.842+0000 7f3d0fe00640 1 -- 172.21.15.42:0/3094801944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d10075f40 msgr2=0x7f3d1019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:15.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.842+0000 7f3d0fe00640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d10075f40 0x7f3d1019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.842+0000 7f3d0fe00640 1 -- 172.21.15.42:0/3094801944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d10109e40 msgr2=0x7f3d101a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:15.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.842+0000 7f3d0fe00640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d10109e40 0x7f3d101a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:15.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.842+0000 7f3d0fe00640 1 -- 172.21.15.42:0/3094801944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d101a20b0 con 0x7f3d10074960 2024-04-20T10:33:15.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.842+0000 7f3d15200640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d10109e40 0x7f3d101a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:15.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.843+0000 7f3d0fe00640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d10074960 0x7f3d1019a490 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000e9f0 tx=0x7f3d0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:15.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.843+0000 7f3d0d600640 1 -- 172.21.15.42:0/3094801944 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0000cd80 con 0x7f3d10074960 2024-04-20T10:33:15.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.843+0000 7f3d17200640 1 -- 172.21.15.42:0/3094801944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d101a23a0 con 0x7f3d10074960 2024-04-20T10:33:15.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.843+0000 7f3d0d600640 1 -- 172.21.15.42:0/3094801944 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d00010910 con 0x7f3d10074960 2024-04-20T10:33:15.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.843+0000 7f3d0d600640 1 -- 172.21.15.42:0/3094801944 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000188f0 con 0x7f3d10074960 2024-04-20T10:33:15.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.843+0000 7f3d17200640 1 -- 172.21.15.42:0/3094801944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d101a2910 con 0x7f3d10074960 2024-04-20T10:33:15.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.844+0000 7f3d17200640 1 -- 172.21.15.42:0/3094801944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d10108c00 con 0x7f3d10074960 2024-04-20T10:33:15.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.846+0000 7f3d0d600640 1 -- 172.21.15.42:0/3094801944 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3d00010ac0 con 0x7f3d10074960 2024-04-20T10:33:15.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.848+0000 7f3d0d600640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3cf40a3c30 0x7f3cf40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:15.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.848+0000 7f3d0d600640 1 -- 172.21.15.42:0/3094801944 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3d000d0730 con 0x7f3d10074960 2024-04-20T10:33:15.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.848+0000 7f3d07e00640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3cf40a3c30 0x7f3cf40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:15.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.849+0000 7f3d07e00640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3cf40a3c30 0x7f3cf40a60f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8005ab0 tx=0x7f3cf802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:15.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:15.852+0000 7f3d0d600640 1 -- 172.21.15.42:0/3094801944 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3d0009caa0 con 0x7f3d10074960 2024-04-20T10:33:16.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.102+0000 7f3d17200640 1 -- 172.21.15.42:0/3094801944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d1002d5d0 con 0x7f3d10074960 2024-04-20T10:33:16.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.104+0000 7f3d0d600640 1 -- 172.21.15.42:0/3094801944 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f3d000a0da0 con 0x7f3d10074960 2024-04-20T10:33:16.105 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:16.105 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:33:16.106 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:33:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.107+0000 7f3d17200640 1 -- 172.21.15.42:0/3094801944 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3cf40a3c30 msgr2=0x7f3cf40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.107+0000 7f3d17200640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3cf40a3c30 0x7f3cf40a60f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8005ab0 tx=0x7f3cf802df90 comp rx=0 tx=0).stop 2024-04-20T10:33:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.107+0000 7f3d17200640 1 -- 172.21.15.42:0/3094801944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d10074960 msgr2=0x7f3d1019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.107+0000 7f3d17200640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d10074960 0x7f3d1019a490 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000e9f0 tx=0x7f3d0000eeb0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.107+0000 7f3d17200640 1 -- 172.21.15.42:0/3094801944 shutdown_connections 2024-04-20T10:33:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.107+0000 7f3d17200640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d10109e40 0x7f3d101a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.107+0000 7f3d17200640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3cf40a3c30 0x7f3cf40a60f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.107+0000 7f3d17200640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3d10075f40 0x7f3d1019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.107+0000 7f3d17200640 1 --2- 172.21.15.42:0/3094801944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d10074960 0x7f3d1019a490 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.107+0000 7f3d17200640 1 -- 172.21.15.42:0/3094801944 >> 172.21.15.42:0/3094801944 conn(0x7f3d100fd540 msgr2=0x7f3d10105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.107+0000 7f3d17200640 1 -- 172.21.15.42:0/3094801944 shutdown_connections 2024-04-20T10:33:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.108+0000 7f3d17200640 1 -- 172.21.15.42:0/3094801944 wait complete. 2024-04-20T10:33:16.129 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:33:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.344+0000 7fa737e00640 1 -- 172.21.15.42:0/3638926797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa738105190 msgr2=0x7fa738107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:16.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.344+0000 7fa737e00640 1 --2- 172.21.15.42:0/3638926797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa738105190 0x7fa738107580 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa724009900 tx=0x7fa72402bc40 comp rx=0 tx=0).stop 2024-04-20T10:33:16.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.345+0000 7fa737e00640 1 -- 172.21.15.42:0/3638926797 shutdown_connections 2024-04-20T10:33:16.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.345+0000 7fa737e00640 1 --2- 172.21.15.42:0/3638926797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa738107ac0 0x7fa738109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.345+0000 7fa737e00640 1 --2- 172.21.15.42:0/3638926797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa738105190 0x7fa738107580 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.345+0000 7fa737e00640 1 --2- 172.21.15.42:0/3638926797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa738102860 0x7fa738104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.345+0000 7fa737e00640 1 -- 172.21.15.42:0/3638926797 >> 172.21.15.42:0/3638926797 conn(0x7fa7380fc230 msgr2=0x7fa7380fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:16.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.345+0000 7fa737e00640 1 -- 172.21.15.42:0/3638926797 shutdown_connections 2024-04-20T10:33:16.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.345+0000 7fa737e00640 1 -- 172.21.15.42:0/3638926797 wait complete. 2024-04-20T10:33:16.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.346+0000 7fa737e00640 1 Processor -- start 2024-04-20T10:33:16.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.346+0000 7fa737e00640 1 -- start start 2024-04-20T10:33:16.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.346+0000 7fa737e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa738102860 0x7fa7381021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:16.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.346+0000 7fa737e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa738105190 0x7fa738100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:16.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.346+0000 7fa737e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa738107ac0 0x7fa738100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:16.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.346+0000 7fa737e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa73810daa0 con 0x7fa738102860 2024-04-20T10:33:16.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.346+0000 7fa737e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa73810d920 con 0x7fa738105190 2024-04-20T10:33:16.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.346+0000 7fa737e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa73810dc20 con 0x7fa738107ac0 2024-04-20T10:33:16.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.347+0000 7fa736a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa738102860 0x7fa7381021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:16.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.347+0000 7fa736a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa738102860 0x7fa7381021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57298/0 (socket says 172.21.15.42:57298) 2024-04-20T10:33:16.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.347+0000 7fa736a00640 1 -- 172.21.15.42:0/4128245051 learned_addr learned my addr 172.21.15.42:0/4128245051 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.347+0000 7fa737400640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa738107ac0 0x7fa738100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.347+0000 7fa736000640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa738105190 0x7fa738100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.347+0000 7fa736a00640 1 -- 172.21.15.42:0/4128245051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa738107ac0 msgr2=0x7fa738100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.347+0000 7fa736a00640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa738107ac0 0x7fa738100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.347+0000 7fa736a00640 1 -- 172.21.15.42:0/4128245051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa738105190 msgr2=0x7fa738100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.347+0000 7fa736a00640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa738105190 0x7fa738100810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.347+0000 7fa736a00640 1 -- 172.21.15.42:0/4128245051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa738101290 con 0x7fa738102860 2024-04-20T10:33:16.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.348+0000 7fa736a00640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa738102860 0x7fa7381021d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa72800e9f0 tx=0x7fa72800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.348+0000 7fa72f400640 1 -- 172.21.15.42:0/4128245051 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72800cd80 con 0x7fa738102860 2024-04-20T10:33:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.348+0000 7fa72f400640 1 -- 172.21.15.42:0/4128245051 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa728010910 con 0x7fa738102860 2024-04-20T10:33:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.348+0000 7fa737e00640 1 -- 172.21.15.42:0/4128245051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa738101580 con 0x7fa738102860 2024-04-20T10:33:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.348+0000 7fa72f400640 1 -- 172.21.15.42:0/4128245051 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728018990 con 0x7fa738102860 2024-04-20T10:33:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.348+0000 7fa737e00640 1 -- 172.21.15.42:0/4128245051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7381a6ba0 con 0x7fa738102860 2024-04-20T10:33:16.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.349+0000 7fa737e00640 1 -- 172.21.15.42:0/4128245051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7380fdd20 con 0x7fa738102860 2024-04-20T10:33:16.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.351+0000 7fa72f400640 1 -- 172.21.15.42:0/4128245051 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa728010ac0 con 0x7fa738102860 2024-04-20T10:33:16.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.353+0000 7fa72f400640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa71c0a3bb0 0x7fa71c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:16.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.353+0000 7fa72f400640 1 -- 172.21.15.42:0/4128245051 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fa7280d0790 con 0x7fa738102860 2024-04-20T10:33:16.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.353+0000 7fa736000640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa71c0a3bb0 0x7fa71c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:16.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.354+0000 7fa736000640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa71c0a3bb0 0x7fa71c0a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa72402c180 tx=0x7fa72402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:16.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.357+0000 7fa72f400640 1 -- 172.21.15.42:0/4128245051 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa72809caa0 con 0x7fa738102860 2024-04-20T10:33:16.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:16 smithi046 ceph-mon[21574]: pgmap v369: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 0 B/s wr, 1 op/s 2024-04-20T10:33:16.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:16 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1217860258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:33:16.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:16 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3094801944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:16.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.563+0000 7fa737e00640 1 -- 172.21.15.42:0/4128245051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa73802d5d0 con 0x7fa738102860 2024-04-20T10:33:16.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.567+0000 7fa72f400640 1 -- 172.21.15.42:0/4128245051 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7fa7280a0da0 con 0x7fa738102860 2024-04-20T10:33:16.568 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:16.568 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:32:31.928088+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:33:16.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.570+0000 7fa737e00640 1 -- 172.21.15.42:0/4128245051 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa71c0a3bb0 msgr2=0x7fa71c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:16.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.570+0000 7fa737e00640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa71c0a3bb0 0x7fa71c0a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa72402c180 tx=0x7fa72402cf90 comp rx=0 tx=0).stop 2024-04-20T10:33:16.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.570+0000 7fa737e00640 1 -- 172.21.15.42:0/4128245051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa738102860 msgr2=0x7fa7381021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:16.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.570+0000 7fa737e00640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa738102860 0x7fa7381021d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa72800e9f0 tx=0x7fa72800eeb0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.570+0000 7fa737e00640 1 -- 172.21.15.42:0/4128245051 shutdown_connections 2024-04-20T10:33:16.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.570+0000 7fa737e00640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fa71c0a3bb0 0x7fa71c0a6070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.570+0000 7fa737e00640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa738107ac0 0x7fa738100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.570+0000 7fa737e00640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa738105190 0x7fa738100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.570+0000 7fa737e00640 1 --2- 172.21.15.42:0/4128245051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa738102860 0x7fa7381021d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.570+0000 7fa737e00640 1 -- 172.21.15.42:0/4128245051 >> 172.21.15.42:0/4128245051 conn(0x7fa7380fc230 msgr2=0x7fa738103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:16.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.570+0000 7fa737e00640 1 -- 172.21.15.42:0/4128245051 shutdown_connections 2024-04-20T10:33:16.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.570+0000 7fa737e00640 1 -- 172.21.15.42:0/4128245051 wait complete. 2024-04-20T10:33:16.586 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:16 smithi042 ceph-mon[17078]: pgmap v369: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 0 B/s wr, 1 op/s 2024-04-20T10:33:16.586 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:16 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1217860258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:33:16.586 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:16 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3094801944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:16.590 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:33:16.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:16 smithi162 ceph-mon[21854]: pgmap v369: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 0 B/s wr, 1 op/s 2024-04-20T10:33:16.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:16 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1217860258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:33:16.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:16 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3094801944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.794+0000 7fc045200640 1 -- 172.21.15.42:0/286624879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc040103fe0 msgr2=0x7fc040104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.794+0000 7fc045200640 1 --2- 172.21.15.42:0/286624879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc040103fe0 0x7fc040104400 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc030009900 tx=0x7fc03002bc50 comp rx=0 tx=0).stop 2024-04-20T10:33:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.795+0000 7fc045200640 1 -- 172.21.15.42:0/286624879 shutdown_connections 2024-04-20T10:33:16.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.795+0000 7fc045200640 1 --2- 172.21.15.42:0/286624879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc040105b90 0x7fc04010c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.795+0000 7fc045200640 1 --2- 172.21.15.42:0/286624879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0401051d0 0x7fc040105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.795+0000 7fc045200640 1 --2- 172.21.15.42:0/286624879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc040103fe0 0x7fc040104400 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.795+0000 7fc045200640 1 -- 172.21.15.42:0/286624879 >> 172.21.15.42:0/286624879 conn(0x7fc0400ff570 msgr2=0x7fc040101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.795+0000 7fc045200640 1 -- 172.21.15.42:0/286624879 shutdown_connections 2024-04-20T10:33:16.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.795+0000 7fc045200640 1 -- 172.21.15.42:0/286624879 wait complete. 2024-04-20T10:33:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.796+0000 7fc045200640 1 Processor -- start 2024-04-20T10:33:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.796+0000 7fc045200640 1 -- start start 2024-04-20T10:33:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.796+0000 7fc045200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc040103fe0 0x7fc04019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:16.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.796+0000 7fc045200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0401051d0 0x7fc04019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.796+0000 7fc045200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc040105b90 0x7fc0401a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.796+0000 7fc045200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc04010fc70 con 0x7fc0401051d0 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.796+0000 7fc045200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc04010faf0 con 0x7fc040105b90 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.796+0000 7fc045200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc04010fdf0 con 0x7fc040103fe0 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.797+0000 7fc03ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0401051d0 0x7fc04019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.797+0000 7fc03f400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc040103fe0 0x7fc04019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.797+0000 7fc03ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0401051d0 0x7fc04019a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57314/0 (socket says 172.21.15.42:57314) 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.797+0000 7fc03f400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc040103fe0 0x7fc04019a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:38492/0 (socket says 172.21.15.42:38492) 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.797+0000 7fc03ea00640 1 -- 172.21.15.42:0/1100103217 learned_addr learned my addr 172.21.15.42:0/1100103217 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.797+0000 7fc03fe00640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc040105b90 0x7fc0401a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.797+0000 7fc03ea00640 1 -- 172.21.15.42:0/1100103217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc040103fe0 msgr2=0x7fc04019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.797+0000 7fc03ea00640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc040103fe0 0x7fc04019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.797+0000 7fc03ea00640 1 -- 172.21.15.42:0/1100103217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc040105b90 msgr2=0x7fc0401a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:16.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.797+0000 7fc03ea00640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc040105b90 0x7fc0401a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.797+0000 7fc03ea00640 1 -- 172.21.15.42:0/1100103217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0401a1f90 con 0x7fc0401051d0 2024-04-20T10:33:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.797+0000 7fc03ea00640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0401051d0 0x7fc04019a8b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00d950 tx=0x7fc02c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:16.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.798+0000 7fc037e00640 1 -- 172.21.15.42:0/1100103217 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c017030 con 0x7fc0401051d0 2024-04-20T10:33:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.798+0000 7fc045200640 1 -- 172.21.15.42:0/1100103217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0401a2280 con 0x7fc0401051d0 2024-04-20T10:33:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.798+0000 7fc045200640 1 -- 172.21.15.42:0/1100103217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0401a2820 con 0x7fc0401051d0 2024-04-20T10:33:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.798+0000 7fc037e00640 1 -- 172.21.15.42:0/1100103217 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc02c004020 con 0x7fc0401051d0 2024-04-20T10:33:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.798+0000 7fc037e00640 1 -- 172.21.15.42:0/1100103217 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0047a0 con 0x7fc0401051d0 2024-04-20T10:33:16.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.799+0000 7fc045200640 1 -- 172.21.15.42:0/1100103217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0200052c0 con 0x7fc0401051d0 2024-04-20T10:33:16.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.802+0000 7fc037e00640 1 -- 172.21.15.42:0/1100103217 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc02c0041d0 con 0x7fc0401051d0 2024-04-20T10:33:16.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.803+0000 7fc037e00640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc0240a3bb0 0x7fc0240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:16.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.803+0000 7fc037e00640 1 -- 172.21.15.42:0/1100103217 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc02c0cfe10 con 0x7fc0401051d0 2024-04-20T10:33:16.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.803+0000 7fc03f400640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc0240a3bb0 0x7fc0240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:16.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.804+0000 7fc03f400640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc0240a3bb0 0x7fc0240a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc030000c00 tx=0x7fc030005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:16.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:16.805+0000 7fc037e00640 1 -- 172.21.15.42:0/1100103217 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc02c022020 con 0x7fc0401051d0 2024-04-20T10:33:17.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.050+0000 7fc045200640 1 -- 172.21.15.42:0/1100103217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0200055b0 con 0x7fc0401051d0 2024-04-20T10:33:17.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.052+0000 7fc037e00640 1 -- 172.21.15.42:0/1100103217 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7fc02c09c800 con 0x7fc0401051d0 2024-04-20T10:33:17.052 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:17.052 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:33:17.053 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:33:17.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.055+0000 7fc045200640 1 -- 172.21.15.42:0/1100103217 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc0240a3bb0 msgr2=0x7fc0240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:17.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.055+0000 7fc045200640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc0240a3bb0 0x7fc0240a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc030000c00 tx=0x7fc030005fd0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.055+0000 7fc045200640 1 -- 172.21.15.42:0/1100103217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0401051d0 msgr2=0x7fc04019a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:17.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.055+0000 7fc045200640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0401051d0 0x7fc04019a8b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00d950 tx=0x7fc02c00de10 comp rx=0 tx=0).stop 2024-04-20T10:33:17.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.055+0000 7fc045200640 1 -- 172.21.15.42:0/1100103217 shutdown_connections 2024-04-20T10:33:17.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.055+0000 7fc045200640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc040105b90 0x7fc0401a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.055+0000 7fc045200640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0401051d0 0x7fc04019a8b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.055+0000 7fc045200640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc0240a3bb0 0x7fc0240a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.055+0000 7fc045200640 1 --2- 172.21.15.42:0/1100103217 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc040103fe0 0x7fc04019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.055+0000 7fc045200640 1 -- 172.21.15.42:0/1100103217 >> 172.21.15.42:0/1100103217 conn(0x7fc0400ff570 msgr2=0x7fc0401084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:17.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.055+0000 7fc045200640 1 -- 172.21.15.42:0/1100103217 shutdown_connections 2024-04-20T10:33:17.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.055+0000 7fc045200640 1 -- 172.21.15.42:0/1100103217 wait complete. 2024-04-20T10:33:17.076 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-20T10:31:35.067527+0000', 'modified': '2024-04-20T10:31:39.263696+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24584}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24568': {'gid': 24568, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.162:6839/3710674499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6838', 'nonce': 3710674499}, {'type': 'v1', 'addr': '172.21.15.162:6839', 'nonce': 3710674499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24584': {'gid': 24584, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6839/2026912868', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6838', 'nonce': 2026912868}, {'type': 'v1', 'addr': '172.21.15.46:6839', 'nonce': 2026912868}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24584, 'qdb_cluster': [24584]} 2024-04-20T10:33:17.076 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T10:33:17.076 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T10:33:17.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.290+0000 7f095ca00640 1 -- 172.21.15.42:0/307281071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09581040a0 msgr2=0x7f09581044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:17.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.290+0000 7f095ca00640 1 --2- 172.21.15.42:0/307281071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09581040a0 0x7f09581044c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0948009900 tx=0x7f094802bc50 comp rx=0 tx=0).stop 2024-04-20T10:33:17.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.290+0000 7f095ca00640 1 -- 172.21.15.42:0/307281071 shutdown_connections 2024-04-20T10:33:17.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.290+0000 7f095ca00640 1 --2- 172.21.15.42:0/307281071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0958105c50 0x7f095810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.290+0000 7f095ca00640 1 --2- 172.21.15.42:0/307281071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0958105290 0x7f0958105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.290+0000 7f095ca00640 1 --2- 172.21.15.42:0/307281071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09581040a0 0x7f09581044c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.290+0000 7f095ca00640 1 -- 172.21.15.42:0/307281071 >> 172.21.15.42:0/307281071 conn(0x7f09580ff610 msgr2=0x7f0958101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:17.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.291+0000 7f095ca00640 1 -- 172.21.15.42:0/307281071 shutdown_connections 2024-04-20T10:33:17.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.291+0000 7f095ca00640 1 -- 172.21.15.42:0/307281071 wait complete. 2024-04-20T10:33:17.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.291+0000 7f095ca00640 1 Processor -- start 2024-04-20T10:33:17.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.292+0000 7f095ca00640 1 -- start start 2024-04-20T10:33:17.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.292+0000 7f095ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09581040a0 0x7f095819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.292+0000 7f095ca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0958105290 0x7f095819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.292+0000 7f095ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0958105c50 0x7f09581a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.292+0000 7f095ca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f095810fd30 con 0x7f0958105c50 2024-04-20T10:33:17.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.292+0000 7f095ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f095810fbb0 con 0x7f09581040a0 2024-04-20T10:33:17.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.292+0000 7f095ca00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f095810feb0 con 0x7f0958105290 2024-04-20T10:33:17.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.292+0000 7f0957200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0958105c50 0x7f09581a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.292+0000 7f0956800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09581040a0 0x7f095819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.292+0000 7f0957200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0958105c50 0x7f09581a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57330/0 (socket says 172.21.15.42:57330) 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.292+0000 7f0957200640 1 -- 172.21.15.42:0/2898033530 learned_addr learned my addr 172.21.15.42:0/2898033530 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.292+0000 7f094de00640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0958105290 0x7f095819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.293+0000 7f0956800640 1 -- 172.21.15.42:0/2898033530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0958105290 msgr2=0x7f095819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.293+0000 7f0956800640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0958105290 0x7f095819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.293+0000 7f0956800640 1 -- 172.21.15.42:0/2898033530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0958105c50 msgr2=0x7f09581a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.293+0000 7f0956800640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0958105c50 0x7f09581a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.293+0000 7f0956800640 1 -- 172.21.15.42:0/2898033530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09581a2050 con 0x7f09581040a0 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.293+0000 7f0956800640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09581040a0 0x7f095819a430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f094802c190 tx=0x7f094802c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.293+0000 7f094fe00640 1 -- 172.21.15.42:0/2898033530 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948042020 con 0x7f09581040a0 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.293+0000 7f095ca00640 1 -- 172.21.15.42:0/2898033530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09581a22e0 con 0x7f09581040a0 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.293+0000 7f094fe00640 1 -- 172.21.15.42:0/2898033530 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0948004450 con 0x7f09581040a0 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.293+0000 7f094fe00640 1 -- 172.21.15.42:0/2898033530 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094803e460 con 0x7f09581040a0 2024-04-20T10:33:17.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.294+0000 7f095ca00640 1 -- 172.21.15.42:0/2898033530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09581a2850 con 0x7f09581040a0 2024-04-20T10:33:17.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f094fe00640 1 -- 172.21.15.42:0/2898033530 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0948045020 con 0x7f09581040a0 2024-04-20T10:33:17.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f0960198740 1 Processor -- start 2024-04-20T10:33:17.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f0960198740 1 -- start start 2024-04-20T10:33:17.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f0960198740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e44b40 0x561352e44f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f0960198740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e45530 0x561352e4a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f0960198740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 0x561352e4cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f0960198740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561352dab9d0 con 0x561352e45530 2024-04-20T10:33:17.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f0960198740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561352d52270 con 0x561352e4a6a0 2024-04-20T10:33:17.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f0960198740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561352da7990 con 0x561352e44b40 2024-04-20T10:33:17.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f0957200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e44b40 0x561352e44f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f094de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 0x561352e4cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f0957200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e44b40 0x561352e44f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:38524/0 (socket says 172.21.15.42:38524) 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f0957200640 1 -- 172.21.15.42:0/2138215031 learned_addr learned my addr 172.21.15.42:0/2138215031 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.297+0000 7f0956800640 1 --2- 172.21.15.42:0/2138215031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e45530 0x561352e4a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.298+0000 7f094de00640 1 -- 172.21.15.42:0/2138215031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e44b40 msgr2=0x561352e44f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.298+0000 7f094de00640 1 --2- 172.21.15.42:0/2138215031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e44b40 0x561352e44f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.298+0000 7f094de00640 1 -- 172.21.15.42:0/2138215031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e45530 msgr2=0x561352e4a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.298+0000 7f094de00640 1 --2- 172.21.15.42:0/2138215031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e45530 0x561352e4a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.298+0000 7f094de00640 1 -- 172.21.15.42:0/2138215031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561352abe2d0 con 0x561352e4a6a0 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.298+0000 7f0957200640 1 --2- 172.21.15.42:0/2138215031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e44b40 0x561352e44f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.298+0000 7f094de00640 1 --2- 172.21.15.42:0/2138215031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 0x561352e4cb30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0940009bf0 tx=0x7f0940011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7426d32c714a8fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.298+0000 7f094fe00640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f093c0a3bb0 0x7f093c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.298+0000 7f094fe00640 1 -- 172.21.15.42:0/2898033530 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f09480f2430 con 0x7f09581040a0 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0957200640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f093c0a3bb0 0x7f093c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.298+0000 7f0947400640 1 -- 172.21.15.42:0/2138215031 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094001e070 con 0x561352e4a6a0 2024-04-20T10:33:17.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0947400640 1 -- 172.21.15.42:0/2138215031 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0940009da0 con 0x561352e4a6a0 2024-04-20T10:33:17.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0947400640 1 -- 172.21.15.42:0/2138215031 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0940018870 con 0x561352e4a6a0 2024-04-20T10:33:17.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0960198740 1 -- 172.21.15.42:0/2138215031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 msgr2=0x561352e4cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:17.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0960198740 1 --2- 172.21.15.42:0/2138215031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 0x561352e4cb30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0940009bf0 tx=0x7f0940011810 comp rx=0 tx=0).stop 2024-04-20T10:33:17.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0957200640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f093c0a3bb0 0x7f093c0a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x561352e45d90 tx=0x7f0950009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:17.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0960198740 1 -- 172.21.15.42:0/2138215031 shutdown_connections 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0960198740 1 --2- 172.21.15.42:0/2138215031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 0x561352e4cb30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0960198740 1 --2- 172.21.15.42:0/2138215031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e45530 0x561352e4a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0960198740 1 --2- 172.21.15.42:0/2138215031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e44b40 0x561352e44f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0960198740 1 -- 172.21.15.42:0/2138215031 >> 172.21.15.42:0/2138215031 conn(0x561352e400b0 msgr2=0x561352e404b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0960198740 1 -- 172.21.15.42:0/2138215031 shutdown_connections 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.299+0000 7f0960198740 1 -- 172.21.15.42:0/2138215031 wait complete. 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.300+0000 7f0960198740 1 -- start start 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.300+0000 7f0960198740 1 Processor -- start 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.300+0000 7f0960198740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e44b40 0x561352e4f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.300+0000 7f0960198740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e45530 0x561352e4f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f0960198740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e4a6a0 0x561352e565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f0960198740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561352e4d650 con 0x561352e45530 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f0960198740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561352d655e0 con 0x561352e44b40 2024-04-20T10:33:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f0960198740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561352e4d7d0 con 0x561352e4a6a0 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f0960198740 10 client.0 _ll_register_callbacks cb 0x561352abdac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f094de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e45530 0x561352e4f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f094de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e45530 0x561352e4f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57346/0 (socket says 172.21.15.42:57346) 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f094de00640 1 -- 172.21.15.42:0/1857497640 learned_addr learned my addr 172.21.15.42:0/1857497640 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f0957200640 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e4a6a0 0x561352e565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f0956800640 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e44b40 0x561352e4f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f094de00640 1 -- 172.21.15.42:0/1857497640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e4a6a0 msgr2=0x561352e565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f094de00640 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e4a6a0 0x561352e565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f094de00640 1 -- 172.21.15.42:0/1857497640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e44b40 msgr2=0x561352e4f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f094de00640 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e44b40 0x561352e4f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f094de00640 1 -- 172.21.15.42:0/1857497640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561352b886b0 con 0x561352e45530 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f0957200640 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e4a6a0 0x561352e565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.301+0000 7f0956800640 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e44b40 0x561352e4f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:33:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.302+0000 7f094de00640 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e45530 0x561352e4f5b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f09400056a0 tx=0x7f09400056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb3b93e17c1585f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.302+0000 7f093be00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.302+0000 7f093be00640 1 -- 172.21.15.42:0/1857497640 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0940027020 con 0x561352e45530 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.302+0000 7f093be00640 1 -- 172.21.15.42:0/1857497640 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f0940003800 con 0x561352e45530 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.302+0000 7f0960198740 1 -- 172.21.15.42:0/1857497640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0940009da0 con 0x561352e45530 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.302+0000 7f093be00640 1 -- 172.21.15.42:0/1857497640 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0940032050 con 0x561352e45530 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.302+0000 7f0960198740 10 client.14874 fetch_fsmap learned FSMap version 19 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.302+0000 7f0960198740 1 -- 172.21.15.42:0/1857497640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561352e599b0 con 0x561352e45530 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.302+0000 7f093be00640 1 -- 172.21.15.42:0/1857497640 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0940009da0 con 0x561352e45530 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.303+0000 7f093be00640 1 -- 172.21.15.42:0/1857497640 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f09400185f0 con 0x561352e45530 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.303+0000 7f0960198740 10 client.14874 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.303+0000 7f0960198740 10 client.14874 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.303+0000 7f0960198740 20 client.14874 populate_metadata read hostname 'smithi042' 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.303+0000 7f0960198740 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561352e5bd90 0x561352e5d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.303+0000 7f0960198740 4 client.14874 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:33:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.303+0000 7f0960198740 1 -- 172.21.15.42:0/1857497640 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561352db93f0 con 0x561352e5bd90 2024-04-20T10:33:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.303+0000 7f0956800640 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561352e5bd90 0x561352e5d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.304+0000 7f0956800640 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561352e5bd90 0x561352e5d200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b0f247af224263 server_cookie=c6f26fe7115aea55 in_seq=0 out_seq=0 2024-04-20T10:33:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.304+0000 7f093be00640 10 client.14874 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:33:17.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.304+0000 7f093be00640 1 -- 172.21.15.42:0/1857497640 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561352db93f0 con 0x561352e5bd90 2024-04-20T10:33:17.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.304+0000 7f093be00640 10 client.14874 handle_command_reply: tid=0 2024-04-20T10:33:17.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f0960198740 1 client.14874 shutdown 2024-04-20T10:33:17.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f0960198740 1 -- 172.21.15.42:0/1857497640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e45530 msgr2=0x561352e4f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:17.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f0960198740 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e45530 0x561352e4f5b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f09400056a0 tx=0x7f09400056d0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f0960198740 1 -- 172.21.15.42:0/1857497640 shutdown_connections 2024-04-20T10:33:17.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f0960198740 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561352e5bd90 0x561352e5d200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f0960198740 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e4a6a0 0x561352e565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f0960198740 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e45530 0x561352e4f5b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f0960198740 1 --2- 172.21.15.42:0/1857497640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e44b40 0x561352e4f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f093be00640 0 client.14874 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:33:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f0960198740 1 -- 172.21.15.42:0/1857497640 >> 172.21.15.42:0/1857497640 conn(0x561352e400b0 msgr2=0x561352e3f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f0960198740 1 -- 172.21.15.42:0/1857497640 shutdown_connections 2024-04-20T10:33:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f0960198740 1 -- 172.21.15.42:0/1857497640 wait complete. 2024-04-20T10:33:17.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.305+0000 7f0960198740 20 client.14874 trim_cache size 0 max 16384 2024-04-20T10:33:17.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.325+0000 7f0960198740 1 Processor -- start 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.325+0000 7f0960198740 1 -- start start 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.325+0000 7f0960198740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e44b40 0x561352e4ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.325+0000 7f0960198740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e45530 0x561352e565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.325+0000 7f0960198740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e4a6a0 0x561352e4dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.325+0000 7f0960198740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561352e5b540 con 0x561352e4a6a0 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.325+0000 7f0960198740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561352e5bc10 con 0x561352e44b40 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.325+0000 7f0960198740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561352e5c2e0 con 0x561352e45530 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.326+0000 7f0957200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e4a6a0 0x561352e4dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.326+0000 7f0957200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e4a6a0 0x561352e4dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57350/0 (socket says 172.21.15.42:57350) 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.326+0000 7f0957200640 1 -- 172.21.15.42:0/413735535 learned_addr learned my addr 172.21.15.42:0/413735535 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.326+0000 7f0956800640 1 --2- 172.21.15.42:0/413735535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e44b40 0x561352e4ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.326+0000 7f094de00640 1 --2- 172.21.15.42:0/413735535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e45530 0x561352e565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.326+0000 7f0957200640 1 -- 172.21.15.42:0/413735535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e45530 msgr2=0x561352e565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:17.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.326+0000 7f0957200640 1 --2- 172.21.15.42:0/413735535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e45530 0x561352e565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.326+0000 7f0957200640 1 -- 172.21.15.42:0/413735535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e44b40 msgr2=0x561352e4ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:17.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.326+0000 7f0957200640 1 --2- 172.21.15.42:0/413735535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e44b40 0x561352e4ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.326+0000 7f0957200640 1 -- 172.21.15.42:0/413735535 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561352e31950 con 0x561352e4a6a0 2024-04-20T10:33:17.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.326+0000 7f0957200640 1 --2- 172.21.15.42:0/413735535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e4a6a0 0x561352e4dba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0950011b70 tx=0x7f0950017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b7bc1e4e6312e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:17.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.327+0000 7f0946a00640 1 -- 172.21.15.42:0/413735535 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095002e070 con 0x561352e4a6a0 2024-04-20T10:33:17.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.327+0000 7f0946a00640 1 -- 172.21.15.42:0/413735535 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0950029410 con 0x561352e4a6a0 2024-04-20T10:33:17.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.327+0000 7f0946a00640 1 -- 172.21.15.42:0/413735535 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095001d400 con 0x561352e4a6a0 2024-04-20T10:33:17.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.327+0000 7f0960198740 1 -- 172.21.15.42:0/413735535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e4a6a0 msgr2=0x561352e4dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.327+0000 7f0960198740 1 --2- 172.21.15.42:0/413735535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e4a6a0 0x561352e4dba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0950011b70 tx=0x7f0950017000 comp rx=0 tx=0).stop 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.327+0000 7f0960198740 1 -- 172.21.15.42:0/413735535 shutdown_connections 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.327+0000 7f0960198740 1 --2- 172.21.15.42:0/413735535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e4a6a0 0x561352e4dba0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.327+0000 7f0960198740 1 --2- 172.21.15.42:0/413735535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e45530 0x561352e565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.327+0000 7f0960198740 1 --2- 172.21.15.42:0/413735535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e44b40 0x561352e4ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.327+0000 7f0960198740 1 -- 172.21.15.42:0/413735535 >> 172.21.15.42:0/413735535 conn(0x561352e400b0 msgr2=0x561352e43ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.327+0000 7f0960198740 1 -- 172.21.15.42:0/413735535 shutdown_connections 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.327+0000 7f0960198740 1 -- 172.21.15.42:0/413735535 wait complete. 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.328+0000 7f0960198740 1 -- start start 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.328+0000 7f0960198740 1 Processor -- start 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.328+0000 7f0960198740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e44b40 0x561352e545a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.328+0000 7f0960198740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e45530 0x561352ebdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.328+0000 7f0960198740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 0x561352ec0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.328+0000 7f0960198740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561352e5d080 con 0x561352e44b40 2024-04-20T10:33:17.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.328+0000 7f0960198740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561352e5c9b0 con 0x561352e4a6a0 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.328+0000 7f0960198740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561352e5d750 con 0x561352e45530 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.328+0000 7f0960198740 10 client.0 _ll_register_callbacks cb 0x7f094001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0956800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e44b40 0x561352e545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0957200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 0x561352ec0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0957200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 0x561352ec0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46750/0 (socket says 172.21.15.42:46750) 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0956800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e44b40 0x561352e545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57358/0 (socket says 172.21.15.42:57358) 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0957200640 1 -- 172.21.15.42:0/3588780678 learned_addr learned my addr 172.21.15.42:0/3588780678 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f094de00640 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e45530 0x561352ebdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0957200640 1 -- 172.21.15.42:0/3588780678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e45530 msgr2=0x561352ebdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0957200640 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e45530 0x561352ebdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0957200640 1 -- 172.21.15.42:0/3588780678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e44b40 msgr2=0x561352e545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0957200640 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e44b40 0x561352e545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0957200640 1 -- 172.21.15.42:0/3588780678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561352ec0930 con 0x561352e4a6a0 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0956800640 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e44b40 0x561352e545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0957200640 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 0x561352ec0270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0950020800 tx=0x7f0950004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71a0756174748f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f093be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f093be00640 1 -- 172.21.15.42:0/3588780678 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095002e070 con 0x561352e4a6a0 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.329+0000 7f0960198740 1 -- 172.21.15.42:0/3588780678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0950029410 con 0x561352e4a6a0 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.330+0000 7f093be00640 1 -- 172.21.15.42:0/3588780678 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f0950020a70 con 0x561352e4a6a0 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.330+0000 7f093be00640 1 -- 172.21.15.42:0/3588780678 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0950004e40 con 0x561352e4a6a0 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.330+0000 7f0960198740 10 client.24989 fetch_fsmap learned FSMap version 19 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.330+0000 7f0960198740 1 -- 172.21.15.42:0/3588780678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561352ec1c80 con 0x561352e4a6a0 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.330+0000 7f093be00640 1 -- 172.21.15.42:0/3588780678 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0950029410 con 0x561352e4a6a0 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.330+0000 7f093be00640 1 -- 172.21.15.42:0/3588780678 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f095000f040 con 0x561352e4a6a0 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.330+0000 7f0960198740 10 client.24989 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:33:17.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.330+0000 7f0960198740 10 client.24989 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.330+0000 7f0960198740 20 client.24989 populate_metadata read hostname 'smithi042' 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.330+0000 7f0960198740 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561352ec6490 0x561352ec51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.330+0000 7f0960198740 4 client.24989 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.330+0000 7f0960198740 1 -- 172.21.15.42:0/3588780678 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "status"}) -- 0x561352db3260 con 0x561352ec6490 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.331+0000 7f0956800640 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561352ec6490 0x561352ec51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.331+0000 7f0956800640 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561352ec6490 0x561352ec51a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2462ba8689fd6700 server_cookie=9ebabe3c71566159 in_seq=0 out_seq=0 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.331+0000 7f093be00640 10 client.24989 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.331+0000 7f093be00640 1 -- 172.21.15.42:0/3588780678 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x561352db3260 con 0x561352ec6490 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.331+0000 7f093be00640 10 client.24989 handle_command_reply: tid=0 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.331+0000 7f0960198740 1 client.24989 shutdown 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f0960198740 1 -- 172.21.15.42:0/3588780678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 msgr2=0x561352ec0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f0960198740 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 0x561352ec0270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0950020800 tx=0x7f0950004a40 comp rx=0 tx=0).stop 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f0960198740 1 -- 172.21.15.42:0/3588780678 shutdown_connections 2024-04-20T10:33:17.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f0960198740 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561352ec6490 0x561352ec51a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f0960198740 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561352e4a6a0 0x561352ec0270 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f0960198740 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561352e45530 0x561352ebdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f093be00640 0 client.24989 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:33:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f0960198740 1 --2- 172.21.15.42:0/3588780678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561352e44b40 0x561352e545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f0960198740 1 -- 172.21.15.42:0/3588780678 >> 172.21.15.42:0/3588780678 conn(0x561352e400b0 msgr2=0x561352e43ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f0960198740 1 -- 172.21.15.42:0/3588780678 shutdown_connections 2024-04-20T10:33:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f0960198740 1 -- 172.21.15.42:0/3588780678 wait complete. 2024-04-20T10:33:17.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f0960198740 20 client.24989 trim_cache size 0 max 16384 2024-04-20T10:33:17.337 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:33:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "83e8f766-feff-11ee-bc92-c7b262605968", 2024-04-20T10:33:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-04-20T10:33:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24584, 2024-04-20T10:33:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-04-20T10:33:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-04-20T10:33:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-04-20T10:33:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 101.608845471, 2024-04-20T10:33:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 19, 2024-04-20T10:33:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 87, 2024-04-20T10:33:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T10:33:17.338 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 107.096825849 2024-04-20T10:33:17.338 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:33:17.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f095ca00640 1 -- 172.21.15.42:0/2898033530 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f093c0a3bb0 msgr2=0x7f093c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:17.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f095ca00640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f093c0a3bb0 0x7f093c0a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x561352e45d90 tx=0x7f0950009680 comp rx=0 tx=0).stop 2024-04-20T10:33:17.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f095ca00640 1 -- 172.21.15.42:0/2898033530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09581040a0 msgr2=0x7f095819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:17.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.332+0000 7f095ca00640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09581040a0 0x7f095819a430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f094802c190 tx=0x7f094802c6e0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.333+0000 7f095ca00640 1 -- 172.21.15.42:0/2898033530 shutdown_connections 2024-04-20T10:33:17.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.333+0000 7f095ca00640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0958105c50 0x7f09581a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.333+0000 7f095ca00640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f093c0a3bb0 0x7f093c0a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.333+0000 7f095ca00640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0958105290 0x7f095819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.333+0000 7f095ca00640 1 --2- 172.21.15.42:0/2898033530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09581040a0 0x7f095819a430 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:17.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.333+0000 7f095ca00640 1 -- 172.21.15.42:0/2898033530 >> 172.21.15.42:0/2898033530 conn(0x7f09580ff610 msgr2=0x7f0958108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:17.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.333+0000 7f095ca00640 1 -- 172.21.15.42:0/2898033530 shutdown_connections 2024-04-20T10:33:17.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:17.333+0000 7f095ca00640 1 -- 172.21.15.42:0/2898033530 wait complete. 2024-04-20T10:33:17.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:17 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4128245051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:17.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:17 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1100103217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:17.349 INFO:tasks.cephadm:Setup complete, yielding 2024-04-20T10:33:17.350 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T10:33:17.361 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-04-20T10:33:17.361 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- bash -c 'ceph orch status' 2024-04-20T10:33:17.551 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:17 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/4128245051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:17.551 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:17 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1100103217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:17.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:17 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/4128245051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:17.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:17 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1100103217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:17.802 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:33:18.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:18 smithi046 ceph-mon[21574]: pgmap v370: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-20T10:33:18.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:18 smithi042 ceph-mon[17078]: pgmap v370: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-20T10:33:18.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:18 smithi162 ceph-mon[21854]: pgmap v370: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-20T10:33:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.131+0000 7f9b34e00700 1 -- 172.21.15.42:0/3203520967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3010aa00 msgr2=0x7f9b3010ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:20.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.131+0000 7f9b34e00700 1 --2- 172.21.15.42:0/3203520967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3010aa00 0x7f9b3010ce70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800b0a0 tx=0x7f9b2800b3b0 comp rx=0 tx=0).stop 2024-04-20T10:33:20.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.132+0000 7f9b34e00700 1 -- 172.21.15.42:0/3203520967 shutdown_connections 2024-04-20T10:33:20.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.132+0000 7f9b34e00700 1 --2- 172.21.15.42:0/3203520967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3010aa00 0x7f9b3010ce70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:20.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.132+0000 7f9b34e00700 1 --2- 172.21.15.42:0/3203520967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b30076a00 0x7f9b30074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:20.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.132+0000 7f9b34e00700 1 --2- 172.21.15.42:0/3203520967 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b30075430 0x7f9b300758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:20.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.132+0000 7f9b34e00700 1 -- 172.21.15.42:0/3203520967 >> 172.21.15.42:0/3203520967 conn(0x7f9b300fe0a0 msgr2=0x7f9b301004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:20.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.132+0000 7f9b34e00700 1 -- 172.21.15.42:0/3203520967 shutdown_connections 2024-04-20T10:33:20.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.132+0000 7f9b34e00700 1 -- 172.21.15.42:0/3203520967 wait complete. 2024-04-20T10:33:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.133+0000 7f9b34e00700 1 Processor -- start 2024-04-20T10:33:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.133+0000 7f9b34e00700 1 -- start start 2024-04-20T10:33:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.133+0000 7f9b34e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b30075430 0x7f9b3019aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.133+0000 7f9b34e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b30076a00 0x7f9b3019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.133+0000 7f9b34e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b3010aa00 0x7f9b301a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.133+0000 7f9b34e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9b30110920 con 0x7f9b3010aa00 2024-04-20T10:33:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.133+0000 7f9b34e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301107a0 con 0x7f9b30076a00 2024-04-20T10:33:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.133+0000 7f9b34e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9b30110aa0 con 0x7f9b30075430 2024-04-20T10:33:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.134+0000 7f9b2de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b30075430 0x7f9b3019aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.134+0000 7f9b2e800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b3010aa00 0x7f9b301a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.134+0000 7f9b2de00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b30075430 0x7f9b3019aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:50596/0 (socket says 172.21.15.42:50596) 2024-04-20T10:33:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.134+0000 7f9b2de00700 1 -- 172.21.15.42:0/1825586365 learned_addr learned my addr 172.21.15.42:0/1825586365 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.134+0000 7f9b25400700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b30076a00 0x7f9b3019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.134+0000 7f9b2de00700 1 -- 172.21.15.42:0/1825586365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b30076a00 msgr2=0x7f9b3019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.134+0000 7f9b2de00700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b30076a00 0x7f9b3019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.134+0000 7f9b2de00700 1 -- 172.21.15.42:0/1825586365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b3010aa00 msgr2=0x7f9b301a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.134+0000 7f9b2de00700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b3010aa00 0x7f9b301a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.134+0000 7f9b2de00700 1 -- 172.21.15.42:0/1825586365 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b301a2b20 con 0x7f9b30075430 2024-04-20T10:33:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.134+0000 7f9b25400700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b30076a00 0x7f9b3019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.135+0000 7f9b2de00700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b30075430 0x7f9b3019aec0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000e9d0 tx=0x7f9b2000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.135+0000 7f9b27400700 1 -- 172.21.15.42:0/1825586365 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2000cbd0 con 0x7f9b30075430 2024-04-20T10:33:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.135+0000 7f9b27400700 1 -- 172.21.15.42:0/1825586365 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b2000ee70 con 0x7f9b30075430 2024-04-20T10:33:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.135+0000 7f9b34e00700 1 -- 172.21.15.42:0/1825586365 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b301a2e10 con 0x7f9b30075430 2024-04-20T10:33:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.135+0000 7f9b27400700 1 -- 172.21.15.42:0/1825586365 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20018660 con 0x7f9b30075430 2024-04-20T10:33:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.135+0000 7f9b34e00700 1 -- 172.21.15.42:0/1825586365 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b301a3270 con 0x7f9b30075430 2024-04-20T10:33:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.136+0000 7f9b34e00700 1 -- 172.21.15.42:0/1825586365 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b30069740 con 0x7f9b30075430 2024-04-20T10:33:20.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.138+0000 7f9b27400700 1 -- 172.21.15.42:0/1825586365 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9b20010430 con 0x7f9b30075430 2024-04-20T10:33:20.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.140+0000 7f9b27400700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9b140a3cc0 0x7f9b140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:20.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.140+0000 7f9b27400700 1 -- 172.21.15.42:0/1825586365 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9b200d07e0 con 0x7f9b30075430 2024-04-20T10:33:20.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.140+0000 7f9b25400700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9b140a3cc0 0x7f9b140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:20.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.141+0000 7f9b25400700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9b140a3cc0 0x7f9b140a6180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9b18011490 tx=0x7f9b18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:20.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.142+0000 7f9b27400700 1 -- 172.21.15.42:0/1825586365 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9b2009d140 con 0x7f9b30075430 2024-04-20T10:33:20.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.396+0000 7f9b34e00700 1 -- 172.21.15.42:0/1825586365 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9b30062e90 con 0x7f9b140a3cc0 2024-04-20T10:33:20.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.399+0000 7f9b27400700 1 -- 172.21.15.42:0/1825586365 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9b30062e90 con 0x7f9b140a3cc0 2024-04-20T10:33:20.399 INFO:teuthology.orchestra.run.smithi042.stdout:Backend: cephadm 2024-04-20T10:33:20.400 INFO:teuthology.orchestra.run.smithi042.stdout:Available: Yes 2024-04-20T10:33:20.400 INFO:teuthology.orchestra.run.smithi042.stdout:Paused: No 2024-04-20T10:33:20.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.402+0000 7f9b34e00700 1 -- 172.21.15.42:0/1825586365 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9b140a3cc0 msgr2=0x7f9b140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:20.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.402+0000 7f9b34e00700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9b140a3cc0 0x7f9b140a6180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9b18011490 tx=0x7f9b18009260 comp rx=0 tx=0).stop 2024-04-20T10:33:20.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.402+0000 7f9b34e00700 1 -- 172.21.15.42:0/1825586365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b30075430 msgr2=0x7f9b3019aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:20.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.402+0000 7f9b34e00700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b30075430 0x7f9b3019aec0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000e9d0 tx=0x7f9b2000ece0 comp rx=0 tx=0).stop 2024-04-20T10:33:20.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.402+0000 7f9b34e00700 1 -- 172.21.15.42:0/1825586365 shutdown_connections 2024-04-20T10:33:20.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.402+0000 7f9b34e00700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9b140a3cc0 0x7f9b140a6180 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:20.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.402+0000 7f9b34e00700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b3010aa00 0x7f9b301a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:20.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.402+0000 7f9b34e00700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b30075430 0x7f9b3019aec0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:20.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.402+0000 7f9b34e00700 1 --2- 172.21.15.42:0/1825586365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b30076a00 0x7f9b3019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:20.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.402+0000 7f9b34e00700 1 -- 172.21.15.42:0/1825586365 >> 172.21.15.42:0/1825586365 conn(0x7f9b300fe0a0 msgr2=0x7f9b30106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:20.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.403+0000 7f9b34e00700 1 -- 172.21.15.42:0/1825586365 shutdown_connections 2024-04-20T10:33:20.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:20.403+0000 7f9b34e00700 1 -- 172.21.15.42:0/1825586365 wait complete. 2024-04-20T10:33:20.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:20 smithi046 ceph-mon[21574]: pgmap v371: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:20.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:20 smithi042 ceph-mon[17078]: pgmap v371: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:20.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:20 smithi162 ceph-mon[21854]: pgmap v371: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:21.305 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-20T10:33:21.784 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:33:22.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:22 smithi046 ceph-mon[21574]: from='client.25015 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:22.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:22 smithi046 ceph-mon[21574]: pgmap v372: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:22.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:22 smithi042 ceph-mon[17078]: from='client.25015 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:22.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:22 smithi042 ceph-mon[17078]: pgmap v372: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:22.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:22 smithi162 ceph-mon[21854]: from='client.25015 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:22.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:22 smithi162 ceph-mon[21854]: pgmap v372: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:24.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.064+0000 7f8ede000700 1 -- 172.21.15.42:0/1980443141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8103540 msgr2=0x7f8ed8105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:24.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.064+0000 7f8ede000700 1 --2- 172.21.15.42:0/1980443141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8103540 0x7f8ed8105930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009990 tx=0x7f8ec8009ca0 comp rx=0 tx=0).stop 2024-04-20T10:33:24.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.064+0000 7f8ede000700 1 -- 172.21.15.42:0/1980443141 shutdown_connections 2024-04-20T10:33:24.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.064+0000 7f8ede000700 1 --2- 172.21.15.42:0/1980443141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8103540 0x7f8ed8105930 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:24.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.064+0000 7f8ede000700 1 --2- 172.21.15.42:0/1980443141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed81087a0 0x7f8ed810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:24.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.064+0000 7f8ede000700 1 --2- 172.21.15.42:0/1980443141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ed8105e70 0x7f8ed8108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:24.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.064+0000 7f8ede000700 1 -- 172.21.15.42:0/1980443141 >> 172.21.15.42:0/1980443141 conn(0x7f8ed80fcf70 msgr2=0x7f8ed80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:24.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.064+0000 7f8ede000700 1 -- 172.21.15.42:0/1980443141 shutdown_connections 2024-04-20T10:33:24.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.064+0000 7f8ede000700 1 -- 172.21.15.42:0/1980443141 wait complete. 2024-04-20T10:33:24.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.066+0000 7f8ede000700 1 Processor -- start 2024-04-20T10:33:24.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.066+0000 7f8ede000700 1 -- start start 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.066+0000 7f8ede000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8103540 0x7f8ed8102e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.066+0000 7f8ede000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ed8105e70 0x7f8ed81014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.066+0000 7f8ede000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed81087a0 0x7f8ed8101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.066+0000 7f8ede000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed810e750 con 0x7f8ed8105e70 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.066+0000 7f8ede000700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed810e5d0 con 0x7f8ed8103540 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.066+0000 7f8ede000700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed810e8d0 con 0x7f8ed81087a0 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.067+0000 7f8ed7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8103540 0x7f8ed8102e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.067+0000 7f8ed7400700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8103540 0x7f8ed8102e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:54518/0 (socket says 172.21.15.42:54518) 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.067+0000 7f8ed7400700 1 -- 172.21.15.42:0/2861715398 learned_addr learned my addr 172.21.15.42:0/2861715398 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.067+0000 7f8ecea00700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ed8105e70 0x7f8ed81014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.067+0000 7f8ed7e00700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed81087a0 0x7f8ed8101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.067+0000 7f8ed7400700 1 -- 172.21.15.42:0/2861715398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed81087a0 msgr2=0x7f8ed8101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:24.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.067+0000 7f8ed7400700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed81087a0 0x7f8ed8101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:24.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.067+0000 7f8ed7400700 1 -- 172.21.15.42:0/2861715398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ed8105e70 msgr2=0x7f8ed81014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:24.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.067+0000 7f8ed7400700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ed8105e70 0x7f8ed81014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:24.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.067+0000 7f8ed7400700 1 -- 172.21.15.42:0/2861715398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed8102300 con 0x7f8ed8103540 2024-04-20T10:33:24.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.068+0000 7f8ed7400700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8103540 0x7f8ed8102e50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8003890 tx=0x7f8ec80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:24.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.068+0000 7f8ed4c00700 1 -- 172.21.15.42:0/2861715398 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8026070 con 0x7f8ed8103540 2024-04-20T10:33:24.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.068+0000 7f8ed4c00700 1 -- 172.21.15.42:0/2861715398 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec8004150 con 0x7f8ed8103540 2024-04-20T10:33:24.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.068+0000 7f8ede000700 1 -- 172.21.15.42:0/2861715398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed81a2e60 con 0x7f8ed8103540 2024-04-20T10:33:24.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.068+0000 7f8ed4c00700 1 -- 172.21.15.42:0/2861715398 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8021410 con 0x7f8ed8103540 2024-04-20T10:33:24.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.068+0000 7f8ede000700 1 -- 172.21.15.42:0/2861715398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed81a3280 con 0x7f8ed8103540 2024-04-20T10:33:24.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.069+0000 7f8ede000700 1 -- 172.21.15.42:0/2861715398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed8069740 con 0x7f8ed8103540 2024-04-20T10:33:24.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.071+0000 7f8ed4c00700 1 -- 172.21.15.42:0/2861715398 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8ec8029070 con 0x7f8ed8103540 2024-04-20T10:33:24.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.073+0000 7f8ed4c00700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8ebc0a3cc0 0x7f8ebc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:24.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.073+0000 7f8ed4c00700 1 -- 172.21.15.42:0/2861715398 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f8ec80d0e50 con 0x7f8ed8103540 2024-04-20T10:33:24.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.073+0000 7f8ecea00700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8ebc0a3cc0 0x7f8ebc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:24.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.074+0000 7f8ecea00700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8ebc0a3cc0 0x7f8ebc0a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0005950 tx=0x7f8ec000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:24.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.077+0000 7f8ed4c00700 1 -- 172.21.15.42:0/2861715398 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8ec8013070 con 0x7f8ed8103540 2024-04-20T10:33:24.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.325+0000 7f8ede000700 1 -- 172.21.15.42:0/2861715398 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8ed8062e90 con 0x7f8ebc0a3cc0 2024-04-20T10:33:24.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:24 smithi042 ceph-mon[17078]: pgmap v373: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:24.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.368+0000 7f8ed4c00700 1 -- 172.21.15.42:0/2861715398 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f8ed8062e90 con 0x7f8ebc0a3cc0 2024-04-20T10:33:24.369 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T10:33:24.369 INFO:teuthology.orchestra.run.smithi042.stdout:mds.a smithi042 running (117s) 23s ago 116s 12.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 957181e577d7 2024-04-20T10:33:24.369 INFO:teuthology.orchestra.run.smithi042.stdout:mds.b smithi046 running (114s) 45s ago 114s 24.8M - 19.0.0-3133-ga8c84f59 76bc6b91b726 bc113dea9960 2024-04-20T10:33:24.369 INFO:teuthology.orchestra.run.smithi042.stdout:mds.c smithi162 running (2m) 45s ago 2m 16.2M - 19.0.0-3133-ga8c84f59 76bc6b91b726 669991ee670d 2024-04-20T10:33:24.369 INFO:teuthology.orchestra.run.smithi042.stdout:mds.d smithi042 running (2m) 23s ago 2m 12.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 3f1f0f598855 2024-04-20T10:33:24.369 INFO:teuthology.orchestra.run.smithi042.stdout:mds.e smithi046 running (2m) 45s ago 2m 15.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 e15407ae1227 2024-04-20T10:33:24.369 INFO:teuthology.orchestra.run.smithi042.stdout:mds.f smithi162 running (2m) 45s ago 2m 13.8M - 19.0.0-3133-ga8c84f59 76bc6b91b726 22ceba1b0f8b 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:mds.g smithi042 running (2m) 23s ago 2m 12.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 45239af92595 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:mds.h smithi046 running (2m) 45s ago 2m 13.8M - 19.0.0-3133-ga8c84f59 76bc6b91b726 f397eec9989b 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:mds.i smithi162 running (2m) 45s ago 2m 18.1M - 19.0.0-3133-ga8c84f59 76bc6b91b726 dd8bfdadbfd4 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:mds.j smithi042 running (2m) 23s ago 2m 12.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 f99cf4e9e713 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:mds.k smithi046 running (2m) 45s ago 2m 14.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 e50c2e20b7c2 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:mds.l smithi162 running (2m) 45s ago 2m 14.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 d0514c17601a 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.x smithi042 *:9283,8765 running (10m) 23s ago 10m 515M - 19.0.0-3133-ga8c84f59 76bc6b91b726 5bd72d6b593f 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.y smithi046 *:8443,8765 running (8m) 45s ago 8m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2c1fd9edaa3b 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.z smithi162 *:8443,8765 running (8m) 45s ago 8m 453M - 19.0.0-3133-ga8c84f59 76bc6b91b726 5fd52e305ebf 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a smithi042 running (10m) 23s ago 11m 47.1M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 f39180ca248a 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b smithi046 running (8m) 45s ago 8m 40.9M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 b88513ffbf0a 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c smithi162 running (8m) 45s ago 8m 41.1M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 78b2674133b1 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (8m) 23s ago 8m 43.7M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 4a56c3de6357 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi046 running (7m) 45s ago 7m 44.9M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 d87c1be6ced8 2024-04-20T10:33:24.370 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi162 running (7m) 45s ago 7m 57.6M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 668860aca425 2024-04-20T10:33:24.371 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi042 running (6m) 23s ago 6m 42.0M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 835509bbc7e7 2024-04-20T10:33:24.371 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi046 running (6m) 45s ago 6m 42.2M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 656aab974eae 2024-04-20T10:33:24.371 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi162 running (5m) 45s ago 5m 53.4M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 f39a63c56810 2024-04-20T10:33:24.371 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 smithi042 running (5m) 23s ago 5m 40.7M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 28c45975bacf 2024-04-20T10:33:24.371 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 smithi046 running (4m) 45s ago 4m 54.6M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 3d5ff6106cdd 2024-04-20T10:33:24.371 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 smithi162 running (4m) 45s ago 4m 54.2M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 9864be425829 2024-04-20T10:33:24.371 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 smithi042 running (3m) 23s ago 3m 52.1M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 9528fd875ce1 2024-04-20T10:33:24.371 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 smithi046 running (3m) 45s ago 3m 55.0M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 7ceaef752129 2024-04-20T10:33:24.371 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 smithi162 running (2m) 45s ago 2m 43.2M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 d446a051b90f 2024-04-20T10:33:24.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.371+0000 7f8ede000700 1 -- 172.21.15.42:0/2861715398 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8ebc0a3cc0 msgr2=0x7f8ebc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:24.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.371+0000 7f8ede000700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8ebc0a3cc0 0x7f8ebc0a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0005950 tx=0x7f8ec000b410 comp rx=0 tx=0).stop 2024-04-20T10:33:24.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.371+0000 7f8ede000700 1 -- 172.21.15.42:0/2861715398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8103540 msgr2=0x7f8ed8102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:24.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.371+0000 7f8ede000700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8103540 0x7f8ed8102e50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8003890 tx=0x7f8ec80047b0 comp rx=0 tx=0).stop 2024-04-20T10:33:24.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.372+0000 7f8ede000700 1 -- 172.21.15.42:0/2861715398 shutdown_connections 2024-04-20T10:33:24.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.372+0000 7f8ede000700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f8ebc0a3cc0 0x7f8ebc0a6180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:24.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.372+0000 7f8ede000700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8103540 0x7f8ed8102e50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:24.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.372+0000 7f8ede000700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed81087a0 0x7f8ed8101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:24.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.372+0000 7f8ede000700 1 --2- 172.21.15.42:0/2861715398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ed8105e70 0x7f8ed81014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:24.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.372+0000 7f8ede000700 1 -- 172.21.15.42:0/2861715398 >> 172.21.15.42:0/2861715398 conn(0x7f8ed80fcf70 msgr2=0x7f8ed8103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:24.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.372+0000 7f8ede000700 1 -- 172.21.15.42:0/2861715398 shutdown_connections 2024-04-20T10:33:24.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:24.372+0000 7f8ede000700 1 -- 172.21.15.42:0/2861715398 wait complete. 2024-04-20T10:33:24.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:24 smithi046 ceph-mon[21574]: pgmap v373: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:24.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:24 smithi162 ceph-mon[21854]: pgmap v373: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:25.146 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-20T10:33:25.577 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:33:26.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:26 smithi042 ceph-mon[17078]: from='client.25001 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:26.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:26 smithi042 ceph-mon[17078]: pgmap v374: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:26.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:26 smithi046 ceph-mon[21574]: from='client.25001 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:26.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:26 smithi046 ceph-mon[21574]: pgmap v374: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:26.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:26 smithi162 ceph-mon[21854]: from='client.25001 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:26.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:26 smithi162 ceph-mon[21854]: pgmap v374: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:27.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.787+0000 7f373a600700 1 -- 172.21.15.42:0/4040621295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f373410a300 msgr2=0x7f373410a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:27.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.787+0000 7f373a600700 1 --2- 172.21.15.42:0/4040621295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f373410a300 0x7f373410a760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f371c009920 tx=0x7f371c009c30 comp rx=0 tx=0).stop 2024-04-20T10:33:27.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.787+0000 7f373a600700 1 -- 172.21.15.42:0/4040621295 shutdown_connections 2024-04-20T10:33:27.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.787+0000 7f373a600700 1 --2- 172.21.15.42:0/4040621295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f373410aca0 0x7f3734111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:27.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.787+0000 7f373a600700 1 --2- 172.21.15.42:0/4040621295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3734109100 0x7f3734109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:27.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.787+0000 7f373a600700 1 --2- 172.21.15.42:0/4040621295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f373410a300 0x7f373410a760 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:27.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.787+0000 7f373a600700 1 -- 172.21.15.42:0/4040621295 >> 172.21.15.42:0/4040621295 conn(0x7f37340780d0 msgr2=0x7f373407a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:27.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.787+0000 7f373a600700 1 -- 172.21.15.42:0/4040621295 shutdown_connections 2024-04-20T10:33:27.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.787+0000 7f373a600700 1 -- 172.21.15.42:0/4040621295 wait complete. 2024-04-20T10:33:27.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.788+0000 7f373a600700 1 Processor -- start 2024-04-20T10:33:27.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.789+0000 7f373a600700 1 -- start start 2024-04-20T10:33:27.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.789+0000 7f373a600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734109100 0x7f373419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:27.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.789+0000 7f373a600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f373410a300 0x7f373419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:27.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.789+0000 7f373a600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373410aca0 0x7f37341a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.789+0000 7f373a600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3734114d20 con 0x7f3734109100 2024-04-20T10:33:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.789+0000 7f373a600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3734114ba0 con 0x7f373410a300 2024-04-20T10:33:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.789+0000 7f373a600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3734114ea0 con 0x7f373410aca0 2024-04-20T10:33:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.789+0000 7f3733400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734109100 0x7f373419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.789+0000 7f3733400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734109100 0x7f373419f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50488/0 (socket says 172.21.15.42:50488) 2024-04-20T10:33:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.789+0000 7f3733400700 1 -- 172.21.15.42:0/879015607 learned_addr learned my addr 172.21.15.42:0/879015607 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.789+0000 7f3733e00700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373410aca0 0x7f37341a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.789+0000 7f372aa00700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f373410a300 0x7f373419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:27.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.790+0000 7f3733400700 1 -- 172.21.15.42:0/879015607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373410aca0 msgr2=0x7f37341a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.790+0000 7f3733400700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373410aca0 0x7f37341a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.790+0000 7f3733400700 1 -- 172.21.15.42:0/879015607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f373410a300 msgr2=0x7f373419f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.790+0000 7f3733400700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f373410a300 0x7f373419f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.790+0000 7f3733400700 1 -- 172.21.15.42:0/879015607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37341a7050 con 0x7f3734109100 2024-04-20T10:33:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.790+0000 7f3733e00700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373410aca0 0x7f37341a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.790+0000 7f3733400700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734109100 0x7f373419f3f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f372400e9a0 tx=0x7f372400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.790+0000 7f3730c00700 1 -- 172.21.15.42:0/879015607 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724004220 con 0x7f3734109100 2024-04-20T10:33:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.790+0000 7f3730c00700 1 -- 172.21.15.42:0/879015607 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37240043c0 con 0x7f3734109100 2024-04-20T10:33:27.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.790+0000 7f373a600700 1 -- 172.21.15.42:0/879015607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37341a7340 con 0x7f3734109100 2024-04-20T10:33:27.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.791+0000 7f3730c00700 1 -- 172.21.15.42:0/879015607 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724003df0 con 0x7f3734109100 2024-04-20T10:33:27.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.791+0000 7f373a600700 1 -- 172.21.15.42:0/879015607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37341a7880 con 0x7f3734109100 2024-04-20T10:33:27.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.792+0000 7f373a600700 1 -- 172.21.15.42:0/879015607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3734069740 con 0x7f3734109100 2024-04-20T10:33:27.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.794+0000 7f3730c00700 1 -- 172.21.15.42:0/879015607 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f372400c810 con 0x7f3734109100 2024-04-20T10:33:27.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.795+0000 7f3730c00700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f37180a3c40 0x7f37180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:27.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.796+0000 7f3730c00700 1 -- 172.21.15.42:0/879015607 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f37240d0710 con 0x7f3734109100 2024-04-20T10:33:27.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.796+0000 7f372aa00700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f37180a3c40 0x7f37180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:27.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.797+0000 7f372aa00700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f37180a3c40 0x7f37180a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f371c00fca0 tx=0x7f371c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:27.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:27.799+0000 7f3730c00700 1 -- 172.21.15.42:0/879015607 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f372409d100 con 0x7f3734109100 2024-04-20T10:33:28.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.044+0000 7f373a600700 1 -- 172.21.15.42:0/879015607 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3734062e90 con 0x7f37180a3c40 2024-04-20T10:33:28.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.054+0000 7f3730c00700 1 -- 172.21.15.42:0/879015607 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3734062e90 con 0x7f37180a3c40 2024-04-20T10:33:28.055 INFO:teuthology.orchestra.run.smithi042.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-20T10:33:28.056 INFO:teuthology.orchestra.run.smithi042.stdout:mds.all 12/12 49s ago 2m smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi046=b;smithi046=e;smithi046=h;smithi046=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l;count:12 2024-04-20T10:33:28.056 INFO:teuthology.orchestra.run.smithi042.stdout:mgr 3/3 49s ago 8m smithi042=x;smithi046=y;smithi162=z;count:3 2024-04-20T10:33:28.056 INFO:teuthology.orchestra.run.smithi042.stdout:mon 3/3 49s ago 9m smithi042:172.21.15.42=a;smithi046:172.21.15.46=b;smithi162:172.21.15.162=c;count:3 2024-04-20T10:33:28.056 INFO:teuthology.orchestra.run.smithi042.stdout:osd 12 49s ago - 2024-04-20T10:33:28.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.057+0000 7f373a600700 1 -- 172.21.15.42:0/879015607 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f37180a3c40 msgr2=0x7f37180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:28.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.057+0000 7f373a600700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f37180a3c40 0x7f37180a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f371c00fca0 tx=0x7f371c019040 comp rx=0 tx=0).stop 2024-04-20T10:33:28.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.057+0000 7f373a600700 1 -- 172.21.15.42:0/879015607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734109100 msgr2=0x7f373419f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:28.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.057+0000 7f373a600700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734109100 0x7f373419f3f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f372400e9a0 tx=0x7f372400ecb0 comp rx=0 tx=0).stop 2024-04-20T10:33:28.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.057+0000 7f373a600700 1 -- 172.21.15.42:0/879015607 shutdown_connections 2024-04-20T10:33:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.057+0000 7f373a600700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f37180a3c40 0x7f37180a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.057+0000 7f373a600700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f373410aca0 0x7f37341a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.057+0000 7f373a600700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734109100 0x7f373419f3f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.057+0000 7f373a600700 1 --2- 172.21.15.42:0/879015607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f373410a300 0x7f373419f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.057+0000 7f373a600700 1 -- 172.21.15.42:0/879015607 >> 172.21.15.42:0/879015607 conn(0x7f37340780d0 msgr2=0x7f373410d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:28.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.057+0000 7f373a600700 1 -- 172.21.15.42:0/879015607 shutdown_connections 2024-04-20T10:33:28.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:28.057+0000 7f373a600700 1 -- 172.21.15.42:0/879015607 wait complete. 2024-04-20T10:33:28.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:28 smithi042 ceph-mon[17078]: pgmap v375: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:28.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:28 smithi046 ceph-mon[21574]: pgmap v375: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:28.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:28 smithi162 ceph-mon[21854]: pgmap v375: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:28.862 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-20T10:33:29.337 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:33:29.521 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:29 smithi042 ceph-mon[17078]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:29.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:29 smithi046 ceph-mon[21574]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:29.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:29 smithi162 ceph-mon[21854]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:30.534 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:30 smithi042 ceph-mon[17078]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:30.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:30 smithi046 ceph-mon[21574]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:30.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:30 smithi162 ceph-mon[21854]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:31.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.541+0000 7efd71c00700 1 -- 172.21.15.42:0/3659152235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c105ef0 msgr2=0x7efd6c106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:31.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.541+0000 7efd71c00700 1 --2- 172.21.15.42:0/3659152235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c105ef0 0x7efd6c106350 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efd58009920 tx=0x7efd58009c30 comp rx=0 tx=0).stop 2024-04-20T10:33:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.541+0000 7efd71c00700 1 -- 172.21.15.42:0/3659152235 shutdown_connections 2024-04-20T10:33:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.541+0000 7efd71c00700 1 --2- 172.21.15.42:0/3659152235 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd6c106890 0x7efd6c10d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.541+0000 7efd71c00700 1 --2- 172.21.15.42:0/3659152235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd6c104cf0 0x7efd6c105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.541+0000 7efd71c00700 1 --2- 172.21.15.42:0/3659152235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c105ef0 0x7efd6c106350 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.541+0000 7efd71c00700 1 -- 172.21.15.42:0/3659152235 >> 172.21.15.42:0/3659152235 conn(0x7efd6c1002b0 msgr2=0x7efd6c1026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.542+0000 7efd71c00700 1 -- 172.21.15.42:0/3659152235 shutdown_connections 2024-04-20T10:33:31.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.542+0000 7efd71c00700 1 -- 172.21.15.42:0/3659152235 wait complete. 2024-04-20T10:33:31.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.543+0000 7efd71c00700 1 Processor -- start 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.543+0000 7efd71c00700 1 -- start start 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.543+0000 7efd71c00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c104cf0 0x7efd6c19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.543+0000 7efd71c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd6c105ef0 0x7efd6c19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.543+0000 7efd71c00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd6c106890 0x7efd6c1a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.543+0000 7efd71c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c110900 con 0x7efd6c105ef0 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.543+0000 7efd71c00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c110780 con 0x7efd6c104cf0 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.543+0000 7efd71c00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c110a80 con 0x7efd6c106890 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.543+0000 7efd6a000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd6c105ef0 0x7efd6c19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.543+0000 7efd6a000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd6c105ef0 0x7efd6c19b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42690/0 (socket says 172.21.15.42:42690) 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.543+0000 7efd6a000700 1 -- 172.21.15.42:0/2260813451 learned_addr learned my addr 172.21.15.42:0/2260813451 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.544+0000 7efd6aa00700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c104cf0 0x7efd6c19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.544+0000 7efd6b400700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd6c106890 0x7efd6c1a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.544+0000 7efd6a000700 1 -- 172.21.15.42:0/2260813451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd6c106890 msgr2=0x7efd6c1a2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:31.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.544+0000 7efd6a000700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd6c106890 0x7efd6c1a2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:31.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.544+0000 7efd6a000700 1 -- 172.21.15.42:0/2260813451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c104cf0 msgr2=0x7efd6c19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:31.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.544+0000 7efd6a000700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c104cf0 0x7efd6c19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:31.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.544+0000 7efd6a000700 1 -- 172.21.15.42:0/2260813451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c1a2bd0 con 0x7efd6c105ef0 2024-04-20T10:33:31.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.544+0000 7efd6b400700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd6c106890 0x7efd6c1a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:31.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.544+0000 7efd6a000700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd6c105ef0 0x7efd6c19b500 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efd58009230 tx=0x7efd58005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:31.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.545+0000 7efd63400700 1 -- 172.21.15.42:0/2260813451 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58005470 con 0x7efd6c105ef0 2024-04-20T10:33:31.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.545+0000 7efd63400700 1 -- 172.21.15.42:0/2260813451 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd58005610 con 0x7efd6c105ef0 2024-04-20T10:33:31.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.545+0000 7efd71c00700 1 -- 172.21.15.42:0/2260813451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6c1a2e60 con 0x7efd6c105ef0 2024-04-20T10:33:31.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.545+0000 7efd63400700 1 -- 172.21.15.42:0/2260813451 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58003c30 con 0x7efd6c105ef0 2024-04-20T10:33:31.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.545+0000 7efd71c00700 1 -- 172.21.15.42:0/2260813451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6c1a3250 con 0x7efd6c105ef0 2024-04-20T10:33:31.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.546+0000 7efd71c00700 1 -- 172.21.15.42:0/2260813451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd6c069740 con 0x7efd6c105ef0 2024-04-20T10:33:31.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.548+0000 7efd63400700 1 -- 172.21.15.42:0/2260813451 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efd5800b560 con 0x7efd6c105ef0 2024-04-20T10:33:31.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.550+0000 7efd63400700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7efd500a3cc0 0x7efd500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:31.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.550+0000 7efd63400700 1 -- 172.21.15.42:0/2260813451 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7efd580d1460 con 0x7efd6c105ef0 2024-04-20T10:33:31.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.550+0000 7efd6aa00700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7efd500a3cc0 0x7efd500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:31.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.551+0000 7efd6aa00700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7efd500a3cc0 0x7efd500a6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efd6c105d50 tx=0x7efd5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:31.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.554+0000 7efd63400700 1 -- 172.21.15.42:0/2260813451 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efd58013030 con 0x7efd6c105ef0 2024-04-20T10:33:31.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.791+0000 7efd71c00700 1 -- 172.21.15.42:0/2260813451 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7efd6c062e90 con 0x7efd500a3cc0 2024-04-20T10:33:31.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.795+0000 7efd63400700 1 -- 172.21.15.42:0/2260813451 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7efd6c062e90 con 0x7efd500a3cc0 2024-04-20T10:33:31.795 INFO:teuthology.orchestra.run.smithi042.stdout:HOST ADDR LABELS STATUS 2024-04-20T10:33:31.796 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 172.21.15.42 2024-04-20T10:33:31.796 INFO:teuthology.orchestra.run.smithi042.stdout:smithi046 172.21.15.46 2024-04-20T10:33:31.796 INFO:teuthology.orchestra.run.smithi042.stdout:smithi162 172.21.15.162 2024-04-20T10:33:31.796 INFO:teuthology.orchestra.run.smithi042.stdout:3 hosts in cluster 2024-04-20T10:33:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.799+0000 7efd71c00700 1 -- 172.21.15.42:0/2260813451 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7efd500a3cc0 msgr2=0x7efd500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.799+0000 7efd71c00700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7efd500a3cc0 0x7efd500a6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efd6c105d50 tx=0x7efd5c00b410 comp rx=0 tx=0).stop 2024-04-20T10:33:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.799+0000 7efd71c00700 1 -- 172.21.15.42:0/2260813451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd6c105ef0 msgr2=0x7efd6c19b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.799+0000 7efd71c00700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd6c105ef0 0x7efd6c19b500 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efd58009230 tx=0x7efd58005250 comp rx=0 tx=0).stop 2024-04-20T10:33:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.799+0000 7efd71c00700 1 -- 172.21.15.42:0/2260813451 shutdown_connections 2024-04-20T10:33:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.799+0000 7efd71c00700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7efd500a3cc0 0x7efd500a6180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.799+0000 7efd71c00700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c104cf0 0x7efd6c19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.799+0000 7efd71c00700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd6c106890 0x7efd6c1a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.799+0000 7efd71c00700 1 --2- 172.21.15.42:0/2260813451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd6c105ef0 0x7efd6c19b500 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.799+0000 7efd71c00700 1 -- 172.21.15.42:0/2260813451 >> 172.21.15.42:0/2260813451 conn(0x7efd6c1002b0 msgr2=0x7efd6c1091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.799+0000 7efd71c00700 1 -- 172.21.15.42:0/2260813451 shutdown_connections 2024-04-20T10:33:31.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:31.799+0000 7efd71c00700 1 -- 172.21.15.42:0/2260813451 wait complete. 2024-04-20T10:33:32.530 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:32 smithi042 ceph-mon[17078]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:32.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:32 smithi046 ceph-mon[21574]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:32.598 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-20T10:33:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:32 smithi162 ceph-mon[21854]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:33.062 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:33:33.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:33 smithi046 ceph-mon[21574]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:33.567 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:33 smithi042 ceph-mon[17078]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:33.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:33 smithi162 ceph-mon[21854]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:34.463 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:34 smithi042 ceph-mon[17078]: pgmap v378: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:34.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:34 smithi046 ceph-mon[21574]: pgmap v378: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:34.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:34 smithi162 ceph-mon[21854]: pgmap v378: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:35.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.323+0000 7f4c55600700 1 -- 172.21.15.42:0/1582136323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c501090f0 msgr2=0x7f4c50109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:35.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.323+0000 7f4c55600700 1 --2- 172.21.15.42:0/1582136323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c501090f0 0x7f4c50109510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4c400099c0 tx=0x7f4c40009cd0 comp rx=0 tx=0).stop 2024-04-20T10:33:35.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.323+0000 7f4c55600700 1 -- 172.21.15.42:0/1582136323 shutdown_connections 2024-04-20T10:33:35.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.323+0000 7f4c55600700 1 --2- 172.21.15.42:0/1582136323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c5010ac90 0x7f4c50111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:35.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.323+0000 7f4c55600700 1 --2- 172.21.15.42:0/1582136323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c501090f0 0x7f4c50109510 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:35.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.323+0000 7f4c55600700 1 --2- 172.21.15.42:0/1582136323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c5010a2f0 0x7f4c5010a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:35.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.323+0000 7f4c55600700 1 -- 172.21.15.42:0/1582136323 >> 172.21.15.42:0/1582136323 conn(0x7f4c500780c0 msgr2=0x7f4c5007a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:35.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.323+0000 7f4c55600700 1 -- 172.21.15.42:0/1582136323 shutdown_connections 2024-04-20T10:33:35.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.323+0000 7f4c55600700 1 -- 172.21.15.42:0/1582136323 wait complete. 2024-04-20T10:33:35.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.325+0000 7f4c55600700 1 Processor -- start 2024-04-20T10:33:35.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.325+0000 7f4c55600700 1 -- start start 2024-04-20T10:33:35.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.325+0000 7f4c55600700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c501090f0 0x7f4c5019f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:35.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.325+0000 7f4c55600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c5010a2f0 0x7f4c5019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:35.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.325+0000 7f4c55600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c5010ac90 0x7f4c501a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:35.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.325+0000 7f4c55600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c50114d00 con 0x7f4c5010a2f0 2024-04-20T10:33:35.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.325+0000 7f4c55600700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c50114b80 con 0x7f4c5010ac90 2024-04-20T10:33:35.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.325+0000 7f4c55600700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4c50114e80 con 0x7f4c501090f0 2024-04-20T10:33:35.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.326+0000 7f4c47e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c5010a2f0 0x7f4c5019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:35.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.326+0000 7f4c4ea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c501090f0 0x7f4c5019f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:35.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.326+0000 7f4c4ea00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c501090f0 0x7f4c5019f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:58036/0 (socket says 172.21.15.42:58036) 2024-04-20T10:33:35.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.326+0000 7f4c4ea00700 1 -- 172.21.15.42:0/48638112 learned_addr learned my addr 172.21.15.42:0/48638112 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:35.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.326+0000 7f4c47e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c5010a2f0 0x7f4c5019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42718/0 (socket says 172.21.15.42:42718) 2024-04-20T10:33:35.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.326+0000 7f4c4f400700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c5010ac90 0x7f4c501a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:35.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.326+0000 7f4c4ea00700 1 -- 172.21.15.42:0/48638112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c5010ac90 msgr2=0x7f4c501a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:35.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.326+0000 7f4c4ea00700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c5010ac90 0x7f4c501a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:35.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.326+0000 7f4c4ea00700 1 -- 172.21.15.42:0/48638112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c5010a2f0 msgr2=0x7f4c5019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:35.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.326+0000 7f4c4ea00700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c5010a2f0 0x7f4c5019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:35.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.326+0000 7f4c4ea00700 1 -- 172.21.15.42:0/48638112 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c501a6f50 con 0x7f4c501090f0 2024-04-20T10:33:35.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.326+0000 7f4c47e00700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c5010a2f0 0x7f4c5019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:35.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.327+0000 7f4c4ea00700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c501090f0 0x7f4c5019f2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4c400057c0 tx=0x7f4c40009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:35.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.327+0000 7f4c47400700 1 -- 172.21.15.42:0/48638112 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40027070 con 0x7f4c501090f0 2024-04-20T10:33:35.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.327+0000 7f4c55600700 1 -- 172.21.15.42:0/48638112 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c501a71e0 con 0x7f4c501090f0 2024-04-20T10:33:35.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.327+0000 7f4c47400700 1 -- 172.21.15.42:0/48638112 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c40004150 con 0x7f4c501090f0 2024-04-20T10:33:35.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.327+0000 7f4c55600700 1 -- 172.21.15.42:0/48638112 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c501a7690 con 0x7f4c501090f0 2024-04-20T10:33:35.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.327+0000 7f4c47400700 1 -- 172.21.15.42:0/48638112 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40022da0 con 0x7f4c501090f0 2024-04-20T10:33:35.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.328+0000 7f4c55600700 1 -- 172.21.15.42:0/48638112 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c50069740 con 0x7f4c501090f0 2024-04-20T10:33:35.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.330+0000 7f4c47400700 1 -- 172.21.15.42:0/48638112 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4c4002a070 con 0x7f4c501090f0 2024-04-20T10:33:35.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.331+0000 7f4c47400700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4c340a3c40 0x7f4c340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:35.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.332+0000 7f4c47400700 1 -- 172.21.15.42:0/48638112 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f4c400d10e0 con 0x7f4c501090f0 2024-04-20T10:33:35.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.332+0000 7f4c47e00700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4c340a3c40 0x7f4c340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:35.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.332+0000 7f4c47e00700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4c340a3c40 0x7f4c340a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4c38010420 tx=0x7f4c380073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:35.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.336+0000 7f4c47400700 1 -- 172.21.15.42:0/48638112 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4c4005eff0 con 0x7f4c501090f0 2024-04-20T10:33:35.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:35 smithi046 ceph-mon[21574]: pgmap v379: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.579+0000 7f4c55600700 1 -- 172.21.15.42:0/48638112 --> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f4c50062e90 con 0x7f4c340a3c40 2024-04-20T10:33:35.580 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:35 smithi042 ceph-mon[17078]: pgmap v379: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:35.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.584+0000 7f4c47400700 1 -- 172.21.15.42:0/48638112 <== mgr.14152 v2:172.21.15.42:6800/3610568749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f4c50062e90 con 0x7f4c340a3c40 2024-04-20T10:33:35.584 INFO:teuthology.orchestra.run.smithi042.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-20T10:33:35.584 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130021480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-20T10:33:35.585 INFO:teuthology.orchestra.run.smithi042.stdout:smithi046 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7512009A480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-20T10:33:35.585 INFO:teuthology.orchestra.run.smithi042.stdout:smithi162 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JU400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-20T10:33:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.586+0000 7f4c55600700 1 -- 172.21.15.42:0/48638112 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4c340a3c40 msgr2=0x7f4c340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.586+0000 7f4c55600700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4c340a3c40 0x7f4c340a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4c38010420 tx=0x7f4c380073d0 comp rx=0 tx=0).stop 2024-04-20T10:33:35.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.586+0000 7f4c55600700 1 -- 172.21.15.42:0/48638112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c501090f0 msgr2=0x7f4c5019f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:35.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.586+0000 7f4c55600700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c501090f0 0x7f4c5019f2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4c400057c0 tx=0x7f4c40009580 comp rx=0 tx=0).stop 2024-04-20T10:33:35.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.586+0000 7f4c55600700 1 -- 172.21.15.42:0/48638112 shutdown_connections 2024-04-20T10:33:35.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.586+0000 7f4c55600700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4c340a3c40 0x7f4c340a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:35.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.587+0000 7f4c55600700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c5010ac90 0x7f4c501a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:35.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.587+0000 7f4c55600700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c501090f0 0x7f4c5019f2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:35.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.587+0000 7f4c55600700 1 --2- 172.21.15.42:0/48638112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c5010a2f0 0x7f4c5019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:35.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.587+0000 7f4c55600700 1 -- 172.21.15.42:0/48638112 >> 172.21.15.42:0/48638112 conn(0x7f4c500780c0 msgr2=0x7f4c5010d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:35.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.587+0000 7f4c55600700 1 -- 172.21.15.42:0/48638112 shutdown_connections 2024-04-20T10:33:35.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:35.587+0000 7f4c55600700 1 -- 172.21.15.42:0/48638112 wait complete. 2024-04-20T10:33:35.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:35 smithi162 ceph-mon[21854]: pgmap v379: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:36.379 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T10:33:36.388 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-04-20T10:33:36.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-20T10:33:36.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:36 smithi046 ceph-mon[21574]: from='client.25036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:36.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:36 smithi042 ceph-mon[17078]: from='client.25036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:36.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:36 smithi162 ceph-mon[21854]: from='client.25036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:33:36.881 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:33:37.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:37 smithi042 ceph-mon[17078]: pgmap v380: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:37.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:37 smithi162 ceph-mon[21854]: pgmap v380: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:37.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:37 smithi046 ceph-mon[21574]: pgmap v380: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.995+0000 7fe15be00700 1 -- 172.21.15.42:0/2633359682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe15c104bf0 msgr2=0x7fe15c105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.995+0000 7fe15be00700 1 --2- 172.21.15.42:0/2633359682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe15c104bf0 0x7fe15c105010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0099c0 tx=0x7fe14c009cd0 comp rx=0 tx=0).stop 2024-04-20T10:33:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.995+0000 7fe15be00700 1 -- 172.21.15.42:0/2633359682 shutdown_connections 2024-04-20T10:33:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.995+0000 7fe15be00700 1 --2- 172.21.15.42:0/2633359682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe15c106790 0x7fe15c10d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.995+0000 7fe15be00700 1 --2- 172.21.15.42:0/2633359682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe15c104bf0 0x7fe15c105010 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.995+0000 7fe15be00700 1 --2- 172.21.15.42:0/2633359682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c105df0 0x7fe15c106250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.995+0000 7fe15be00700 1 -- 172.21.15.42:0/2633359682 >> 172.21.15.42:0/2633359682 conn(0x7fe15c100190 msgr2=0x7fe15c1025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.995+0000 7fe15be00700 1 -- 172.21.15.42:0/2633359682 shutdown_connections 2024-04-20T10:33:38.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.995+0000 7fe15be00700 1 -- 172.21.15.42:0/2633359682 wait complete. 2024-04-20T10:33:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.997+0000 7fe15be00700 1 Processor -- start 2024-04-20T10:33:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.997+0000 7fe15be00700 1 -- start start 2024-04-20T10:33:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.997+0000 7fe15be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe15c104bf0 0x7fe15c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.997+0000 7fe15be00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe15c105df0 0x7fe15c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:38.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.997+0000 7fe15be00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c1a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.997+0000 7fe15be00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c110800 con 0x7fe15c104bf0 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.997+0000 7fe15be00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c110680 con 0x7fe15c106790 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.997+0000 7fe15be00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c110980 con 0x7fe15c105df0 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.997+0000 7fe15aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe15c104bf0 0x7fe15c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.997+0000 7fe15aa00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe15c104bf0 0x7fe15c19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59726/0 (socket says 172.21.15.42:59726) 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.998+0000 7fe15aa00700 1 -- 172.21.15.42:0/943579220 learned_addr learned my addr 172.21.15.42:0/943579220 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.998+0000 7fe15a000700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe15c105df0 0x7fe15c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.998+0000 7fe15b400700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c1a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.998+0000 7fe15a000700 1 -- 172.21.15.42:0/943579220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 msgr2=0x7fe15c1a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.998+0000 7fe15a000700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c1a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.998+0000 7fe15a000700 1 -- 172.21.15.42:0/943579220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe15c104bf0 msgr2=0x7fe15c19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.998+0000 7fe15a000700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe15c104bf0 0x7fe15c19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.998+0000 7fe15a000700 1 -- 172.21.15.42:0/943579220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c1a2aa0 con 0x7fe15c105df0 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.998+0000 7fe15b400700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c1a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:39.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.998+0000 7fe15aa00700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe15c104bf0 0x7fe15c19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:39.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.998+0000 7fe15a000700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe15c105df0 0x7fe15c19b380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe14800ea20 tx=0x7fe14800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:39.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.999+0000 7fe153400700 1 -- 172.21.15.42:0/943579220 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14800cbd0 con 0x7fe15c105df0 2024-04-20T10:33:39.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.999+0000 7fe153400700 1 -- 172.21.15.42:0/943579220 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe14800ee50 con 0x7fe15c105df0 2024-04-20T10:33:39.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.999+0000 7fe153400700 1 -- 172.21.15.42:0/943579220 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148018660 con 0x7fe15c105df0 2024-04-20T10:33:39.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.999+0000 7fe15be00700 1 -- 172.21.15.42:0/943579220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15c1a2d90 con 0x7fe15c105df0 2024-04-20T10:33:39.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:38.999+0000 7fe15be00700 1 -- 172.21.15.42:0/943579220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe15c1a3250 con 0x7fe15c105df0 2024-04-20T10:33:39.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.000+0000 7fe15be00700 1 -- 172.21.15.42:0/943579220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe15c069740 con 0x7fe15c105df0 2024-04-20T10:33:39.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.002+0000 7fe153400700 1 -- 172.21.15.42:0/943579220 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe148010430 con 0x7fe15c105df0 2024-04-20T10:33:39.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.003+0000 7fe153400700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe1400a3cc0 0x7fe1400a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:39.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.004+0000 7fe153400700 1 -- 172.21.15.42:0/943579220 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe1480d0470 con 0x7fe15c105df0 2024-04-20T10:33:39.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.004+0000 7fe15aa00700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe1400a3cc0 0x7fe1400a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:39.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.004+0000 7fe15aa00700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe1400a3cc0 0x7fe1400a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0038c0 tx=0x7fe14c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.008+0000 7fe153400700 1 -- 172.21.15.42:0/943579220 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe14809cdd0 con 0x7fe15c105df0 2024-04-20T10:33:39.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.287+0000 7fe15be00700 1 -- 172.21.15.42:0/943579220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe15c063660 con 0x7fe15c105df0 2024-04-20T10:33:39.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.288+0000 7fe153400700 1 -- 172.21.15.42:0/943579220 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+3023 (secure 0 0 0) 0x7fe14809c6d0 con 0x7fe15c105df0 2024-04-20T10:33:39.290 INFO:teuthology.orchestra.run.smithi042.stdout:e19 2024-04-20T10:33:39.290 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 19 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-04-20T10:31:35.067527+0000 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-04-20T10:31:39.263696+0000 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 300 2024-04-20T10:33:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:snap_rstat 1 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24584} 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-04-20T10:33:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 0 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 24584 members: 24584 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.b{0:24584} state up:active seq 3 addr [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.i{0:24568} state up:standby-replay seq 1 addr [v2:172.21.15.162:6838/3710674499,v1:172.21.15.162:6839/3710674499] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.g{-1:14454} state up:standby seq 1 addr [v2:172.21.15.42:6838/2523412756,v1:172.21.15.42:6839/2523412756] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T10:33:39.293 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.a{-1:14472} state up:standby seq 1 addr [v2:172.21.15.42:6840/3615312070,v1:172.21.15.42:6841/3615312070] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T10:33:39.294 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.k{-1:24449} state up:standby seq 1 addr [v2:172.21.15.46:6832/1854750771,v1:172.21.15.46:6833/1854750771] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T10:33:39.294 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.l{-1:24463} state up:standby seq 1 addr [v2:172.21.15.162:6832/4094679184,v1:172.21.15.162:6833/4094679184] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T10:33:39.294 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.j{-1:24487} state up:standby seq 1 addr [v2:172.21.15.42:6834/2947757102,v1:172.21.15.42:6835/2947757102] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T10:33:39.294 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.e{-1:24494} state up:standby seq 1 addr [v2:172.21.15.46:6834/4194525549,v1:172.21.15.46:6835/4194525549] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T10:33:39.294 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.d{-1:24505} state up:standby seq 1 addr [v2:172.21.15.42:6836/2898732601,v1:172.21.15.42:6837/2898732601] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T10:33:39.294 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.h{-1:24524} state up:standby seq 1 addr [v2:172.21.15.46:6836/3787981189,v1:172.21.15.46:6837/3787981189] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T10:33:39.294 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.f{-1:24526} state up:standby seq 1 addr [v2:172.21.15.162:6834/2486205910,v1:172.21.15.162:6835/2486205910] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T10:33:39.294 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.c{-1:24538} state up:standby seq 1 addr [v2:172.21.15.162:6836/2056054864,v1:172.21.15.162:6837/2056054864] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T10:33:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.291+0000 7fe15be00700 1 -- 172.21.15.42:0/943579220 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe1400a3cc0 msgr2=0x7fe1400a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.291+0000 7fe15be00700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe1400a3cc0 0x7fe1400a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0038c0 tx=0x7fe14c005fb0 comp rx=0 tx=0).stop 2024-04-20T10:33:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.291+0000 7fe15be00700 1 -- 172.21.15.42:0/943579220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe15c105df0 msgr2=0x7fe15c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.291+0000 7fe15be00700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe15c105df0 0x7fe15c19b380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe14800ea20 tx=0x7fe14800ed30 comp rx=0 tx=0).stop 2024-04-20T10:33:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.292+0000 7fe15be00700 1 -- 172.21.15.42:0/943579220 shutdown_connections 2024-04-20T10:33:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.292+0000 7fe15be00700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fe1400a3cc0 0x7fe1400a6180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.292+0000 7fe15be00700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe15c106790 0x7fe15c1a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.292+0000 7fe15be00700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe15c104bf0 0x7fe15c19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.292+0000 7fe15be00700 1 --2- 172.21.15.42:0/943579220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe15c105df0 0x7fe15c19b380 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.292+0000 7fe15be00700 1 -- 172.21.15.42:0/943579220 >> 172.21.15.42:0/943579220 conn(0x7fe15c100190 msgr2=0x7fe15c109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.292+0000 7fe15be00700 1 -- 172.21.15.42:0/943579220 shutdown_connections 2024-04-20T10:33:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:39.292+0000 7fe15be00700 1 -- 172.21.15.42:0/943579220 wait complete. 2024-04-20T10:33:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:33:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:39 smithi162 ceph-mon[21854]: pgmap v381: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:39.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:39 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/943579220' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T10:33:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:39 smithi046 ceph-mon[21574]: pgmap v381: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:39 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/943579220' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T10:33:40.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:39 smithi042 ceph-mon[17078]: pgmap v381: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:40.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:39 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/943579220' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T10:33:40.210 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-20T10:33:40.665 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:33:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:41 smithi042 ceph-mon[17078]: pgmap v382: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:41 smithi162 ceph-mon[21854]: pgmap v382: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:41 smithi046 ceph-mon[21574]: pgmap v382: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:42.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.767+0000 7f81c5a00700 1 -- 172.21.15.42:0/4191612435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81c0103540 msgr2=0x7f81c0105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:42.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.767+0000 7f81c5a00700 1 --2- 172.21.15.42:0/4191612435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81c0103540 0x7f81c0105930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f81b000b0a0 tx=0x7f81b000b3b0 comp rx=0 tx=0).stop 2024-04-20T10:33:42.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.767+0000 7f81c5a00700 1 -- 172.21.15.42:0/4191612435 shutdown_connections 2024-04-20T10:33:42.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.767+0000 7f81c5a00700 1 --2- 172.21.15.42:0/4191612435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c01087a0 0x7f81c010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:42.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.767+0000 7f81c5a00700 1 --2- 172.21.15.42:0/4191612435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81c0103540 0x7f81c0105930 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:42.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.767+0000 7f81c5a00700 1 --2- 172.21.15.42:0/4191612435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81c0105e70 0x7f81c0108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:42.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.767+0000 7f81c5a00700 1 -- 172.21.15.42:0/4191612435 >> 172.21.15.42:0/4191612435 conn(0x7f81c00fcf50 msgr2=0x7f81c00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:42.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.767+0000 7f81c5a00700 1 -- 172.21.15.42:0/4191612435 shutdown_connections 2024-04-20T10:33:42.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.767+0000 7f81c5a00700 1 -- 172.21.15.42:0/4191612435 wait complete. 2024-04-20T10:33:42.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.768+0000 7f81c5a00700 1 Processor -- start 2024-04-20T10:33:42.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.768+0000 7f81c5a00700 1 -- start start 2024-04-20T10:33:42.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.769+0000 7f81c5a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0103540 0x7f81c019af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:42.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.769+0000 7f81c5a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81c0105e70 0x7f81c019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:42.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.769+0000 7f81c5a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81c01087a0 0x7f81c01a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:42.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.769+0000 7f81c5a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f81c010e6d0 con 0x7f81c01087a0 2024-04-20T10:33:42.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.769+0000 7f81c5a00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81c010e550 con 0x7f81c0103540 2024-04-20T10:33:42.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.769+0000 7f81c5a00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f81c010e850 con 0x7f81c0105e70 2024-04-20T10:33:42.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.769+0000 7f81bf400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81c01087a0 0x7f81c01a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:42.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.769+0000 7f81bea00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0103540 0x7f81c019af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:42.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.769+0000 7f81b6000700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81c0105e70 0x7f81c019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.769+0000 7f81bf400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81c01087a0 0x7f81c01a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59756/0 (socket says 172.21.15.42:59756) 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.769+0000 7f81bf400700 1 -- 172.21.15.42:0/3097662447 learned_addr learned my addr 172.21.15.42:0/3097662447 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.770+0000 7f81b6000700 1 -- 172.21.15.42:0/3097662447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0103540 msgr2=0x7f81c019af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.770+0000 7f81b6000700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0103540 0x7f81c019af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.770+0000 7f81b6000700 1 -- 172.21.15.42:0/3097662447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81c01087a0 msgr2=0x7f81c01a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.770+0000 7f81b6000700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81c01087a0 0x7f81c01a2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.770+0000 7f81b6000700 1 -- 172.21.15.42:0/3097662447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c01a2bf0 con 0x7f81c0105e70 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.770+0000 7f81bea00700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0103540 0x7f81c019af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.770+0000 7f81b6000700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81c0105e70 0x7f81c019b4d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f81a800ea20 tx=0x7f81a800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.770+0000 7f81b7e00700 1 -- 172.21.15.42:0/3097662447 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a800cbd0 con 0x7f81c0105e70 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.770+0000 7f81b7e00700 1 -- 172.21.15.42:0/3097662447 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a800ee60 con 0x7f81c0105e70 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.770+0000 7f81b7e00700 1 -- 172.21.15.42:0/3097662447 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8018680 con 0x7f81c0105e70 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.770+0000 7f81c5a00700 1 -- 172.21.15.42:0/3097662447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c01a2ee0 con 0x7f81c0105e70 2024-04-20T10:33:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.770+0000 7f81c5a00700 1 -- 172.21.15.42:0/3097662447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c01a3340 con 0x7f81c0105e70 2024-04-20T10:33:42.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.772+0000 7f81c5a00700 1 -- 172.21.15.42:0/3097662447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c0069780 con 0x7f81c0105e70 2024-04-20T10:33:42.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.774+0000 7f81b7e00700 1 -- 172.21.15.42:0/3097662447 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f81a8018820 con 0x7f81c0105e70 2024-04-20T10:33:42.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.775+0000 7f81b7e00700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f81a40a3cc0 0x7f81a40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:42.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.775+0000 7f81b7e00700 1 -- 172.21.15.42:0/3097662447 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f81a80d04f0 con 0x7f81c0105e70 2024-04-20T10:33:42.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.775+0000 7f81bea00700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f81a40a3cc0 0x7f81a40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:42.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.776+0000 7f81bea00700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f81a40a3cc0 0x7f81a40a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009bc0 tx=0x7f81b000b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:42.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:42.779+0000 7f81b7e00700 1 -- 172.21.15.42:0/3097662447 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f81a809ce50 con 0x7f81c0105e70 2024-04-20T10:33:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.014+0000 7f81c5a00700 1 -- 172.21.15.42:0/3097662447 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f81c0063660 con 0x7f81c0105e70 2024-04-20T10:33:43.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.015+0000 7f81b7e00700 1 -- 172.21.15.42:0/3097662447 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5329 (secure 0 0 0) 0x7f81a809c750 con 0x7f81c0105e70 2024-04-20T10:33:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 87 2024-04-20T10:33:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:fsid 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:33:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-04-20T10:22:21.221805+0000 2024-04-20T10:33:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-04-20T10:32:31.928088+0000 2024-04-20T10:33:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-20T10:33:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:crush_version 26 2024-04-20T10:33:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:full_ratio 0.95 2024-04-20T10:33:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:backfillfull_ratio 0.9 2024-04-20T10:33:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:nearfull_ratio 0.85 2024-04-20T10:33:43.017 INFO:teuthology.orchestra.run.smithi042.stdout:require_min_compat_client luminous 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:min_compat_client luminous 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:require_osd_release squid 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:stretch_mode_enabled false 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:max_osd 12 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6802/1817026493,v1:172.21.15.42:6803/1817026493] [v2:172.21.15.42:6804/1817026493,v1:172.21.15.42:6805/1817026493] exists,up 47367eda-0111-48e6-891e-e8c78dab4d43 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6800/1972645025,v1:172.21.15.46:6801/1972645025] [v2:172.21.15.46:6802/1972645025,v1:172.21.15.46:6803/1972645025] exists,up a4764a30-900f-4c2c-a5e4-99d346d82f17 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6800/1877464810,v1:172.21.15.162:6801/1877464810] [v2:172.21.15.162:6802/1877464810,v1:172.21.15.162:6803/1877464810] exists,up f69aba92-96e7-45c8-a47a-114aafa47178 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 up in weight 1 up_from 25 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6810/2352652467,v1:172.21.15.42:6811/2352652467] [v2:172.21.15.42:6812/2352652467,v1:172.21.15.42:6813/2352652467] exists,up 620d380c-c93c-4b80-b7c2-f6db56a31e5e 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6808/4097597758,v1:172.21.15.46:6809/4097597758] [v2:172.21.15.46:6810/4097597758,v1:172.21.15.46:6811/4097597758] exists,up d67ed7c7-d9b2-4e22-ad08-633fa00b77f8 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6808/2005776096,v1:172.21.15.162:6809/2005776096] [v2:172.21.15.162:6810/2005776096,v1:172.21.15.162:6811/2005776096] exists,up fe973de7-c11c-402d-89ce-138f33c0a706 2024-04-20T10:33:43.018 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6818/2702679714,v1:172.21.15.42:6819/2702679714] [v2:172.21.15.42:6820/2702679714,v1:172.21.15.42:6821/2702679714] exists,up 67420dab-1d28-4626-9f42-6774ce3b19af 2024-04-20T10:33:43.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6816/2431095643,v1:172.21.15.46:6817/2431095643] [v2:172.21.15.46:6818/2431095643,v1:172.21.15.46:6819/2431095643] exists,up 13021f92-8ad1-4ff6-b0c7-52d1a2d174a5 2024-04-20T10:33:43.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6816/3373880377,v1:172.21.15.162:6817/3373880377] [v2:172.21.15.162:6818/3373880377,v1:172.21.15.162:6819/3373880377] exists,up 8185b6db-3e13-40ed-950c-b330c983331e 2024-04-20T10:33:43.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6826/661029076,v1:172.21.15.42:6827/661029076] [v2:172.21.15.42:6828/661029076,v1:172.21.15.42:6829/661029076] exists,up 1cd34bd8-f661-4903-88cb-7c44fd178f1d 2024-04-20T10:33:43.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6824/1339678617,v1:172.21.15.46:6825/1339678617] [v2:172.21.15.46:6826/1339678617,v1:172.21.15.46:6827/1339678617] exists,up 6d4fba56-28b6-449d-9b0c-fc927cbe6aa7 2024-04-20T10:33:43.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6824/1519508339,v1:172.21.15.162:6825/1519508339] [v2:172.21.15.162:6826/1519508339,v1:172.21.15.162:6827/1519508339] exists,up 6b31e91e-bfbb-4672-b2c3-55fc49d9aae8 2024-04-20T10:33:43.019 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.8 [2,11] 2024-04-20T10:33:43.019 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.b [8,11] 2024-04-20T10:33:43.019 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.1b [2,4] 2024-04-20T10:33:43.019 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.3 [4,9] 2024-04-20T10:33:43.019 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.1a [5,6] 2024-04-20T10:33:43.019 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.1e [3,9] 2024-04-20T10:33:43.019 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.27 [4,7] 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.31 [4,8] 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.34 [4,9] 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.39 [5,8] 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6801/2743527860 expires 2024-04-21T10:23:30.366298+0000 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6800/2743527860 expires 2024-04-21T10:23:30.366298+0000 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/3840708146 expires 2024-04-21T10:23:30.366298+0000 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/1852528586 expires 2024-04-21T10:23:30.366298+0000 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/2022912579 expires 2024-04-21T10:22:55.593824+0000 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/2862444953 expires 2024-04-21T10:22:55.593824+0000 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6800/2556893432 expires 2024-04-21T10:22:55.593824+0000 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/2734400038 expires 2024-04-21T10:23:30.366298+0000 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/3551554049 expires 2024-04-21T10:22:55.593824+0000 2024-04-20T10:33:43.020 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6801/2556893432 expires 2024-04-21T10:22:55.593824+0000 2024-04-20T10:33:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.019+0000 7f81c5a00700 1 -- 172.21.15.42:0/3097662447 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f81a40a3cc0 msgr2=0x7f81a40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.019+0000 7f81c5a00700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f81a40a3cc0 0x7f81a40a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009bc0 tx=0x7f81b000b700 comp rx=0 tx=0).stop 2024-04-20T10:33:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.019+0000 7f81c5a00700 1 -- 172.21.15.42:0/3097662447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81c0105e70 msgr2=0x7f81c019b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.019+0000 7f81c5a00700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81c0105e70 0x7f81c019b4d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f81a800ea20 tx=0x7f81a800ed30 comp rx=0 tx=0).stop 2024-04-20T10:33:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.019+0000 7f81c5a00700 1 -- 172.21.15.42:0/3097662447 shutdown_connections 2024-04-20T10:33:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.019+0000 7f81c5a00700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f81a40a3cc0 0x7f81a40a6180 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.019+0000 7f81c5a00700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0103540 0x7f81c019af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:43.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.019+0000 7f81c5a00700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81c01087a0 0x7f81c01a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:43.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.019+0000 7f81c5a00700 1 --2- 172.21.15.42:0/3097662447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81c0105e70 0x7f81c019b4d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:43.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.019+0000 7f81c5a00700 1 -- 172.21.15.42:0/3097662447 >> 172.21.15.42:0/3097662447 conn(0x7f81c00fcf50 msgr2=0x7f81c0102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:43.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.020+0000 7f81c5a00700 1 -- 172.21.15.42:0/3097662447 shutdown_connections 2024-04-20T10:33:43.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:43.020+0000 7f81c5a00700 1 -- 172.21.15.42:0/3097662447 wait complete. 2024-04-20T10:33:43.743 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:43 smithi042 ceph-mon[17078]: pgmap v383: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:43.744 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:43 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3097662447' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-20T10:33:43.876 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-20T10:33:43.889 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:33:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:43 smithi162 ceph-mon[21854]: pgmap v383: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:43 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3097662447' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-20T10:33:44.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:43 smithi046 ceph-mon[21574]: pgmap v383: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:44.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:43 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3097662447' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-20T10:33:44.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.116+0000 7fc888c00640 1 -- 172.21.15.42:0/1267082357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc884105b90 msgr2=0x7fc88410c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:44.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.116+0000 7fc888c00640 1 --2- 172.21.15.42:0/1267082357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc884105b90 0x7fc88410c420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009be0 tx=0x7fc87c02bef0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.117+0000 7fc888c00640 1 -- 172.21.15.42:0/1267082357 shutdown_connections 2024-04-20T10:33:44.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.117+0000 7fc888c00640 1 --2- 172.21.15.42:0/1267082357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc884105b90 0x7fc88410c420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.117+0000 7fc888c00640 1 --2- 172.21.15.42:0/1267082357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8841051d0 0x7fc884105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.117+0000 7fc888c00640 1 --2- 172.21.15.42:0/1267082357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc884103fe0 0x7fc884104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.117+0000 7fc888c00640 1 -- 172.21.15.42:0/1267082357 >> 172.21.15.42:0/1267082357 conn(0x7fc8840ff570 msgr2=0x7fc884101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:44.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.117+0000 7fc888c00640 1 -- 172.21.15.42:0/1267082357 shutdown_connections 2024-04-20T10:33:44.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.117+0000 7fc888c00640 1 -- 172.21.15.42:0/1267082357 wait complete. 2024-04-20T10:33:44.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.118+0000 7fc888c00640 1 Processor -- start 2024-04-20T10:33:44.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.118+0000 7fc888c00640 1 -- start start 2024-04-20T10:33:44.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.119+0000 7fc888c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc884103fe0 0x7fc88419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:44.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.119+0000 7fc888c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8841051d0 0x7fc88419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:44.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.119+0000 7fc888c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc884105b90 0x7fc8841a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:44.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.119+0000 7fc888c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc88410fcc0 con 0x7fc8841051d0 2024-04-20T10:33:44.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.119+0000 7fc888c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc88410fb40 con 0x7fc884105b90 2024-04-20T10:33:44.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.119+0000 7fc888c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc88410fe40 con 0x7fc884103fe0 2024-04-20T10:33:44.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.119+0000 7fc87be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8841051d0 0x7fc88419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:44.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.119+0000 7fc883400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc884103fe0 0x7fc88419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:44.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.119+0000 7fc883e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc884105b90 0x7fc8841a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:44.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.119+0000 7fc883400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc884103fe0 0x7fc88419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37322/0 (socket says 172.21.15.42:37322) 2024-04-20T10:33:44.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.119+0000 7fc883400640 1 -- 172.21.15.42:0/685884507 learned_addr learned my addr 172.21.15.42:0/685884507 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:44.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.119+0000 7fc883400640 1 -- 172.21.15.42:0/685884507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc884105b90 msgr2=0x7fc8841a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:44.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.120+0000 7fc883400640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc884105b90 0x7fc8841a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.120+0000 7fc883400640 1 -- 172.21.15.42:0/685884507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8841051d0 msgr2=0x7fc88419a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:44.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.120+0000 7fc883400640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8841051d0 0x7fc88419a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.120+0000 7fc883400640 1 -- 172.21.15.42:0/685884507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8841a1eb0 con 0x7fc884103fe0 2024-04-20T10:33:44.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.120+0000 7fc883e00640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc884105b90 0x7fc8841a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:44.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.120+0000 7fc883400640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc884103fe0 0x7fc88419a290 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc87400e9c0 tx=0x7fc87400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:44.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.120+0000 7fc880c00640 1 -- 172.21.15.42:0/685884507 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87400ccc0 con 0x7fc884103fe0 2024-04-20T10:33:44.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.120+0000 7fc880c00640 1 -- 172.21.15.42:0/685884507 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8740035f0 con 0x7fc884103fe0 2024-04-20T10:33:44.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.120+0000 7fc880c00640 1 -- 172.21.15.42:0/685884507 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87400ce60 con 0x7fc884103fe0 2024-04-20T10:33:44.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.120+0000 7fc888c00640 1 -- 172.21.15.42:0/685884507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8841a21a0 con 0x7fc884103fe0 2024-04-20T10:33:44.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.120+0000 7fc888c00640 1 -- 172.21.15.42:0/685884507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8841a2740 con 0x7fc884103fe0 2024-04-20T10:33:44.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.122+0000 7fc888c00640 1 -- 172.21.15.42:0/685884507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc88410b2a0 con 0x7fc884103fe0 2024-04-20T10:33:44.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.124+0000 7fc880c00640 1 -- 172.21.15.42:0/685884507 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc8740037a0 con 0x7fc884103fe0 2024-04-20T10:33:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.125+0000 7fc880c00640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc8680a3c30 0x7fc8680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.125+0000 7fc880c00640 1 -- 172.21.15.42:0/685884507 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc8740d0730 con 0x7fc884103fe0 2024-04-20T10:33:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.125+0000 7fc87be00640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc8680a3c30 0x7fc8680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:44.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.126+0000 7fc87be00640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc8680a3c30 0x7fc8680a60f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc88419b7a0 tx=0x7fc86c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:44.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.129+0000 7fc880c00640 1 -- 172.21.15.42:0/685884507 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc87409d090 con 0x7fc884103fe0 2024-04-20T10:33:44.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.342+0000 7fc888c00640 1 -- 172.21.15.42:0/685884507 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc884062b60 con 0x7fc884103fe0 2024-04-20T10:33:44.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.344+0000 7fc880c00640 1 -- 172.21.15.42:0/685884507 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7fc87409c990 con 0x7fc884103fe0 2024-04-20T10:33:44.345 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:44.346 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:32:31.928088+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:33:44.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.347+0000 7fc888c00640 1 -- 172.21.15.42:0/685884507 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc8680a3c30 msgr2=0x7fc8680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:44.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.347+0000 7fc888c00640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc8680a3c30 0x7fc8680a60f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc88419b7a0 tx=0x7fc86c009260 comp rx=0 tx=0).stop 2024-04-20T10:33:44.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.347+0000 7fc888c00640 1 -- 172.21.15.42:0/685884507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc884103fe0 msgr2=0x7fc88419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:44.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.347+0000 7fc888c00640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc884103fe0 0x7fc88419a290 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc87400e9c0 tx=0x7fc87400edc0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.347+0000 7fc888c00640 1 -- 172.21.15.42:0/685884507 shutdown_connections 2024-04-20T10:33:44.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.347+0000 7fc888c00640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc884105b90 0x7fc8841a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.347+0000 7fc888c00640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8841051d0 0x7fc88419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.347+0000 7fc888c00640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fc8680a3c30 0x7fc8680a60f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.347+0000 7fc888c00640 1 --2- 172.21.15.42:0/685884507 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc884103fe0 0x7fc88419a290 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.347+0000 7fc888c00640 1 -- 172.21.15.42:0/685884507 >> 172.21.15.42:0/685884507 conn(0x7fc8840ff570 msgr2=0x7fc884108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:44.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.347+0000 7fc888c00640 1 -- 172.21.15.42:0/685884507 shutdown_connections 2024-04-20T10:33:44.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.347+0000 7fc888c00640 1 -- 172.21.15.42:0/685884507 wait complete. 2024-04-20T10:33:44.370 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:26:16.537072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:31:33.523692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:31:34.019519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:33:44.370 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:33:44.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.589+0000 7fad4a800640 1 -- 172.21.15.42:0/3337798321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad44075fa0 msgr2=0x7fad440743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:44.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.589+0000 7fad4a800640 1 --2- 172.21.15.42:0/3337798321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad44075fa0 0x7fad440743f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fad34009860 tx=0x7fad3402bb70 comp rx=0 tx=0).stop 2024-04-20T10:33:44.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.590+0000 7fad4a800640 1 -- 172.21.15.42:0/3337798321 shutdown_connections 2024-04-20T10:33:44.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.590+0000 7fad4a800640 1 --2- 172.21.15.42:0/3337798321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad4410e560 0x7fad44110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.590+0000 7fad4a800640 1 --2- 172.21.15.42:0/3337798321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad440749c0 0x7fad44074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.590+0000 7fad4a800640 1 --2- 172.21.15.42:0/3337798321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad44075fa0 0x7fad440743f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.590+0000 7fad4a800640 1 -- 172.21.15.42:0/3337798321 >> 172.21.15.42:0/3337798321 conn(0x7fad440fd520 msgr2=0x7fad440ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:44.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.590+0000 7fad4a800640 1 -- 172.21.15.42:0/3337798321 shutdown_connections 2024-04-20T10:33:44.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.590+0000 7fad4a800640 1 -- 172.21.15.42:0/3337798321 wait complete. 2024-04-20T10:33:44.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.591+0000 7fad4a800640 1 Processor -- start 2024-04-20T10:33:44.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.591+0000 7fad4a800640 1 -- start start 2024-04-20T10:33:44.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad4a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad440749c0 0x7fad4419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad4a800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad44075fa0 0x7fad4419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad4a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad4410e560 0x7fad441a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad4a800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fad44114220 con 0x7fad440749c0 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad4a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad441140a0 con 0x7fad4410e560 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad4a800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fad441143a0 con 0x7fad44075fa0 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad43400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad440749c0 0x7fad4419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad43400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad440749c0 0x7fad4419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59788/0 (socket says 172.21.15.42:59788) 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad43400640 1 -- 172.21.15.42:0/3161114595 learned_addr learned my addr 172.21.15.42:0/3161114595 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad42a00640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad44075fa0 0x7fad4419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad43e00640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad4410e560 0x7fad441a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad43400640 1 -- 172.21.15.42:0/3161114595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad44075fa0 msgr2=0x7fad4419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad43400640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad44075fa0 0x7fad4419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad43400640 1 -- 172.21.15.42:0/3161114595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad4410e560 msgr2=0x7fad441a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:44.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad43400640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad4410e560 0x7fad441a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad43400640 1 -- 172.21.15.42:0/3161114595 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad441a64d0 con 0x7fad440749c0 2024-04-20T10:33:44.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.592+0000 7fad43e00640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad4410e560 0x7fad441a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:33:44.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.593+0000 7fad43400640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad440749c0 0x7fad4419e8b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fad34005940 tx=0x7fad34034930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:44.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.593+0000 7fad3be00640 1 -- 172.21.15.42:0/3161114595 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34041020 con 0x7fad440749c0 2024-04-20T10:33:44.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.593+0000 7fad4a800640 1 -- 172.21.15.42:0/3161114595 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad441a6760 con 0x7fad440749c0 2024-04-20T10:33:44.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.593+0000 7fad3be00640 1 -- 172.21.15.42:0/3161114595 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad3402c9f0 con 0x7fad440749c0 2024-04-20T10:33:44.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.593+0000 7fad3be00640 1 -- 172.21.15.42:0/3161114595 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34039040 con 0x7fad440749c0 2024-04-20T10:33:44.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.593+0000 7fad4a800640 1 -- 172.21.15.42:0/3161114595 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad441a6c50 con 0x7fad440749c0 2024-04-20T10:33:44.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.595+0000 7fad4a800640 1 -- 172.21.15.42:0/3161114595 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad44104730 con 0x7fad440749c0 2024-04-20T10:33:44.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.596+0000 7fad3be00640 1 -- 172.21.15.42:0/3161114595 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fad34044020 con 0x7fad440749c0 2024-04-20T10:33:44.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.598+0000 7fad3be00640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fad280a3bb0 0x7fad280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:44.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.598+0000 7fad3be00640 1 -- 172.21.15.42:0/3161114595 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fad340f1eb0 con 0x7fad440749c0 2024-04-20T10:33:44.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.598+0000 7fad42a00640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fad280a3bb0 0x7fad280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:44.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.599+0000 7fad42a00640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fad280a3bb0 0x7fad280a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fad4419fdc0 tx=0x7fad30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:44.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.603+0000 7fad3be00640 1 -- 172.21.15.42:0/3161114595 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fad3404b020 con 0x7fad440749c0 2024-04-20T10:33:44.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.810+0000 7fad4a800640 1 -- 172.21.15.42:0/3161114595 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fad44062b60 con 0x7fad440749c0 2024-04-20T10:33:44.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.811+0000 7fad3be00640 1 -- 172.21.15.42:0/3161114595 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fad340be810 con 0x7fad440749c0 2024-04-20T10:33:44.812 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-20T10:33:44.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.814+0000 7fad4a800640 1 -- 172.21.15.42:0/3161114595 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fad280a3bb0 msgr2=0x7fad280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:44.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.814+0000 7fad4a800640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fad280a3bb0 0x7fad280a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fad4419fdc0 tx=0x7fad30009260 comp rx=0 tx=0).stop 2024-04-20T10:33:44.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.814+0000 7fad4a800640 1 -- 172.21.15.42:0/3161114595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad440749c0 msgr2=0x7fad4419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:44.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.814+0000 7fad4a800640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad440749c0 0x7fad4419e8b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fad34005940 tx=0x7fad34034930 comp rx=0 tx=0).stop 2024-04-20T10:33:44.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.814+0000 7fad4a800640 1 -- 172.21.15.42:0/3161114595 shutdown_connections 2024-04-20T10:33:44.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.814+0000 7fad4a800640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad4410e560 0x7fad441a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.814+0000 7fad4a800640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fad280a3bb0 0x7fad280a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.814+0000 7fad4a800640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad44075fa0 0x7fad4419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.814+0000 7fad4a800640 1 --2- 172.21.15.42:0/3161114595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad440749c0 0x7fad4419e8b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:44.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.814+0000 7fad4a800640 1 -- 172.21.15.42:0/3161114595 >> 172.21.15.42:0/3161114595 conn(0x7fad440fd520 msgr2=0x7fad44101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:44.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.814+0000 7fad4a800640 1 -- 172.21.15.42:0/3161114595 shutdown_connections 2024-04-20T10:33:44.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:44.814+0000 7fad4a800640 1 -- 172.21.15.42:0/3161114595 wait complete. 2024-04-20T10:33:44.835 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:33:44.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:44 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/685884507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:44.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:44 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/685884507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:45.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.049+0000 7f3b53e00640 1 -- 172.21.15.42:0/3678341972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b4c075f20 msgr2=0x7f3b4c074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.049+0000 7f3b53e00640 1 --2- 172.21.15.42:0/3678341972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b4c075f20 0x7f3b4c074370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009900 tx=0x7f3b4002bcc0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.049+0000 7f3b53e00640 1 -- 172.21.15.42:0/3678341972 shutdown_connections 2024-04-20T10:33:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.049+0000 7f3b53e00640 1 --2- 172.21.15.42:0/3678341972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b4c109e60 0x7f3b4c10c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.049+0000 7f3b53e00640 1 --2- 172.21.15.42:0/3678341972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b4c074940 0x7f3b4c074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.049+0000 7f3b53e00640 1 --2- 172.21.15.42:0/3678341972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b4c075f20 0x7f3b4c074370 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.049+0000 7f3b53e00640 1 -- 172.21.15.42:0/3678341972 >> 172.21.15.42:0/3678341972 conn(0x7f3b4c0fd540 msgr2=0x7f3b4c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.050+0000 7f3b53e00640 1 -- 172.21.15.42:0/3678341972 shutdown_connections 2024-04-20T10:33:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.050+0000 7f3b53e00640 1 -- 172.21.15.42:0/3678341972 wait complete. 2024-04-20T10:33:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.051+0000 7f3b53e00640 1 Processor -- start 2024-04-20T10:33:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.051+0000 7f3b53e00640 1 -- start start 2024-04-20T10:33:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.051+0000 7f3b53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b4c074940 0x7f3b4c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.051+0000 7f3b53e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b4c075f20 0x7f3b4c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.051+0000 7f3b53e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b4c109e60 0x7f3b4c1a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:45.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.051+0000 7f3b53e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c10fe70 con 0x7f3b4c074940 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.051+0000 7f3b53e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c10fcf0 con 0x7f3b4c075f20 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.051+0000 7f3b53e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c10fff0 con 0x7f3b4c109e60 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.051+0000 7f3b51400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b4c074940 0x7f3b4c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.051+0000 7f3b51400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b4c074940 0x7f3b4c19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59810/0 (socket says 172.21.15.42:59810) 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.051+0000 7f3b51400640 1 -- 172.21.15.42:0/1207946794 learned_addr learned my addr 172.21.15.42:0/1207946794 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.052+0000 7f3b50a00640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b4c075f20 0x7f3b4c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.052+0000 7f3b51e00640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b4c109e60 0x7f3b4c1a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.052+0000 7f3b51400640 1 -- 172.21.15.42:0/1207946794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b4c109e60 msgr2=0x7f3b4c1a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.052+0000 7f3b51400640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b4c109e60 0x7f3b4c1a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.052+0000 7f3b51400640 1 -- 172.21.15.42:0/1207946794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b4c075f20 msgr2=0x7f3b4c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.052+0000 7f3b51400640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b4c075f20 0x7f3b4c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.052+0000 7f3b51400640 1 -- 172.21.15.42:0/1207946794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c1a2100 con 0x7f3b4c074940 2024-04-20T10:33:45.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.052+0000 7f3b51400640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b4c074940 0x7f3b4c19a4e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b40000c00 tx=0x7f3b4002c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:45.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.052+0000 7f3b4a000640 1 -- 172.21.15.42:0/1207946794 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40043020 con 0x7f3b4c074940 2024-04-20T10:33:45.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.053+0000 7f3b53e00640 1 -- 172.21.15.42:0/1207946794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c1a2390 con 0x7f3b4c074940 2024-04-20T10:33:45.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.053+0000 7f3b4a000640 1 -- 172.21.15.42:0/1207946794 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b400054e0 con 0x7f3b4c074940 2024-04-20T10:33:45.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.053+0000 7f3b4a000640 1 -- 172.21.15.42:0/1207946794 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40003c20 con 0x7f3b4c074940 2024-04-20T10:33:45.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.053+0000 7f3b53e00640 1 -- 172.21.15.42:0/1207946794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c1a28e0 con 0x7f3b4c074940 2024-04-20T10:33:45.055 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:44 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/685884507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:45.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.054+0000 7f3b53e00640 1 -- 172.21.15.42:0/1207946794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c108c20 con 0x7f3b4c074940 2024-04-20T10:33:45.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.056+0000 7f3b4a000640 1 -- 172.21.15.42:0/1207946794 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3b40046020 con 0x7f3b4c074940 2024-04-20T10:33:45.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.057+0000 7f3b4a000640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3b340a3bb0 0x7f3b340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:45.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.058+0000 7f3b4a000640 1 -- 172.21.15.42:0/1207946794 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3b400f24a0 con 0x7f3b4c074940 2024-04-20T10:33:45.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.058+0000 7f3b50a00640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3b340a3bb0 0x7f3b340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:45.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.058+0000 7f3b50a00640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3b340a3bb0 0x7f3b340a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c19b9f0 tx=0x7f3b3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:45.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.061+0000 7f3b4a000640 1 -- 172.21.15.42:0/1207946794 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3b4004d020 con 0x7f3b4c074940 2024-04-20T10:33:45.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.266+0000 7f3b53e00640 1 -- 172.21.15.42:0/1207946794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3b4c062b60 con 0x7f3b4c074940 2024-04-20T10:33:45.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.267+0000 7f3b4a000640 1 -- 172.21.15.42:0/1207946794 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f3b400bee00 con 0x7f3b4c074940 2024-04-20T10:33:45.268 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-20T10:33:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.270+0000 7f3b53e00640 1 -- 172.21.15.42:0/1207946794 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3b340a3bb0 msgr2=0x7f3b340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.270+0000 7f3b53e00640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3b340a3bb0 0x7f3b340a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c19b9f0 tx=0x7f3b3c005f90 comp rx=0 tx=0).stop 2024-04-20T10:33:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.270+0000 7f3b53e00640 1 -- 172.21.15.42:0/1207946794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b4c074940 msgr2=0x7f3b4c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.270+0000 7f3b53e00640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b4c074940 0x7f3b4c19a4e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b40000c00 tx=0x7f3b4002c6e0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.270+0000 7f3b53e00640 1 -- 172.21.15.42:0/1207946794 shutdown_connections 2024-04-20T10:33:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.270+0000 7f3b53e00640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f3b340a3bb0 0x7f3b340a6070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.270+0000 7f3b53e00640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b4c109e60 0x7f3b4c1a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.270+0000 7f3b53e00640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b4c075f20 0x7f3b4c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.270+0000 7f3b53e00640 1 --2- 172.21.15.42:0/1207946794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b4c074940 0x7f3b4c19a4e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.270+0000 7f3b53e00640 1 -- 172.21.15.42:0/1207946794 >> 172.21.15.42:0/1207946794 conn(0x7f3b4c0fd540 msgr2=0x7f3b4c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.270+0000 7f3b53e00640 1 -- 172.21.15.42:0/1207946794 shutdown_connections 2024-04-20T10:33:45.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.270+0000 7f3b53e00640 1 -- 172.21.15.42:0/1207946794 wait complete. 2024-04-20T10:33:45.290 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:33:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.500+0000 7fcce1200640 1 -- 172.21.15.42:0/1254938626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccdc078500 msgr2=0x7fccdc078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.500+0000 7fcce1200640 1 --2- 172.21.15.42:0/1254938626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccdc078500 0x7fccdc078da0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009900 tx=0x7fccd402bc40 comp rx=0 tx=0).stop 2024-04-20T10:33:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.500+0000 7fcce1200640 1 -- 172.21.15.42:0/1254938626 shutdown_connections 2024-04-20T10:33:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.500+0000 7fcce1200640 1 --2- 172.21.15.42:0/1254938626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccdc078500 0x7fccdc078da0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.500+0000 7fcce1200640 1 --2- 172.21.15.42:0/1254938626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccdc077b60 0x7fccdc077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.500+0000 7fcce1200640 1 --2- 172.21.15.42:0/1254938626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccdc076920 0x7fccdc076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.500+0000 7fcce1200640 1 -- 172.21.15.42:0/1254938626 >> 172.21.15.42:0/1254938626 conn(0x7fccdc0ff530 msgr2=0x7fccdc101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:45.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.500+0000 7fcce1200640 1 -- 172.21.15.42:0/1254938626 shutdown_connections 2024-04-20T10:33:45.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.500+0000 7fcce1200640 1 -- 172.21.15.42:0/1254938626 wait complete. 2024-04-20T10:33:45.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.501+0000 7fcce1200640 1 Processor -- start 2024-04-20T10:33:45.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.501+0000 7fcce1200640 1 -- start start 2024-04-20T10:33:45.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.502+0000 7fcce1200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccdc076920 0x7fccdc19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:45.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.502+0000 7fcce1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccdc077b60 0x7fccdc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:45.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.502+0000 7fcce1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccdc078500 0x7fccdc1a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:45.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.502+0000 7fcce1200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc114090 con 0x7fccdc078500 2024-04-20T10:33:45.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.502+0000 7fcce1200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc113f10 con 0x7fccdc077b60 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.502+0000 7fcce1200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc114210 con 0x7fccdc076920 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.502+0000 7fccdbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccdc078500 0x7fccdc1a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.502+0000 7fccdbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccdc078500 0x7fccdc1a5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59828/0 (socket says 172.21.15.42:59828) 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.502+0000 7fccdbe00640 1 -- 172.21.15.42:0/1220138080 learned_addr learned my addr 172.21.15.42:0/1220138080 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.502+0000 7fccdb400640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccdc076920 0x7fccdc19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.502+0000 7fccd2a00640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccdc077b60 0x7fccdc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.503+0000 7fccdbe00640 1 -- 172.21.15.42:0/1220138080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccdc076920 msgr2=0x7fccdc19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.503+0000 7fccdbe00640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccdc076920 0x7fccdc19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.503+0000 7fccdbe00640 1 -- 172.21.15.42:0/1220138080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccdc077b60 msgr2=0x7fccdc19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.503+0000 7fccdbe00640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccdc077b60 0x7fccdc19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.503+0000 7fccdbe00640 1 -- 172.21.15.42:0/1220138080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccdc1a6240 con 0x7fccdc078500 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.503+0000 7fccdb400640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccdc076920 0x7fccdc19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:33:45.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.503+0000 7fccdbe00640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccdc078500 0x7fccdc1a5bd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fccd402c180 tx=0x7fccd402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:45.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.503+0000 7fccd8c00640 1 -- 172.21.15.42:0/1220138080 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4041070 con 0x7fccdc078500 2024-04-20T10:33:45.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.503+0000 7fcce1200640 1 -- 172.21.15.42:0/1220138080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccdc1a64d0 con 0x7fccdc078500 2024-04-20T10:33:45.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.503+0000 7fccd8c00640 1 -- 172.21.15.42:0/1220138080 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd40054e0 con 0x7fccdc078500 2024-04-20T10:33:45.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.503+0000 7fccd8c00640 1 -- 172.21.15.42:0/1220138080 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4034440 con 0x7fccdc078500 2024-04-20T10:33:45.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.503+0000 7fcce1200640 1 -- 172.21.15.42:0/1220138080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccdc1a6a10 con 0x7fccdc078500 2024-04-20T10:33:45.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.504+0000 7fcce1200640 1 -- 172.21.15.42:0/1220138080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccdc110930 con 0x7fccdc078500 2024-04-20T10:33:45.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.506+0000 7fccd8c00640 1 -- 172.21.15.42:0/1220138080 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fccd4044070 con 0x7fccdc078500 2024-04-20T10:33:45.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.508+0000 7fccd8c00640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fccc00a3bb0 0x7fccc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:45.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.508+0000 7fccd8c00640 1 -- 172.21.15.42:0/1220138080 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fccd40f2060 con 0x7fccdc078500 2024-04-20T10:33:45.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.508+0000 7fccdb400640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fccc00a3bb0 0x7fccc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:45.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.509+0000 7fccdb400640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fccc00a3bb0 0x7fccc00a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcccc005fd0 tx=0x7fcccc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:45.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.512+0000 7fccd8c00640 1 -- 172.21.15.42:0/1220138080 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fccd40bea20 con 0x7fccdc078500 2024-04-20T10:33:45.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.710+0000 7fcce1200640 1 -- 172.21.15.42:0/1220138080 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fccdc062b60 con 0x7fccdc078500 2024-04-20T10:33:45.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.711+0000 7fccd8c00640 1 -- 172.21.15.42:0/1220138080 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fccd40be320 con 0x7fccdc078500 2024-04-20T10:33:45.712 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-20T10:33:45.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.714+0000 7fcce1200640 1 -- 172.21.15.42:0/1220138080 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fccc00a3bb0 msgr2=0x7fccc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.714+0000 7fcce1200640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fccc00a3bb0 0x7fccc00a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcccc005fd0 tx=0x7fcccc009500 comp rx=0 tx=0).stop 2024-04-20T10:33:45.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.714+0000 7fcce1200640 1 -- 172.21.15.42:0/1220138080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccdc078500 msgr2=0x7fccdc1a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.714+0000 7fcce1200640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccdc078500 0x7fccdc1a5bd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fccd402c180 tx=0x7fccd402c660 comp rx=0 tx=0).stop 2024-04-20T10:33:45.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.714+0000 7fcce1200640 1 -- 172.21.15.42:0/1220138080 shutdown_connections 2024-04-20T10:33:45.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.714+0000 7fcce1200640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccdc078500 0x7fccdc1a5bd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.714+0000 7fcce1200640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccdc077b60 0x7fccdc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.714+0000 7fcce1200640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fccc00a3bb0 0x7fccc00a6070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.714+0000 7fcce1200640 1 --2- 172.21.15.42:0/1220138080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fccdc076920 0x7fccdc19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.714+0000 7fcce1200640 1 -- 172.21.15.42:0/1220138080 >> 172.21.15.42:0/1220138080 conn(0x7fccdc0ff530 msgr2=0x7fccdc101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:45.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.714+0000 7fcce1200640 1 -- 172.21.15.42:0/1220138080 shutdown_connections 2024-04-20T10:33:45.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.714+0000 7fcce1200640 1 -- 172.21.15.42:0/1220138080 wait complete. 2024-04-20T10:33:45.733 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:33:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:45 smithi162 ceph-mon[21854]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:45 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3161114595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:33:45.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:45 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1207946794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:33:45.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.939+0000 7f0d4aa00640 1 -- 172.21.15.42:0/3466340842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d44100fb0 msgr2=0x7f0d44101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.939+0000 7f0d4aa00640 1 --2- 172.21.15.42:0/3466340842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d44100fb0 0x7f0d44101410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009900 tx=0x7f0d3002bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.939+0000 7f0d4aa00640 1 -- 172.21.15.42:0/3466340842 shutdown_connections 2024-04-20T10:33:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.939+0000 7f0d4aa00640 1 --2- 172.21.15.42:0/3466340842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d44101950 0x7f0d44101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.939+0000 7f0d4aa00640 1 --2- 172.21.15.42:0/3466340842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d44100fb0 0x7f0d44101410 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.939+0000 7f0d4aa00640 1 --2- 172.21.15.42:0/3466340842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d44100650 0x7f0d44100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.939+0000 7f0d4aa00640 1 -- 172.21.15.42:0/3466340842 >> 172.21.15.42:0/3466340842 conn(0x7f0d440fc270 msgr2=0x7f0d440fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.939+0000 7f0d4aa00640 1 -- 172.21.15.42:0/3466340842 shutdown_connections 2024-04-20T10:33:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.939+0000 7f0d4aa00640 1 -- 172.21.15.42:0/3466340842 wait complete. 2024-04-20T10:33:45.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.940+0000 7f0d4aa00640 1 Processor -- start 2024-04-20T10:33:45.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.940+0000 7f0d4aa00640 1 -- start start 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.940+0000 7f0d4aa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d44100650 0x7f0d4419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.940+0000 7f0d4aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d44100fb0 0x7f0d4419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.940+0000 7f0d4aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d44101950 0x7f0d441a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.940+0000 7f0d4aa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4410fcd0 con 0x7f0d44101950 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.940+0000 7f0d4aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4410fb50 con 0x7f0d44100fb0 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.940+0000 7f0d4aa00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4410fe50 con 0x7f0d44100650 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.941+0000 7f0d4a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d44101950 0x7f0d441a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.941+0000 7f0d4a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d44101950 0x7f0d441a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59842/0 (socket says 172.21.15.42:59842) 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.941+0000 7f0d4a000640 1 -- 172.21.15.42:0/383837488 learned_addr learned my addr 172.21.15.42:0/383837488 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.941+0000 7f0d49600640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d44100650 0x7f0d4419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.941+0000 7f0d48c00640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d44100fb0 0x7f0d4419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.941+0000 7f0d4a000640 1 -- 172.21.15.42:0/383837488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d44100650 msgr2=0x7f0d4419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.941+0000 7f0d4a000640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d44100650 0x7f0d4419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.941+0000 7f0d4a000640 1 -- 172.21.15.42:0/383837488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d44100fb0 msgr2=0x7f0d4419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.941+0000 7f0d4a000640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d44100fb0 0x7f0d4419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.941+0000 7f0d4a000640 1 -- 172.21.15.42:0/383837488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d441a1f50 con 0x7f0d44101950 2024-04-20T10:33:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.941+0000 7f0d49600640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d44100650 0x7f0d4419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.942+0000 7f0d4a000640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d44101950 0x7f0d441a18e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00efd0 tx=0x7f0d3c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.942+0000 7f0d42000640 1 -- 172.21.15.42:0/383837488 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c003b70 con 0x7f0d44101950 2024-04-20T10:33:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.942+0000 7f0d42000640 1 -- 172.21.15.42:0/383837488 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3c003d10 con 0x7f0d44101950 2024-04-20T10:33:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.942+0000 7f0d4aa00640 1 -- 172.21.15.42:0/383837488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d441a2240 con 0x7f0d44101950 2024-04-20T10:33:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.942+0000 7f0d42000640 1 -- 172.21.15.42:0/383837488 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c0109f0 con 0x7f0d44101950 2024-04-20T10:33:45.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.942+0000 7f0d4aa00640 1 -- 172.21.15.42:0/383837488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d441a2760 con 0x7f0d44101950 2024-04-20T10:33:45.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.943+0000 7f0d4aa00640 1 -- 172.21.15.42:0/383837488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4410b580 con 0x7f0d44101950 2024-04-20T10:33:45.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.945+0000 7f0d42000640 1 -- 172.21.15.42:0/383837488 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0d3c004020 con 0x7f0d44101950 2024-04-20T10:33:45.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.948+0000 7f0d42000640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0d2c0a3c30 0x7f0d2c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:45.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.948+0000 7f0d42000640 1 -- 172.21.15.42:0/383837488 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f0d3c0d0820 con 0x7f0d44101950 2024-04-20T10:33:45.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.948+0000 7f0d49600640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0d2c0a3c30 0x7f0d2c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:45.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.949+0000 7f0d49600640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0d2c0a3c30 0x7f0d2c0a60f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009fc0 tx=0x7f0d38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:45.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:45.952+0000 7f0d42000640 1 -- 172.21.15.42:0/383837488 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0d3c09d180 con 0x7f0d44101950 2024-04-20T10:33:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:45 smithi046 ceph-mon[21574]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:45 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3161114595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:33:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:45 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1207946794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:33:46.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:45 smithi042 ceph-mon[17078]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:46.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:45 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3161114595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:33:46.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:45 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1207946794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:33:46.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.201+0000 7f0d4aa00640 1 -- 172.21.15.42:0/383837488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d4402d5d0 con 0x7f0d44101950 2024-04-20T10:33:46.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.203+0000 7f0d42000640 1 -- 172.21.15.42:0/383837488 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f0d3c09ca80 con 0x7f0d44101950 2024-04-20T10:33:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:46.204 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:33:46.204 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:33:46.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.206+0000 7f0d4aa00640 1 -- 172.21.15.42:0/383837488 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0d2c0a3c30 msgr2=0x7f0d2c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:46.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.206+0000 7f0d4aa00640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0d2c0a3c30 0x7f0d2c0a60f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009fc0 tx=0x7f0d38005f90 comp rx=0 tx=0).stop 2024-04-20T10:33:46.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.206+0000 7f0d4aa00640 1 -- 172.21.15.42:0/383837488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d44101950 msgr2=0x7f0d441a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:46.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.206+0000 7f0d4aa00640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d44101950 0x7f0d441a18e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00efd0 tx=0x7f0d3c00c6a0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.206+0000 7f0d4aa00640 1 -- 172.21.15.42:0/383837488 shutdown_connections 2024-04-20T10:33:46.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.206+0000 7f0d4aa00640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d44101950 0x7f0d441a18e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.206+0000 7f0d4aa00640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d44100fb0 0x7f0d4419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.206+0000 7f0d4aa00640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f0d2c0a3c30 0x7f0d2c0a60f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.206+0000 7f0d4aa00640 1 --2- 172.21.15.42:0/383837488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d44100650 0x7f0d4419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.206+0000 7f0d4aa00640 1 -- 172.21.15.42:0/383837488 >> 172.21.15.42:0/383837488 conn(0x7f0d440fc270 msgr2=0x7f0d440fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:46.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.206+0000 7f0d4aa00640 1 -- 172.21.15.42:0/383837488 shutdown_connections 2024-04-20T10:33:46.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.206+0000 7f0d4aa00640 1 -- 172.21.15.42:0/383837488 wait complete. 2024-04-20T10:33:46.228 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:33:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.430+0000 7fbc12a00640 1 -- 172.21.15.42:0/1783996499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc0c103fe0 msgr2=0x7fbc0c104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.430+0000 7fbc12a00640 1 --2- 172.21.15.42:0/1783996499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc0c103fe0 0x7fbc0c104400 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009900 tx=0x7fbc0002bc50 comp rx=0 tx=0).stop 2024-04-20T10:33:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.430+0000 7fbc12a00640 1 -- 172.21.15.42:0/1783996499 shutdown_connections 2024-04-20T10:33:46.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.430+0000 7fbc12a00640 1 --2- 172.21.15.42:0/1783996499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc0c105b90 0x7fbc0c10c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.430+0000 7fbc12a00640 1 --2- 172.21.15.42:0/1783996499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc0c1051d0 0x7fbc0c105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.430+0000 7fbc12a00640 1 --2- 172.21.15.42:0/1783996499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc0c103fe0 0x7fbc0c104400 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.430+0000 7fbc12a00640 1 -- 172.21.15.42:0/1783996499 >> 172.21.15.42:0/1783996499 conn(0x7fbc0c0ff570 msgr2=0x7fbc0c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.430+0000 7fbc12a00640 1 -- 172.21.15.42:0/1783996499 shutdown_connections 2024-04-20T10:33:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.431+0000 7fbc12a00640 1 -- 172.21.15.42:0/1783996499 wait complete. 2024-04-20T10:33:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.432+0000 7fbc12a00640 1 Processor -- start 2024-04-20T10:33:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.432+0000 7fbc12a00640 1 -- start start 2024-04-20T10:33:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.432+0000 7fbc12a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc0c103fe0 0x7fbc0c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.432+0000 7fbc12a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc0c1051d0 0x7fbc0c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.432+0000 7fbc12a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc0c105b90 0x7fbc0c1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.432+0000 7fbc12a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c10fc80 con 0x7fbc0c105b90 2024-04-20T10:33:46.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.432+0000 7fbc12a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c10fb00 con 0x7fbc0c103fe0 2024-04-20T10:33:46.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.432+0000 7fbc12a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c10fe00 con 0x7fbc0c1051d0 2024-04-20T10:33:46.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.433+0000 7fbc10c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc0c1051d0 0x7fbc0c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:46.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.433+0000 7fbc11600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc0c103fe0 0x7fbc0c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:46.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.433+0000 7fbc10c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc0c1051d0 0x7fbc0c19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37392/0 (socket says 172.21.15.42:37392) 2024-04-20T10:33:46.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.433+0000 7fbc10c00640 1 -- 172.21.15.42:0/1858975777 learned_addr learned my addr 172.21.15.42:0/1858975777 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:46.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.433+0000 7fbc11600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc0c103fe0 0x7fbc0c19a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47148/0 (socket says 172.21.15.42:47148) 2024-04-20T10:33:46.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.433+0000 7fbc12000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc0c105b90 0x7fbc0c1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.433+0000 7fbc10c00640 1 -- 172.21.15.42:0/1858975777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc0c103fe0 msgr2=0x7fbc0c19a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.433+0000 7fbc10c00640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc0c103fe0 0x7fbc0c19a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.433+0000 7fbc10c00640 1 -- 172.21.15.42:0/1858975777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc0c105b90 msgr2=0x7fbc0c1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.433+0000 7fbc10c00640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc0c105b90 0x7fbc0c1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.433+0000 7fbc10c00640 1 -- 172.21.15.42:0/1858975777 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0c1a1f40 con 0x7fbc0c1051d0 2024-04-20T10:33:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.433+0000 7fbc11600640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc0c103fe0 0x7fbc0c19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:46.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.434+0000 7fbc10c00640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc0c1051d0 0x7fbc0c19a860 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800ea60 tx=0x7fbbf800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:46.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.434+0000 7fbc0a000640 1 -- 172.21.15.42:0/1858975777 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800cd20 con 0x7fbc0c1051d0 2024-04-20T10:33:46.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.434+0000 7fbc0a000640 1 -- 172.21.15.42:0/1858975777 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf80035f0 con 0x7fbc0c1051d0 2024-04-20T10:33:46.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.434+0000 7fbc12a00640 1 -- 172.21.15.42:0/1858975777 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0c1a2230 con 0x7fbc0c1051d0 2024-04-20T10:33:46.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.434+0000 7fbc0a000640 1 -- 172.21.15.42:0/1858975777 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8010690 con 0x7fbc0c1051d0 2024-04-20T10:33:46.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.434+0000 7fbc12a00640 1 -- 172.21.15.42:0/1858975777 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0c1a27d0 con 0x7fbc0c1051d0 2024-04-20T10:33:46.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.435+0000 7fbc12a00640 1 -- 172.21.15.42:0/1858975777 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0c10b380 con 0x7fbc0c1051d0 2024-04-20T10:33:46.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.437+0000 7fbc0a000640 1 -- 172.21.15.42:0/1858975777 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbbf80037a0 con 0x7fbc0c1051d0 2024-04-20T10:33:46.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.439+0000 7fbc0a000640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fbbf40a3c30 0x7fbbf40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:46.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.439+0000 7fbc0a000640 1 -- 172.21.15.42:0/1858975777 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fbbf80d0850 con 0x7fbc0c1051d0 2024-04-20T10:33:46.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.439+0000 7fbc11600640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fbbf40a3c30 0x7fbbf40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:46.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.440+0000 7fbc11600640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fbbf40a3c30 0x7fbbf40a60f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbc0002c190 tx=0x7fbc00000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:46.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.441+0000 7fbc0a000640 1 -- 172.21.15.42:0/1858975777 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbbf809d210 con 0x7fbc0c1051d0 2024-04-20T10:33:46.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.651+0000 7fbc12a00640 1 -- 172.21.15.42:0/1858975777 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc0c02d820 con 0x7fbc0c1051d0 2024-04-20T10:33:46.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.660+0000 7fbc0a000640 1 -- 172.21.15.42:0/1858975777 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7fbbf809cb10 con 0x7fbc0c1051d0 2024-04-20T10:33:46.661 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:46.661 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:32:31.928088+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:33:46.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.662+0000 7fbc12a00640 1 -- 172.21.15.42:0/1858975777 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fbbf40a3c30 msgr2=0x7fbbf40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:46.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.662+0000 7fbc12a00640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fbbf40a3c30 0x7fbbf40a60f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbc0002c190 tx=0x7fbc00000bc0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.662+0000 7fbc12a00640 1 -- 172.21.15.42:0/1858975777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc0c1051d0 msgr2=0x7fbc0c19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:46.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.662+0000 7fbc12a00640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc0c1051d0 0x7fbc0c19a860 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800ea60 tx=0x7fbbf800ee60 comp rx=0 tx=0).stop 2024-04-20T10:33:46.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.663+0000 7fbc12a00640 1 -- 172.21.15.42:0/1858975777 shutdown_connections 2024-04-20T10:33:46.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.663+0000 7fbc12a00640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc0c105b90 0x7fbc0c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.663+0000 7fbc12a00640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fbbf40a3c30 0x7fbbf40a60f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.663+0000 7fbc12a00640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc0c1051d0 0x7fbc0c19a860 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.663+0000 7fbc12a00640 1 --2- 172.21.15.42:0/1858975777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc0c103fe0 0x7fbc0c19a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.663+0000 7fbc12a00640 1 -- 172.21.15.42:0/1858975777 >> 172.21.15.42:0/1858975777 conn(0x7fbc0c0ff570 msgr2=0x7fbc0c108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:46.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.663+0000 7fbc12a00640 1 -- 172.21.15.42:0/1858975777 shutdown_connections 2024-04-20T10:33:46.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.663+0000 7fbc12a00640 1 -- 172.21.15.42:0/1858975777 wait complete. 2024-04-20T10:33:46.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:26:16.537072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:31:33.523692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:31:34.019519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:33:46.684 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:33:46.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.891+0000 7f5adf400640 1 -- 172.21.15.42:0/3900289063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad8077b60 msgr2=0x7f5ad8077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:46.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.891+0000 7f5adf400640 1 --2- 172.21.15.42:0/3900289063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad8077b60 0x7f5ad8077fc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009900 tx=0x7f5ac402bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:46.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.891+0000 7f5adf400640 1 -- 172.21.15.42:0/3900289063 shutdown_connections 2024-04-20T10:33:46.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.891+0000 7f5adf400640 1 --2- 172.21.15.42:0/3900289063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ad8078500 0x7f5ad8078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.891+0000 7f5adf400640 1 --2- 172.21.15.42:0/3900289063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad8077b60 0x7f5ad8077fc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.891+0000 7f5adf400640 1 --2- 172.21.15.42:0/3900289063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ad8076920 0x7f5ad8076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.891+0000 7f5adf400640 1 -- 172.21.15.42:0/3900289063 >> 172.21.15.42:0/3900289063 conn(0x7f5ad80ff750 msgr2=0x7f5ad8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:46.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.891+0000 7f5adf400640 1 -- 172.21.15.42:0/3900289063 shutdown_connections 2024-04-20T10:33:46.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.891+0000 7f5adf400640 1 -- 172.21.15.42:0/3900289063 wait complete. 2024-04-20T10:33:46.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.892+0000 7f5adf400640 1 Processor -- start 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5adf400640 1 -- start start 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5adf400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ad8076920 0x7f5ad819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5adf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad8077b60 0x7f5ad819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5adf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ad8078500 0x7f5ad81a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5adf400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8114250 con 0x7f5ad8078500 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5adf400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81140d0 con 0x7f5ad8077b60 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5adf400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81143d0 con 0x7f5ad8076920 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5acfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad8077b60 0x7f5ad819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5adca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ad8076920 0x7f5ad819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5add400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ad8078500 0x7f5ad81a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5acfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad8077b60 0x7f5ad819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47160/0 (socket says 172.21.15.42:47160) 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5adca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ad8076920 0x7f5ad819e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37412/0 (socket says 172.21.15.42:37412) 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.893+0000 7f5acfe00640 1 -- 172.21.15.42:0/4168845884 learned_addr learned my addr 172.21.15.42:0/4168845884 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:46.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.894+0000 7f5acfe00640 1 -- 172.21.15.42:0/4168845884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ad8076920 msgr2=0x7f5ad819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:46.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.894+0000 7f5acfe00640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ad8076920 0x7f5ad819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.894+0000 7f5acfe00640 1 -- 172.21.15.42:0/4168845884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ad8078500 msgr2=0x7f5ad81a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:46.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.894+0000 7f5acfe00640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ad8078500 0x7f5ad81a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:46.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.894+0000 7f5acfe00640 1 -- 172.21.15.42:0/4168845884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad81a64b0 con 0x7f5ad8077b60 2024-04-20T10:33:46.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.894+0000 7f5add400640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ad8078500 0x7f5ad81a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:46.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.894+0000 7f5acfe00640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad8077b60 0x7f5ad819edd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4000c00 tx=0x7f5ac402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:46.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.895+0000 7f5ad6000640 1 -- 172.21.15.42:0/4168845884 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4043020 con 0x7f5ad8077b60 2024-04-20T10:33:46.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.895+0000 7f5adf400640 1 -- 172.21.15.42:0/4168845884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad81a6740 con 0x7f5ad8077b60 2024-04-20T10:33:46.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.895+0000 7f5ad6000640 1 -- 172.21.15.42:0/4168845884 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac40054e0 con 0x7f5ad8077b60 2024-04-20T10:33:46.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.895+0000 7f5ad6000640 1 -- 172.21.15.42:0/4168845884 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4003bd0 con 0x7f5ad8077b60 2024-04-20T10:33:46.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.895+0000 7f5adf400640 1 -- 172.21.15.42:0/4168845884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad81a6c90 con 0x7f5ad8077b60 2024-04-20T10:33:46.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.896+0000 7f5adf400640 1 -- 172.21.15.42:0/4168845884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad8110b30 con 0x7f5ad8077b60 2024-04-20T10:33:46.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.898+0000 7f5ad6000640 1 -- 172.21.15.42:0/4168845884 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5ac4046020 con 0x7f5ad8077b60 2024-04-20T10:33:46.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.899+0000 7f5ad6000640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f5abc0a3bb0 0x7f5abc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:46.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.900+0000 7f5ad6000640 1 -- 172.21.15.42:0/4168845884 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f5ac40f24e0 con 0x7f5ad8077b60 2024-04-20T10:33:46.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.900+0000 7f5adca00640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f5abc0a3bb0 0x7f5abc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:46.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.900+0000 7f5adca00640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f5abc0a3bb0 0x7f5abc0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009bd0 tx=0x7f5ac8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:46.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:46.904+0000 7f5ad6000640 1 -- 172.21.15.42:0/4168845884 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5ac404d020 con 0x7f5ad8077b60 2024-04-20T10:33:47.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:46 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1220138080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:33:47.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:46 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/383837488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:47.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:46 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1220138080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:33:47.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:46 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/383837488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:47.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.116+0000 7f5adf400640 1 -- 172.21.15.42:0/4168845884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ad8062b60 con 0x7f5ad8077b60 2024-04-20T10:33:47.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.117+0000 7f5ad6000640 1 -- 172.21.15.42:0/4168845884 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f5ac40bee40 con 0x7f5ad8077b60 2024-04-20T10:33:47.117 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-20T10:33:47.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.120+0000 7f5adf400640 1 -- 172.21.15.42:0/4168845884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f5abc0a3bb0 msgr2=0x7f5abc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:47.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.120+0000 7f5adf400640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f5abc0a3bb0 0x7f5abc0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009bd0 tx=0x7f5ac8005f90 comp rx=0 tx=0).stop 2024-04-20T10:33:47.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.120+0000 7f5adf400640 1 -- 172.21.15.42:0/4168845884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad8077b60 msgr2=0x7f5ad819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:47.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.120+0000 7f5adf400640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad8077b60 0x7f5ad819edd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4000c00 tx=0x7f5ac402c650 comp rx=0 tx=0).stop 2024-04-20T10:33:47.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.121+0000 7f5adf400640 1 -- 172.21.15.42:0/4168845884 shutdown_connections 2024-04-20T10:33:47.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.121+0000 7f5adf400640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ad8078500 0x7f5ad81a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.121+0000 7f5adf400640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad8077b60 0x7f5ad819edd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.121+0000 7f5adf400640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f5abc0a3bb0 0x7f5abc0a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.121+0000 7f5adf400640 1 --2- 172.21.15.42:0/4168845884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ad8076920 0x7f5ad819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.121+0000 7f5adf400640 1 -- 172.21.15.42:0/4168845884 >> 172.21.15.42:0/4168845884 conn(0x7f5ad80ff750 msgr2=0x7f5ad8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:47.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.121+0000 7f5adf400640 1 -- 172.21.15.42:0/4168845884 shutdown_connections 2024-04-20T10:33:47.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.121+0000 7f5adf400640 1 -- 172.21.15.42:0/4168845884 wait complete. 2024-04-20T10:33:47.141 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:33:47.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:46 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1220138080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:33:47.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:46 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/383837488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:47.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.361+0000 7f9be3200640 1 -- 172.21.15.42:0/2714916360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9be4103050 msgr2=0x7f9be41054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:47.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.361+0000 7f9be3200640 1 --2- 172.21.15.42:0/2714916360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9be4103050 0x7f9be41054c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009900 tx=0x7f9bd002bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:47.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.361+0000 7f9be3200640 1 -- 172.21.15.42:0/2714916360 shutdown_connections 2024-04-20T10:33:47.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.361+0000 7f9be3200640 1 --2- 172.21.15.42:0/2714916360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be4105a00 0x7f9be4105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.361+0000 7f9be3200640 1 --2- 172.21.15.42:0/2714916360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9be4103050 0x7f9be41054c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.361+0000 7f9be3200640 1 --2- 172.21.15.42:0/2714916360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9be41006e0 0x7f9be4102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.361+0000 7f9be3200640 1 -- 172.21.15.42:0/2714916360 >> 172.21.15.42:0/2714916360 conn(0x7f9be40fc300 msgr2=0x7f9be40fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:47.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.361+0000 7f9be3200640 1 -- 172.21.15.42:0/2714916360 shutdown_connections 2024-04-20T10:33:47.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.361+0000 7f9be3200640 1 -- 172.21.15.42:0/2714916360 wait complete. 2024-04-20T10:33:47.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.362+0000 7f9be3200640 1 Processor -- start 2024-04-20T10:33:47.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.362+0000 7f9be3200640 1 -- start start 2024-04-20T10:33:47.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be3200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be41006e0 0x7f9be419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:47.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9be4105a00 0x7f9be419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:47.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9be41a3ef0 0x7f9be41a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:47.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9be4114160 con 0x7f9be41a3ef0 2024-04-20T10:33:47.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9be4113fe0 con 0x7f9be4105a00 2024-04-20T10:33:47.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be3200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9be41142e0 con 0x7f9be41006e0 2024-04-20T10:33:47.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be1e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be41006e0 0x7f9be419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:47.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9be41a3ef0 0x7f9be41a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:47.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be1e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be41006e0 0x7f9be419e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37432/0 (socket says 172.21.15.42:37432) 2024-04-20T10:33:47.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be1e00640 1 -- 172.21.15.42:0/3476149564 learned_addr learned my addr 172.21.15.42:0/3476149564 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:47.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be1400640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9be4105a00 0x7f9be419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:47.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be1e00640 1 -- 172.21.15.42:0/3476149564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9be4105a00 msgr2=0x7f9be419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:47.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be1e00640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9be4105a00 0x7f9be419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be1e00640 1 -- 172.21.15.42:0/3476149564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9be41a3ef0 msgr2=0x7f9be41a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be1e00640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9be41a3ef0 0x7f9be41a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.363+0000 7f9be1e00640 1 -- 172.21.15.42:0/3476149564 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be41a6950 con 0x7f9be41006e0 2024-04-20T10:33:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.364+0000 7f9be1e00640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be41006e0 0x7f9be419e950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400e9f0 tx=0x7f9bd400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.364+0000 7f9bdaa00640 1 -- 172.21.15.42:0/3476149564 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400cd80 con 0x7f9be41006e0 2024-04-20T10:33:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.364+0000 7f9bdaa00640 1 -- 172.21.15.42:0/3476149564 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bd4010910 con 0x7f9be41006e0 2024-04-20T10:33:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.364+0000 7f9be3200640 1 -- 172.21.15.42:0/3476149564 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be41a6c40 con 0x7f9be41006e0 2024-04-20T10:33:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.364+0000 7f9bdaa00640 1 -- 172.21.15.42:0/3476149564 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4018990 con 0x7f9be41006e0 2024-04-20T10:33:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.364+0000 7f9be3200640 1 -- 172.21.15.42:0/3476149564 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be41a7160 con 0x7f9be41006e0 2024-04-20T10:33:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.366+0000 7f9be3200640 1 -- 172.21.15.42:0/3476149564 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be40fddf0 con 0x7f9be41006e0 2024-04-20T10:33:47.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.368+0000 7f9bdaa00640 1 -- 172.21.15.42:0/3476149564 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9bd4010ac0 con 0x7f9be41006e0 2024-04-20T10:33:47.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.369+0000 7f9bdaa00640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9bc80a3bb0 0x7f9bc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:47.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.369+0000 7f9bdaa00640 1 -- 172.21.15.42:0/3476149564 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9bd40d0930 con 0x7f9be41006e0 2024-04-20T10:33:47.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.370+0000 7f9be1400640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9bc80a3bb0 0x7f9bc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:47.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.370+0000 7f9be1400640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9bc80a3bb0 0x7f9bc80a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009230 tx=0x7f9bd002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:47.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.372+0000 7f9bdaa00640 1 -- 172.21.15.42:0/3476149564 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9bd409cc40 con 0x7f9be41006e0 2024-04-20T10:33:47.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.577+0000 7f9be3200640 1 -- 172.21.15.42:0/3476149564 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9be402d860 con 0x7f9be41006e0 2024-04-20T10:33:47.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.578+0000 7f9bdaa00640 1 -- 172.21.15.42:0/3476149564 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f9bd40a0f40 con 0x7f9be41006e0 2024-04-20T10:33:47.578 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-20T10:33:47.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.581+0000 7f9be3200640 1 -- 172.21.15.42:0/3476149564 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9bc80a3bb0 msgr2=0x7f9bc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:47.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.581+0000 7f9be3200640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9bc80a3bb0 0x7f9bc80a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009230 tx=0x7f9bd002df90 comp rx=0 tx=0).stop 2024-04-20T10:33:47.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.581+0000 7f9be3200640 1 -- 172.21.15.42:0/3476149564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be41006e0 msgr2=0x7f9be419e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:47.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.581+0000 7f9be3200640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be41006e0 0x7f9be419e950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400e9f0 tx=0x7f9bd400eeb0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.581+0000 7f9be3200640 1 -- 172.21.15.42:0/3476149564 shutdown_connections 2024-04-20T10:33:47.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.581+0000 7f9be3200640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9be41a3ef0 0x7f9be41a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.581+0000 7f9be3200640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9be4105a00 0x7f9be419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.581+0000 7f9be3200640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f9bc80a3bb0 0x7f9bc80a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.581+0000 7f9be3200640 1 --2- 172.21.15.42:0/3476149564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9be41006e0 0x7f9be419e950 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.581+0000 7f9be3200640 1 -- 172.21.15.42:0/3476149564 >> 172.21.15.42:0/3476149564 conn(0x7f9be40fc300 msgr2=0x7f9be41010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:47.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.581+0000 7f9be3200640 1 -- 172.21.15.42:0/3476149564 shutdown_connections 2024-04-20T10:33:47.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.581+0000 7f9be3200640 1 -- 172.21.15.42:0/3476149564 wait complete. 2024-04-20T10:33:47.601 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:33:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.817+0000 7fb496e00640 1 -- 172.21.15.42:0/3127219985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb490101da0 msgr2=0x7fb4901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.817+0000 7fb496e00640 1 --2- 172.21.15.42:0/3127219985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb490101da0 0x7fb4901021c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb484009ca0 tx=0x7fb48402c100 comp rx=0 tx=0).stop 2024-04-20T10:33:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.817+0000 7fb496e00640 1 -- 172.21.15.42:0/3127219985 shutdown_connections 2024-04-20T10:33:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.817+0000 7fb496e00640 1 --2- 172.21.15.42:0/3127219985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb490103950 0x7fb49010a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.817+0000 7fb496e00640 1 --2- 172.21.15.42:0/3127219985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490102f90 0x7fb490103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.817+0000 7fb496e00640 1 --2- 172.21.15.42:0/3127219985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb490101da0 0x7fb4901021c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.817+0000 7fb496e00640 1 -- 172.21.15.42:0/3127219985 >> 172.21.15.42:0/3127219985 conn(0x7fb4900fd310 msgr2=0x7fb4900ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.817+0000 7fb496e00640 1 -- 172.21.15.42:0/3127219985 shutdown_connections 2024-04-20T10:33:47.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.817+0000 7fb496e00640 1 -- 172.21.15.42:0/3127219985 wait complete. 2024-04-20T10:33:47.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.819+0000 7fb496e00640 1 Processor -- start 2024-04-20T10:33:47.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.819+0000 7fb496e00640 1 -- start start 2024-04-20T10:33:47.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.819+0000 7fb496e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb490101da0 0x7fb49019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:47.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.819+0000 7fb496e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb490102f90 0x7fb49019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:47.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.819+0000 7fb496e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490103950 0x7fb4901a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.819+0000 7fb496e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb49010d9f0 con 0x7fb490101da0 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.819+0000 7fb496e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb49010d870 con 0x7fb490103950 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.819+0000 7fb496e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb49010db70 con 0x7fb490102f90 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.819+0000 7fb495a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb490101da0 0x7fb49019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.819+0000 7fb495a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb490101da0 0x7fb49019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59912/0 (socket says 172.21.15.42:59912) 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.819+0000 7fb495a00640 1 -- 172.21.15.42:0/3784290242 learned_addr learned my addr 172.21.15.42:0/3784290242 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.820+0000 7fb496400640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490103950 0x7fb4901a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.820+0000 7fb495000640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb490102f90 0x7fb49019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.820+0000 7fb495a00640 1 -- 172.21.15.42:0/3784290242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb490102f90 msgr2=0x7fb49019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.820+0000 7fb495a00640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb490102f90 0x7fb49019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.820+0000 7fb495a00640 1 -- 172.21.15.42:0/3784290242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490103950 msgr2=0x7fb4901a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.820+0000 7fb495a00640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490103950 0x7fb4901a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:47.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.820+0000 7fb495a00640 1 -- 172.21.15.42:0/3784290242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4901a1ea0 con 0x7fb490101da0 2024-04-20T10:33:47.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.820+0000 7fb495a00640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb490101da0 0x7fb49019a280 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb4840324c0 tx=0x7fb484035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:47.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.821+0000 7fb48e000640 1 -- 172.21.15.42:0/3784290242 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484039040 con 0x7fb490101da0 2024-04-20T10:33:47.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.821+0000 7fb48e000640 1 -- 172.21.15.42:0/3784290242 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb484044070 con 0x7fb490101da0 2024-04-20T10:33:47.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.821+0000 7fb496e00640 1 -- 172.21.15.42:0/3784290242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4901a2130 con 0x7fb490101da0 2024-04-20T10:33:47.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.821+0000 7fb48e000640 1 -- 172.21.15.42:0/3784290242 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484004370 con 0x7fb490101da0 2024-04-20T10:33:47.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.821+0000 7fb496e00640 1 -- 172.21.15.42:0/3784290242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4901a2620 con 0x7fb490101da0 2024-04-20T10:33:47.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.822+0000 7fb496e00640 1 -- 172.21.15.42:0/3784290242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4901a2940 con 0x7fb490101da0 2024-04-20T10:33:47.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.824+0000 7fb48e000640 1 -- 172.21.15.42:0/3784290242 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb484004510 con 0x7fb490101da0 2024-04-20T10:33:47.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.826+0000 7fb48e000640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb4780a3bb0 0x7fb4780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:47.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.826+0000 7fb48e000640 1 -- 172.21.15.42:0/3784290242 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fb4840f2a40 con 0x7fb490101da0 2024-04-20T10:33:47.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.826+0000 7fb495000640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb4780a3bb0 0x7fb4780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:47.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.827+0000 7fb495000640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb4780a3bb0 0x7fb4780a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0095f0 tx=0x7fb47c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:47.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:47.830+0000 7fb48e000640 1 -- 172.21.15.42:0/3784290242 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb4840bf3a0 con 0x7fb490101da0 2024-04-20T10:33:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:47 smithi042 ceph-mon[17078]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:47 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1858975777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:47 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4168845884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:33:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:47 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3476149564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:33:48.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.038+0000 7fb496e00640 1 -- 172.21.15.42:0/3784290242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb4901a2cd0 con 0x7fb490101da0 2024-04-20T10:33:48.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.039+0000 7fb48e000640 1 -- 172.21.15.42:0/3784290242 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fb4840beca0 con 0x7fb490101da0 2024-04-20T10:33:48.040 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-20T10:33:48.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.042+0000 7fb496e00640 1 -- 172.21.15.42:0/3784290242 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb4780a3bb0 msgr2=0x7fb4780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:48.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.042+0000 7fb496e00640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb4780a3bb0 0x7fb4780a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0095f0 tx=0x7fb47c005f90 comp rx=0 tx=0).stop 2024-04-20T10:33:48.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.042+0000 7fb496e00640 1 -- 172.21.15.42:0/3784290242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb490101da0 msgr2=0x7fb49019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:48.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.042+0000 7fb496e00640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb490101da0 0x7fb49019a280 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb4840324c0 tx=0x7fb484035d90 comp rx=0 tx=0).stop 2024-04-20T10:33:48.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.042+0000 7fb496e00640 1 -- 172.21.15.42:0/3784290242 shutdown_connections 2024-04-20T10:33:48.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.042+0000 7fb496e00640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490103950 0x7fb4901a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.042+0000 7fb496e00640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fb4780a3bb0 0x7fb4780a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.042+0000 7fb496e00640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb490102f90 0x7fb49019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.042+0000 7fb496e00640 1 --2- 172.21.15.42:0/3784290242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb490101da0 0x7fb49019a280 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.042+0000 7fb496e00640 1 -- 172.21.15.42:0/3784290242 >> 172.21.15.42:0/3784290242 conn(0x7fb4900fd310 msgr2=0x7fb490074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:48.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.042+0000 7fb496e00640 1 -- 172.21.15.42:0/3784290242 shutdown_connections 2024-04-20T10:33:48.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.042+0000 7fb496e00640 1 -- 172.21.15.42:0/3784290242 wait complete. 2024-04-20T10:33:48.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:47 smithi162 ceph-mon[21854]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:48.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:47 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1858975777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:48.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:47 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/4168845884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:33:48.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:47 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3476149564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:33:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:47 smithi046 ceph-mon[21574]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:33:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:47 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1858975777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:47 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/4168845884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:33:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:47 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3476149564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:33:48.064 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:33:48.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.271+0000 7ff12b600640 1 -- 172.21.15.42:0/3956092498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1241085c0 msgr2=0x7ff1241089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:48.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.271+0000 7ff12b600640 1 --2- 172.21.15.42:0/3956092498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1241085c0 0x7ff1241089e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff118009900 tx=0x7ff11802bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:48.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.272+0000 7ff12b600640 1 -- 172.21.15.42:0/3956092498 shutdown_connections 2024-04-20T10:33:48.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.272+0000 7ff12b600640 1 --2- 172.21.15.42:0/3956092498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff12410a170 0x7ff124110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.272+0000 7ff12b600640 1 --2- 172.21.15.42:0/3956092498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1241097b0 0x7ff124109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.272+0000 7ff12b600640 1 --2- 172.21.15.42:0/3956092498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1241085c0 0x7ff1241089e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.272+0000 7ff12b600640 1 -- 172.21.15.42:0/3956092498 >> 172.21.15.42:0/3956092498 conn(0x7ff124077580 msgr2=0x7ff1240799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:48.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.272+0000 7ff12b600640 1 -- 172.21.15.42:0/3956092498 shutdown_connections 2024-04-20T10:33:48.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.272+0000 7ff12b600640 1 -- 172.21.15.42:0/3956092498 wait complete. 2024-04-20T10:33:48.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.273+0000 7ff12b600640 1 Processor -- start 2024-04-20T10:33:48.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.273+0000 7ff12b600640 1 -- start start 2024-04-20T10:33:48.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.273+0000 7ff12b600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1241085c0 0x7ff12419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:48.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.273+0000 7ff12b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1241097b0 0x7ff12419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:48.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.273+0000 7ff12b600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff12410a170 0x7ff1241a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:48.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.273+0000 7ff12b600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff124114290 con 0x7ff1241085c0 2024-04-20T10:33:48.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.273+0000 7ff12b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff124114110 con 0x7ff1241097b0 2024-04-20T10:33:48.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.273+0000 7ff12b600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff124114410 con 0x7ff12410a170 2024-04-20T10:33:48.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.274+0000 7ff128c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1241085c0 0x7ff12419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:48.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.274+0000 7ff128c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1241085c0 0x7ff12419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59920/0 (socket says 172.21.15.42:59920) 2024-04-20T10:33:48.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.274+0000 7ff128c00640 1 -- 172.21.15.42:0/3678386802 learned_addr learned my addr 172.21.15.42:0/3678386802 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.274+0000 7ff123e00640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1241097b0 0x7ff12419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.274+0000 7ff129600640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff12410a170 0x7ff1241a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.274+0000 7ff128c00640 1 -- 172.21.15.42:0/3678386802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff12410a170 msgr2=0x7ff1241a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.274+0000 7ff128c00640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff12410a170 0x7ff1241a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.274+0000 7ff128c00640 1 -- 172.21.15.42:0/3678386802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1241097b0 msgr2=0x7ff12419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.274+0000 7ff128c00640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1241097b0 0x7ff12419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.274+0000 7ff128c00640 1 -- 172.21.15.42:0/3678386802 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1241a6520 con 0x7ff1241085c0 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.274+0000 7ff129600640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff12410a170 0x7ff1241a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.274+0000 7ff128c00640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1241085c0 0x7ff12419e900 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff118000c00 tx=0x7ff11802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.275+0000 7ff121600640 1 -- 172.21.15.42:0/3678386802 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118043020 con 0x7ff1241085c0 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.275+0000 7ff121600640 1 -- 172.21.15.42:0/3678386802 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1180054e0 con 0x7ff1241085c0 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.275+0000 7ff12b600640 1 -- 172.21.15.42:0/3678386802 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1241a67b0 con 0x7ff1241085c0 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.275+0000 7ff121600640 1 -- 172.21.15.42:0/3678386802 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118003c40 con 0x7ff1241085c0 2024-04-20T10:33:48.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.275+0000 7ff12b600640 1 -- 172.21.15.42:0/3678386802 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1241a6d00 con 0x7ff1241085c0 2024-04-20T10:33:48.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.276+0000 7ff12b600640 1 -- 172.21.15.42:0/3678386802 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12410f8e0 con 0x7ff1241085c0 2024-04-20T10:33:48.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.278+0000 7ff121600640 1 -- 172.21.15.42:0/3678386802 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff118046020 con 0x7ff1241085c0 2024-04-20T10:33:48.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.279+0000 7ff121600640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff10c0a3bb0 0x7ff10c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:48.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.280+0000 7ff121600640 1 -- 172.21.15.42:0/3678386802 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff1180f2420 con 0x7ff1241085c0 2024-04-20T10:33:48.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.280+0000 7ff123e00640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff10c0a3bb0 0x7ff10c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:48.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.281+0000 7ff123e00640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff10c0a3bb0 0x7ff10c0a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff12419fe10 tx=0x7ff110005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:48.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.284+0000 7ff121600640 1 -- 172.21.15.42:0/3678386802 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff11804d020 con 0x7ff1241085c0 2024-04-20T10:33:48.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.533+0000 7ff12b600640 1 -- 172.21.15.42:0/3678386802 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff12402d7f0 con 0x7ff1241085c0 2024-04-20T10:33:48.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.535+0000 7ff121600640 1 -- 172.21.15.42:0/3678386802 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7ff1180bed80 con 0x7ff1241085c0 2024-04-20T10:33:48.536 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:48.536 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:33:48.536 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:33:48.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.538+0000 7ff12b600640 1 -- 172.21.15.42:0/3678386802 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff10c0a3bb0 msgr2=0x7ff10c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:48.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.538+0000 7ff12b600640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff10c0a3bb0 0x7ff10c0a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff12419fe10 tx=0x7ff110005f90 comp rx=0 tx=0).stop 2024-04-20T10:33:48.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.538+0000 7ff12b600640 1 -- 172.21.15.42:0/3678386802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1241085c0 msgr2=0x7ff12419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:48.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.538+0000 7ff12b600640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1241085c0 0x7ff12419e900 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff118000c00 tx=0x7ff11802c650 comp rx=0 tx=0).stop 2024-04-20T10:33:48.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.538+0000 7ff12b600640 1 -- 172.21.15.42:0/3678386802 shutdown_connections 2024-04-20T10:33:48.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.538+0000 7ff12b600640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff10c0a3bb0 0x7ff10c0a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.538+0000 7ff12b600640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff12410a170 0x7ff1241a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.538+0000 7ff12b600640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1241097b0 0x7ff12419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.538+0000 7ff12b600640 1 --2- 172.21.15.42:0/3678386802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1241085c0 0x7ff12419e900 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.538+0000 7ff12b600640 1 -- 172.21.15.42:0/3678386802 >> 172.21.15.42:0/3678386802 conn(0x7ff124077580 msgr2=0x7ff12410cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.538+0000 7ff12b600640 1 -- 172.21.15.42:0/3678386802 shutdown_connections 2024-04-20T10:33:48.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.539+0000 7ff12b600640 1 -- 172.21.15.42:0/3678386802 wait complete. 2024-04-20T10:33:48.559 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:33:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.763+0000 7fdcabc00640 1 -- 172.21.15.42:0/1168819656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca4105d90 msgr2=0x7fdca410c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.763+0000 7fdcabc00640 1 --2- 172.21.15.42:0/1168819656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca4105d90 0x7fdca410c620 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009be0 tx=0x7fdc9c02bef0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.763+0000 7fdcabc00640 1 -- 172.21.15.42:0/1168819656 shutdown_connections 2024-04-20T10:33:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.763+0000 7fdcabc00640 1 --2- 172.21.15.42:0/1168819656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca4105d90 0x7fdca410c620 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.763+0000 7fdcabc00640 1 --2- 172.21.15.42:0/1168819656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca41053d0 0x7fdca4105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.763+0000 7fdcabc00640 1 --2- 172.21.15.42:0/1168819656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdca41041e0 0x7fdca4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.763+0000 7fdcabc00640 1 -- 172.21.15.42:0/1168819656 >> 172.21.15.42:0/1168819656 conn(0x7fdca40ff750 msgr2=0x7fdca4101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.763+0000 7fdcabc00640 1 -- 172.21.15.42:0/1168819656 shutdown_connections 2024-04-20T10:33:48.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.763+0000 7fdcabc00640 1 -- 172.21.15.42:0/1168819656 wait complete. 2024-04-20T10:33:48.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.765+0000 7fdcabc00640 1 Processor -- start 2024-04-20T10:33:48.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.765+0000 7fdcabc00640 1 -- start start 2024-04-20T10:33:48.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.765+0000 7fdcabc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca41041e0 0x7fdca419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:48.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.765+0000 7fdcabc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca41053d0 0x7fdca419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:48.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.765+0000 7fdcabc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdca4105d90 0x7fdca41a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:48.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.765+0000 7fdcabc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdca410fe90 con 0x7fdca4105d90 2024-04-20T10:33:48.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.765+0000 7fdcabc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdca410fd10 con 0x7fdca41053d0 2024-04-20T10:33:48.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.765+0000 7fdcabc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4110010 con 0x7fdca41041e0 2024-04-20T10:33:48.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.765+0000 7fdca9200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca41041e0 0x7fdca419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:48.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.766+0000 7fdca9200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca41041e0 0x7fdca419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37488/0 (socket says 172.21.15.42:37488) 2024-04-20T10:33:48.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.766+0000 7fdca9200640 1 -- 172.21.15.42:0/437473957 learned_addr learned my addr 172.21.15.42:0/437473957 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:48.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.766+0000 7fdca9c00640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdca4105d90 0x7fdca41a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:48.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.766+0000 7fdca3e00640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca41053d0 0x7fdca419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:48.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.766+0000 7fdca9200640 1 -- 172.21.15.42:0/437473957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca41053d0 msgr2=0x7fdca419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:48.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.766+0000 7fdca9200640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca41053d0 0x7fdca419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.766+0000 7fdca9200640 1 -- 172.21.15.42:0/437473957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdca4105d90 msgr2=0x7fdca41a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:48.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.766+0000 7fdca9200640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdca4105d90 0x7fdca41a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.766+0000 7fdca9200640 1 -- 172.21.15.42:0/437473957 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca41a2060 con 0x7fdca41041e0 2024-04-20T10:33:48.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.766+0000 7fdca9200640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca41041e0 0x7fdca419a440 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800e9a0 tx=0x7fdc9800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:48.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.767+0000 7fdca1600640 1 -- 172.21.15.42:0/437473957 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9800cd80 con 0x7fdca41041e0 2024-04-20T10:33:48.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.767+0000 7fdca1600640 1 -- 172.21.15.42:0/437473957 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc98010910 con 0x7fdca41041e0 2024-04-20T10:33:48.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.767+0000 7fdcabc00640 1 -- 172.21.15.42:0/437473957 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca41a2350 con 0x7fdca41041e0 2024-04-20T10:33:48.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.767+0000 7fdca1600640 1 -- 172.21.15.42:0/437473957 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98018a30 con 0x7fdca41041e0 2024-04-20T10:33:48.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.767+0000 7fdcabc00640 1 -- 172.21.15.42:0/437473957 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca41a28f0 con 0x7fdca41041e0 2024-04-20T10:33:48.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.768+0000 7fdcabc00640 1 -- 172.21.15.42:0/437473957 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca410b4a0 con 0x7fdca41041e0 2024-04-20T10:33:48.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.770+0000 7fdca1600640 1 -- 172.21.15.42:0/437473957 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdc98010ac0 con 0x7fdca41041e0 2024-04-20T10:33:48.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.771+0000 7fdca1600640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdc8c0a3c30 0x7fdc8c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.771+0000 7fdca1600640 1 -- 172.21.15.42:0/437473957 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fdc980d0790 con 0x7fdca41041e0 2024-04-20T10:33:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.772+0000 7fdca3e00640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdc8c0a3c30 0x7fdc8c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.772+0000 7fdca3e00640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdc8c0a3c30 0x7fdc8c0a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdca419b950 tx=0x7fdc90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:48.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.775+0000 7fdca1600640 1 -- 172.21.15.42:0/437473957 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdc98053ff0 con 0x7fdca41041e0 2024-04-20T10:33:48.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:48 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3784290242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:33:48.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:48 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3678386802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:48.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.985+0000 7fdcabc00640 1 -- 172.21.15.42:0/437473957 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdca4062b60 con 0x7fdca41041e0 2024-04-20T10:33:48.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.987+0000 7fdca1600640 1 -- 172.21.15.42:0/437473957 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7fdc9809d0f0 con 0x7fdca41041e0 2024-04-20T10:33:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:48.988 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:32:31.928088+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:33:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.989+0000 7fdcabc00640 1 -- 172.21.15.42:0/437473957 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdc8c0a3c30 msgr2=0x7fdc8c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.990+0000 7fdcabc00640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdc8c0a3c30 0x7fdc8c0a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdca419b950 tx=0x7fdc90005f90 comp rx=0 tx=0).stop 2024-04-20T10:33:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.990+0000 7fdcabc00640 1 -- 172.21.15.42:0/437473957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca41041e0 msgr2=0x7fdca419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.990+0000 7fdcabc00640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca41041e0 0x7fdca419a440 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800e9a0 tx=0x7fdc9800ee60 comp rx=0 tx=0).stop 2024-04-20T10:33:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.990+0000 7fdcabc00640 1 -- 172.21.15.42:0/437473957 shutdown_connections 2024-04-20T10:33:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.990+0000 7fdcabc00640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdca4105d90 0x7fdca41a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.990+0000 7fdcabc00640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca41053d0 0x7fdca419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.990+0000 7fdcabc00640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdc8c0a3c30 0x7fdc8c0a60f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.990+0000 7fdcabc00640 1 --2- 172.21.15.42:0/437473957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca41041e0 0x7fdca419a440 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.990+0000 7fdcabc00640 1 -- 172.21.15.42:0/437473957 >> 172.21.15.42:0/437473957 conn(0x7fdca40ff750 msgr2=0x7fdca4108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.990+0000 7fdcabc00640 1 -- 172.21.15.42:0/437473957 shutdown_connections 2024-04-20T10:33:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:48.990+0000 7fdcabc00640 1 -- 172.21.15.42:0/437473957 wait complete. 2024-04-20T10:33:49.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-20T10:31:35.067527+0000', 'modified': '2024-04-20T10:31:39.263696+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24584}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24568': {'gid': 24568, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.162:6839/3710674499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6838', 'nonce': 3710674499}, {'type': 'v1', 'addr': '172.21.15.162:6839', 'nonce': 3710674499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24584': {'gid': 24584, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6839/2026912868', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6838', 'nonce': 2026912868}, {'type': 'v1', 'addr': '172.21.15.46:6839', 'nonce': 2026912868}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24584, 'qdb_cluster': [24584]} 2024-04-20T10:33:49.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T10:33:49.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T10:33:49.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:48 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3784290242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:33:49.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:48 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3678386802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:49.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:48 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3784290242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:33:49.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:48 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3678386802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:33:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.228+0000 7fdce2c00640 1 -- 172.21.15.42:0/475199483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc105290 msgr2=0x7fdcdc105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.228+0000 7fdce2c00640 1 --2- 172.21.15.42:0/475199483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc105290 0x7fdcdc105710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009900 tx=0x7fdcc802bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.228+0000 7fdce2c00640 1 -- 172.21.15.42:0/475199483 shutdown_connections 2024-04-20T10:33:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.228+0000 7fdce2c00640 1 --2- 172.21.15.42:0/475199483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcdc105c50 0x7fdcdc10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.228+0000 7fdce2c00640 1 --2- 172.21.15.42:0/475199483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc105290 0x7fdcdc105710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.228+0000 7fdce2c00640 1 --2- 172.21.15.42:0/475199483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcdc1040a0 0x7fdcdc1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.228+0000 7fdce2c00640 1 -- 172.21.15.42:0/475199483 >> 172.21.15.42:0/475199483 conn(0x7fdcdc0ff610 msgr2=0x7fdcdc101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.228+0000 7fdce2c00640 1 -- 172.21.15.42:0/475199483 shutdown_connections 2024-04-20T10:33:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.228+0000 7fdce2c00640 1 -- 172.21.15.42:0/475199483 wait complete. 2024-04-20T10:33:49.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.229+0000 7fdce2c00640 1 Processor -- start 2024-04-20T10:33:49.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.229+0000 7fdce2c00640 1 -- start start 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce2c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc1040a0 0x7fdcdc19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce2c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcdc105290 0x7fdcdc19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce2c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcdc105c50 0x7fdcdc1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce2c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc10fd80 con 0x7fdcdc105290 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce2c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc10fc00 con 0x7fdcdc1040a0 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce2c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc10ff00 con 0x7fdcdc105c50 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcdc105290 0x7fdcdc19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc1040a0 0x7fdcdc19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce2200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcdc105c50 0x7fdcdc1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc1040a0 0x7fdcdc19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40022/0 (socket says 172.21.15.42:40022) 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcdc105290 0x7fdcdc19a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59466/0 (socket says 172.21.15.42:59466) 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce1800640 1 -- 172.21.15.42:0/3561988854 learned_addr learned my addr 172.21.15.42:0/3561988854 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:49.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce2200640 1 -- 172.21.15.42:0/3561988854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc1040a0 msgr2=0x7fdcdc19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce2200640 1 --2- 172.21.15.42:0/3561988854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc1040a0 0x7fdcdc19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce2200640 1 -- 172.21.15.42:0/3561988854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcdc105290 msgr2=0x7fdcdc19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.230+0000 7fdce2200640 1 --2- 172.21.15.42:0/3561988854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcdc105290 0x7fdcdc19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.231+0000 7fdce2200640 1 -- 172.21.15.42:0/3561988854 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc1a1fc0 con 0x7fdcdc105c50 2024-04-20T10:33:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.231+0000 7fdce2200640 1 --2- 172.21.15.42:0/3561988854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcdc105c50 0x7fdcdc1a1950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009fd0 tx=0x7fdcd400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.231+0000 7fdcda000640 1 -- 172.21.15.42:0/3561988854 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400ce20 con 0x7fdcdc105c50 2024-04-20T10:33:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.231+0000 7fdcda000640 1 -- 172.21.15.42:0/3561988854 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd4005020 con 0x7fdcdc105c50 2024-04-20T10:33:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.231+0000 7fdce2c00640 1 -- 172.21.15.42:0/3561988854 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc1a22b0 con 0x7fdcdc105c50 2024-04-20T10:33:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.231+0000 7fdcda000640 1 -- 172.21.15.42:0/3561988854 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4010720 con 0x7fdcdc105c50 2024-04-20T10:33:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.231+0000 7fdce2c00640 1 -- 172.21.15.42:0/3561988854 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc1a27d0 con 0x7fdcdc105c50 2024-04-20T10:33:49.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.234+0000 7fdce6fef740 1 Processor -- start 2024-04-20T10:33:49.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.234+0000 7fdcda000640 1 -- 172.21.15.42:0/3561988854 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdcd4005540 con 0x7fdcdc105c50 2024-04-20T10:33:49.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce6fef740 1 -- start start 2024-04-20T10:33:49.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce6fef740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 0x56350675bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce6fef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675c530 0x563506761030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce6fef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635067616a0 0x563506763b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce6fef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5635066c29d0 con 0x5635067616a0 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce6fef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563506669270 con 0x56350675c530 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce6fef740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5635066be990 con 0x56350675bb40 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce0e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 0x56350675bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675c530 0x563506761030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce0e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 0x56350675bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:40988/0 (socket says 172.21.15.42:40988) 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675c530 0x563506761030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40032/0 (socket says 172.21.15.42:40032) 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce0e00640 1 -- 172.21.15.42:0/1983545002 learned_addr learned my addr 172.21.15.42:0/1983545002 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.235+0000 7fdce0e00640 1 --2- 172.21.15.42:0/1983545002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635067616a0 0x563506763b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.236+0000 7fdce1800640 1 -- 172.21.15.42:0/1983545002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 msgr2=0x56350675bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.236+0000 7fdce1800640 1 --2- 172.21.15.42:0/1983545002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 0x56350675bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.236+0000 7fdce1800640 1 -- 172.21.15.42:0/1983545002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635067616a0 msgr2=0x563506763b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.236+0000 7fdce1800640 1 --2- 172.21.15.42:0/1983545002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635067616a0 0x563506763b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.236+0000 7fdce1800640 1 -- 172.21.15.42:0/1983545002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635063d52d0 con 0x56350675c530 2024-04-20T10:33:49.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.236+0000 7fdce1800640 1 --2- 172.21.15.42:0/1983545002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675c530 0x563506761030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00058c0 tx=0x7fdcd0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0dd331451272bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.236+0000 7fdcda000640 1 --2- 172.21.15.42:0/3561988854 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdcc40a3c30 0x7fdcc40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.236+0000 7fdccea00640 1 -- 172.21.15.42:0/1983545002 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0004220 con 0x56350675c530 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.237+0000 7fdccea00640 1 -- 172.21.15.42:0/1983545002 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd00043c0 con 0x56350675c530 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.237+0000 7fdccea00640 1 -- 172.21.15.42:0/1983545002 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0004db0 con 0x56350675c530 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.237+0000 7fdcda000640 1 -- 172.21.15.42:0/3561988854 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fdcd40d0ac0 con 0x7fdcdc105c50 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.237+0000 7fdce0e00640 1 --2- 172.21.15.42:0/3561988854 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdcc40a3c30 0x7fdcc40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.237+0000 7fdce6fef740 1 -- 172.21.15.42:0/1983545002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675c530 msgr2=0x563506761030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.237+0000 7fdce6fef740 1 --2- 172.21.15.42:0/1983545002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675c530 0x563506761030 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00058c0 tx=0x7fdcd0011810 comp rx=0 tx=0).stop 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.237+0000 7fdce0e00640 1 --2- 172.21.15.42:0/3561988854 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdcc40a3c30 0x7fdcc40a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8005ab0 tx=0x7fdcc8045040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.238+0000 7fdce6fef740 1 -- 172.21.15.42:0/1983545002 shutdown_connections 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.238+0000 7fdce6fef740 1 --2- 172.21.15.42:0/1983545002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5635067616a0 0x563506763b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.238+0000 7fdce6fef740 1 --2- 172.21.15.42:0/1983545002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675c530 0x563506761030 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.238+0000 7fdce6fef740 1 --2- 172.21.15.42:0/1983545002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 0x56350675bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.238+0000 7fdce6fef740 1 -- 172.21.15.42:0/1983545002 >> 172.21.15.42:0/1983545002 conn(0x5635067570b0 msgr2=0x5635067574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:49.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.238+0000 7fdce6fef740 1 -- 172.21.15.42:0/1983545002 shutdown_connections 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.238+0000 7fdce6fef740 1 -- 172.21.15.42:0/1983545002 wait complete. 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.238+0000 7fdce6fef740 1 -- start start 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.238+0000 7fdce6fef740 1 Processor -- start 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce6fef740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 0x563506766040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce6fef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675c530 0x563506766580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce6fef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635067616a0 0x56350676d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce6fef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563506764650 con 0x56350675c530 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce6fef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56350667c5e0 con 0x5635067616a0 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce6fef740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5635067647d0 con 0x56350675bb40 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce6fef740 10 client.0 _ll_register_callbacks cb 0x5635063d4ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce1800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 0x563506766040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675c530 0x563506766580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce1800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 0x563506766040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:41000/0 (socket says 172.21.15.42:41000) 2024-04-20T10:33:49.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce1800640 1 -- 172.21.15.42:0/3587550320 learned_addr learned my addr 172.21.15.42:0/3587550320 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce2200640 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635067616a0 0x56350676d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce1800640 1 -- 172.21.15.42:0/3587550320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635067616a0 msgr2=0x56350676d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce1800640 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635067616a0 0x56350676d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce1800640 1 -- 172.21.15.42:0/3587550320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675c530 msgr2=0x563506766580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.239+0000 7fdce1800640 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675c530 0x563506766580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.240+0000 7fdce1800640 1 -- 172.21.15.42:0/3587550320 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350649f6b0 con 0x56350675bb40 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.240+0000 7fdce0e00640 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675c530 0x563506766580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.240+0000 7fdce1800640 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 0x563506766040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0011d10 tx=0x7fdcd0005210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=804778f5a790e697 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.240+0000 7fdcc3400640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.240+0000 7fdcc3400640 1 -- 172.21.15.42:0/3587550320 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0027030 con 0x56350675bb40 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.240+0000 7fdcc3400640 1 -- 172.21.15.42:0/3587550320 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fdcd0019430 con 0x56350675bb40 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.240+0000 7fdce6fef740 1 -- 172.21.15.42:0/3587550320 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd00043c0 con 0x56350675bb40 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.240+0000 7fdcc3400640 1 -- 172.21.15.42:0/3587550320 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdcd0032070 con 0x56350675bb40 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.240+0000 7fdce6fef740 10 client.25126 fetch_fsmap learned FSMap version 19 2024-04-20T10:33:49.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.241+0000 7fdce6fef740 1 -- 172.21.15.42:0/3587550320 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563506770930 con 0x56350675bb40 2024-04-20T10:33:49.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.241+0000 7fdcc3400640 1 -- 172.21.15.42:0/3587550320 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd00043c0 con 0x56350675bb40 2024-04-20T10:33:49.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.241+0000 7fdcc3400640 1 -- 172.21.15.42:0/3587550320 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fdcd0019c50 con 0x56350675bb40 2024-04-20T10:33:49.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.241+0000 7fdce6fef740 10 client.25126 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:33:49.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.241+0000 7fdce6fef740 10 client.25126 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T10:33:49.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.241+0000 7fdce6fef740 20 client.25126 populate_metadata read hostname 'smithi042' 2024-04-20T10:33:49.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.241+0000 7fdce6fef740 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563506772d90 0x563506774200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.241+0000 7fdce6fef740 4 client.25126 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:33:49.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.241+0000 7fdce6fef740 1 -- 172.21.15.42:0/3587550320 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635066d03f0 con 0x563506772d90 2024-04-20T10:33:49.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.242+0000 7fdce0e00640 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563506772d90 0x563506774200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.242+0000 7fdce0e00640 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563506772d90 0x563506774200 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b517cab585a8c38 server_cookie=bbce183f3ffe4eb0 in_seq=0 out_seq=0 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.242+0000 7fdcc3400640 10 client.25126 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.243+0000 7fdcc3400640 1 -- 172.21.15.42:0/3587550320 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635066d03f0 con 0x563506772d90 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.243+0000 7fdcc3400640 10 client.25126 handle_command_reply: tid=0 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.243+0000 7fdce6fef740 1 client.25126 shutdown 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.243+0000 7fdce6fef740 1 -- 172.21.15.42:0/3587550320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 msgr2=0x563506766040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.243+0000 7fdce6fef740 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 0x563506766040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0011d10 tx=0x7fdcd0005210 comp rx=0 tx=0).stop 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.243+0000 7fdce6fef740 1 -- 172.21.15.42:0/3587550320 shutdown_connections 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.243+0000 7fdce6fef740 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563506772d90 0x563506774200 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.243+0000 7fdce6fef740 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635067616a0 0x56350676d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.243+0000 7fdce6fef740 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675c530 0x563506766580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.243+0000 7fdcc3400640 0 client.25126 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.243+0000 7fdce6fef740 1 --2- 172.21.15.42:0/3587550320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675bb40 0x563506766040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.243+0000 7fdce6fef740 1 -- 172.21.15.42:0/3587550320 >> 172.21.15.42:0/3587550320 conn(0x5635067570b0 msgr2=0x5635067563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:49.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.244+0000 7fdce6fef740 1 -- 172.21.15.42:0/3587550320 shutdown_connections 2024-04-20T10:33:49.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.244+0000 7fdce6fef740 1 -- 172.21.15.42:0/3587550320 wait complete. 2024-04-20T10:33:49.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.244+0000 7fdce6fef740 20 client.25126 trim_cache size 0 max 16384 2024-04-20T10:33:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.264+0000 7fdce6fef740 1 Processor -- start 2024-04-20T10:33:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.264+0000 7fdce6fef740 1 -- start start 2024-04-20T10:33:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.264+0000 7fdce6fef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675bb40 0x563506763a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.264+0000 7fdce6fef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675c530 0x56350676d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.264+0000 7fdce6fef740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635067616a0 0x563506771d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.264+0000 7fdce6fef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563506772540 con 0x56350675c530 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce6fef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563506772c10 con 0x56350675bb40 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce6fef740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5635067732e0 con 0x5635067616a0 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675bb40 0x563506763a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675bb40 0x563506763a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40060/0 (socket says 172.21.15.42:40060) 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce1800640 1 -- 172.21.15.42:0/2984254914 learned_addr learned my addr 172.21.15.42:0/2984254914 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce0e00640 1 --2- 172.21.15.42:0/2984254914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675c530 0x56350676d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce2200640 1 --2- 172.21.15.42:0/2984254914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635067616a0 0x563506771d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce1800640 1 -- 172.21.15.42:0/2984254914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635067616a0 msgr2=0x563506771d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce1800640 1 --2- 172.21.15.42:0/2984254914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635067616a0 0x563506771d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce1800640 1 -- 172.21.15.42:0/2984254914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675c530 msgr2=0x56350676d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce1800640 1 --2- 172.21.15.42:0/2984254914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675c530 0x56350676d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce1800640 1 -- 172.21.15.42:0/2984254914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350675c910 con 0x56350675bb40 2024-04-20T10:33:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.265+0000 7fdce0e00640 1 --2- 172.21.15.42:0/2984254914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675c530 0x56350676d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:33:49.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.266+0000 7fdce1800640 1 --2- 172.21.15.42:0/2984254914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675bb40 0x563506763a90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0011f50 tx=0x7fdcd0004e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdfd57320edcdc2c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:49.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.266+0000 7fdcce000640 1 -- 172.21.15.42:0/2984254914 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0027040 con 0x56350675bb40 2024-04-20T10:33:49.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.266+0000 7fdcce000640 1 -- 172.21.15.42:0/2984254914 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd0005280 con 0x56350675bb40 2024-04-20T10:33:49.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.266+0000 7fdcce000640 1 -- 172.21.15.42:0/2984254914 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd001d040 con 0x56350675bb40 2024-04-20T10:33:49.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.266+0000 7fdce6fef740 1 -- 172.21.15.42:0/2984254914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675bb40 msgr2=0x563506763a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:49.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.266+0000 7fdce6fef740 1 --2- 172.21.15.42:0/2984254914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675bb40 0x563506763a90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0011f50 tx=0x7fdcd0004e80 comp rx=0 tx=0).stop 2024-04-20T10:33:49.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.267+0000 7fdce6fef740 1 -- 172.21.15.42:0/2984254914 shutdown_connections 2024-04-20T10:33:49.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.267+0000 7fdce6fef740 1 --2- 172.21.15.42:0/2984254914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5635067616a0 0x563506771d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.267+0000 7fdce6fef740 1 --2- 172.21.15.42:0/2984254914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675c530 0x56350676d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.267+0000 7fdce6fef740 1 --2- 172.21.15.42:0/2984254914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56350675bb40 0x563506763a90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.267+0000 7fdce6fef740 1 -- 172.21.15.42:0/2984254914 >> 172.21.15.42:0/2984254914 conn(0x5635067570b0 msgr2=0x56350675aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.267+0000 7fdce6fef740 1 -- 172.21.15.42:0/2984254914 shutdown_connections 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.267+0000 7fdce6fef740 1 -- 172.21.15.42:0/2984254914 wait complete. 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.267+0000 7fdce6fef740 1 -- start start 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.267+0000 7fdce6fef740 1 Processor -- start 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce6fef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675bb40 0x56350675fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce6fef740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675c530 0x563506760170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce6fef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635067616a0 0x56350676bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce6fef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563506774080 con 0x56350675bb40 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce6fef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5635067739b0 con 0x5635067616a0 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce6fef740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563506774750 con 0x56350675c530 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce6fef740 10 client.0 _ll_register_callbacks cb 0x7fdcc8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce0e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675c530 0x563506760170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675bb40 0x56350675fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce0e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675c530 0x563506760170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:41012/0 (socket says 172.21.15.42:41012) 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce0e00640 1 -- 172.21.15.42:0/2634117759 learned_addr learned my addr 172.21.15.42:0/2634117759 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce2200640 1 --2- 172.21.15.42:0/2634117759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635067616a0 0x56350676bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce0e00640 1 -- 172.21.15.42:0/2634117759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635067616a0 msgr2=0x56350676bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce0e00640 1 --2- 172.21.15.42:0/2634117759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635067616a0 0x56350676bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce0e00640 1 -- 172.21.15.42:0/2634117759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675bb40 msgr2=0x56350675fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce0e00640 1 --2- 172.21.15.42:0/2634117759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675bb40 0x56350675fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.268+0000 7fdce0e00640 1 -- 172.21.15.42:0/2634117759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350676c1b0 con 0x56350675c530 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.269+0000 7fdce0e00640 1 --2- 172.21.15.42:0/2634117759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675c530 0x563506760170 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdcc802c240 tx=0x7fdcc802df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2dc4c38aeb25a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.269+0000 7fdcc3400640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.269+0000 7fdcc3400640 1 -- 172.21.15.42:0/2634117759 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc805b060 con 0x56350675c530 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.269+0000 7fdcc3400640 1 -- 172.21.15.42:0/2634117759 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fdcc8056b50 con 0x56350675c530 2024-04-20T10:33:49.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.269+0000 7fdce6fef740 1 -- 172.21.15.42:0/2634117759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd0005280 con 0x56350675c530 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.269+0000 7fdcc3400640 1 -- 172.21.15.42:0/2634117759 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdcc8056d50 con 0x56350675c530 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.269+0000 7fdce6fef740 10 client.25132 fetch_fsmap learned FSMap version 19 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.269+0000 7fdce6fef740 1 -- 172.21.15.42:0/2634117759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56350676d530 con 0x56350675c530 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.270+0000 7fdcc3400640 1 -- 172.21.15.42:0/2634117759 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd0005280 con 0x56350675c530 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.270+0000 7fdcc3400640 1 -- 172.21.15.42:0/2634117759 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fdcc804b040 con 0x56350675c530 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.270+0000 7fdce6fef740 10 client.25132 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.270+0000 7fdce6fef740 10 client.25132 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.270+0000 7fdce6fef740 20 client.25132 populate_metadata read hostname 'smithi042' 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.270+0000 7fdce6fef740 1 --2- 172.21.15.42:0/2634117759 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5635067e5bd0 0x5635067e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.270+0000 7fdce6fef740 4 client.25132 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.270+0000 7fdce6fef740 1 -- 172.21.15.42:0/2634117759 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "status"}) -- 0x5635066ca260 con 0x5635067e5bd0 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.270+0000 7fdce1800640 1 --2- 172.21.15.42:0/2634117759 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5635067e5bd0 0x5635067e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdce1800640 1 --2- 172.21.15.42:0/2634117759 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5635067e5bd0 0x5635067e48e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9511233fdc0054f3 server_cookie=60287baa02fd6805 in_seq=0 out_seq=0 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdcc3400640 10 client.25132 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:33:49.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdcc3400640 1 -- 172.21.15.42:0/2634117759 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5635066ca260 con 0x5635067e5bd0 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdcc3400640 10 client.25132 handle_command_reply: tid=0 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdce6fef740 1 client.25132 shutdown 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdce6fef740 1 -- 172.21.15.42:0/2634117759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675c530 msgr2=0x563506760170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdce6fef740 1 --2- 172.21.15.42:0/2634117759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675c530 0x563506760170 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdcc802c240 tx=0x7fdcc802df90 comp rx=0 tx=0).stop 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdce6fef740 1 -- 172.21.15.42:0/2634117759 shutdown_connections 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdce6fef740 1 --2- 172.21.15.42:0/2634117759 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5635067e5bd0 0x5635067e48e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdce6fef740 1 --2- 172.21.15.42:0/2634117759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635067616a0 0x56350676bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdce6fef740 1 --2- 172.21.15.42:0/2634117759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56350675c530 0x563506760170 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdce6fef740 1 --2- 172.21.15.42:0/2634117759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56350675bb40 0x56350675fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdce6fef740 1 -- 172.21.15.42:0/2634117759 >> 172.21.15.42:0/2634117759 conn(0x5635067570b0 msgr2=0x56350675aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.271+0000 7fdcc3400640 0 client.25132 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce6fef740 1 -- 172.21.15.42:0/2634117759 shutdown_connections 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce6fef740 1 -- 172.21.15.42:0/2634117759 wait complete. 2024-04-20T10:33:49.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce6fef740 20 client.25132 trim_cache size 0 max 16384 2024-04-20T10:33:49.277 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:33:49.277 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "83e8f766-feff-11ee-bc92-c7b262605968", 2024-04-20T10:33:49.277 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-04-20T10:33:49.277 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24584, 2024-04-20T10:33:49.277 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-04-20T10:33:49.277 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-04-20T10:33:49.277 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-04-20T10:33:49.277 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 133.54846810699999, 2024-04-20T10:33:49.277 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 19, 2024-04-20T10:33:49.277 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 87, 2024-04-20T10:33:49.277 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T10:33:49.278 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 139.03644625000001 2024-04-20T10:33:49.278 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:33:49.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce2c00640 1 -- 172.21.15.42:0/3561988854 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdcc40a3c30 msgr2=0x7fdcc40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:49.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce2c00640 1 --2- 172.21.15.42:0/3561988854 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdcc40a3c30 0x7fdcc40a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8005ab0 tx=0x7fdcc8045040 comp rx=0 tx=0).stop 2024-04-20T10:33:49.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce2c00640 1 -- 172.21.15.42:0/3561988854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcdc105c50 msgr2=0x7fdcdc1a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:49.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce2c00640 1 --2- 172.21.15.42:0/3561988854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcdc105c50 0x7fdcdc1a1950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009fd0 tx=0x7fdcd400ef10 comp rx=0 tx=0).stop 2024-04-20T10:33:49.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce2c00640 1 -- 172.21.15.42:0/3561988854 shutdown_connections 2024-04-20T10:33:49.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce2c00640 1 --2- 172.21.15.42:0/3561988854 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fdcc40a3c30 0x7fdcc40a60f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce2c00640 1 --2- 172.21.15.42:0/3561988854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcdc105c50 0x7fdcdc1a1950 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce2c00640 1 --2- 172.21.15.42:0/3561988854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdcdc105290 0x7fdcdc19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce2c00640 1 --2- 172.21.15.42:0/3561988854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc1040a0 0x7fdcdc19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.272+0000 7fdce2c00640 1 -- 172.21.15.42:0/3561988854 >> 172.21.15.42:0/3561988854 conn(0x7fdcdc0ff610 msgr2=0x7fdcdc108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:49.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.273+0000 7fdce2c00640 1 -- 172.21.15.42:0/3561988854 shutdown_connections 2024-04-20T10:33:49.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.273+0000 7fdce2c00640 1 -- 172.21.15.42:0/3561988854 wait complete. 2024-04-20T10:33:49.291 INFO:teuthology.run_tasks:Running task sequential... 2024-04-20T10:33:49.301 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-20T10:33:49.301 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-20T10:33:49.302 INFO:teuthology.task.print:Enabling mgr modules 2024-04-20T10:33:49.302 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-20T10:33:49.302 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T10:33:49.302 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-04-20T10:33:49.303 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-20T10:33:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.492+0000 7f4edea00640 1 -- 172.21.15.42:0/265063660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed8101ef0 msgr2=0x7f4ed8102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:49.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.492+0000 7f4edea00640 1 --2- 172.21.15.42:0/265063660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed8101ef0 0x7f4ed8102370 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009900 tx=0x7f4ec402bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.492+0000 7f4edea00640 1 -- 172.21.15.42:0/265063660 shutdown_connections 2024-04-20T10:33:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.492+0000 7f4edea00640 1 --2- 172.21.15.42:0/265063660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ed81028b0 0x7f4ed8109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.492+0000 7f4edea00640 1 --2- 172.21.15.42:0/265063660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed8101ef0 0x7f4ed8102370 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.492+0000 7f4edea00640 1 --2- 172.21.15.42:0/265063660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ed8100d00 0x7f4ed8101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.493+0000 7f4edea00640 1 -- 172.21.15.42:0/265063660 >> 172.21.15.42:0/265063660 conn(0x7f4ed80fc290 msgr2=0x7f4ed80fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.493+0000 7f4edea00640 1 -- 172.21.15.42:0/265063660 shutdown_connections 2024-04-20T10:33:49.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.493+0000 7f4edea00640 1 -- 172.21.15.42:0/265063660 wait complete. 2024-04-20T10:33:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.494+0000 7f4edea00640 1 Processor -- start 2024-04-20T10:33:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.494+0000 7f4edea00640 1 -- start start 2024-04-20T10:33:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.494+0000 7f4edea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ed8100d00 0x7f4ed819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.494+0000 7f4edea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ed8101ef0 0x7f4ed819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.494+0000 7f4edea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed81028b0 0x7f4ed81a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.494+0000 7f4edea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed810db00 con 0x7f4ed81028b0 2024-04-20T10:33:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.494+0000 7f4edea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed810d980 con 0x7f4ed8101ef0 2024-04-20T10:33:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.494+0000 7f4edea00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed810dc80 con 0x7f4ed8100d00 2024-04-20T10:33:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4edd600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ed8100d00 0x7f4ed819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4edd600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ed8100d00 0x7f4ed819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:41030/0 (socket says 172.21.15.42:41030) 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4edd600640 1 -- 172.21.15.42:0/845304438 learned_addr learned my addr 172.21.15.42:0/845304438 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4ede000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed81028b0 0x7f4ed81a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4edcc00640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ed8101ef0 0x7f4ed819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4edd600640 1 -- 172.21.15.42:0/845304438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ed8101ef0 msgr2=0x7f4ed819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4edd600640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ed8101ef0 0x7f4ed819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4edd600640 1 -- 172.21.15.42:0/845304438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed81028b0 msgr2=0x7f4ed81a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4edd600640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed81028b0 0x7f4ed81a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4edd600640 1 -- 172.21.15.42:0/845304438 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed81a1f50 con 0x7f4ed8100d00 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4edd600640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ed8100d00 0x7f4ed819a380 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00e9c0 tx=0x7f4ecc00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4ed6000640 1 -- 172.21.15.42:0/845304438 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00cd80 con 0x7f4ed8100d00 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4ed6000640 1 -- 172.21.15.42:0/845304438 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ecc010910 con 0x7f4ed8100d00 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.495+0000 7f4edea00640 1 -- 172.21.15.42:0/845304438 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed81a2240 con 0x7f4ed8100d00 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.496+0000 7f4ed6000640 1 -- 172.21.15.42:0/845304438 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc018950 con 0x7f4ed8100d00 2024-04-20T10:33:49.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.496+0000 7f4edea00640 1 -- 172.21.15.42:0/845304438 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed81a27b0 con 0x7f4ed8100d00 2024-04-20T10:33:49.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.497+0000 7f4edea00640 1 -- 172.21.15.42:0/845304438 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed81a2a10 con 0x7f4ed8100d00 2024-04-20T10:33:49.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.499+0000 7f4ed6000640 1 -- 172.21.15.42:0/845304438 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4ecc01f020 con 0x7f4ed8100d00 2024-04-20T10:33:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.500+0000 7f4ed6000640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4ec00a3c20 0x7f4ec00a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:49.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.500+0000 7f4ed6000640 1 -- 172.21.15.42:0/845304438 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f4ecc0d0490 con 0x7f4ed8100d00 2024-04-20T10:33:49.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.500+0000 7f4edcc00640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4ec00a3c20 0x7f4ec00a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:49.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.501+0000 7f4edcc00640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4ec00a3c20 0x7f4ec00a60e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4ec402c240 tx=0x7f4ec402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:49.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.504+0000 7f4ed6000640 1 -- 172.21.15.42:0/845304438 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4ecc09cdf0 con 0x7f4ed8100d00 2024-04-20T10:33:49.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:49.769+0000 7f4edea00640 1 -- 172.21.15.42:0/845304438 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f4ed81a3160 con 0x7f4ed8100d00 2024-04-20T10:33:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:49 smithi046 ceph-mon[21574]: pgmap v386: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:49 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/437473957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:49 smithi042 ceph-mon[17078]: pgmap v386: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:49 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/437473957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:49 smithi162 ceph-mon[21854]: pgmap v386: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:33:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:49 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/437473957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:33:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.691+0000 7f4ed6000640 1 -- 172.21.15.42:0/845304438 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f4ecc010430 con 0x7f4ed8100d00 2024-04-20T10:33:50.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.692+0000 7f4ed6000640 1 -- 172.21.15.42:0/845304438 <== mon.1 v2:172.21.15.162:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f4ecc064c90 con 0x7f4ed8100d00 2024-04-20T10:33:50.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.697+0000 7f4edea00640 1 -- 172.21.15.42:0/845304438 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4ec00a3c20 msgr2=0x7f4ec00a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:50.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.697+0000 7f4edea00640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4ec00a3c20 0x7f4ec00a60e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4ec402c240 tx=0x7f4ec402df90 comp rx=0 tx=0).stop 2024-04-20T10:33:50.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.697+0000 7f4edea00640 1 -- 172.21.15.42:0/845304438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ed8100d00 msgr2=0x7f4ed819a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:50.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.697+0000 7f4edea00640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ed8100d00 0x7f4ed819a380 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00e9c0 tx=0x7f4ecc00edc0 comp rx=0 tx=0).stop 2024-04-20T10:33:50.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.697+0000 7f4edea00640 1 -- 172.21.15.42:0/845304438 shutdown_connections 2024-04-20T10:33:50.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.697+0000 7f4edea00640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed81028b0 0x7f4ed81a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:50.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.697+0000 7f4edea00640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ed8101ef0 0x7f4ed819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:50.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.697+0000 7f4edea00640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f4ec00a3c20 0x7f4ec00a60e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:50.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.697+0000 7f4edea00640 1 --2- 172.21.15.42:0/845304438 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ed8100d00 0x7f4ed819a380 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:50.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.697+0000 7f4edea00640 1 -- 172.21.15.42:0/845304438 >> 172.21.15.42:0/845304438 conn(0x7f4ed80fc290 msgr2=0x7f4ed8107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:50.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.697+0000 7f4edea00640 1 -- 172.21.15.42:0/845304438 shutdown_connections 2024-04-20T10:33:50.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.697+0000 7f4edea00640 1 -- 172.21.15.42:0/845304438 wait complete. 2024-04-20T10:33:50.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-20T10:33:50.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.902+0000 7f49fd600640 1 -- 172.21.15.42:0/3034005461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f8103b90 msgr2=0x7f49f810a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:50.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.903+0000 7f49fd600640 1 --2- 172.21.15.42:0/3034005461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f8103b90 0x7f49f810a420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009ca0 tx=0x7f49f002c100 comp rx=0 tx=0).stop 2024-04-20T10:33:50.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.903+0000 7f49fd600640 1 -- 172.21.15.42:0/3034005461 shutdown_connections 2024-04-20T10:33:50.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.903+0000 7f49fd600640 1 --2- 172.21.15.42:0/3034005461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f8103b90 0x7f49f810a420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:50.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.903+0000 7f49fd600640 1 --2- 172.21.15.42:0/3034005461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f81031d0 0x7f49f8103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:50.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.903+0000 7f49fd600640 1 --2- 172.21.15.42:0/3034005461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f8101fe0 0x7f49f8102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:50.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.903+0000 7f49fd600640 1 -- 172.21.15.42:0/3034005461 >> 172.21.15.42:0/3034005461 conn(0x7f49f80fd550 msgr2=0x7f49f80ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:50.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.903+0000 7f49fd600640 1 -- 172.21.15.42:0/3034005461 shutdown_connections 2024-04-20T10:33:50.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.903+0000 7f49fd600640 1 -- 172.21.15.42:0/3034005461 wait complete. 2024-04-20T10:33:50.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.904+0000 7f49fd600640 1 Processor -- start 2024-04-20T10:33:50.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.904+0000 7f49fd600640 1 -- start start 2024-04-20T10:33:50.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.904+0000 7f49fd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f8101fe0 0x7f49f819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:50.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.904+0000 7f49fd600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f81031d0 0x7f49f819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:50.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.904+0000 7f49fd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f8103b90 0x7f49f81a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.904+0000 7f49fd600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f49f810dc70 con 0x7f49f8101fe0 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.904+0000 7f49fd600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49f810daf0 con 0x7f49f8103b90 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.904+0000 7f49fd600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f49f810ddf0 con 0x7f49f81031d0 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.905+0000 7f49f6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f8101fe0 0x7f49f819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.905+0000 7f49f6000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f81031d0 0x7f49f819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.905+0000 7f49f6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f8101fe0 0x7f49f819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59546/0 (socket says 172.21.15.42:59546) 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.905+0000 7f49f6a00640 1 -- 172.21.15.42:0/907811241 learned_addr learned my addr 172.21.15.42:0/907811241 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.905+0000 7f49f6000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f81031d0 0x7f49f819a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:41048/0 (socket says 172.21.15.42:41048) 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.905+0000 7f49f7400640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f8103b90 0x7f49f81a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.905+0000 7f49f6000640 1 -- 172.21.15.42:0/907811241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f8103b90 msgr2=0x7f49f81a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.905+0000 7f49f6000640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f8103b90 0x7f49f81a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.905+0000 7f49f6000640 1 -- 172.21.15.42:0/907811241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f8101fe0 msgr2=0x7f49f819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.905+0000 7f49f6000640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f8101fe0 0x7f49f819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.905+0000 7f49f6000640 1 -- 172.21.15.42:0/907811241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f81a2010 con 0x7f49f81031d0 2024-04-20T10:33:50.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.905+0000 7f49f6000640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f81031d0 0x7f49f819a930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f49e000e9f0 tx=0x7f49e000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:50.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.906+0000 7f49ef400640 1 -- 172.21.15.42:0/907811241 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e000ccc0 con 0x7f49f81031d0 2024-04-20T10:33:50.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.906+0000 7f49ef400640 1 -- 172.21.15.42:0/907811241 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e0005020 con 0x7f49f81031d0 2024-04-20T10:33:50.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.906+0000 7f49fd600640 1 -- 172.21.15.42:0/907811241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f81a2300 con 0x7f49f81031d0 2024-04-20T10:33:50.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.906+0000 7f49ef400640 1 -- 172.21.15.42:0/907811241 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e00049e0 con 0x7f49f81031d0 2024-04-20T10:33:50.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.906+0000 7f49fd600640 1 -- 172.21.15.42:0/907811241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f81a2800 con 0x7f49f81031d0 2024-04-20T10:33:50.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.907+0000 7f49fd600640 1 -- 172.21.15.42:0/907811241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f81a2ac0 con 0x7f49f81031d0 2024-04-20T10:33:50.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.909+0000 7f49ef400640 1 -- 172.21.15.42:0/907811241 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f49e0005560 con 0x7f49f81031d0 2024-04-20T10:33:50.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.910+0000 7f49ef400640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f49dc0a3ca0 0x7f49dc0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:50.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.911+0000 7f49ef400640 1 -- 172.21.15.42:0/907811241 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f49e00d0460 con 0x7f49f81031d0 2024-04-20T10:33:50.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.911+0000 7f49f6a00640 1 -- 172.21.15.42:0/907811241 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f49dc0a3ca0 msgr2=0x7f49dc0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3610568749 2024-04-20T10:33:50.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.911+0000 7f49f6a00640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f49dc0a3ca0 0x7f49dc0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:33:50.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:50.914+0000 7f49ef400640 1 -- 172.21.15.42:0/907811241 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f49e00051d0 con 0x7f49f81031d0 2024-04-20T10:33:51.012 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:50 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/845304438' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T10:33:51.012 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:50 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T10:33:51.012 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: ignoring --setuser ceph since I am not root 2024-04-20T10:33:51.012 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: ignoring --setgroup ceph since I am not root 2024-04-20T10:33:51.012 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:51 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:51.010+0000 7fd504023200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:33:51.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:50 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/845304438' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T10:33:51.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:50 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T10:33:51.052 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:50 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: ignoring --setuser ceph since I am not root 2024-04-20T10:33:51.052 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:50 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: ignoring --setgroup ceph since I am not root 2024-04-20T10:33:51.052 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:50 smithi162 ceph-mgr[23744]: -- 172.21.15.162:0/700124185 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596b14765a0 con 0x5596b144a800 2024-04-20T10:33:51.052 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:50 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:50.861+0000 7fcf94e26200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:33:51.052 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:51.051+0000 7fcf94e26200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:33:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:50 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/845304438' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T10:33:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:50 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T10:33:51.053 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:50 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: ignoring --setuser ceph since I am not root 2024-04-20T10:33:51.053 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:50 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: ignoring --setgroup ceph since I am not root 2024-04-20T10:33:51.053 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:50 smithi046 ceph-mgr[22974]: -- 172.21.15.46:0/2256740493 <== mon.2 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556414c3e5a0 con 0x556414c12c00 2024-04-20T10:33:51.053 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:50 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:50.864+0000 7fe98f8bf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:33:51.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.112+0000 7f49f6a00640 1 -- 172.21.15.42:0/907811241 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f49dc0a3ca0 msgr2=0x7f49dc0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3610568749 2024-04-20T10:33:51.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.112+0000 7f49f6a00640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f49dc0a3ca0 0x7f49dc0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:33:51.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.125+0000 7f49fd600640 1 -- 172.21.15.42:0/907811241 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f49f81a3200 con 0x7f49f81031d0 2024-04-20T10:33:51.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.132+0000 7f49ef400640 1 -- 172.21.15.42:0/907811241 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f49e009cdc0 con 0x7f49f81031d0 2024-04-20T10:33:51.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.134+0000 7f49fd600640 1 -- 172.21.15.42:0/907811241 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f49dc0a3ca0 msgr2=0x7f49dc0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:33:51.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.134+0000 7f49fd600640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f49dc0a3ca0 0x7f49dc0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.134+0000 7f49fd600640 1 -- 172.21.15.42:0/907811241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f81031d0 msgr2=0x7f49f819a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:51.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.134+0000 7f49fd600640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f81031d0 0x7f49f819a930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f49e000e9f0 tx=0x7f49e000eeb0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.135+0000 7f49fd600640 1 -- 172.21.15.42:0/907811241 shutdown_connections 2024-04-20T10:33:51.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.135+0000 7f49fd600640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f8103b90 0x7f49f81a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.135+0000 7f49fd600640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f49dc0a3ca0 0x7f49dc0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.135+0000 7f49fd600640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49f81031d0 0x7f49f819a930 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.135+0000 7f49fd600640 1 --2- 172.21.15.42:0/907811241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f8101fe0 0x7f49f819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.135+0000 7f49fd600640 1 -- 172.21.15.42:0/907811241 >> 172.21.15.42:0/907811241 conn(0x7f49f80fd550 msgr2=0x7f49f8074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:51.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.135+0000 7f49fd600640 1 -- 172.21.15.42:0/907811241 shutdown_connections 2024-04-20T10:33:51.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.135+0000 7f49fd600640 1 -- 172.21.15.42:0/907811241 wait complete. 2024-04-20T10:33:51.154 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-20T10:33:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.336+0000 7f0465e00640 1 -- 172.21.15.42:0/2142700952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0460102b00 msgr2=0x7f0460104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.336+0000 7f0465e00640 1 --2- 172.21.15.42:0/2142700952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0460102b00 0x7f0460104ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0450009900 tx=0x7f045002bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.336+0000 7f0465e00640 1 -- 172.21.15.42:0/2142700952 shutdown_connections 2024-04-20T10:33:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.336+0000 7f0465e00640 1 --2- 172.21.15.42:0/2142700952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0460107d60 0x7f046010a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.336+0000 7f0465e00640 1 --2- 172.21.15.42:0/2142700952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0460105430 0x7f0460107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.336+0000 7f0465e00640 1 --2- 172.21.15.42:0/2142700952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0460102b00 0x7f0460104ef0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.336+0000 7f0465e00640 1 -- 172.21.15.42:0/2142700952 >> 172.21.15.42:0/2142700952 conn(0x7f04600fc450 msgr2=0x7f04600fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:51.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.336+0000 7f0465e00640 1 -- 172.21.15.42:0/2142700952 shutdown_connections 2024-04-20T10:33:51.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.336+0000 7f0465e00640 1 -- 172.21.15.42:0/2142700952 wait complete. 2024-04-20T10:33:51.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.337+0000 7f0465e00640 1 Processor -- start 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.337+0000 7f0465e00640 1 -- start start 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.337+0000 7f0465e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0460102b00 0x7f0460198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f0465e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0460105430 0x7f0460198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f0465e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0460107d60 0x7f046019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f0465e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f046010dcb0 con 0x7f0460105430 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f0465e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f046010db30 con 0x7f0460107d60 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f0465e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f046010de30 con 0x7f0460102b00 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f045ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0460102b00 0x7f0460198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f045e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0460105430 0x7f0460198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f045ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0460102b00 0x7f0460198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:41062/0 (socket says 172.21.15.42:41062) 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f045ea00640 1 -- 172.21.15.42:0/1867203945 learned_addr learned my addr 172.21.15.42:0/1867203945 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f045f400640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0460107d60 0x7f046019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:51.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f045ea00640 1 -- 172.21.15.42:0/1867203945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0460107d60 msgr2=0x7f046019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:51.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f045ea00640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0460107d60 0x7f046019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f045ea00640 1 -- 172.21.15.42:0/1867203945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0460105430 msgr2=0x7f0460198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:51.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f045ea00640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0460105430 0x7f0460198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f045ea00640 1 -- 172.21.15.42:0/1867203945 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046019fe40 con 0x7f0460102b00 2024-04-20T10:33:51.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.338+0000 7f045e000640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0460105430 0x7f0460198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:51.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.339+0000 7f045ea00640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0460102b00 0x7f0460198220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0450000c00 tx=0x7f045002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:51.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.339+0000 7f0464a00640 1 -- 172.21.15.42:0/1867203945 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450043020 con 0x7f0460102b00 2024-04-20T10:33:51.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.339+0000 7f0464a00640 1 -- 172.21.15.42:0/1867203945 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04500054e0 con 0x7f0460102b00 2024-04-20T10:33:51.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.339+0000 7f0465e00640 1 -- 172.21.15.42:0/1867203945 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04601a00d0 con 0x7f0460102b00 2024-04-20T10:33:51.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.339+0000 7f0464a00640 1 -- 172.21.15.42:0/1867203945 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450003c40 con 0x7f0460102b00 2024-04-20T10:33:51.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.339+0000 7f0465e00640 1 -- 172.21.15.42:0/1867203945 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04601a0670 con 0x7f0460102b00 2024-04-20T10:33:51.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.340+0000 7f0465e00640 1 -- 172.21.15.42:0/1867203945 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04600fdf40 con 0x7f0460102b00 2024-04-20T10:33:51.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.342+0000 7f0464a00640 1 -- 172.21.15.42:0/1867203945 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0450046020 con 0x7f0460102b00 2024-04-20T10:33:51.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.343+0000 7f0464a00640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f04440a3d20 0x7f04440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:51.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.343+0000 7f0464a00640 1 -- 172.21.15.42:0/1867203945 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f04500f2480 con 0x7f0460102b00 2024-04-20T10:33:51.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.343+0000 7f045e000640 1 -- 172.21.15.42:0/1867203945 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f04440a3d20 msgr2=0x7f04440a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3610568749 2024-04-20T10:33:51.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.343+0000 7f045e000640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f04440a3d20 0x7f04440a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:33:51.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.347+0000 7f0464a00640 1 -- 172.21.15.42:0/1867203945 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f045004d020 con 0x7f0460102b00 2024-04-20T10:33:51.399 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:51.156+0000 7fcf94e26200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:33:51.482 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:51 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:51.056+0000 7fe98f8bf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:33:51.482 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:51 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:51.162+0000 7fe98f8bf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:33:51.482 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:51 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:51.482+0000 7fe98f8bf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:33:51.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.544+0000 7f045e000640 1 -- 172.21.15.42:0/1867203945 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f04440a3d20 msgr2=0x7f04440a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3610568749 2024-04-20T10:33:51.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.544+0000 7f045e000640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f04440a3d20 0x7f04440a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:33:51.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.552+0000 7f0465e00640 1 -- 172.21.15.42:0/1867203945 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f046002d5d0 con 0x7f0460102b00 2024-04-20T10:33:51.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.560+0000 7f0464a00640 1 -- 172.21.15.42:0/1867203945 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f04500bede0 con 0x7f0460102b00 2024-04-20T10:33:51.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.562+0000 7f0465e00640 1 -- 172.21.15.42:0/1867203945 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f04440a3d20 msgr2=0x7f04440a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:33:51.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.562+0000 7f0465e00640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f04440a3d20 0x7f04440a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.562+0000 7f0465e00640 1 -- 172.21.15.42:0/1867203945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0460102b00 msgr2=0x7f0460198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:51.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.562+0000 7f0465e00640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0460102b00 0x7f0460198220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0450000c00 tx=0x7f045002c650 comp rx=0 tx=0).stop 2024-04-20T10:33:51.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.563+0000 7f0465e00640 1 -- 172.21.15.42:0/1867203945 shutdown_connections 2024-04-20T10:33:51.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.563+0000 7f0465e00640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0460107d60 0x7f046019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.563+0000 7f0465e00640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0460105430 0x7f0460198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.563+0000 7f0465e00640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7f04440a3d20 0x7f04440a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.563+0000 7f0465e00640 1 --2- 172.21.15.42:0/1867203945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0460102b00 0x7f0460198220 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.563+0000 7f0465e00640 1 -- 172.21.15.42:0/1867203945 >> 172.21.15.42:0/1867203945 conn(0x7f04600fc450 msgr2=0x7f0460103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.563+0000 7f0465e00640 1 -- 172.21.15.42:0/1867203945 shutdown_connections 2024-04-20T10:33:51.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:51.563+0000 7f0465e00640 1 -- 172.21.15.42:0/1867203945 wait complete. 2024-04-20T10:33:51.577 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:51 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:51.261+0000 7fd504023200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:33:51.578 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:51 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:51.400+0000 7fd504023200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:33:51.580 INFO:teuthology.run_tasks:Running task kclient... 2024-04-20T10:33:51.606 INFO:tasks.kclient:Mounting kernel clients... 2024-04-20T10:33:51.606 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-20T10:33:51.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:33:51.606 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-04-20T10:33:51.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:33:51.662 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-04-20T10:33:51.741 INFO:teuthology.orchestra.run.smithi042.stderr:Cannot find device "ceph-brx" 2024-04-20T10:33:51.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:33:51.745 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-20T10:33:51.745 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T10:33:51.745 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T10:33:51.745 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph fs ls 2024-04-20T10:33:51.803 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:51 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:51.570+0000 7fe98f8bf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:33:51.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:51 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-20T10:33:51.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:51 smithi046 ceph-mon[21574]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-20T10:33:51.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:51 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' 2024-04-20T10:33:51.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:51 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' 2024-04-20T10:33:51.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:51 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:51.820+0000 7fd504023200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:33:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:51 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-20T10:33:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:51 smithi042 ceph-mon[17078]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-20T10:33:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:51 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' 2024-04-20T10:33:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:51 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' 2024-04-20T10:33:51.899 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:51.472+0000 7fcf94e26200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:33:51.899 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:51.559+0000 7fcf94e26200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:33:51.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:51 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-20T10:33:51.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:51 smithi162 ceph-mon[21854]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-20T10:33:51.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:51 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' 2024-04-20T10:33:51.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:51 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' 2024-04-20T10:33:52.199 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:33:52.348 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:51 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:51.938+0000 7fd504023200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:33:53.097 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:52.755+0000 7fcf94e26200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:33:53.097 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:52.927+0000 7fcf94e26200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:33:53.097 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:53.096+0000 7fcf94e26200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:33:53.163 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:52 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:52.811+0000 7fe98f8bf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:33:53.163 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:52 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:52.988+0000 7fe98f8bf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:33:53.649 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:53.386+0000 7fcf94e26200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:33:53.649 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:53.481+0000 7fcf94e26200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:33:53.766 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:53.163+0000 7fe98f8bf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:33:53.766 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:53.462+0000 7fe98f8bf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:33:53.766 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:53.557+0000 7fe98f8bf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:33:53.767 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:53.766+0000 7fe98f8bf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:33:53.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:53 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:53.561+0000 7fd504023200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:33:53.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:53 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:53.804+0000 7fd504023200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:33:54.053 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:53.967+0000 7fe98f8bf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:33:54.149 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:53.685+0000 7fcf94e26200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:33:54.149 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:53.880+0000 7fcf94e26200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:33:54.191 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:54.030+0000 7fd504023200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:33:54.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.370+0000 7fac52e00700 1 -- 172.21.15.42:0/3368489411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c1044c0 msgr2=0x7fac4c10c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:54.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.370+0000 7fac52e00700 1 --2- 172.21.15.42:0/3368489411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c1044c0 0x7fac4c10c9b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fac38009990 tx=0x7fac38009ca0 comp rx=0 tx=0).stop 2024-04-20T10:33:54.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.371+0000 7fac52e00700 1 -- 172.21.15.42:0/3368489411 shutdown_connections 2024-04-20T10:33:54.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.371+0000 7fac52e00700 1 --2- 172.21.15.42:0/3368489411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac4c10cf80 0x7fac4c10f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:54.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.371+0000 7fac52e00700 1 --2- 172.21.15.42:0/3368489411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c103ba0 0x7fac4c103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:54.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.371+0000 7fac52e00700 1 --2- 172.21.15.42:0/3368489411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c1044c0 0x7fac4c10c9b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:54.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.371+0000 7fac52e00700 1 -- 172.21.15.42:0/3368489411 >> 172.21.15.42:0/3368489411 conn(0x7fac4c0fd370 msgr2=0x7fac4c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:54.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.371+0000 7fac52e00700 1 -- 172.21.15.42:0/3368489411 shutdown_connections 2024-04-20T10:33:54.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.371+0000 7fac52e00700 1 -- 172.21.15.42:0/3368489411 wait complete. 2024-04-20T10:33:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.372+0000 7fac52e00700 1 Processor -- start 2024-04-20T10:33:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.372+0000 7fac52e00700 1 -- start start 2024-04-20T10:33:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac52e00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c103ba0 0x7fac4c101800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac52e00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c1044c0 0x7fac4c101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac52e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac4c10cf80 0x7fac4c102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac52e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c112c90 con 0x7fac4c10cf80 2024-04-20T10:33:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac52e00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c112b10 con 0x7fac4c1044c0 2024-04-20T10:33:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac52e00700 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c112e10 con 0x7fac4c103ba0 2024-04-20T10:33:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac51a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c103ba0 0x7fac4c101800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac51000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c1044c0 0x7fac4c101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:54.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac51a00700 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c103ba0 0x7fac4c101800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:41066/0 (socket says 172.21.15.42:41066) 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac51a00700 1 -- 172.21.15.42:0/280758139 learned_addr learned my addr 172.21.15.42:0/280758139 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac52400700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac4c10cf80 0x7fac4c102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac51a00700 1 -- 172.21.15.42:0/280758139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c1044c0 msgr2=0x7fac4c101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac51a00700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c1044c0 0x7fac4c101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac51a00700 1 -- 172.21.15.42:0/280758139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac4c10cf80 msgr2=0x7fac4c102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac51a00700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac4c10cf80 0x7fac4c102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.373+0000 7fac51a00700 1 -- 172.21.15.42:0/280758139 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4c1a7690 con 0x7fac4c103ba0 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.374+0000 7fac51000700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c1044c0 0x7fac4c101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.374+0000 7fac51a00700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c103ba0 0x7fac4c101800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fac4000e9a0 tx=0x7fac4000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.374+0000 7fac4a000700 1 -- 172.21.15.42:0/280758139 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4000ca10 con 0x7fac4c103ba0 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.374+0000 7fac4a000700 1 -- 172.21.15.42:0/280758139 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac4000cbb0 con 0x7fac4c103ba0 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.374+0000 7fac4a000700 1 -- 172.21.15.42:0/280758139 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40018680 con 0x7fac4c103ba0 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.374+0000 7fac52e00700 1 -- 172.21.15.42:0/280758139 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4c1a7980 con 0x7fac4c103ba0 2024-04-20T10:33:54.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.374+0000 7fac52e00700 1 -- 172.21.15.42:0/280758139 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4c1a7ee0 con 0x7fac4c103ba0 2024-04-20T10:33:54.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.376+0000 7fac52e00700 1 -- 172.21.15.42:0/280758139 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac4c102d30 con 0x7fac4c103ba0 2024-04-20T10:33:54.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.377+0000 7fac4a000700 1 -- 172.21.15.42:0/280758139 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fac40010430 con 0x7fac4c103ba0 2024-04-20T10:33:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.379+0000 7fac4a000700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fac340a3d10 0x7fac340a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.379+0000 7fac51000700 1 -- 172.21.15.42:0/280758139 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fac340a3d10 msgr2=0x7fac340a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3610568749 2024-04-20T10:33:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.379+0000 7fac4a000700 1 -- 172.21.15.42:0/280758139 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fac400d0660 con 0x7fac4c103ba0 2024-04-20T10:33:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.379+0000 7fac51000700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fac340a3d10 0x7fac340a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:33:54.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.383+0000 7fac4a000700 1 -- 172.21.15.42:0/280758139 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fac4009cfc0 con 0x7fac4c103ba0 2024-04-20T10:33:54.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.580+0000 7fac51000700 1 -- 172.21.15.42:0/280758139 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fac340a3d10 msgr2=0x7fac340a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3610568749 2024-04-20T10:33:54.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.580+0000 7fac51000700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fac340a3d10 0x7fac340a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:33:54.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.694+0000 7fac52e00700 1 -- 172.21.15.42:0/280758139 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fac4c02df90 con 0x7fac4c103ba0 2024-04-20T10:33:54.695 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:54.465+0000 7fd504023200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:33:54.695 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:54.625+0000 7fd504023200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T10:33:54.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.694+0000 7fac4a000700 1 -- 172.21.15.42:0/280758139 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+73 (secure 0 0 0) 0x7fac4009c8c0 con 0x7fac4c103ba0 2024-04-20T10:33:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.697+0000 7fac52e00700 1 -- 172.21.15.42:0/280758139 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fac340a3d10 msgr2=0x7fac340a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:33:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.697+0000 7fac52e00700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fac340a3d10 0x7fac340a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.697+0000 7fac52e00700 1 -- 172.21.15.42:0/280758139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c103ba0 msgr2=0x7fac4c101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.697+0000 7fac52e00700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c103ba0 0x7fac4c101800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fac4000e9a0 tx=0x7fac4000ecb0 comp rx=0 tx=0).stop 2024-04-20T10:33:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.697+0000 7fac52e00700 1 -- 172.21.15.42:0/280758139 shutdown_connections 2024-04-20T10:33:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.697+0000 7fac52e00700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fac340a3d10 0x7fac340a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.697+0000 7fac52e00700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac4c10cf80 0x7fac4c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.697+0000 7fac52e00700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c103ba0 0x7fac4c101800 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.697+0000 7fac52e00700 1 --2- 172.21.15.42:0/280758139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c1044c0 0x7fac4c101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.697+0000 7fac52e00700 1 -- 172.21.15.42:0/280758139 >> 172.21.15.42:0/280758139 conn(0x7fac4c0fd370 msgr2=0x7fac4c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.697+0000 7fac52e00700 1 -- 172.21.15.42:0/280758139 shutdown_connections 2024-04-20T10:33:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:54.698+0000 7fac52e00700 1 -- 172.21.15.42:0/280758139 wait complete. 2024-04-20T10:33:54.747 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:54.490+0000 7fcf94e26200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:33:55.015 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:54 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/280758139' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T10:33:55.015 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:54.916+0000 7fcf94e26200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:33:55.015 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:55.014+0000 7fcf94e26200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:33:55.026 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:54 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/280758139' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T10:33:55.027 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:54 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:54.590+0000 7fe98f8bf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:33:55.104 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:54.939+0000 7fd504023200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:33:55.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:54 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/280758139' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T10:33:55.303 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:55.026+0000 7fe98f8bf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:33:55.303 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:55.126+0000 7fe98f8bf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:33:55.399 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:55.218+0000 7fcf94e26200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:33:55.454 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:55 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:55.216+0000 7fd504023200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:33:55.498 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:33:55.498 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:33:55.498 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-04-20T10:33:55.498 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T10:33:55.498 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T10:33:55.498 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:33:55.498 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:33:55.498 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:33:55.498 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:33:55.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:33:55.499 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-04-20T10:33:55.548 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:33:55.548 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:33:55.548 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:33:55.548 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:33:55.548 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-04-20T10:33:55.548 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:33:55.548 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:33:55.548 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-04-20T10:33:55.548 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 41889sec preferred_lft 41889sec 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:33:55.549 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-04-20T10:33:55.550 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T10:33:55.551 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:33:55.551 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-04-20T10:33:55.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add name ceph-brx type bridge 2024-04-20T10:33:55.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr flush dev ceph-brx 2024-04-20T10:33:55.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set ceph-brx up 2024-04-20T10:33:55.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T10:33:55.551 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-04-20T10:33:55.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:33:55.803 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:55.335+0000 7fe98f8bf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:33:56.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:33:56.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:33:56.028 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:33:56.081 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:55.741+0000 7fcf94e26200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:33:56.082 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:55.834+0000 7fcf94e26200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:33:56.095 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-04-20T10:33:56.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:33:56.099 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-04-20T10:33:56.150 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-04-20T10:33:56.150 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:33:56.150 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-20T10:33:56.150 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-20T10:33:56.150 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:33:56.151 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:33:56.151 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-04-20T10:33:56.152 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-20T10:33:56.152 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-20T10:33:56.152 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-20T10:33:56.152 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-04-20T10:33:56.227 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:55.880+0000 7fe98f8bf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:33:56.227 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:55.975+0000 7fe98f8bf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:33:56.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:33:56.265 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:56 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:56.053+0000 7fd504023200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:33:56.399 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:56.080+0000 7fcf94e26200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:33:56.399 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:56.217+0000 7fcf94e26200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:33:56.399 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:56.312+0000 7fcf94e26200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:33:56.553 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:56 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:56.226+0000 7fe98f8bf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:33:56.553 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:56 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:56.363+0000 7fe98f8bf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:33:56.553 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:56 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:56.462+0000 7fe98f8bf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:33:56.737 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:56.429+0000 7fcf94e26200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:33:56.737 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:56.736+0000 7fcf94e26200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:33:56.784 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:56 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:56.646+0000 7fd504023200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:33:56.784 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:56 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:56.782+0000 7fd504023200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:33:56.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:33:56.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:33:56.877 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-04-20T10:33:56.891 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:56 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:56.581+0000 7fe98f8bf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:33:56.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:33:56.922 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2024-04-20T10:33:57.016 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:33:57.016 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-04-20T10:33:57.016 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:33:57.016 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-20T10:33:57.016 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-04-20T10:33:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:33:57.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:33:57.190 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-20T10:33:57.190 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:33:57.190 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-04-20T10:33:57.190 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-20T10:33:57.191 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T10:33:57.191 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-20T10:33:57.191 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-20T10:33:57.191 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-20T10:33:57.191 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-04-20T10:33:57.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:33:57.303 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:56 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:56.891+0000 7fe98f8bf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:33:57.343 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:57 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:57.069+0000 7fd504023200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:33:57.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:33:57.472 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:33:57.472 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-04-20T10:33:57.472 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-04-20T10:33:57.472 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T10:33:57.472 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-04-20T10:33:57.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:33:57.634 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:57.347+0000 7fcf94e26200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:33:57.634 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:33:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z[23740]: 2024-04-20T10:33:57.564+0000 7fcf94e26200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:33:57.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:33:57.670 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:33:57.670 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T10:33:57.708 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:33:57.708 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:33:57.708 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:33:57.800 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-20T10:33:57.800 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-20T10:33:57.800 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/bb321639-ad29-40c2-98d4-8ea3b9b7ace4 2024-04-20T10:33:57.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:33:57.800 DEBUG:teuthology.orchestra.run.smithi042:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/bb321639-ad29-40c2-98d4-8ea3b9b7ace4 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-20T10:33:57.803 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:57.517+0000 7fe98f8bf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:33:57.803 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:33:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y[22970]: 2024-04-20T10:33:57.744+0000 7fe98f8bf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:33:57.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:57 smithi046 ceph-mon[21574]: Standby manager daemon z restarted 2024-04-20T10:33:57.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:57 smithi046 ceph-mon[21574]: Standby manager daemon z started 2024-04-20T10:33:57.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:57 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.162:0/1728693289' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T10:33:57.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:57 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.162:0/1728693289' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:33:57.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:57 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.162:0/1728693289' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T10:33:57.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:57 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.162:0/1728693289' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:33:57.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:57 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:57.776+0000 7fd504023200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:33:57.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:57 smithi042 ceph-mon[17078]: Standby manager daemon z restarted 2024-04-20T10:33:57.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:57 smithi042 ceph-mon[17078]: Standby manager daemon z started 2024-04-20T10:33:57.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:57 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.162:0/1728693289' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T10:33:57.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:57 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.162:0/1728693289' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:33:57.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:57 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.162:0/1728693289' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T10:33:57.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:57 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.162:0/1728693289' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:33:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:57 smithi162 ceph-mon[21854]: Standby manager daemon z restarted 2024-04-20T10:33:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:57 smithi162 ceph-mon[21854]: Standby manager daemon z started 2024-04-20T10:33:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:57 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.162:0/1728693289' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T10:33:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:57 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.162:0/1728693289' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:33:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:57 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.162:0/1728693289' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T10:33:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:57 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.162:0/1728693289' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:33:57.917 INFO:teuthology.orchestra.run.smithi042.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-20T10:33:57.917 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-20T10:33:57.917 INFO:teuthology.orchestra.run.smithi042.stdout:invalid new device string format 2024-04-20T10:33:58.217 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:57 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:57.911+0000 7fd504023200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:33:58.592 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:58.265+0000 7fd504023200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:33:58.593 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:58.457+0000 7fd504023200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:33:58.593 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:58.591+0000 7fd504023200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:33:58.722 INFO:teuthology.orchestra.run.smithi042.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-20T10:33:58.722 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-20T10:33:58.722 INFO:teuthology.orchestra.run.smithi042.stdout:invalid new device string format 2024-04-20T10:33:58.722 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: resolved to: "172.21.15.42:3300,172.21.15.46:3300,172.21.15.162:3300" 2024-04-20T10:33:58.722 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.42:3300,172.21.15.46:3300,172.21.15.162:3300:/volumes/qa/sv_1/bb321639-ad29-40c2-98d4-8ea3b9b7ace4 2024-04-20T10:33:58.722 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=83e8f766-feff-11ee-bc92-c7b262605968" will pass to kernel 2024-04-20T10:33:58.724 INFO:teuthology.orchestra.run.smithi042.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-20T10:33:58.725 INFO:teuthology.orchestra.run.smithi042.stdout: You just mounted a file system that supports labels which does not 2024-04-20T10:33:58.725 INFO:teuthology.orchestra.run.smithi042.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-20T10:33:58.725 INFO:teuthology.orchestra.run.smithi042.stdout: applications will generate AVC messages and not be allowed access to 2024-04-20T10:33:58.725 INFO:teuthology.orchestra.run.smithi042.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-20T10:33:58.728 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-20T10:33:58.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:33:58.729 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:33:58.797 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T10:33:58.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:58.762+0000 7fd504023200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:33:58.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:58 smithi042 ceph-mon[17078]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-20T10:33:58.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:58 smithi042 ceph-mon[17078]: Standby manager daemon y restarted 2024-04-20T10:33:58.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:58 smithi042 ceph-mon[17078]: Standby manager daemon y started 2024-04-20T10:33:58.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:58 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.46:0/2794782119' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T10:33:58.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:58 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.46:0/2794782119' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:33:58.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:58 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.46:0/2794782119' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T10:33:58.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:58 smithi042 ceph-mon[17078]: from='mgr.? 172.21.15.46:0/2794782119' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:33:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T10:33:58.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:58 smithi162 ceph-mon[21854]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-20T10:33:58.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:58 smithi162 ceph-mon[21854]: Standby manager daemon y restarted 2024-04-20T10:33:58.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:58 smithi162 ceph-mon[21854]: Standby manager daemon y started 2024-04-20T10:33:58.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:58 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.46:0/2794782119' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T10:33:58.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:58 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.46:0/2794782119' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:33:58.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:58 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.46:0/2794782119' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T10:33:58.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:58 smithi162 ceph-mon[21854]: from='mgr.? 172.21.15.46:0/2794782119' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:33:58.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:33:58.912 INFO:teuthology.orchestra.run.smithi042.stdout:client15045 2024-04-20T10:33:58.912 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:33:58.912 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:33:58.962 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T10:33:59.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:58 smithi046 ceph-mon[21574]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-20T10:33:59.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:58 smithi046 ceph-mon[21574]: Standby manager daemon y restarted 2024-04-20T10:33:59.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:58 smithi046 ceph-mon[21574]: Standby manager daemon y started 2024-04-20T10:33:59.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:58 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.46:0/2794782119' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T10:33:59.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:58 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.46:0/2794782119' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T10:33:59.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:58 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.46:0/2794782119' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T10:33:59.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:58 smithi046 ceph-mon[21574]: from='mgr.? 172.21.15.46:0/2794782119' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T10:33:59.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T10:33:59.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:33:59.080 INFO:teuthology.orchestra.run.smithi042.stdout:client15045 2024-04-20T10:33:59.080 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T10:33:59.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T10:33:59.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:33:59.192 INFO:teuthology.orchestra.run.smithi042.stdout:client15045 2024-04-20T10:33:59.192 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/83e8f766-feff-11ee-bc92-c7b262605968.client15045/status') 2024-04-20T10:33:59.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:33:59.314 INFO:teuthology.orchestra.run.smithi042.stdout:instance: client.15045 (3)192.168.0.1:0/236301845 2024-04-20T10:33:59.314 INFO:teuthology.orchestra.run.smithi042.stdout:blocklisted: false 2024-04-20T10:33:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:0+1 records in 2024-04-20T10:33:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:0+1 records out 2024-04-20T10:33:59.314 INFO:teuthology.orchestra.run.smithi042.stderr:69 bytes copied, 8.8117e-05 s, 783 kB/s 2024-04-20T10:33:59.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:33:59.327 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-20T10:33:59.339 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-20T10:33:59.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:33:59.460 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:59 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:59.145+0000 7fd504023200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:33:59.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.592+0000 7ff9bfe00640 1 -- 172.21.15.42:0/4148377811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9c0101e00 msgr2=0x7ff9c0102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:59.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.592+0000 7ff9bfe00640 1 --2- 172.21.15.42:0/4148377811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9c0101e00 0x7ff9c0102220 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009900 tx=0x7ff9b002bc30 comp rx=0 tx=0).stop 2024-04-20T10:33:59.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.592+0000 7ff9bfe00640 1 -- 172.21.15.42:0/4148377811 shutdown_connections 2024-04-20T10:33:59.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.592+0000 7ff9bfe00640 1 --2- 172.21.15.42:0/4148377811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c01039b0 0x7ff9c010a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:59.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.592+0000 7ff9bfe00640 1 --2- 172.21.15.42:0/4148377811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9c0102ff0 0x7ff9c0103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:59.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.592+0000 7ff9bfe00640 1 --2- 172.21.15.42:0/4148377811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9c0101e00 0x7ff9c0102220 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:59.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.592+0000 7ff9bfe00640 1 -- 172.21.15.42:0/4148377811 >> 172.21.15.42:0/4148377811 conn(0x7ff9c00fd370 msgr2=0x7ff9c00ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:59.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.592+0000 7ff9bfe00640 1 -- 172.21.15.42:0/4148377811 shutdown_connections 2024-04-20T10:33:59.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.593+0000 7ff9bfe00640 1 -- 172.21.15.42:0/4148377811 wait complete. 2024-04-20T10:33:59.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.593+0000 7ff9bfe00640 1 Processor -- start 2024-04-20T10:33:59.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.594+0000 7ff9bfe00640 1 -- start start 2024-04-20T10:33:59.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.594+0000 7ff9bfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9c0101e00 0x7ff9c019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:59.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.594+0000 7ff9bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c0102ff0 0x7ff9c019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:59.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.594+0000 7ff9bfe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9c01039b0 0x7ff9c01a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:59.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.594+0000 7ff9bfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c010daa0 con 0x7ff9c0101e00 2024-04-20T10:33:59.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.594+0000 7ff9bfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c010d920 con 0x7ff9c0102ff0 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.594+0000 7ff9bfe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c010dc20 con 0x7ff9c01039b0 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.594+0000 7ff9bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9c0101e00 0x7ff9c019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.594+0000 7ff9bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9c0101e00 0x7ff9c019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33920/0 (socket says 172.21.15.42:33920) 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.594+0000 7ff9bea00640 1 -- 172.21.15.42:0/859246417 learned_addr learned my addr 172.21.15.42:0/859246417 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.594+0000 7ff9be000640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c0102ff0 0x7ff9c019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.594+0000 7ff9bf400640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9c01039b0 0x7ff9c01a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.595+0000 7ff9be000640 1 -- 172.21.15.42:0/859246417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9c01039b0 msgr2=0x7ff9c01a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.595+0000 7ff9be000640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9c01039b0 0x7ff9c01a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.595+0000 7ff9be000640 1 -- 172.21.15.42:0/859246417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9c0101e00 msgr2=0x7ff9c019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.595+0000 7ff9be000640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9c0101e00 0x7ff9c019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.595+0000 7ff9be000640 1 -- 172.21.15.42:0/859246417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c01a1f40 con 0x7ff9c0102ff0 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.595+0000 7ff9bea00640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9c0101e00 0x7ff9c019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.595+0000 7ff9be000640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c0102ff0 0x7ff9c019a860 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00e9f0 tx=0x7ff9ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:33:59.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.595+0000 7ff9b7400640 1 -- 172.21.15.42:0/859246417 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac00ccc0 con 0x7ff9c0102ff0 2024-04-20T10:33:59.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.595+0000 7ff9bfe00640 1 -- 172.21.15.42:0/859246417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c01a2230 con 0x7ff9c0102ff0 2024-04-20T10:33:59.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.595+0000 7ff9b7400640 1 -- 172.21.15.42:0/859246417 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9ac010910 con 0x7ff9c0102ff0 2024-04-20T10:33:59.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.595+0000 7ff9b7400640 1 -- 172.21.15.42:0/859246417 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac018960 con 0x7ff9c0102ff0 2024-04-20T10:33:59.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.596+0000 7ff9bfe00640 1 -- 172.21.15.42:0/859246417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c01a2780 con 0x7ff9c0102ff0 2024-04-20T10:33:59.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.597+0000 7ff9bfe00640 1 -- 172.21.15.42:0/859246417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c01a29e0 con 0x7ff9c0102ff0 2024-04-20T10:33:59.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.599+0000 7ff9b7400640 1 -- 172.21.15.42:0/859246417 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 21) ==== 138663+0+0 (secure 0 0 0) 0x7ff9ac010ac0 con 0x7ff9c0102ff0 2024-04-20T10:33:59.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.601+0000 7ff9b7400640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff9a40a3c00 0x7ff9a40a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:33:59.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.601+0000 7ff9bea00640 1 -- 172.21.15.42:0/859246417 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff9a40a3c00 msgr2=0x7ff9a40a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3610568749 2024-04-20T10:33:59.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.601+0000 7ff9b7400640 1 -- 172.21.15.42:0/859246417 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff9ac0d0860 con 0x7ff9c0102ff0 2024-04-20T10:33:59.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.601+0000 7ff9bea00640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff9a40a3c00 0x7ff9a40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:33:59.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.603+0000 7ff9b7400640 1 -- 172.21.15.42:0/859246417 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff9ac09cb70 con 0x7ff9c0102ff0 2024-04-20T10:33:59.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.802+0000 7ff9bea00640 1 -- 172.21.15.42:0/859246417 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff9a40a3c00 msgr2=0x7ff9a40a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3610568749 2024-04-20T10:33:59.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.802+0000 7ff9bea00640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff9a40a3c00 0x7ff9a40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:33:59.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.818+0000 7ff9bfe00640 1 -- 172.21.15.42:0/859246417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9c01a2cd0 con 0x7ff9c0102ff0 2024-04-20T10:33:59.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.820+0000 7ff9b7400640 1 -- 172.21.15.42:0/859246417 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7ff9ac0a0e70 con 0x7ff9c0102ff0 2024-04-20T10:33:59.821 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:33:59.821 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:32:31.928088+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:33:59.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.822+0000 7ff9bfe00640 1 -- 172.21.15.42:0/859246417 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff9a40a3c00 msgr2=0x7ff9a40a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:33:59.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.822+0000 7ff9bfe00640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff9a40a3c00 0x7ff9a40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.822+0000 7ff9bfe00640 1 -- 172.21.15.42:0/859246417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c0102ff0 msgr2=0x7ff9c019a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:33:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.822+0000 7ff9bfe00640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c0102ff0 0x7ff9c019a860 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00e9f0 tx=0x7ff9ac00eeb0 comp rx=0 tx=0).stop 2024-04-20T10:33:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.823+0000 7ff9bfe00640 1 -- 172.21.15.42:0/859246417 shutdown_connections 2024-04-20T10:33:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.823+0000 7ff9bfe00640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7ff9a40a3c00 0x7ff9a40a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.823+0000 7ff9bfe00640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9c01039b0 0x7ff9c01a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.823+0000 7ff9bfe00640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c0102ff0 0x7ff9c019a860 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.823+0000 7ff9bfe00640 1 --2- 172.21.15.42:0/859246417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9c0101e00 0x7ff9c019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:33:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.823+0000 7ff9bfe00640 1 -- 172.21.15.42:0/859246417 >> 172.21.15.42:0/859246417 conn(0x7ff9c00fd370 msgr2=0x7ff9c0074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:33:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.823+0000 7ff9bfe00640 1 -- 172.21.15.42:0/859246417 shutdown_connections 2024-04-20T10:33:59.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:33:59.823+0000 7ff9bfe00640 1 -- 172.21.15.42:0/859246417 wait complete. 2024-04-20T10:33:59.840 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:33:59 smithi042 ceph-mon[17078]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-04-20T10:33:59.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:26:16.537072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:31:33.523692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:31:34.019519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:33:59.844 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:34:00.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.038+0000 7fd842800640 1 -- 172.21.15.42:0/3718636783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c1041c0 msgr2=0x7fd83c1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:00.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.038+0000 7fd842800640 1 --2- 172.21.15.42:0/3718636783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c1041c0 0x7fd83c1045e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009900 tx=0x7fd82c02bc40 comp rx=0 tx=0).stop 2024-04-20T10:34:00.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.038+0000 7fd842800640 1 -- 172.21.15.42:0/3718636783 shutdown_connections 2024-04-20T10:34:00.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.038+0000 7fd842800640 1 --2- 172.21.15.42:0/3718636783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83c105d70 0x7fd83c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.038+0000 7fd842800640 1 --2- 172.21.15.42:0/3718636783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c1053b0 0x7fd83c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.038+0000 7fd842800640 1 --2- 172.21.15.42:0/3718636783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c1041c0 0x7fd83c1045e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.038+0000 7fd842800640 1 -- 172.21.15.42:0/3718636783 >> 172.21.15.42:0/3718636783 conn(0x7fd83c0ff750 msgr2=0x7fd83c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:00.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.038+0000 7fd842800640 1 -- 172.21.15.42:0/3718636783 shutdown_connections 2024-04-20T10:34:00.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.038+0000 7fd842800640 1 -- 172.21.15.42:0/3718636783 wait complete. 2024-04-20T10:34:00.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd842800640 1 Processor -- start 2024-04-20T10:34:00.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd842800640 1 -- start start 2024-04-20T10:34:00.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd842800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c1041c0 0x7fd83c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:00.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd842800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c1053b0 0x7fd83c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:00.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd842800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83c105d70 0x7fd83c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:00.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd842800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c10fe20 con 0x7fd83c1041c0 2024-04-20T10:34:00.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd842800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c10fca0 con 0x7fd83c1053b0 2024-04-20T10:34:00.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd842800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd83c10ffa0 con 0x7fd83c105d70 2024-04-20T10:34:00.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd83b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c1041c0 0x7fd83c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:00.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd83aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c1053b0 0x7fd83c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:00.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd83b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c1041c0 0x7fd83c19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33942/0 (socket says 172.21.15.42:33942) 2024-04-20T10:34:00.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd83b400640 1 -- 172.21.15.42:0/4223117537 learned_addr learned my addr 172.21.15.42:0/4223117537 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:00.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.039+0000 7fd83aa00640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c1053b0 0x7fd83c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:51758/0 (socket says 172.21.15.42:51758) 2024-04-20T10:34:00.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.040+0000 7fd83be00640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83c105d70 0x7fd83c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:00.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.040+0000 7fd83aa00640 1 -- 172.21.15.42:0/4223117537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83c105d70 msgr2=0x7fd83c1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:00.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.040+0000 7fd83aa00640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83c105d70 0x7fd83c1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.040+0000 7fd83aa00640 1 -- 172.21.15.42:0/4223117537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c1041c0 msgr2=0x7fd83c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:00.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.040+0000 7fd83aa00640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c1041c0 0x7fd83c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.040+0000 7fd83aa00640 1 -- 172.21.15.42:0/4223117537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83c1a20c0 con 0x7fd83c1053b0 2024-04-20T10:34:00.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.040+0000 7fd83aa00640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c1053b0 0x7fd83c19aa30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd82800e9f0 tx=0x7fd82800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:00.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.040+0000 7fd833e00640 1 -- 172.21.15.42:0/4223117537 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82800cd80 con 0x7fd83c1053b0 2024-04-20T10:34:00.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.041+0000 7fd833e00640 1 -- 172.21.15.42:0/4223117537 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8280035f0 con 0x7fd83c1053b0 2024-04-20T10:34:00.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.041+0000 7fd842800640 1 -- 172.21.15.42:0/4223117537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83c1a23b0 con 0x7fd83c1053b0 2024-04-20T10:34:00.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.041+0000 7fd833e00640 1 -- 172.21.15.42:0/4223117537 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828010690 con 0x7fd83c1053b0 2024-04-20T10:34:00.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.041+0000 7fd842800640 1 -- 172.21.15.42:0/4223117537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd83c1a2900 con 0x7fd83c1053b0 2024-04-20T10:34:00.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.042+0000 7fd842800640 1 -- 172.21.15.42:0/4223117537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd83c10b580 con 0x7fd83c1053b0 2024-04-20T10:34:00.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.045+0000 7fd833e00640 1 -- 172.21.15.42:0/4223117537 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 21) ==== 138663+0+0 (secure 0 0 0) 0x7fd828003b30 con 0x7fd83c1053b0 2024-04-20T10:34:00.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.046+0000 7fd833e00640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd8200a3c00 0x7fd8200a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:00.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.046+0000 7fd83b400640 1 -- 172.21.15.42:0/4223117537 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd8200a3c00 msgr2=0x7fd8200a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3610568749 2024-04-20T10:34:00.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.046+0000 7fd833e00640 1 -- 172.21.15.42:0/4223117537 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd8280d08b0 con 0x7fd83c1053b0 2024-04-20T10:34:00.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.046+0000 7fd83b400640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd8200a3c00 0x7fd8200a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:34:00.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.048+0000 7fd833e00640 1 -- 172.21.15.42:0/4223117537 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd828043e20 con 0x7fd83c1053b0 2024-04-20T10:34:00.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:33:59 smithi046 ceph-mon[21574]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-04-20T10:34:00.102 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:33:59 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:33:59.987+0000 7fd504023200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:34:00.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:33:59 smithi162 ceph-mon[21854]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-04-20T10:34:00.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.247+0000 7fd83b400640 1 -- 172.21.15.42:0/4223117537 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd8200a3c00 msgr2=0x7fd8200a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3610568749 2024-04-20T10:34:00.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.247+0000 7fd83b400640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd8200a3c00 0x7fd8200a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:34:00.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.251+0000 7fd842800640 1 -- 172.21.15.42:0/4223117537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd83c02d760 con 0x7fd83c1053b0 2024-04-20T10:34:00.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.251+0000 7fd833e00640 1 -- 172.21.15.42:0/4223117537 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fd82809d270 con 0x7fd83c1053b0 2024-04-20T10:34:00.252 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-20T10:34:00.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.254+0000 7fd842800640 1 -- 172.21.15.42:0/4223117537 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd8200a3c00 msgr2=0x7fd8200a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:34:00.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.254+0000 7fd842800640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd8200a3c00 0x7fd8200a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.254+0000 7fd842800640 1 -- 172.21.15.42:0/4223117537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c1053b0 msgr2=0x7fd83c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:00.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.254+0000 7fd842800640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c1053b0 0x7fd83c19aa30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd82800e9f0 tx=0x7fd82800eeb0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.254+0000 7fd842800640 1 -- 172.21.15.42:0/4223117537 shutdown_connections 2024-04-20T10:34:00.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.254+0000 7fd842800640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.42:6800/3610568749,v1:172.21.15.42:6801/3610568749] conn(0x7fd8200a3c00 0x7fd8200a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.254+0000 7fd842800640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd83c105d70 0x7fd83c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.254+0000 7fd842800640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd83c1053b0 0x7fd83c19aa30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.254+0000 7fd842800640 1 --2- 172.21.15.42:0/4223117537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd83c1041c0 0x7fd83c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.254+0000 7fd842800640 1 -- 172.21.15.42:0/4223117537 >> 172.21.15.42:0/4223117537 conn(0x7fd83c0ff750 msgr2=0x7fd83c108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:00.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.254+0000 7fd842800640 1 -- 172.21.15.42:0/4223117537 shutdown_connections 2024-04-20T10:34:00.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.254+0000 7fd842800640 1 -- 172.21.15.42:0/4223117537 wait complete. 2024-04-20T10:34:00.272 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:34:00.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.478+0000 7f3fffe00640 1 -- 172.21.15.42:0/651716986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4000075f40 msgr2=0x7f4000074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:00.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.478+0000 7f3fffe00640 1 --2- 172.21.15.42:0/651716986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4000075f40 0x7f4000074390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009900 tx=0x7f3ff002bd10 comp rx=0 tx=0).stop 2024-04-20T10:34:00.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.479+0000 7f3fffe00640 1 -- 172.21.15.42:0/651716986 shutdown_connections 2024-04-20T10:34:00.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.479+0000 7f3fffe00640 1 --2- 172.21.15.42:0/651716986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4000109d20 0x7f400010c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.479+0000 7f3fffe00640 1 --2- 172.21.15.42:0/651716986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4000074960 0x7f4000074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.479+0000 7f3fffe00640 1 --2- 172.21.15.42:0/651716986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4000075f40 0x7f4000074390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.479+0000 7f3fffe00640 1 -- 172.21.15.42:0/651716986 >> 172.21.15.42:0/651716986 conn(0x7f40000fd400 msgr2=0x7f40000ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:00.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.479+0000 7f3fffe00640 1 -- 172.21.15.42:0/651716986 shutdown_connections 2024-04-20T10:34:00.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.479+0000 7f3fffe00640 1 -- 172.21.15.42:0/651716986 wait complete. 2024-04-20T10:34:00.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.480+0000 7f3fffe00640 1 Processor -- start 2024-04-20T10:34:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.480+0000 7f3fffe00640 1 -- start start 2024-04-20T10:34:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.480+0000 7f3fffe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4000074960 0x7f4000105210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.480+0000 7f3fffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4000075f40 0x7f4000105750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.480+0000 7f3fffe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4000109d20 0x7f4000101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.480+0000 7f3fffe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f400010fd70 con 0x7f4000074960 2024-04-20T10:34:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.480+0000 7f3fffe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f400010fbf0 con 0x7f4000075f40 2024-04-20T10:34:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.480+0000 7f3fffe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f400010fef0 con 0x7f4000109d20 2024-04-20T10:34:00.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.481+0000 7f3ffe000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4000075f40 0x7f4000105750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:00.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.481+0000 7f3ffea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4000074960 0x7f4000105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:00.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.481+0000 7f3ffe000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4000075f40 0x7f4000105750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:51838/0 (socket says 172.21.15.42:51838) 2024-04-20T10:34:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.481+0000 7f3ffe000640 1 -- 172.21.15.42:0/780904024 learned_addr learned my addr 172.21.15.42:0/780904024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.481+0000 7f3fff400640 1 --2- 172.21.15.42:0/780904024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4000109d20 0x7f4000101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.481+0000 7f3ffea00640 1 -- 172.21.15.42:0/780904024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4000109d20 msgr2=0x7f4000101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.481+0000 7f3ffea00640 1 --2- 172.21.15.42:0/780904024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4000109d20 0x7f4000101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.481+0000 7f3ffea00640 1 -- 172.21.15.42:0/780904024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4000075f40 msgr2=0x7f4000105750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.481+0000 7f3ffea00640 1 --2- 172.21.15.42:0/780904024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4000075f40 0x7f4000105750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.481+0000 7f3ffea00640 1 -- 172.21.15.42:0/780904024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4000102640 con 0x7f4000074960 2024-04-20T10:34:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.481+0000 7f3ffe000640 1 --2- 172.21.15.42:0/780904024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4000075f40 0x7f4000105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.482+0000 7f3ffea00640 1 --2- 172.21.15.42:0/780904024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4000074960 0x7f4000105210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3ff002c250 tx=0x7f3ff002ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.482+0000 7f3ff7400640 1 -- 172.21.15.42:0/780904024 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0041020 con 0x7f4000074960 2024-04-20T10:34:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.482+0000 7f3ff7400640 1 -- 172.21.15.42:0/780904024 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff002e2e0 con 0x7f4000074960 2024-04-20T10:34:00.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.482+0000 7f3ff7400640 1 -- 172.21.15.42:0/780904024 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0005220 con 0x7f4000074960 2024-04-20T10:34:00.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.482+0000 7f3fffe00640 1 -- 172.21.15.42:0/780904024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40001a6760 con 0x7f4000074960 2024-04-20T10:34:00.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.482+0000 7f3fffe00640 1 -- 172.21.15.42:0/780904024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40001a6b10 con 0x7f4000074960 2024-04-20T10:34:00.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.483+0000 7f3fffe00640 1 -- 172.21.15.42:0/780904024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4000068ca0 con 0x7f4000074960 2024-04-20T10:34:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.489+0000 7f3ff7400640 1 -- 172.21.15.42:0/780904024 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f3ff0044020 con 0x7f4000074960 2024-04-20T10:34:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.495+0000 7f3ff7400640 1 -- 172.21.15.42:0/780904024 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3ff00f1900 con 0x7f4000074960 2024-04-20T10:34:00.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.499+0000 7f3ff7400640 1 -- 172.21.15.42:0/780904024 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3ff004b020 con 0x7f4000074960 2024-04-20T10:34:00.534 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:34:00.295+0000 7fd504023200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:34:00.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.750+0000 7f3fffe00640 1 -- 172.21.15.42:0/780904024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4000062b60 con 0x7f4000074960 2024-04-20T10:34:00.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.751+0000 7f3ff7400640 1 -- 172.21.15.42:0/780904024 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f3ff00be560 con 0x7f4000074960 2024-04-20T10:34:00.752 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-20T10:34:00.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.754+0000 7f3fffe00640 1 -- 172.21.15.42:0/780904024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4000074960 msgr2=0x7f4000105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.754+0000 7f3fffe00640 1 --2- 172.21.15.42:0/780904024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4000074960 0x7f4000105210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3ff002c250 tx=0x7f3ff002ec90 comp rx=0 tx=0).stop 2024-04-20T10:34:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.754+0000 7f3fffe00640 1 -- 172.21.15.42:0/780904024 shutdown_connections 2024-04-20T10:34:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.754+0000 7f3fffe00640 1 --2- 172.21.15.42:0/780904024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4000109d20 0x7f4000101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.754+0000 7f3fffe00640 1 --2- 172.21.15.42:0/780904024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4000075f40 0x7f4000105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.754+0000 7f3fffe00640 1 --2- 172.21.15.42:0/780904024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4000074960 0x7f4000105210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.754+0000 7f3fffe00640 1 -- 172.21.15.42:0/780904024 >> 172.21.15.42:0/780904024 conn(0x7f40000fd400 msgr2=0x7f4000105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.754+0000 7f3fffe00640 1 -- 172.21.15.42:0/780904024 shutdown_connections 2024-04-20T10:34:00.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.754+0000 7f3fffe00640 1 -- 172.21.15.42:0/780904024 wait complete. 2024-04-20T10:34:00.772 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:34:00.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/859246417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:34:00.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/4223117537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: Active manager daemon x restarted 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: Activating manager daemon x 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: osdmap e88: 12 total, 12 up, 12 in 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: mgrmap e22: x(active, starting, since 0.0173863s), standbys: y, z 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T10:34:00.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:34:00.805 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: Manager daemon x is now available 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T10:34:00.806 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:00 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T10:34:00.829 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/859246417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:34:00.829 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4223117537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: Active manager daemon x restarted 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: Activating manager daemon x 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: osdmap e88: 12 total, 12 up, 12 in 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: mgrmap e22: x(active, starting, since 0.0173863s), standbys: y, z 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T10:34:00.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T10:34:00.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T10:34:00.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T10:34:00.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T10:34:00.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T10:34:00.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T10:34:00.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T10:34:00.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T10:34:00.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T10:34:00.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:34:00.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:34:00.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:34:00.831 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:34:00.642+0000 7fd455e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:34:00.642+0000 7fd455e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:34:00.642+0000 7fd455e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:34:00.642+0000 7fd455e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:34:00.642+0000 7fd455e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:34:00.643+0000 7fd45a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:34:00.643+0000 7fd45a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:34:00.643+0000 7fd45a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:34:00.643+0000 7fd45a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:34:00.643+0000 7fd45a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:34:00.832 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: Manager daemon x is now available 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T10:34:00.833 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:00 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T10:34:00.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/859246417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:34:00.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/4223117537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:34:00.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: Active manager daemon x restarted 2024-04-20T10:34:00.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: Activating manager daemon x 2024-04-20T10:34:00.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: osdmap e88: 12 total, 12 up, 12 in 2024-04-20T10:34:00.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: mgrmap e22: x(active, starting, since 0.0173863s), standbys: y, z 2024-04-20T10:34:00.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T10:34:00.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T10:34:00.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T10:34:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T10:34:00.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T10:34:00.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T10:34:00.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T10:34:00.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: Manager daemon x is now available 2024-04-20T10:34:00.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:34:00.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:34:00.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T10:34:00.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T10:34:00.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:00 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T10:34:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.956+0000 7ff18ec00640 1 -- 172.21.15.42:0/89891049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff188103fc0 msgr2=0x7ff1881043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.956+0000 7ff18ec00640 1 --2- 172.21.15.42:0/89891049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff188103fc0 0x7ff1881043e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009900 tx=0x7ff17c02bc50 comp rx=0 tx=0).stop 2024-04-20T10:34:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.957+0000 7ff18ec00640 1 -- 172.21.15.42:0/89891049 shutdown_connections 2024-04-20T10:34:00.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.957+0000 7ff18ec00640 1 --2- 172.21.15.42:0/89891049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff188105b70 0x7ff18810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.957+0000 7ff18ec00640 1 --2- 172.21.15.42:0/89891049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1881051b0 0x7ff188105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.957+0000 7ff18ec00640 1 --2- 172.21.15.42:0/89891049 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff188103fc0 0x7ff1881043e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.957+0000 7ff18ec00640 1 -- 172.21.15.42:0/89891049 >> 172.21.15.42:0/89891049 conn(0x7ff1880ff530 msgr2=0x7ff188101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:00.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.957+0000 7ff18ec00640 1 -- 172.21.15.42:0/89891049 shutdown_connections 2024-04-20T10:34:00.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.957+0000 7ff18ec00640 1 -- 172.21.15.42:0/89891049 wait complete. 2024-04-20T10:34:00.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.958+0000 7ff18ec00640 1 Processor -- start 2024-04-20T10:34:00.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.958+0000 7ff18ec00640 1 -- start start 2024-04-20T10:34:00.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.958+0000 7ff18ec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff188103fc0 0x7ff18819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:00.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.958+0000 7ff18ec00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1881051b0 0x7ff18819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:00.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.958+0000 7ff18ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff188105b70 0x7ff1881a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:00.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.958+0000 7ff18ec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff18810fc50 con 0x7ff188103fc0 2024-04-20T10:34:00.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.958+0000 7ff18ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff18810fad0 con 0x7ff188105b70 2024-04-20T10:34:00.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.958+0000 7ff18ec00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff18810fdd0 con 0x7ff1881051b0 2024-04-20T10:34:00.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.959+0000 7ff18ce00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1881051b0 0x7ff18819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:00.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.959+0000 7ff18d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff188103fc0 0x7ff18819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:00.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.959+0000 7ff18e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff188105b70 0x7ff1881a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:00.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.959+0000 7ff18ce00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1881051b0 0x7ff18819a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:55488/0 (socket says 172.21.15.42:55488) 2024-04-20T10:34:00.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.959+0000 7ff18ce00640 1 -- 172.21.15.42:0/470782826 learned_addr learned my addr 172.21.15.42:0/470782826 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:00.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.959+0000 7ff18ce00640 1 -- 172.21.15.42:0/470782826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff188105b70 msgr2=0x7ff1881a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:00.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.959+0000 7ff18ce00640 1 --2- 172.21.15.42:0/470782826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff188105b70 0x7ff1881a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.959+0000 7ff18ce00640 1 -- 172.21.15.42:0/470782826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff188103fc0 msgr2=0x7ff18819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:00.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.959+0000 7ff18ce00640 1 --2- 172.21.15.42:0/470782826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff188103fc0 0x7ff18819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:00.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.959+0000 7ff18ce00640 1 -- 172.21.15.42:0/470782826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1881a1f80 con 0x7ff1881051b0 2024-04-20T10:34:00.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.959+0000 7ff18ce00640 1 --2- 172.21.15.42:0/470782826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1881051b0 0x7ff18819a8a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff17400e9f0 tx=0x7ff17400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:00.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.960+0000 7ff186000640 1 -- 172.21.15.42:0/470782826 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17400cd80 con 0x7ff1881051b0 2024-04-20T10:34:00.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.960+0000 7ff186000640 1 -- 172.21.15.42:0/470782826 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff174010910 con 0x7ff1881051b0 2024-04-20T10:34:00.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.960+0000 7ff18ec00640 1 -- 172.21.15.42:0/470782826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1881a2270 con 0x7ff1881051b0 2024-04-20T10:34:00.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.960+0000 7ff186000640 1 -- 172.21.15.42:0/470782826 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174018990 con 0x7ff1881051b0 2024-04-20T10:34:00.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.960+0000 7ff18ec00640 1 -- 172.21.15.42:0/470782826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1881a2810 con 0x7ff1881051b0 2024-04-20T10:34:00.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.961+0000 7ff18ec00640 1 -- 172.21.15.42:0/470782826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff18810b2e0 con 0x7ff1881051b0 2024-04-20T10:34:00.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.963+0000 7ff186000640 1 -- 172.21.15.42:0/470782826 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7ff174010ac0 con 0x7ff1881051b0 2024-04-20T10:34:00.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.964+0000 7ff186000640 1 -- 172.21.15.42:0/470782826 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff1740d0280 con 0x7ff1881051b0 2024-04-20T10:34:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:00.968+0000 7ff186000640 1 -- 172.21.15.42:0/470782826 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff17409c890 con 0x7ff1881051b0 2024-04-20T10:34:01.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.194+0000 7ff18ec00640 1 -- 172.21.15.42:0/470782826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff188062b60 con 0x7ff1881051b0 2024-04-20T10:34:01.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.194+0000 7ff186000640 1 -- 172.21.15.42:0/470782826 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7ff1740a0b90 con 0x7ff1881051b0 2024-04-20T10:34:01.195 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-20T10:34:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.197+0000 7ff18ec00640 1 -- 172.21.15.42:0/470782826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1881051b0 msgr2=0x7ff18819a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.197+0000 7ff18ec00640 1 --2- 172.21.15.42:0/470782826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1881051b0 0x7ff18819a8a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff17400e9f0 tx=0x7ff17400eeb0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.197+0000 7ff18ec00640 1 -- 172.21.15.42:0/470782826 shutdown_connections 2024-04-20T10:34:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.197+0000 7ff18ec00640 1 --2- 172.21.15.42:0/470782826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff188105b70 0x7ff1881a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.197+0000 7ff18ec00640 1 --2- 172.21.15.42:0/470782826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1881051b0 0x7ff18819a8a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.197+0000 7ff18ec00640 1 --2- 172.21.15.42:0/470782826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff188103fc0 0x7ff18819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.197+0000 7ff18ec00640 1 -- 172.21.15.42:0/470782826 >> 172.21.15.42:0/470782826 conn(0x7ff1880ff530 msgr2=0x7ff1881084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.197+0000 7ff18ec00640 1 -- 172.21.15.42:0/470782826 shutdown_connections 2024-04-20T10:34:01.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.197+0000 7ff18ec00640 1 -- 172.21.15.42:0/470782826 wait complete. 2024-04-20T10:34:01.221 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:34:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.419+0000 7f0b10c00640 1 -- 172.21.15.42:0/327246124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b0c1041c0 msgr2=0x7f0b0c1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.419+0000 7f0b10c00640 1 --2- 172.21.15.42:0/327246124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b0c1041c0 0x7f0b0c1045e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009860 tx=0x7f0afc02bb70 comp rx=0 tx=0).stop 2024-04-20T10:34:01.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.420+0000 7f0b10c00640 1 -- 172.21.15.42:0/327246124 shutdown_connections 2024-04-20T10:34:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.420+0000 7f0b10c00640 1 --2- 172.21.15.42:0/327246124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b0c105d70 0x7f0b0c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.420+0000 7f0b10c00640 1 --2- 172.21.15.42:0/327246124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b0c1053b0 0x7f0b0c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.420+0000 7f0b10c00640 1 --2- 172.21.15.42:0/327246124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b0c1041c0 0x7f0b0c1045e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.420+0000 7f0b10c00640 1 -- 172.21.15.42:0/327246124 >> 172.21.15.42:0/327246124 conn(0x7f0b0c0ff750 msgr2=0x7f0b0c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.420+0000 7f0b10c00640 1 -- 172.21.15.42:0/327246124 shutdown_connections 2024-04-20T10:34:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.420+0000 7f0b10c00640 1 -- 172.21.15.42:0/327246124 wait complete. 2024-04-20T10:34:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.421+0000 7f0b10c00640 1 Processor -- start 2024-04-20T10:34:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.421+0000 7f0b10c00640 1 -- start start 2024-04-20T10:34:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.421+0000 7f0b10c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b0c1041c0 0x7f0b0c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.421+0000 7f0b10c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b0c1053b0 0x7f0b0c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.421+0000 7f0b10c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b0c105d70 0x7f0b0c1a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.421+0000 7f0b10c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c10fe50 con 0x7f0b0c105d70 2024-04-20T10:34:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.421+0000 7f0b10c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c10fcd0 con 0x7f0b0c1053b0 2024-04-20T10:34:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.421+0000 7f0b10c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c10ffd0 con 0x7f0b0c1041c0 2024-04-20T10:34:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.421+0000 7f0b09400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b0c1053b0 0x7f0b0c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.422+0000 7f0b09e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b0c1041c0 0x7f0b0c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.422+0000 7f0b09400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b0c1053b0 0x7f0b0c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:51938/0 (socket says 172.21.15.42:51938) 2024-04-20T10:34:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.422+0000 7f0b09400640 1 -- 172.21.15.42:0/2446470520 learned_addr learned my addr 172.21.15.42:0/2446470520 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.422+0000 7f0b0a800640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b0c105d70 0x7f0b0c1a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.422+0000 7f0b09400640 1 -- 172.21.15.42:0/2446470520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b0c1041c0 msgr2=0x7f0b0c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.422+0000 7f0b09400640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b0c1041c0 0x7f0b0c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.422+0000 7f0b09400640 1 -- 172.21.15.42:0/2446470520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b0c105d70 msgr2=0x7f0b0c1a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.422+0000 7f0b09400640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b0c105d70 0x7f0b0c1a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.422+0000 7f0b09400640 1 -- 172.21.15.42:0/2446470520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c1a2100 con 0x7f0b0c1053b0 2024-04-20T10:34:01.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.422+0000 7f0b09400640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b0c1053b0 0x7f0b0c19aa20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0af800e9c0 tx=0x7f0af800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:01.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.422+0000 7f0b02a00640 1 -- 172.21.15.42:0/2446470520 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af800cde0 con 0x7f0b0c1053b0 2024-04-20T10:34:01.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.423+0000 7f0b02a00640 1 -- 172.21.15.42:0/2446470520 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af80035f0 con 0x7f0b0c1053b0 2024-04-20T10:34:01.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.423+0000 7f0b10c00640 1 -- 172.21.15.42:0/2446470520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c1a23f0 con 0x7f0b0c1053b0 2024-04-20T10:34:01.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.423+0000 7f0b02a00640 1 -- 172.21.15.42:0/2446470520 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8010690 con 0x7f0b0c1053b0 2024-04-20T10:34:01.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.423+0000 7f0b10c00640 1 -- 172.21.15.42:0/2446470520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c1a2990 con 0x7f0b0c1053b0 2024-04-20T10:34:01.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.424+0000 7f0b10c00640 1 -- 172.21.15.42:0/2446470520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0c10b530 con 0x7f0b0c1053b0 2024-04-20T10:34:01.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.426+0000 7f0b02a00640 1 -- 172.21.15.42:0/2446470520 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f0af80037a0 con 0x7f0b0c1053b0 2024-04-20T10:34:01.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.428+0000 7f0b02a00640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0af00a3a30 0x7f0af00a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:01.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.428+0000 7f0b02a00640 1 -- 172.21.15.42:0/2446470520 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0af80d0450 con 0x7f0b0c1053b0 2024-04-20T10:34:01.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.428+0000 7f0b09e00640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0af00a3a30 0x7f0af00a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:01.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.429+0000 7f0b09e00640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0af00a3a30 0x7f0af00a5ef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009830 tx=0x7f0afc034ad0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:01.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.432+0000 7f0b02a00640 1 -- 172.21.15.42:0/2446470520 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0af809cfc0 con 0x7f0b0c1053b0 2024-04-20T10:34:01.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.638+0000 7f0b10c00640 1 -- 172.21.15.42:0/2446470520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b0c062b60 con 0x7f0b0c1053b0 2024-04-20T10:34:01.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.639+0000 7f0b02a00640 1 -- 172.21.15.42:0/2446470520 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23100 (secure 0 0 0) 0x7f0af809c8c0 con 0x7f0b0c1053b0 2024-04-20T10:34:01.641 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:34:01.641 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":88,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:34:00.299452+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/230562472":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/1321607472":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/3804122415":"2024-04-21T10:34:00.299359+0000","172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/3610568749":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/869471821":"2024-04-21T10:34:00.299359+0000","172.21.15.42:6800/3610568749":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:34:01.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.642+0000 7f0b10c00640 1 -- 172.21.15.42:0/2446470520 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0af00a3a30 msgr2=0x7f0af00a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:01.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.642+0000 7f0b10c00640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0af00a3a30 0x7f0af00a5ef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009830 tx=0x7f0afc034ad0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.642+0000 7f0b10c00640 1 -- 172.21.15.42:0/2446470520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b0c1053b0 msgr2=0x7f0b0c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:01.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.642+0000 7f0b10c00640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b0c1053b0 0x7f0b0c19aa20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0af800e9c0 tx=0x7f0af800ee80 comp rx=0 tx=0).stop 2024-04-20T10:34:01.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.642+0000 7f0b10c00640 1 -- 172.21.15.42:0/2446470520 shutdown_connections 2024-04-20T10:34:01.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.642+0000 7f0b10c00640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0af00a3a30 0x7f0af00a5ef0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.642+0000 7f0b10c00640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b0c105d70 0x7f0b0c1a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.642+0000 7f0b10c00640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b0c1053b0 0x7f0b0c19aa20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.642+0000 7f0b10c00640 1 --2- 172.21.15.42:0/2446470520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b0c1041c0 0x7f0b0c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.642+0000 7f0b10c00640 1 -- 172.21.15.42:0/2446470520 >> 172.21.15.42:0/2446470520 conn(0x7f0b0c0ff750 msgr2=0x7f0b0c108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:01.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.642+0000 7f0b10c00640 1 -- 172.21.15.42:0/2446470520 shutdown_connections 2024-04-20T10:34:01.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.643+0000 7f0b10c00640 1 -- 172.21.15.42:0/2446470520 wait complete. 2024-04-20T10:34:01.662 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:26:16.537072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:31:33.523692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:31:34.019519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:34:01.663 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:34:01.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.856+0000 7f8718c00640 1 -- 172.21.15.42:0/911096401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141041c0 msgr2=0x7f87141045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:01.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.856+0000 7f8718c00640 1 --2- 172.21.15.42:0/911096401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141041c0 0x7f87141045e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8704009900 tx=0x7f870402bc40 comp rx=0 tx=0).stop 2024-04-20T10:34:01.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.857+0000 7f8718c00640 1 -- 172.21.15.42:0/911096401 shutdown_connections 2024-04-20T10:34:01.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.857+0000 7f8718c00640 1 --2- 172.21.15.42:0/911096401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8714105d70 0x7f871410c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.857+0000 7f8718c00640 1 --2- 172.21.15.42:0/911096401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87141053b0 0x7f8714105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.857+0000 7f8718c00640 1 --2- 172.21.15.42:0/911096401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87141041c0 0x7f87141045e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.857+0000 7f8718c00640 1 -- 172.21.15.42:0/911096401 >> 172.21.15.42:0/911096401 conn(0x7f87140ff750 msgr2=0x7f8714101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:01.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.857+0000 7f8718c00640 1 -- 172.21.15.42:0/911096401 shutdown_connections 2024-04-20T10:34:01.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.857+0000 7f8718c00640 1 -- 172.21.15.42:0/911096401 wait complete. 2024-04-20T10:34:01.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.858+0000 7f8718c00640 1 Processor -- start 2024-04-20T10:34:01.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.858+0000 7f8718c00640 1 -- start start 2024-04-20T10:34:01.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.859+0000 7f8718c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87141041c0 0x7f871419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:01.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.859+0000 7f8718c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87141053b0 0x7f871419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:01.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.859+0000 7f8718c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8714105d70 0x7f87141a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:01.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.859+0000 7f8718c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8714077540 con 0x7f8714105d70 2024-04-20T10:34:01.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.859+0000 7f8718c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f87140773c0 con 0x7f87141041c0 2024-04-20T10:34:01.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.859+0000 7f8718c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f87140776c0 con 0x7f87141053b0 2024-04-20T10:34:01.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.859+0000 7f8712800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8714105d70 0x7f87141a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:01.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.859+0000 7f8711e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87141041c0 0x7f871419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:01.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.859+0000 7f8711e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87141041c0 0x7f871419e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:51950/0 (socket says 172.21.15.42:51950) 2024-04-20T10:34:01.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.859+0000 7f8711e00640 1 -- 172.21.15.42:0/2779402965 learned_addr learned my addr 172.21.15.42:0/2779402965 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:01.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.859+0000 7f870be00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87141053b0 0x7f871419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:01.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.860+0000 7f8711e00640 1 -- 172.21.15.42:0/2779402965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87141053b0 msgr2=0x7f871419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:01.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.860+0000 7f8711e00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87141053b0 0x7f871419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.860+0000 7f8711e00640 1 -- 172.21.15.42:0/2779402965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8714105d70 msgr2=0x7f87141a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:01.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.860+0000 7f8711e00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8714105d70 0x7f87141a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:01.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.860+0000 7f8711e00640 1 -- 172.21.15.42:0/2779402965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87141a64f0 con 0x7f87141041c0 2024-04-20T10:34:01.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.860+0000 7f8712800640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8714105d70 0x7f87141a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.860+0000 7f8711e00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87141041c0 0x7f871419e8d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f870402efd0 tx=0x7f870402c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.860+0000 7f870aa00640 1 -- 172.21.15.42:0/2779402965 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704041070 con 0x7f87141041c0 2024-04-20T10:34:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.860+0000 7f8718c00640 1 -- 172.21.15.42:0/2779402965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87141a6780 con 0x7f87141041c0 2024-04-20T10:34:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.860+0000 7f870aa00640 1 -- 172.21.15.42:0/2779402965 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8704004450 con 0x7f87141041c0 2024-04-20T10:34:01.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.860+0000 7f870aa00640 1 -- 172.21.15.42:0/2779402965 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704003e50 con 0x7f87141041c0 2024-04-20T10:34:01.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.860+0000 7f8718c00640 1 -- 172.21.15.42:0/2779402965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87141a6c90 con 0x7f87141041c0 2024-04-20T10:34:01.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.862+0000 7f8718c00640 1 -- 172.21.15.42:0/2779402965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87140691d0 con 0x7f87141041c0 2024-04-20T10:34:01.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.863+0000 7f870aa00640 1 -- 172.21.15.42:0/2779402965 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f8704044070 con 0x7f87141041c0 2024-04-20T10:34:01.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.865+0000 7f870aa00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f86f80a3b50 0x7f86f80a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:01.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.865+0000 7f870aa00640 1 -- 172.21.15.42:0/2779402965 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f87040f1c60 con 0x7f87141041c0 2024-04-20T10:34:01.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.865+0000 7f870be00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f86f80a3b50 0x7f86f80a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:01.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.866+0000 7f870be00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f86f80a3b50 0x7f86f80a6010 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f871419fde0 tx=0x7f86fc005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:01.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:01.869+0000 7f870aa00640 1 -- 172.21.15.42:0/2779402965 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f87040be180 con 0x7f87141041c0 2024-04-20T10:34:01.929 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:01 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/780904024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:34:01.930 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:01 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/470782826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:34:01.930 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:01 smithi162 ceph-mon[21854]: mgrmap e23: x(active, since 1.03047s), standbys: y, z 2024-04-20T10:34:01.930 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:01 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:01.930 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:01 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:01.930 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:01 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:01.930 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:01 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:01.930 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:01 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2446470520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:34:02.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:01 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/780904024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:34:02.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:01 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/470782826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:34:02.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:01 smithi046 ceph-mon[21574]: mgrmap e23: x(active, since 1.03047s), standbys: y, z 2024-04-20T10:34:02.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:01 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:02.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:01 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:02.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:01 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:02.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:01 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:02.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:01 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2446470520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:34:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.101+0000 7f8718c00640 1 -- 172.21.15.42:0/2779402965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8714062b60 con 0x7f87141041c0 2024-04-20T10:34:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.102+0000 7f870aa00640 1 -- 172.21.15.42:0/2779402965 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f87040c2480 con 0x7f87141041c0 2024-04-20T10:34:02.102 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-20T10:34:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:01 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/780904024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:34:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:01 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/470782826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:34:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:01 smithi042 ceph-mon[17078]: mgrmap e23: x(active, since 1.03047s), standbys: y, z 2024-04-20T10:34:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:01 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:01 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:01 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:01 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:02.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:01 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2446470520' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:34:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.106+0000 7f8718c00640 1 -- 172.21.15.42:0/2779402965 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f86f80a3b50 msgr2=0x7f86f80a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.106+0000 7f8718c00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f86f80a3b50 0x7f86f80a6010 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f871419fde0 tx=0x7f86fc005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.106+0000 7f8718c00640 1 -- 172.21.15.42:0/2779402965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87141041c0 msgr2=0x7f871419e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.106+0000 7f8718c00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87141041c0 0x7f871419e8d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f870402efd0 tx=0x7f870402c660 comp rx=0 tx=0).stop 2024-04-20T10:34:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.106+0000 7f8718c00640 1 -- 172.21.15.42:0/2779402965 shutdown_connections 2024-04-20T10:34:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.106+0000 7f8718c00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f86f80a3b50 0x7f86f80a6010 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.106+0000 7f8718c00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8714105d70 0x7f87141a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.106+0000 7f8718c00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87141053b0 0x7f871419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.106+0000 7f8718c00640 1 --2- 172.21.15.42:0/2779402965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87141041c0 0x7f871419e8d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.106+0000 7f8718c00640 1 -- 172.21.15.42:0/2779402965 >> 172.21.15.42:0/2779402965 conn(0x7f87140ff750 msgr2=0x7f8714079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:02.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.106+0000 7f8718c00640 1 -- 172.21.15.42:0/2779402965 shutdown_connections 2024-04-20T10:34:02.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.106+0000 7f8718c00640 1 -- 172.21.15.42:0/2779402965 wait complete. 2024-04-20T10:34:02.125 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:34:02.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.342+0000 7f2c8fe00640 1 -- 172.21.15.42:0/3101599627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c90100d70 msgr2=0x7f2c90101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:02.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.342+0000 7f2c8fe00640 1 --2- 172.21.15.42:0/3101599627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c90100d70 0x7f2c90101190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009900 tx=0x7f2c8002bc30 comp rx=0 tx=0).stop 2024-04-20T10:34:02.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.342+0000 7f2c8fe00640 1 -- 172.21.15.42:0/3101599627 shutdown_connections 2024-04-20T10:34:02.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.342+0000 7f2c8fe00640 1 --2- 172.21.15.42:0/3101599627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c90102920 0x7f2c901091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.342+0000 7f2c8fe00640 1 --2- 172.21.15.42:0/3101599627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c90101f60 0x7f2c901023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.342+0000 7f2c8fe00640 1 --2- 172.21.15.42:0/3101599627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c90100d70 0x7f2c90101190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.342+0000 7f2c8fe00640 1 -- 172.21.15.42:0/3101599627 >> 172.21.15.42:0/3101599627 conn(0x7f2c900fc300 msgr2=0x7f2c900fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:02.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.343+0000 7f2c8fe00640 1 -- 172.21.15.42:0/3101599627 shutdown_connections 2024-04-20T10:34:02.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.343+0000 7f2c8fe00640 1 -- 172.21.15.42:0/3101599627 wait complete. 2024-04-20T10:34:02.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.344+0000 7f2c8fe00640 1 Processor -- start 2024-04-20T10:34:02.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.344+0000 7f2c8fe00640 1 -- start start 2024-04-20T10:34:02.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c90100d70 0x7f2c90198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:02.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c90101f60 0x7f2c90198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:02.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c90102920 0x7f2c9019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:02.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9010c9c0 con 0x7f2c90101f60 2024-04-20T10:34:02.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9010c840 con 0x7f2c90100d70 2024-04-20T10:34:02.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9010cb40 con 0x7f2c90102920 2024-04-20T10:34:02.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c90101f60 0x7f2c90198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:02.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c90100d70 0x7f2c90198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:02.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c90100d70 0x7f2c90198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:51970/0 (socket says 172.21.15.42:51970) 2024-04-20T10:34:02.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8ea00640 1 -- 172.21.15.42:0/933054376 learned_addr learned my addr 172.21.15.42:0/933054376 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8f400640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c90102920 0x7f2c9019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8e000640 1 -- 172.21.15.42:0/933054376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c90102920 msgr2=0x7f2c9019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8e000640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c90102920 0x7f2c9019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8e000640 1 -- 172.21.15.42:0/933054376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c90100d70 msgr2=0x7f2c90198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8e000640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c90100d70 0x7f2c90198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8e000640 1 -- 172.21.15.42:0/933054376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9019fd70 con 0x7f2c90101f60 2024-04-20T10:34:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.345+0000 7f2c8ea00640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c90100d70 0x7f2c90198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.346+0000 7f2c8e000640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c90101f60 0x7f2c90198690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00e9f0 tx=0x7f2c7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:02.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.346+0000 7f2c94a00640 1 -- 172.21.15.42:0/933054376 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00cd80 con 0x7f2c90101f60 2024-04-20T10:34:02.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.346+0000 7f2c8fe00640 1 -- 172.21.15.42:0/933054376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c901a0060 con 0x7f2c90101f60 2024-04-20T10:34:02.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.346+0000 7f2c94a00640 1 -- 172.21.15.42:0/933054376 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7c0035f0 con 0x7f2c90101f60 2024-04-20T10:34:02.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.346+0000 7f2c94a00640 1 -- 172.21.15.42:0/933054376 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c005220 con 0x7f2c90101f60 2024-04-20T10:34:02.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.346+0000 7f2c8fe00640 1 -- 172.21.15.42:0/933054376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c901a05b0 con 0x7f2c90101f60 2024-04-20T10:34:02.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.348+0000 7f2c8fe00640 1 -- 172.21.15.42:0/933054376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c901a0870 con 0x7f2c90101f60 2024-04-20T10:34:02.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.349+0000 7f2c94a00640 1 -- 172.21.15.42:0/933054376 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f2c7c0037a0 con 0x7f2c90101f60 2024-04-20T10:34:02.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.352+0000 7f2c94a00640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2c740a39b0 0x7f2c740a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:02.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.352+0000 7f2c94a00640 1 -- 172.21.15.42:0/933054376 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f2c7c0d0370 con 0x7f2c90101f60 2024-04-20T10:34:02.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.352+0000 7f2c8ea00640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2c740a39b0 0x7f2c740a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:02.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.353+0000 7f2c8ea00640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2c740a39b0 0x7f2c740a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2c8002c8e0 tx=0x7f2c80005b50 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:02.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.356+0000 7f2c94a00640 1 -- 172.21.15.42:0/933054376 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2c7c09cee0 con 0x7f2c90101f60 2024-04-20T10:34:02.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.587+0000 7f2c8fe00640 1 -- 172.21.15.42:0/933054376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2c901a0a40 con 0x7f2c90101f60 2024-04-20T10:34:02.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.588+0000 7f2c94a00640 1 -- 172.21.15.42:0/933054376 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f2c7c09c7e0 con 0x7f2c90101f60 2024-04-20T10:34:02.589 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-20T10:34:02.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.591+0000 7f2c8fe00640 1 -- 172.21.15.42:0/933054376 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2c740a39b0 msgr2=0x7f2c740a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:02.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.591+0000 7f2c8fe00640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2c740a39b0 0x7f2c740a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2c8002c8e0 tx=0x7f2c80005b50 comp rx=0 tx=0).stop 2024-04-20T10:34:02.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.591+0000 7f2c8fe00640 1 -- 172.21.15.42:0/933054376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c90101f60 msgr2=0x7f2c90198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:02.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.591+0000 7f2c8fe00640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c90101f60 0x7f2c90198690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00e9f0 tx=0x7f2c7c00eeb0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.591+0000 7f2c8fe00640 1 -- 172.21.15.42:0/933054376 shutdown_connections 2024-04-20T10:34:02.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.591+0000 7f2c8fe00640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2c740a39b0 0x7f2c740a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.591+0000 7f2c8fe00640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c90102920 0x7f2c9019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.591+0000 7f2c8fe00640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c90101f60 0x7f2c90198690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.591+0000 7f2c8fe00640 1 --2- 172.21.15.42:0/933054376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c90100d70 0x7f2c90198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.591+0000 7f2c8fe00640 1 -- 172.21.15.42:0/933054376 >> 172.21.15.42:0/933054376 conn(0x7f2c900fc300 msgr2=0x7f2c900fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:02.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.591+0000 7f2c8fe00640 1 -- 172.21.15.42:0/933054376 shutdown_connections 2024-04-20T10:34:02.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.591+0000 7f2c8fe00640 1 -- 172.21.15.42:0/933054376 wait complete. 2024-04-20T10:34:02.611 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:34:02.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.803+0000 7efd76c00640 1 -- 172.21.15.42:0/2531682623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd70078500 msgr2=0x7efd70078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:02.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.803+0000 7efd76c00640 1 --2- 172.21.15.42:0/2531682623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd70078500 0x7efd70078da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd68009ca0 tx=0x7efd6802c100 comp rx=0 tx=0).stop 2024-04-20T10:34:02.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.803+0000 7efd76c00640 1 -- 172.21.15.42:0/2531682623 shutdown_connections 2024-04-20T10:34:02.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.803+0000 7efd76c00640 1 --2- 172.21.15.42:0/2531682623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd70078500 0x7efd70078da0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.803+0000 7efd76c00640 1 --2- 172.21.15.42:0/2531682623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd70077b60 0x7efd70077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.803+0000 7efd76c00640 1 --2- 172.21.15.42:0/2531682623 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd70076920 0x7efd70076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.803+0000 7efd76c00640 1 -- 172.21.15.42:0/2531682623 >> 172.21.15.42:0/2531682623 conn(0x7efd700ff750 msgr2=0x7efd70101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.803+0000 7efd76c00640 1 -- 172.21.15.42:0/2531682623 shutdown_connections 2024-04-20T10:34:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.803+0000 7efd76c00640 1 -- 172.21.15.42:0/2531682623 wait complete. 2024-04-20T10:34:02.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.804+0000 7efd76c00640 1 Processor -- start 2024-04-20T10:34:02.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.804+0000 7efd76c00640 1 -- start start 2024-04-20T10:34:02.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.804+0000 7efd76c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd70076920 0x7efd7019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:02.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.804+0000 7efd76c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd70077b60 0x7efd7019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.804+0000 7efd76c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd70078500 0x7efd701a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.804+0000 7efd76c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efd70114250 con 0x7efd70077b60 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.804+0000 7efd76c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efd701140d0 con 0x7efd70076920 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.804+0000 7efd76c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efd701143d0 con 0x7efd70078500 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.805+0000 7efd6fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd70076920 0x7efd7019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.805+0000 7efd6fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd70076920 0x7efd7019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:51998/0 (socket says 172.21.15.42:51998) 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.805+0000 7efd6fe00640 1 -- 172.21.15.42:0/1007185576 learned_addr learned my addr 172.21.15.42:0/1007185576 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.805+0000 7efd67400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd70077b60 0x7efd7019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.805+0000 7efd74c00640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd70078500 0x7efd701a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.805+0000 7efd6fe00640 1 -- 172.21.15.42:0/1007185576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd70078500 msgr2=0x7efd701a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.805+0000 7efd6fe00640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd70078500 0x7efd701a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.805+0000 7efd6fe00640 1 -- 172.21.15.42:0/1007185576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd70077b60 msgr2=0x7efd7019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.805+0000 7efd6fe00640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd70077b60 0x7efd7019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.805+0000 7efd6fe00640 1 -- 172.21.15.42:0/1007185576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd701a6410 con 0x7efd70076920 2024-04-20T10:34:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.805+0000 7efd6fe00640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd70076920 0x7efd7019e7f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd6000e9f0 tx=0x7efd6000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.806+0000 7efd6d600640 1 -- 172.21.15.42:0/1007185576 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6000cd80 con 0x7efd70076920 2024-04-20T10:34:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.806+0000 7efd76c00640 1 -- 172.21.15.42:0/1007185576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd701a6700 con 0x7efd70076920 2024-04-20T10:34:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.806+0000 7efd6d600640 1 -- 172.21.15.42:0/1007185576 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd60005500 con 0x7efd70076920 2024-04-20T10:34:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.806+0000 7efd6d600640 1 -- 172.21.15.42:0/1007185576 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60010690 con 0x7efd70076920 2024-04-20T10:34:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.806+0000 7efd76c00640 1 -- 172.21.15.42:0/1007185576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd701a6c50 con 0x7efd70076920 2024-04-20T10:34:02.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.807+0000 7efd76c00640 1 -- 172.21.15.42:0/1007185576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd70110b30 con 0x7efd70076920 2024-04-20T10:34:02.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.809+0000 7efd6d600640 1 -- 172.21.15.42:0/1007185576 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7efd600046c0 con 0x7efd70076920 2024-04-20T10:34:02.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.810+0000 7efd6d600640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7efd540a3d20 0x7efd540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:02.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.811+0000 7efd6d600640 1 -- 172.21.15.42:0/1007185576 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7efd600d0d40 con 0x7efd70076920 2024-04-20T10:34:02.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.811+0000 7efd67400640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7efd540a3d20 0x7efd540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:02.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.811+0000 7efd67400640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7efd540a3d20 0x7efd540a61e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efd7019fd00 tx=0x7efd58005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:02.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:02.812+0000 7efd6d600640 1 -- 172.21.15.42:0/1007185576 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efd6009d380 con 0x7efd70076920 2024-04-20T10:34:02.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:02 smithi042 ceph-mon[17078]: [20/Apr/2024:10:34:01] ENGINE Bus STARTING 2024-04-20T10:34:02.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:02 smithi042 ceph-mon[17078]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:34:02.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:02 smithi042 ceph-mon[17078]: [20/Apr/2024:10:34:01] ENGINE Serving on http://172.21.15.42:8765 2024-04-20T10:34:02.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:02 smithi042 ceph-mon[17078]: [20/Apr/2024:10:34:01] ENGINE Serving on https://172.21.15.42:7150 2024-04-20T10:34:02.931 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:02 smithi042 ceph-mon[17078]: [20/Apr/2024:10:34:01] ENGINE Bus STARTED 2024-04-20T10:34:02.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:02 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2779402965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:34:02.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:02 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:02.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:02 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/933054376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:34:02.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:02 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:03.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.027+0000 7efd76c00640 1 -- 172.21.15.42:0/1007185576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd70062b60 con 0x7efd70076920 2024-04-20T10:34:03.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.028+0000 7efd6d600640 1 -- 172.21.15.42:0/1007185576 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7efd6009cc80 con 0x7efd70076920 2024-04-20T10:34:03.028 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-20T10:34:03.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.030+0000 7efd76c00640 1 -- 172.21.15.42:0/1007185576 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7efd540a3d20 msgr2=0x7efd540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:03.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.030+0000 7efd76c00640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7efd540a3d20 0x7efd540a61e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efd7019fd00 tx=0x7efd58005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:03.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.030+0000 7efd76c00640 1 -- 172.21.15.42:0/1007185576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd70076920 msgr2=0x7efd7019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:03.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.030+0000 7efd76c00640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd70076920 0x7efd7019e7f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd6000e9f0 tx=0x7efd6000edf0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.031+0000 7efd76c00640 1 -- 172.21.15.42:0/1007185576 shutdown_connections 2024-04-20T10:34:03.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.031+0000 7efd76c00640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7efd540a3d20 0x7efd540a61e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.031+0000 7efd76c00640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd70078500 0x7efd701a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.031+0000 7efd76c00640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd70077b60 0x7efd7019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.031+0000 7efd76c00640 1 --2- 172.21.15.42:0/1007185576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd70076920 0x7efd7019e7f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.031+0000 7efd76c00640 1 -- 172.21.15.42:0/1007185576 >> 172.21.15.42:0/1007185576 conn(0x7efd700ff750 msgr2=0x7efd70101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:03.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.031+0000 7efd76c00640 1 -- 172.21.15.42:0/1007185576 shutdown_connections 2024-04-20T10:34:03.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.031+0000 7efd76c00640 1 -- 172.21.15.42:0/1007185576 wait complete. 2024-04-20T10:34:03.050 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-20T10:34:03.050 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:34:03.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:02 smithi046 ceph-mon[21574]: [20/Apr/2024:10:34:01] ENGINE Bus STARTING 2024-04-20T10:34:03.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:02 smithi046 ceph-mon[21574]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:34:03.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:02 smithi046 ceph-mon[21574]: [20/Apr/2024:10:34:01] ENGINE Serving on http://172.21.15.42:8765 2024-04-20T10:34:03.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:02 smithi046 ceph-mon[21574]: [20/Apr/2024:10:34:01] ENGINE Serving on https://172.21.15.42:7150 2024-04-20T10:34:03.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:02 smithi046 ceph-mon[21574]: [20/Apr/2024:10:34:01] ENGINE Bus STARTED 2024-04-20T10:34:03.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:02 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2779402965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:34:03.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:02 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:03.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:02 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/933054376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:34:03.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:02 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:03.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:02 smithi162 ceph-mon[21854]: [20/Apr/2024:10:34:01] ENGINE Bus STARTING 2024-04-20T10:34:03.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:02 smithi162 ceph-mon[21854]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:34:03.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:02 smithi162 ceph-mon[21854]: [20/Apr/2024:10:34:01] ENGINE Serving on http://172.21.15.42:8765 2024-04-20T10:34:03.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:02 smithi162 ceph-mon[21854]: [20/Apr/2024:10:34:01] ENGINE Serving on https://172.21.15.42:7150 2024-04-20T10:34:03.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:02 smithi162 ceph-mon[21854]: [20/Apr/2024:10:34:01] ENGINE Bus STARTED 2024-04-20T10:34:03.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:02 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2779402965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:34:03.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:02 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:03.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:02 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/933054376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:34:03.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:02 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:03.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.245+0000 7eff8a000640 1 -- 172.21.15.42:0/2011029171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff84102850 msgr2=0x7eff84104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:03.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.245+0000 7eff8a000640 1 --2- 172.21.15.42:0/2011029171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff84102850 0x7eff84104c40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7eff78009900 tx=0x7eff7802bc40 comp rx=0 tx=0).stop 2024-04-20T10:34:03.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.245+0000 7eff8a000640 1 -- 172.21.15.42:0/2011029171 shutdown_connections 2024-04-20T10:34:03.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.245+0000 7eff8a000640 1 --2- 172.21.15.42:0/2011029171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff84107ab0 0x7eff84109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.245+0000 7eff8a000640 1 --2- 172.21.15.42:0/2011029171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff84105180 0x7eff84107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.245+0000 7eff8a000640 1 --2- 172.21.15.42:0/2011029171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff84102850 0x7eff84104c40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.245+0000 7eff8a000640 1 -- 172.21.15.42:0/2011029171 >> 172.21.15.42:0/2011029171 conn(0x7eff840fc240 msgr2=0x7eff840fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:03.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.246+0000 7eff8a000640 1 -- 172.21.15.42:0/2011029171 shutdown_connections 2024-04-20T10:34:03.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.246+0000 7eff8a000640 1 -- 172.21.15.42:0/2011029171 wait complete. 2024-04-20T10:34:03.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.246+0000 7eff8a000640 1 Processor -- start 2024-04-20T10:34:03.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.247+0000 7eff8a000640 1 -- start start 2024-04-20T10:34:03.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.247+0000 7eff8a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff84102850 0x7eff841021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:03.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.247+0000 7eff8a000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff84105180 0x7eff84100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:03.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.247+0000 7eff8a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff84107ab0 0x7eff84100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:03.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.247+0000 7eff8a000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7eff8410dac0 con 0x7eff84107ab0 2024-04-20T10:34:03.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.247+0000 7eff8a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff8410d940 con 0x7eff84102850 2024-04-20T10:34:03.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.247+0000 7eff8a000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7eff8410dc40 con 0x7eff84105180 2024-04-20T10:34:03.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.247+0000 7eff88c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff84102850 0x7eff841021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:03.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.247+0000 7eff89600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff84107ab0 0x7eff84100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:03.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.247+0000 7eff83e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff84105180 0x7eff84100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.247+0000 7eff88c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff84102850 0x7eff841021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52022/0 (socket says 172.21.15.42:52022) 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.247+0000 7eff88c00640 1 -- 172.21.15.42:0/875804641 learned_addr learned my addr 172.21.15.42:0/875804641 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.248+0000 7eff88c00640 1 -- 172.21.15.42:0/875804641 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff84105180 msgr2=0x7eff84100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.248+0000 7eff88c00640 1 --2- 172.21.15.42:0/875804641 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff84105180 0x7eff84100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.248+0000 7eff88c00640 1 -- 172.21.15.42:0/875804641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff84107ab0 msgr2=0x7eff84100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.248+0000 7eff88c00640 1 --2- 172.21.15.42:0/875804641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff84107ab0 0x7eff84100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.248+0000 7eff88c00640 1 -- 172.21.15.42:0/875804641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff84101280 con 0x7eff84102850 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.248+0000 7eff88c00640 1 --2- 172.21.15.42:0/875804641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff84102850 0x7eff841021c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7eff7802c8f0 tx=0x7eff7802c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.248+0000 7eff81600640 1 -- 172.21.15.42:0/875804641 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78041070 con 0x7eff84102850 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.248+0000 7eff8a000640 1 -- 172.21.15.42:0/875804641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff84101510 con 0x7eff84102850 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.248+0000 7eff81600640 1 -- 172.21.15.42:0/875804641 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff78004450 con 0x7eff84102850 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.248+0000 7eff81600640 1 -- 172.21.15.42:0/875804641 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78034440 con 0x7eff84102850 2024-04-20T10:34:03.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.248+0000 7eff8a000640 1 -- 172.21.15.42:0/875804641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff841a6b60 con 0x7eff84102850 2024-04-20T10:34:03.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.249+0000 7eff8a000640 1 -- 172.21.15.42:0/875804641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff840fdd30 con 0x7eff84102850 2024-04-20T10:34:03.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.251+0000 7eff81600640 1 -- 172.21.15.42:0/875804641 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7eff78044070 con 0x7eff84102850 2024-04-20T10:34:03.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.253+0000 7eff81600640 1 --2- 172.21.15.42:0/875804641 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:03.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.253+0000 7eff81600640 1 -- 172.21.15.42:0/875804641 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7eff780f2200 con 0x7eff84102850 2024-04-20T10:34:03.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.253+0000 7eff83e00640 1 --2- 172.21.15.42:0/875804641 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:03.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.254+0000 7eff83e00640 1 --2- 172.21.15.42:0/875804641 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff84101f30 tx=0x7eff70005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:03.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.257+0000 7eff81600640 1 -- 172.21.15.42:0/875804641 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7eff780be300 con 0x7eff84102850 2024-04-20T10:34:03.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.522+0000 7eff8a000640 1 -- 172.21.15.42:0/875804641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff8402d5d0 con 0x7eff84102850 2024-04-20T10:34:03.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.523+0000 7eff81600640 1 -- 172.21.15.42:0/875804641 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7eff780c2600 con 0x7eff84102850 2024-04-20T10:34:03.524 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:34:03.524 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:34:03.525 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:34:03.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.525+0000 7eff8a000640 1 -- 172.21.15.42:0/875804641 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7eff6c0a3d20 msgr2=0x7eff6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:03.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.525+0000 7eff8a000640 1 --2- 172.21.15.42:0/875804641 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff84101f30 tx=0x7eff70005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:03.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.526+0000 7eff8a000640 1 -- 172.21.15.42:0/875804641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff84102850 msgr2=0x7eff841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:03.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.526+0000 7eff8a000640 1 --2- 172.21.15.42:0/875804641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff84102850 0x7eff841021c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7eff7802c8f0 tx=0x7eff7802c660 comp rx=0 tx=0).stop 2024-04-20T10:34:03.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.526+0000 7eff8a000640 1 -- 172.21.15.42:0/875804641 shutdown_connections 2024-04-20T10:34:03.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.526+0000 7eff8a000640 1 --2- 172.21.15.42:0/875804641 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7eff6c0a3d20 0x7eff6c0a61e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.526+0000 7eff8a000640 1 --2- 172.21.15.42:0/875804641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff84107ab0 0x7eff84100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.526+0000 7eff8a000640 1 --2- 172.21.15.42:0/875804641 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff84105180 0x7eff84100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.526+0000 7eff8a000640 1 --2- 172.21.15.42:0/875804641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff84102850 0x7eff841021c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.526+0000 7eff8a000640 1 -- 172.21.15.42:0/875804641 >> 172.21.15.42:0/875804641 conn(0x7eff840fc240 msgr2=0x7eff84103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:03.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.526+0000 7eff8a000640 1 -- 172.21.15.42:0/875804641 shutdown_connections 2024-04-20T10:34:03.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.526+0000 7eff8a000640 1 -- 172.21.15.42:0/875804641 wait complete. 2024-04-20T10:34:03.551 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-20T10:34:03.551 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-20T10:34:03.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:34:03.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.748+0000 7faa02800640 1 -- 172.21.15.42:0/2076247767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9fc1041c0 msgr2=0x7fa9fc1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:03.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.748+0000 7faa02800640 1 --2- 172.21.15.42:0/2076247767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9fc1041c0 0x7fa9fc1045e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009900 tx=0x7fa9ec02bc30 comp rx=0 tx=0).stop 2024-04-20T10:34:03.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.748+0000 7faa02800640 1 -- 172.21.15.42:0/2076247767 shutdown_connections 2024-04-20T10:34:03.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.748+0000 7faa02800640 1 --2- 172.21.15.42:0/2076247767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9fc105d70 0x7fa9fc10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.748+0000 7faa02800640 1 --2- 172.21.15.42:0/2076247767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9fc1053b0 0x7fa9fc105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.748+0000 7faa02800640 1 --2- 172.21.15.42:0/2076247767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9fc1041c0 0x7fa9fc1045e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.748+0000 7faa02800640 1 -- 172.21.15.42:0/2076247767 >> 172.21.15.42:0/2076247767 conn(0x7fa9fc0ff750 msgr2=0x7fa9fc101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:03.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.748+0000 7faa02800640 1 -- 172.21.15.42:0/2076247767 shutdown_connections 2024-04-20T10:34:03.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.748+0000 7faa02800640 1 -- 172.21.15.42:0/2076247767 wait complete. 2024-04-20T10:34:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.749+0000 7faa02800640 1 Processor -- start 2024-04-20T10:34:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.750+0000 7faa02800640 1 -- start start 2024-04-20T10:34:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.750+0000 7faa02800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9fc1041c0 0x7fa9fc079d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.750+0000 7faa02800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9fc1053b0 0x7fa9fc07a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.750+0000 7faa02800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9fc105d70 0x7fa9fc076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.750+0000 7faa02800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc10fea0 con 0x7fa9fc105d70 2024-04-20T10:34:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.750+0000 7faa02800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc10fd20 con 0x7fa9fc1053b0 2024-04-20T10:34:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.750+0000 7faa02800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc110020 con 0x7fa9fc1041c0 2024-04-20T10:34:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.750+0000 7fa9fbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9fc105d70 0x7fa9fc076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.750+0000 7fa9faa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9fc1053b0 0x7fa9fc07a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.750+0000 7fa9fbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9fc105d70 0x7fa9fc076880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34262/0 (socket says 172.21.15.42:34262) 2024-04-20T10:34:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.750+0000 7fa9fbe00640 1 -- 172.21.15.42:0/777987734 learned_addr learned my addr 172.21.15.42:0/777987734 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.750+0000 7fa9fb400640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9fc1041c0 0x7fa9fc079d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.751+0000 7fa9fb400640 1 -- 172.21.15.42:0/777987734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9fc1053b0 msgr2=0x7fa9fc07a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.751+0000 7fa9fb400640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9fc1053b0 0x7fa9fc07a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.751+0000 7fa9fb400640 1 -- 172.21.15.42:0/777987734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9fc105d70 msgr2=0x7fa9fc076880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.751+0000 7fa9fb400640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9fc105d70 0x7fa9fc076880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.751+0000 7fa9fb400640 1 -- 172.21.15.42:0/777987734 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc077140 con 0x7fa9fc1041c0 2024-04-20T10:34:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.751+0000 7fa9fbe00640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9fc105d70 0x7fa9fc076880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.751+0000 7fa9fb400640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9fc1041c0 0x7fa9fc079d10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009230 tx=0x7fa9ec02c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.751+0000 7fa9f3e00640 1 -- 172.21.15.42:0/777987734 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec043020 con 0x7fa9fc1041c0 2024-04-20T10:34:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.751+0000 7fa9f3e00640 1 -- 172.21.15.42:0/777987734 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ec0054e0 con 0x7fa9fc1041c0 2024-04-20T10:34:03.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.751+0000 7fa9f3e00640 1 -- 172.21.15.42:0/777987734 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec003b60 con 0x7fa9fc1041c0 2024-04-20T10:34:03.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.751+0000 7faa02800640 1 -- 172.21.15.42:0/777987734 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9fc1a6970 con 0x7fa9fc1041c0 2024-04-20T10:34:03.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.752+0000 7faa02800640 1 -- 172.21.15.42:0/777987734 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9fc1a6d70 con 0x7fa9fc1041c0 2024-04-20T10:34:03.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.753+0000 7faa02800640 1 -- 172.21.15.42:0/777987734 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9fc0691d0 con 0x7fa9fc1041c0 2024-04-20T10:34:03.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.754+0000 7fa9f3e00640 1 -- 172.21.15.42:0/777987734 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7fa9ec046020 con 0x7fa9fc1041c0 2024-04-20T10:34:03.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.756+0000 7fa9f3e00640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa9e00a3da0 0x7fa9e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.756+0000 7fa9f3e00640 1 -- 172.21.15.42:0/777987734 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fa9ec0f2680 con 0x7fa9fc1041c0 2024-04-20T10:34:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.756+0000 7fa9faa00640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa9e00a3da0 0x7fa9e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.757+0000 7fa9faa00640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa9e00a3da0 0x7fa9e00a6260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc077c90 tx=0x7fa9e8005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:03.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.760+0000 7fa9f3e00640 1 -- 172.21.15.42:0/777987734 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa9ec04d020 con 0x7fa9fc1041c0 2024-04-20T10:34:03.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:34:03.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-20T10:34:03.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1007185576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:34:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/875804641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:34:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:03.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:03 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:03.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:03.988+0000 7faa02800640 1 -- 172.21.15.42:0/777987734 --> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9fc0623a0 con 0x7fa9e00a3da0 2024-04-20T10:34:04.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.001+0000 7fa9f3e00640 1 -- 172.21.15.42:0/777987734 <== mgr.15024 v2:172.21.15.42:6800/170978916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257971 (secure 0 0 0) 0x7fa9fc0623a0 con 0x7fa9e00a3da0 2024-04-20T10:34:04.005 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:34:04.006 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-04-20T10:34:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.005+0000 7faa02800640 1 -- 172.21.15.42:0/777987734 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa9e00a3da0 msgr2=0x7fa9e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.005+0000 7faa02800640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa9e00a3da0 0x7fa9e00a6260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc077c90 tx=0x7fa9e8005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.005+0000 7faa02800640 1 -- 172.21.15.42:0/777987734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9fc1041c0 msgr2=0x7fa9fc079d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.005+0000 7faa02800640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9fc1041c0 0x7fa9fc079d10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009230 tx=0x7fa9ec02c5d0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.005+0000 7faa02800640 1 -- 172.21.15.42:0/777987734 shutdown_connections 2024-04-20T10:34:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.005+0000 7faa02800640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa9e00a3da0 0x7fa9e00a6260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.006+0000 7faa02800640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9fc105d70 0x7fa9fc076880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.006+0000 7faa02800640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9fc1053b0 0x7fa9fc07a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.006+0000 7faa02800640 1 --2- 172.21.15.42:0/777987734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9fc1041c0 0x7fa9fc079d10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.006+0000 7faa02800640 1 -- 172.21.15.42:0/777987734 >> 172.21.15.42:0/777987734 conn(0x7fa9fc0ff750 msgr2=0x7fa9fc1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.006+0000 7faa02800640 1 -- 172.21.15.42:0/777987734 shutdown_connections 2024-04-20T10:34:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.006+0000 7faa02800640 1 -- 172.21.15.42:0/777987734 wait complete. 2024-04-20T10:34:04.028 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-20T10:34:02.327235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1183976,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":529,"num_read_kb":417,"num_write":384,"num_write_kb":4345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369180,"kb_used_data":11664,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572644,"statfs":{"total":1151940427776,"available":1151562387456,"internally_reserved":0,"allocated":11943936,"data_stored":9092096,"data_compressed":281912,"data_compressed_allocated":376832,"data_compressed_original":3604480,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.983813"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279951+0000","last_change":"2024-04-20T10:31:35.649585+0000","last_active":"2024-04-20T10:34:01.279951+0000","last_peered":"2024-04-20T10:34:01.279951+0000","last_clean":"2024-04-20T10:34:01.279951+0000","last_became_active":"2024-04-20T10:31:35.649282+0000","last_became_peered":"2024-04-20T10:31:35.649282+0000","last_unstale":"2024-04-20T10:34:01.279951+0000","last_undegraded":"2024-04-20T10:34:01.279951+0000","last_fullsized":"2024-04-20T10:34:01.279951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:01:00.517383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313417+0000","last_change":"2024-04-20T10:31:35.648299+0000","last_active":"2024-04-20T10:34:00.313417+0000","last_peered":"2024-04-20T10:34:00.313417+0000","last_clean":"2024-04-20T10:34:00.313417+0000","last_became_active":"2024-04-20T10:31:35.647283+0000","last_became_peered":"2024-04-20T10:31:35.647283+0000","last_unstale":"2024-04-20T10:34:00.313417+0000","last_undegraded":"2024-04-20T10:34:00.313417+0000","last_fullsized":"2024-04-20T10:34:00.313417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:59.006234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.314500+0000","last_change":"2024-04-20T10:31:35.647029+0000","last_active":"2024-04-20T10:34:00.314500+0000","last_peered":"2024-04-20T10:34:00.314500+0000","last_clean":"2024-04-20T10:34:00.314500+0000","last_became_active":"2024-04-20T10:31:35.646649+0000","last_became_peered":"2024-04-20T10:31:35.646649+0000","last_unstale":"2024-04-20T10:34:00.314500+0000","last_undegraded":"2024-04-20T10:34:00.314500+0000","last_fullsized":"2024-04-20T10:34:00.314500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:03:23.420078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272555+0000","last_change":"2024-04-20T10:31:35.624628+0000","last_active":"2024-04-20T10:34:01.272555+0000","last_peered":"2024-04-20T10:34:01.272555+0000","last_clean":"2024-04-20T10:34:01.272555+0000","last_became_active":"2024-04-20T10:31:35.623607+0000","last_became_peered":"2024-04-20T10:31:35.623607+0000","last_unstale":"2024-04-20T10:34:01.272555+0000","last_undegraded":"2024-04-20T10:34:01.272555+0000","last_fullsized":"2024-04-20T10:34:01.272555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:44:46.768584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.445865+0000","last_change":"2024-04-20T10:31:35.638293+0000","last_active":"2024-04-20T10:34:00.445865+0000","last_peered":"2024-04-20T10:34:00.445865+0000","last_clean":"2024-04-20T10:34:00.445865+0000","last_became_active":"2024-04-20T10:31:35.637967+0000","last_became_peered":"2024-04-20T10:31:35.637967+0000","last_unstale":"2024-04-20T10:34:00.445865+0000","last_undegraded":"2024-04-20T10:34:00.445865+0000","last_fullsized":"2024-04-20T10:34:00.445865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:58:00.765015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.435951+0000","last_change":"2024-04-20T10:31:35.634565+0000","last_active":"2024-04-20T10:34:00.435951+0000","last_peered":"2024-04-20T10:34:00.435951+0000","last_clean":"2024-04-20T10:34:00.435951+0000","last_became_active":"2024-04-20T10:31:35.634200+0000","last_became_peered":"2024-04-20T10:31:35.634200+0000","last_unstale":"2024-04-20T10:34:00.435951+0000","last_undegraded":"2024-04-20T10:34:00.435951+0000","last_fullsized":"2024-04-20T10:34:00.435951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:14.298732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272189+0000","last_change":"2024-04-20T10:32:31.949166+0000","last_active":"2024-04-20T10:34:01.272189+0000","last_peered":"2024-04-20T10:34:01.272189+0000","last_clean":"2024-04-20T10:34:01.272189+0000","last_became_active":"2024-04-20T10:32:31.948825+0000","last_became_peered":"2024-04-20T10:32:31.948825+0000","last_unstale":"2024-04-20T10:34:01.272189+0000","last_undegraded":"2024-04-20T10:34:01.272189+0000","last_fullsized":"2024-04-20T10:34:01.272189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:05.337941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.438962+0000","last_change":"2024-04-20T10:31:35.631996+0000","last_active":"2024-04-20T10:34:00.438962+0000","last_peered":"2024-04-20T10:34:00.438962+0000","last_clean":"2024-04-20T10:34:00.438962+0000","last_became_active":"2024-04-20T10:31:35.631576+0000","last_became_peered":"2024-04-20T10:31:35.631576+0000","last_unstale":"2024-04-20T10:34:00.438962+0000","last_undegraded":"2024-04-20T10:34:00.438962+0000","last_fullsized":"2024-04-20T10:34:00.438962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:51.448025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.652006+0000","last_change":"2024-04-20T10:31:35.624430+0000","last_active":"2024-04-20T10:34:00.652006+0000","last_peered":"2024-04-20T10:34:00.652006+0000","last_clean":"2024-04-20T10:34:00.652006+0000","last_became_active":"2024-04-20T10:31:35.623018+0000","last_became_peered":"2024-04-20T10:31:35.623018+0000","last_unstale":"2024-04-20T10:34:00.652006+0000","last_undegraded":"2024-04-20T10:34:00.652006+0000","last_fullsized":"2024-04-20T10:34:00.652006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:43.100925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.436193+0000","last_change":"2024-04-20T10:31:35.619038+0000","last_active":"2024-04-20T10:34:00.436193+0000","last_peered":"2024-04-20T10:34:00.436193+0000","last_clean":"2024-04-20T10:34:00.436193+0000","last_became_active":"2024-04-20T10:31:35.618699+0000","last_became_peered":"2024-04-20T10:31:35.618699+0000","last_unstale":"2024-04-20T10:34:00.436193+0000","last_undegraded":"2024-04-20T10:34:00.436193+0000","last_fullsized":"2024-04-20T10:34:00.436193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:27.205634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272793+0000","last_change":"2024-04-20T10:31:35.619908+0000","last_active":"2024-04-20T10:34:01.272793+0000","last_peered":"2024-04-20T10:34:01.272793+0000","last_clean":"2024-04-20T10:34:01.272793+0000","last_became_active":"2024-04-20T10:31:35.619580+0000","last_became_peered":"2024-04-20T10:31:35.619580+0000","last_unstale":"2024-04-20T10:34:01.272793+0000","last_undegraded":"2024-04-20T10:34:01.272793+0000","last_fullsized":"2024-04-20T10:34:01.272793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:54:48.999925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.309895+0000","last_change":"2024-04-20T10:32:31.952701+0000","last_active":"2024-04-20T10:34:00.309895+0000","last_peered":"2024-04-20T10:34:00.309895+0000","last_clean":"2024-04-20T10:34:00.309895+0000","last_became_active":"2024-04-20T10:32:31.951329+0000","last_became_peered":"2024-04-20T10:32:31.951329+0000","last_unstale":"2024-04-20T10:34:00.309895+0000","last_undegraded":"2024-04-20T10:34:00.309895+0000","last_fullsized":"2024-04-20T10:34:00.309895+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:29.442878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307970+0000","last_change":"2024-04-20T10:31:35.619594+0000","last_active":"2024-04-20T10:34:00.307970+0000","last_peered":"2024-04-20T10:34:00.307970+0000","last_clean":"2024-04-20T10:34:00.307970+0000","last_became_active":"2024-04-20T10:31:35.618831+0000","last_became_peered":"2024-04-20T10:31:35.618831+0000","last_unstale":"2024-04-20T10:34:00.307970+0000","last_undegraded":"2024-04-20T10:34:00.307970+0000","last_fullsized":"2024-04-20T10:34:00.307970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:07.625225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313280+0000","last_change":"2024-04-20T10:31:35.647476+0000","last_active":"2024-04-20T10:34:00.313280+0000","last_peered":"2024-04-20T10:34:00.313280+0000","last_clean":"2024-04-20T10:34:00.313280+0000","last_became_active":"2024-04-20T10:31:35.647009+0000","last_became_peered":"2024-04-20T10:31:35.647009+0000","last_unstale":"2024-04-20T10:34:00.313280+0000","last_undegraded":"2024-04-20T10:34:00.313280+0000","last_fullsized":"2024-04-20T10:34:00.313280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:11:37.526115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313153+0000","last_change":"2024-04-20T10:32:31.958321+0000","last_active":"2024-04-20T10:34:00.313153+0000","last_peered":"2024-04-20T10:34:00.313153+0000","last_clean":"2024-04-20T10:34:00.313153+0000","last_became_active":"2024-04-20T10:32:31.957681+0000","last_became_peered":"2024-04-20T10:32:31.957681+0000","last_unstale":"2024-04-20T10:34:00.313153+0000","last_undegraded":"2024-04-20T10:34:00.313153+0000","last_fullsized":"2024-04-20T10:34:00.313153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:49:31.154909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650987+0000","last_change":"2024-04-20T10:31:35.643142+0000","last_active":"2024-04-20T10:34:00.650987+0000","last_peered":"2024-04-20T10:34:00.650987+0000","last_clean":"2024-04-20T10:34:00.650987+0000","last_became_active":"2024-04-20T10:31:35.642742+0000","last_became_peered":"2024-04-20T10:31:35.642742+0000","last_unstale":"2024-04-20T10:34:00.650987+0000","last_undegraded":"2024-04-20T10:34:00.650987+0000","last_fullsized":"2024-04-20T10:34:00.650987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:40.818808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.280039+0000","last_change":"2024-04-20T10:31:35.650333+0000","last_active":"2024-04-20T10:34:01.280039+0000","last_peered":"2024-04-20T10:34:01.280039+0000","last_clean":"2024-04-20T10:34:01.280039+0000","last_became_active":"2024-04-20T10:31:35.649933+0000","last_became_peered":"2024-04-20T10:31:35.649933+0000","last_unstale":"2024-04-20T10:34:01.280039+0000","last_undegraded":"2024-04-20T10:34:01.280039+0000","last_fullsized":"2024-04-20T10:34:01.280039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:34.513990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.315466+0000","last_change":"2024-04-20T10:31:35.637315+0000","last_active":"2024-04-20T10:34:00.315466+0000","last_peered":"2024-04-20T10:34:00.315466+0000","last_clean":"2024-04-20T10:34:00.315466+0000","last_became_active":"2024-04-20T10:31:35.635909+0000","last_became_peered":"2024-04-20T10:31:35.635909+0000","last_unstale":"2024-04-20T10:34:00.315466+0000","last_undegraded":"2024-04-20T10:34:00.315466+0000","last_fullsized":"2024-04-20T10:34:00.315466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:24:07.400032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650759+0000","last_change":"2024-04-20T10:31:35.638888+0000","last_active":"2024-04-20T10:34:00.650759+0000","last_peered":"2024-04-20T10:34:00.650759+0000","last_clean":"2024-04-20T10:34:00.650759+0000","last_became_active":"2024-04-20T10:31:35.638568+0000","last_became_peered":"2024-04-20T10:31:35.638568+0000","last_unstale":"2024-04-20T10:34:00.650759+0000","last_undegraded":"2024-04-20T10:34:00.650759+0000","last_fullsized":"2024-04-20T10:34:00.650759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:48.363874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.315517+0000","last_change":"2024-04-20T10:31:34.597814+0000","last_active":"2024-04-20T10:34:00.315517+0000","last_peered":"2024-04-20T10:34:00.315517+0000","last_clean":"2024-04-20T10:34:00.315517+0000","last_became_active":"2024-04-20T10:31:34.597480+0000","last_became_peered":"2024-04-20T10:31:34.597480+0000","last_unstale":"2024-04-20T10:34:00.315517+0000","last_undegraded":"2024-04-20T10:34:00.315517+0000","last_fullsized":"2024-04-20T10:34:00.315517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:48:26.102787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.309886+0000","last_change":"2024-04-20T10:32:31.956771+0000","last_active":"2024-04-20T10:34:00.309886+0000","last_peered":"2024-04-20T10:34:00.309886+0000","last_clean":"2024-04-20T10:34:00.309886+0000","last_became_active":"2024-04-20T10:32:31.955413+0000","last_became_peered":"2024-04-20T10:32:31.955413+0000","last_unstale":"2024-04-20T10:34:00.309886+0000","last_undegraded":"2024-04-20T10:34:00.309886+0000","last_fullsized":"2024-04-20T10:34:00.309886+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:37.199764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.438965+0000","last_change":"2024-04-20T10:31:34.609942+0000","last_active":"2024-04-20T10:34:00.438965+0000","last_peered":"2024-04-20T10:34:00.438965+0000","last_clean":"2024-04-20T10:34:00.438965+0000","last_became_active":"2024-04-20T10:31:34.608994+0000","last_became_peered":"2024-04-20T10:31:34.608994+0000","last_unstale":"2024-04-20T10:34:00.438965+0000","last_undegraded":"2024-04-20T10:34:00.438965+0000","last_fullsized":"2024-04-20T10:34:00.438965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:26:53.488091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.314873+0000","last_change":"2024-04-20T10:31:35.647710+0000","last_active":"2024-04-20T10:34:00.314873+0000","last_peered":"2024-04-20T10:34:00.314873+0000","last_clean":"2024-04-20T10:34:00.314873+0000","last_became_active":"2024-04-20T10:31:35.646037+0000","last_became_peered":"2024-04-20T10:31:35.646037+0000","last_unstale":"2024-04-20T10:34:00.314873+0000","last_undegraded":"2024-04-20T10:34:00.314873+0000","last_fullsized":"2024-04-20T10:34:00.314873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:50:21.092892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446278+0000","last_change":"2024-04-20T10:31:34.593529+0000","last_active":"2024-04-20T10:34:00.446278+0000","last_peered":"2024-04-20T10:34:00.446278+0000","last_clean":"2024-04-20T10:34:00.446278+0000","last_became_active":"2024-04-20T10:31:34.592583+0000","last_became_peered":"2024-04-20T10:31:34.592583+0000","last_unstale":"2024-04-20T10:34:00.446278+0000","last_undegraded":"2024-04-20T10:34:00.446278+0000","last_fullsized":"2024-04-20T10:34:00.446278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:51:20.570499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.651292+0000","last_change":"2024-04-20T10:31:35.630739+0000","last_active":"2024-04-20T10:34:00.651292+0000","last_peered":"2024-04-20T10:34:00.651292+0000","last_clean":"2024-04-20T10:34:00.651292+0000","last_became_active":"2024-04-20T10:31:35.630424+0000","last_became_peered":"2024-04-20T10:31:35.630424+0000","last_unstale":"2024-04-20T10:34:00.651292+0000","last_undegraded":"2024-04-20T10:34:00.651292+0000","last_fullsized":"2024-04-20T10:34:00.651292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:48:17.485677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446129+0000","last_change":"2024-04-20T10:31:34.591652+0000","last_active":"2024-04-20T10:34:00.446129+0000","last_peered":"2024-04-20T10:34:00.446129+0000","last_clean":"2024-04-20T10:34:00.446129+0000","last_became_active":"2024-04-20T10:31:34.588945+0000","last_became_peered":"2024-04-20T10:31:34.588945+0000","last_unstale":"2024-04-20T10:34:00.446129+0000","last_undegraded":"2024-04-20T10:34:00.446129+0000","last_fullsized":"2024-04-20T10:34:00.446129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:15.137046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456803+0000","last_change":"2024-04-20T10:31:35.622462+0000","last_active":"2024-04-20T10:34:00.456803+0000","last_peered":"2024-04-20T10:34:00.456803+0000","last_clean":"2024-04-20T10:34:00.456803+0000","last_became_active":"2024-04-20T10:31:35.622125+0000","last_became_peered":"2024-04-20T10:31:35.622125+0000","last_unstale":"2024-04-20T10:34:00.456803+0000","last_undegraded":"2024-04-20T10:34:00.456803+0000","last_fullsized":"2024-04-20T10:34:00.456803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:27.894564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456509+0000","last_change":"2024-04-20T10:32:32.043629+0000","last_active":"2024-04-20T10:34:00.456509+0000","last_peered":"2024-04-20T10:34:00.456509+0000","last_clean":"2024-04-20T10:34:00.456509+0000","last_became_active":"2024-04-20T10:32:32.042849+0000","last_became_peered":"2024-04-20T10:32:32.042849+0000","last_unstale":"2024-04-20T10:34:00.456509+0000","last_undegraded":"2024-04-20T10:34:00.456509+0000","last_fullsized":"2024-04-20T10:34:00.456509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:52.646201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.445985+0000","last_change":"2024-04-20T10:32:31.951658+0000","last_active":"2024-04-20T10:34:00.445985+0000","last_peered":"2024-04-20T10:34:00.445985+0000","last_clean":"2024-04-20T10:34:00.445985+0000","last_became_active":"2024-04-20T10:32:31.951307+0000","last_became_peered":"2024-04-20T10:32:31.951307+0000","last_unstale":"2024-04-20T10:34:00.445985+0000","last_undegraded":"2024-04-20T10:34:00.445985+0000","last_fullsized":"2024-04-20T10:34:00.445985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:42.743969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.435759+0000","last_change":"2024-04-20T10:31:34.612038+0000","last_active":"2024-04-20T10:34:00.435759+0000","last_peered":"2024-04-20T10:34:00.435759+0000","last_clean":"2024-04-20T10:34:00.435759+0000","last_became_active":"2024-04-20T10:31:34.611671+0000","last_became_peered":"2024-04-20T10:31:34.611671+0000","last_unstale":"2024-04-20T10:34:00.435759+0000","last_undegraded":"2024-04-20T10:34:00.435759+0000","last_fullsized":"2024-04-20T10:34:00.435759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:47.862702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456602+0000","last_change":"2024-04-20T10:31:35.643930+0000","last_active":"2024-04-20T10:34:00.456602+0000","last_peered":"2024-04-20T10:34:00.456602+0000","last_clean":"2024-04-20T10:34:00.456602+0000","last_became_active":"2024-04-20T10:31:35.643470+0000","last_became_peered":"2024-04-20T10:31:35.643470+0000","last_unstale":"2024-04-20T10:34:00.456602+0000","last_undegraded":"2024-04-20T10:34:00.456602+0000","last_fullsized":"2024-04-20T10:34:00.456602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:55.570176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307827+0000","last_change":"2024-04-20T10:31:34.613224+0000","last_active":"2024-04-20T10:34:00.307827+0000","last_peered":"2024-04-20T10:34:00.307827+0000","last_clean":"2024-04-20T10:34:00.307827+0000","last_became_active":"2024-04-20T10:31:34.612871+0000","last_became_peered":"2024-04-20T10:31:34.612871+0000","last_unstale":"2024-04-20T10:34:00.307827+0000","last_undegraded":"2024-04-20T10:34:00.307827+0000","last_fullsized":"2024-04-20T10:34:00.307827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:05:57.284870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.438784+0000","last_change":"2024-04-20T10:31:35.643824+0000","last_active":"2024-04-20T10:34:00.438784+0000","last_peered":"2024-04-20T10:34:00.438784+0000","last_clean":"2024-04-20T10:34:00.438784+0000","last_became_active":"2024-04-20T10:31:35.643425+0000","last_became_peered":"2024-04-20T10:31:35.643425+0000","last_unstale":"2024-04-20T10:34:00.438784+0000","last_undegraded":"2024-04-20T10:34:00.438784+0000","last_fullsized":"2024-04-20T10:34:00.438784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:34.008374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456641+0000","last_change":"2024-04-20T10:31:34.593071+0000","last_active":"2024-04-20T10:34:00.456641+0000","last_peered":"2024-04-20T10:34:00.456641+0000","last_clean":"2024-04-20T10:34:00.456641+0000","last_became_active":"2024-04-20T10:31:34.592721+0000","last_became_peered":"2024-04-20T10:31:34.592721+0000","last_unstale":"2024-04-20T10:34:00.456641+0000","last_undegraded":"2024-04-20T10:34:00.456641+0000","last_fullsized":"2024-04-20T10:34:00.456641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:23:43.749586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.310079+0000","last_change":"2024-04-20T10:31:35.647524+0000","last_active":"2024-04-20T10:34:00.310079+0000","last_peered":"2024-04-20T10:34:00.310079+0000","last_clean":"2024-04-20T10:34:00.310079+0000","last_became_active":"2024-04-20T10:31:35.647122+0000","last_became_peered":"2024-04-20T10:31:35.647122+0000","last_unstale":"2024-04-20T10:34:00.310079+0000","last_undegraded":"2024-04-20T10:34:00.310079+0000","last_fullsized":"2024-04-20T10:34:00.310079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.436118+0000","last_change":"2024-04-20T10:31:34.604500+0000","last_active":"2024-04-20T10:34:00.436118+0000","last_peered":"2024-04-20T10:34:00.436118+0000","last_clean":"2024-04-20T10:34:00.436118+0000","last_became_active":"2024-04-20T10:31:34.604148+0000","last_became_peered":"2024-04-20T10:31:34.604148+0000","last_unstale":"2024-04-20T10:34:00.436118+0000","last_undegraded":"2024-04-20T10:34:00.436118+0000","last_fullsized":"2024-04-20T10:34:00.436118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:49:15.174414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.652037+0000","last_change":"2024-04-20T10:31:35.636751+0000","last_active":"2024-04-20T10:34:00.652037+0000","last_peered":"2024-04-20T10:34:00.652037+0000","last_clean":"2024-04-20T10:34:00.652037+0000","last_became_active":"2024-04-20T10:31:35.636103+0000","last_became_peered":"2024-04-20T10:31:35.636103+0000","last_unstale":"2024-04-20T10:34:00.652037+0000","last_undegraded":"2024-04-20T10:34:00.652037+0000","last_fullsized":"2024-04-20T10:34:00.652037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:22:40.580512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.273088+0000","last_change":"2024-04-20T10:31:34.595207+0000","last_active":"2024-04-20T10:34:01.273088+0000","last_peered":"2024-04-20T10:34:01.273088+0000","last_clean":"2024-04-20T10:34:01.273088+0000","last_became_active":"2024-04-20T10:31:34.594860+0000","last_became_peered":"2024-04-20T10:31:34.594860+0000","last_unstale":"2024-04-20T10:34:01.273088+0000","last_undegraded":"2024-04-20T10:34:01.273088+0000","last_fullsized":"2024-04-20T10:34:01.273088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:50.054454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.314770+0000","last_change":"2024-04-20T10:31:35.648086+0000","last_active":"2024-04-20T10:34:00.314770+0000","last_peered":"2024-04-20T10:34:00.314770+0000","last_clean":"2024-04-20T10:34:00.314770+0000","last_became_active":"2024-04-20T10:31:35.647275+0000","last_became_peered":"2024-04-20T10:31:35.647275+0000","last_unstale":"2024-04-20T10:34:00.314770+0000","last_undegraded":"2024-04-20T10:34:00.314770+0000","last_fullsized":"2024-04-20T10:34:00.314770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:27.669008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.457023+0000","last_change":"2024-04-20T10:31:34.601674+0000","last_active":"2024-04-20T10:34:00.457023+0000","last_peered":"2024-04-20T10:34:00.457023+0000","last_clean":"2024-04-20T10:34:00.457023+0000","last_became_active":"2024-04-20T10:31:34.601255+0000","last_became_peered":"2024-04-20T10:31:34.601255+0000","last_unstale":"2024-04-20T10:34:00.457023+0000","last_undegraded":"2024-04-20T10:34:00.457023+0000","last_fullsized":"2024-04-20T10:34:00.457023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:57.821635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.651600+0000","last_change":"2024-04-20T10:31:35.621888+0000","last_active":"2024-04-20T10:34:00.651600+0000","last_peered":"2024-04-20T10:34:00.651600+0000","last_clean":"2024-04-20T10:34:00.651600+0000","last_became_active":"2024-04-20T10:31:35.621503+0000","last_became_peered":"2024-04-20T10:31:35.621503+0000","last_unstale":"2024-04-20T10:34:00.651600+0000","last_undegraded":"2024-04-20T10:34:00.651600+0000","last_fullsized":"2024-04-20T10:34:00.651600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:15:21.118638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.185784+0000","last_change":"2024-04-20T10:31:34.590558+0000","last_active":"2024-04-20T10:34:01.185784+0000","last_peered":"2024-04-20T10:34:01.185784+0000","last_clean":"2024-04-20T10:34:01.185784+0000","last_became_active":"2024-04-20T10:31:34.590140+0000","last_became_peered":"2024-04-20T10:31:34.590140+0000","last_unstale":"2024-04-20T10:34:01.185784+0000","last_undegraded":"2024-04-20T10:34:01.185784+0000","last_fullsized":"2024-04-20T10:34:01.185784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:15:18.591748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67760,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":297,"num_read_kb":215,"num_write":34,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446214+0000","last_change":"2024-04-20T10:31:35.620780+0000","last_active":"2024-04-20T10:34:00.446214+0000","last_peered":"2024-04-20T10:34:00.446214+0000","last_clean":"2024-04-20T10:34:00.446214+0000","last_became_active":"2024-04-20T10:31:35.620429+0000","last_became_peered":"2024-04-20T10:31:35.620429+0000","last_unstale":"2024-04-20T10:34:00.446214+0000","last_undegraded":"2024-04-20T10:34:00.446214+0000","last_fullsized":"2024-04-20T10:34:00.446214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:47:45.587920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650597+0000","last_change":"2024-04-20T10:31:34.602135+0000","last_active":"2024-04-20T10:34:00.650597+0000","last_peered":"2024-04-20T10:34:00.650597+0000","last_clean":"2024-04-20T10:34:00.650597+0000","last_became_active":"2024-04-20T10:31:34.601460+0000","last_became_peered":"2024-04-20T10:31:34.601460+0000","last_unstale":"2024-04-20T10:34:00.650597+0000","last_undegraded":"2024-04-20T10:34:00.650597+0000","last_fullsized":"2024-04-20T10:34:00.650597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:00:00.832702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.308031+0000","last_change":"2024-04-20T10:31:35.635526+0000","last_active":"2024-04-20T10:34:00.308031+0000","last_peered":"2024-04-20T10:34:00.308031+0000","last_clean":"2024-04-20T10:34:00.308031+0000","last_became_active":"2024-04-20T10:31:35.635143+0000","last_became_peered":"2024-04-20T10:31:35.635143+0000","last_unstale":"2024-04-20T10:34:00.308031+0000","last_undegraded":"2024-04-20T10:34:00.308031+0000","last_fullsized":"2024-04-20T10:34:00.308031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:57:24.509318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456493+0000","last_change":"2024-04-20T10:31:34.604561+0000","last_active":"2024-04-20T10:34:00.456493+0000","last_peered":"2024-04-20T10:34:00.456493+0000","last_clean":"2024-04-20T10:34:00.456493+0000","last_became_active":"2024-04-20T10:31:34.604232+0000","last_became_peered":"2024-04-20T10:31:34.604232+0000","last_unstale":"2024-04-20T10:34:00.456493+0000","last_undegraded":"2024-04-20T10:34:00.456493+0000","last_fullsized":"2024-04-20T10:34:00.456493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:39.317473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272869+0000","last_change":"2024-04-20T10:31:35.632840+0000","last_active":"2024-04-20T10:34:01.272869+0000","last_peered":"2024-04-20T10:34:01.272869+0000","last_clean":"2024-04-20T10:34:01.272869+0000","last_became_active":"2024-04-20T10:31:35.632471+0000","last_became_peered":"2024-04-20T10:31:35.632471+0000","last_unstale":"2024-04-20T10:34:01.272869+0000","last_undegraded":"2024-04-20T10:34:01.272869+0000","last_fullsized":"2024-04-20T10:34:01.272869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:06.679967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272715+0000","last_change":"2024-04-20T10:31:34.591696+0000","last_active":"2024-04-20T10:34:01.272715+0000","last_peered":"2024-04-20T10:34:01.272715+0000","last_clean":"2024-04-20T10:34:01.272715+0000","last_became_active":"2024-04-20T10:31:34.591330+0000","last_became_peered":"2024-04-20T10:31:34.591330+0000","last_unstale":"2024-04-20T10:34:01.272715+0000","last_undegraded":"2024-04-20T10:34:01.272715+0000","last_fullsized":"2024-04-20T10:34:01.272715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:44:36.385963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272119+0000","last_change":"2024-04-20T10:31:35.627201+0000","last_active":"2024-04-20T10:34:01.272119+0000","last_peered":"2024-04-20T10:34:01.272119+0000","last_clean":"2024-04-20T10:34:01.272119+0000","last_became_active":"2024-04-20T10:31:35.626880+0000","last_became_peered":"2024-04-20T10:31:35.626880+0000","last_unstale":"2024-04-20T10:34:01.272119+0000","last_undegraded":"2024-04-20T10:34:01.272119+0000","last_fullsized":"2024-04-20T10:34:01.272119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:44.562723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.445834+0000","last_change":"2024-04-20T10:31:34.600683+0000","last_active":"2024-04-20T10:34:00.445834+0000","last_peered":"2024-04-20T10:34:00.445834+0000","last_clean":"2024-04-20T10:34:00.445834+0000","last_became_active":"2024-04-20T10:31:34.600305+0000","last_became_peered":"2024-04-20T10:31:34.600305+0000","last_unstale":"2024-04-20T10:34:00.445834+0000","last_undegraded":"2024-04-20T10:34:00.445834+0000","last_fullsized":"2024-04-20T10:34:00.445834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:05.441158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456458+0000","last_change":"2024-04-20T10:31:35.619835+0000","last_active":"2024-04-20T10:34:00.456458+0000","last_peered":"2024-04-20T10:34:00.456458+0000","last_clean":"2024-04-20T10:34:00.456458+0000","last_became_active":"2024-04-20T10:31:35.619554+0000","last_became_peered":"2024-04-20T10:31:35.619554+0000","last_unstale":"2024-04-20T10:34:00.456458+0000","last_undegraded":"2024-04-20T10:34:00.456458+0000","last_fullsized":"2024-04-20T10:34:00.456458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:33.161563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446313+0000","last_change":"2024-04-20T10:31:34.591128+0000","last_active":"2024-04-20T10:34:00.446313+0000","last_peered":"2024-04-20T10:34:00.446313+0000","last_clean":"2024-04-20T10:34:00.446313+0000","last_became_active":"2024-04-20T10:31:34.587772+0000","last_became_peered":"2024-04-20T10:31:34.587772+0000","last_unstale":"2024-04-20T10:34:00.446313+0000","last_undegraded":"2024-04-20T10:34:00.446313+0000","last_fullsized":"2024-04-20T10:34:00.446313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:12:42.541111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279821+0000","last_change":"2024-04-20T10:31:35.616743+0000","last_active":"2024-04-20T10:34:01.279821+0000","last_peered":"2024-04-20T10:34:01.279821+0000","last_clean":"2024-04-20T10:34:01.279821+0000","last_became_active":"2024-04-20T10:31:35.616419+0000","last_became_peered":"2024-04-20T10:31:35.616419+0000","last_unstale":"2024-04-20T10:34:01.279821+0000","last_undegraded":"2024-04-20T10:34:01.279821+0000","last_fullsized":"2024-04-20T10:34:01.279821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:37.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.310071+0000","last_change":"2024-04-20T10:31:34.603734+0000","last_active":"2024-04-20T10:34:00.310071+0000","last_peered":"2024-04-20T10:34:00.310071+0000","last_clean":"2024-04-20T10:34:00.310071+0000","last_became_active":"2024-04-20T10:31:34.603388+0000","last_became_peered":"2024-04-20T10:31:34.603388+0000","last_unstale":"2024-04-20T10:34:00.310071+0000","last_undegraded":"2024-04-20T10:34:00.310071+0000","last_fullsized":"2024-04-20T10:34:00.310071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:58:57.199173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.436119+0000","last_change":"2024-04-20T10:31:35.638996+0000","last_active":"2024-04-20T10:34:00.436119+0000","last_peered":"2024-04-20T10:34:00.436119+0000","last_clean":"2024-04-20T10:34:00.436119+0000","last_became_active":"2024-04-20T10:31:35.638571+0000","last_became_peered":"2024-04-20T10:31:35.638571+0000","last_unstale":"2024-04-20T10:34:00.436119+0000","last_undegraded":"2024-04-20T10:34:00.436119+0000","last_fullsized":"2024-04-20T10:34:00.436119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:59.541738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.273244+0000","last_change":"2024-04-20T10:31:34.612501+0000","last_active":"2024-04-20T10:34:01.273244+0000","last_peered":"2024-04-20T10:34:01.273244+0000","last_clean":"2024-04-20T10:34:01.273244+0000","last_became_active":"2024-04-20T10:31:34.612150+0000","last_became_peered":"2024-04-20T10:31:34.612150+0000","last_unstale":"2024-04-20T10:34:01.273244+0000","last_undegraded":"2024-04-20T10:34:01.273244+0000","last_fullsized":"2024-04-20T10:34:01.273244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:28.172851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.435952+0000","last_change":"2024-04-20T10:32:31.957767+0000","last_active":"2024-04-20T10:34:00.435952+0000","last_peered":"2024-04-20T10:34:00.435952+0000","last_clean":"2024-04-20T10:34:00.435952+0000","last_became_active":"2024-04-20T10:32:31.957319+0000","last_became_peered":"2024-04-20T10:32:31.957319+0000","last_unstale":"2024-04-20T10:34:00.435952+0000","last_undegraded":"2024-04-20T10:34:00.435952+0000","last_fullsized":"2024-04-20T10:34:00.435952+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:56.375705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456794+0000","last_change":"2024-04-20T10:31:35.617775+0000","last_active":"2024-04-20T10:34:00.456794+0000","last_peered":"2024-04-20T10:34:00.456794+0000","last_clean":"2024-04-20T10:34:00.456794+0000","last_became_active":"2024-04-20T10:31:35.617465+0000","last_became_peered":"2024-04-20T10:31:35.617465+0000","last_unstale":"2024-04-20T10:34:00.456794+0000","last_undegraded":"2024-04-20T10:34:00.456794+0000","last_fullsized":"2024-04-20T10:34:00.456794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:37.830953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307653+0000","last_change":"2024-04-20T10:31:34.615044+0000","last_active":"2024-04-20T10:34:00.307653+0000","last_peered":"2024-04-20T10:34:00.307653+0000","last_clean":"2024-04-20T10:34:00.307653+0000","last_became_active":"2024-04-20T10:31:34.612875+0000","last_became_peered":"2024-04-20T10:31:34.612875+0000","last_unstale":"2024-04-20T10:34:00.307653+0000","last_undegraded":"2024-04-20T10:34:00.307653+0000","last_fullsized":"2024-04-20T10:34:00.307653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:59.867501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439537+0000","last_change":"2024-04-20T10:31:35.630909+0000","last_active":"2024-04-20T10:34:00.439537+0000","last_peered":"2024-04-20T10:34:00.439537+0000","last_clean":"2024-04-20T10:34:00.439537+0000","last_became_active":"2024-04-20T10:31:35.630455+0000","last_became_peered":"2024-04-20T10:31:35.630455+0000","last_unstale":"2024-04-20T10:34:00.439537+0000","last_undegraded":"2024-04-20T10:34:00.439537+0000","last_fullsized":"2024-04-20T10:34:00.439537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:02.522485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272616+0000","last_change":"2024-04-20T10:31:34.602466+0000","last_active":"2024-04-20T10:34:01.272616+0000","last_peered":"2024-04-20T10:34:01.272616+0000","last_clean":"2024-04-20T10:34:01.272616+0000","last_became_active":"2024-04-20T10:31:34.602089+0000","last_became_peered":"2024-04-20T10:31:34.602089+0000","last_unstale":"2024-04-20T10:34:01.272616+0000","last_undegraded":"2024-04-20T10:34:01.272616+0000","last_fullsized":"2024-04-20T10:34:01.272616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:19:09.849840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272126+0000","last_change":"2024-04-20T10:30:14.724347+0000","last_active":"2024-04-20T10:34:01.272126+0000","last_peered":"2024-04-20T10:34:01.272126+0000","last_clean":"2024-04-20T10:34:01.272126+0000","last_became_active":"2024-04-20T10:30:14.723877+0000","last_became_peered":"2024-04-20T10:30:14.723877+0000","last_unstale":"2024-04-20T10:34:01.272126+0000","last_undegraded":"2024-04-20T10:34:01.272126+0000","last_fullsized":"2024-04-20T10:34:01.272126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_clean_scrub_stamp":"2024-04-20T10:26:16.803986+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:30.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307565+0000","last_change":"2024-04-20T10:31:35.626505+0000","last_active":"2024-04-20T10:34:00.307565+0000","last_peered":"2024-04-20T10:34:00.307565+0000","last_clean":"2024-04-20T10:34:00.307565+0000","last_became_active":"2024-04-20T10:31:35.626141+0000","last_became_peered":"2024-04-20T10:31:35.626141+0000","last_unstale":"2024-04-20T10:34:00.307565+0000","last_undegraded":"2024-04-20T10:34:00.307565+0000","last_fullsized":"2024-04-20T10:34:00.307565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:29.827225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272242+0000","last_change":"2024-04-20T10:31:34.596936+0000","last_active":"2024-04-20T10:34:01.272242+0000","last_peered":"2024-04-20T10:34:01.272242+0000","last_clean":"2024-04-20T10:34:01.272242+0000","last_became_active":"2024-04-20T10:31:34.596692+0000","last_became_peered":"2024-04-20T10:31:34.596692+0000","last_unstale":"2024-04-20T10:34:01.272242+0000","last_undegraded":"2024-04-20T10:34:01.272242+0000","last_fullsized":"2024-04-20T10:34:01.272242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:02:39.819096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446098+0000","last_change":"2024-04-20T10:31:34.601407+0000","last_active":"2024-04-20T10:34:00.446098+0000","last_peered":"2024-04-20T10:34:00.446098+0000","last_clean":"2024-04-20T10:34:00.446098+0000","last_became_active":"2024-04-20T10:31:34.601007+0000","last_became_peered":"2024-04-20T10:31:34.601007+0000","last_unstale":"2024-04-20T10:34:00.446098+0000","last_undegraded":"2024-04-20T10:34:00.446098+0000","last_fullsized":"2024-04-20T10:34:00.446098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:05:03.037841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.308057+0000","last_change":"2024-04-20T10:31:35.641046+0000","last_active":"2024-04-20T10:34:00.308057+0000","last_peered":"2024-04-20T10:34:00.308057+0000","last_clean":"2024-04-20T10:34:00.308057+0000","last_became_active":"2024-04-20T10:31:35.640666+0000","last_became_peered":"2024-04-20T10:31:35.640666+0000","last_unstale":"2024-04-20T10:34:00.308057+0000","last_undegraded":"2024-04-20T10:34:00.308057+0000","last_fullsized":"2024-04-20T10:34:00.308057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:42.496037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'45","reported_seq":90,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.606523+0000","last_change":"2024-04-20T10:31:34.604643+0000","last_active":"2024-04-20T10:34:00.606523+0000","last_peered":"2024-04-20T10:34:00.606523+0000","last_clean":"2024-04-20T10:34:00.606523+0000","last_became_active":"2024-04-20T10:31:34.604210+0000","last_became_peered":"2024-04-20T10:31:34.604210+0000","last_unstale":"2024-04-20T10:34:00.606523+0000","last_undegraded":"2024-04-20T10:34:00.606523+0000","last_fullsized":"2024-04-20T10:34:00.606523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:24:09.748012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":81,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439226+0000","last_change":"2024-04-20T10:31:35.641981+0000","last_active":"2024-04-20T10:34:00.439226+0000","last_peered":"2024-04-20T10:34:00.439226+0000","last_clean":"2024-04-20T10:34:00.439226+0000","last_became_active":"2024-04-20T10:31:35.641607+0000","last_became_peered":"2024-04-20T10:31:35.641607+0000","last_unstale":"2024-04-20T10:34:00.439226+0000","last_undegraded":"2024-04-20T10:34:00.439226+0000","last_fullsized":"2024-04-20T10:34:00.439226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:48.520813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272757+0000","last_change":"2024-04-20T10:31:34.611218+0000","last_active":"2024-04-20T10:34:01.272757+0000","last_peered":"2024-04-20T10:34:01.272757+0000","last_clean":"2024-04-20T10:34:01.272757+0000","last_became_active":"2024-04-20T10:31:34.610950+0000","last_became_peered":"2024-04-20T10:31:34.610950+0000","last_unstale":"2024-04-20T10:34:01.272757+0000","last_undegraded":"2024-04-20T10:34:01.272757+0000","last_fullsized":"2024-04-20T10:34:01.272757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:14:11.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313611+0000","last_change":"2024-04-20T10:31:35.647531+0000","last_active":"2024-04-20T10:34:00.313611+0000","last_peered":"2024-04-20T10:34:00.313611+0000","last_clean":"2024-04-20T10:34:00.313611+0000","last_became_active":"2024-04-20T10:31:35.646389+0000","last_became_peered":"2024-04-20T10:31:35.646389+0000","last_unstale":"2024-04-20T10:34:00.313611+0000","last_undegraded":"2024-04-20T10:34:00.313611+0000","last_fullsized":"2024-04-20T10:34:00.313611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:50.141385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.651038+0000","last_change":"2024-04-20T10:32:31.957675+0000","last_active":"2024-04-20T10:34:00.651038+0000","last_peered":"2024-04-20T10:34:00.651038+0000","last_clean":"2024-04-20T10:34:00.651038+0000","last_became_active":"2024-04-20T10:32:31.957202+0000","last_became_peered":"2024-04-20T10:32:31.957202+0000","last_unstale":"2024-04-20T10:34:00.651038+0000","last_undegraded":"2024-04-20T10:34:00.651038+0000","last_fullsized":"2024-04-20T10:34:00.651038+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:28.001088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.445870+0000","last_change":"2024-04-20T10:31:35.616751+0000","last_active":"2024-04-20T10:34:00.445870+0000","last_peered":"2024-04-20T10:34:00.445870+0000","last_clean":"2024-04-20T10:34:00.445870+0000","last_became_active":"2024-04-20T10:31:35.616432+0000","last_became_peered":"2024-04-20T10:31:35.616432+0000","last_unstale":"2024-04-20T10:34:00.445870+0000","last_undegraded":"2024-04-20T10:34:00.445870+0000","last_fullsized":"2024-04-20T10:34:00.445870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:25.052140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439413+0000","last_change":"2024-04-20T10:31:34.609139+0000","last_active":"2024-04-20T10:34:00.439413+0000","last_peered":"2024-04-20T10:34:00.439413+0000","last_clean":"2024-04-20T10:34:00.439413+0000","last_became_active":"2024-04-20T10:31:34.608818+0000","last_became_peered":"2024-04-20T10:31:34.608818+0000","last_unstale":"2024-04-20T10:34:00.439413+0000","last_undegraded":"2024-04-20T10:34:00.439413+0000","last_fullsized":"2024-04-20T10:34:00.439413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:42:39.581059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279576+0000","last_change":"2024-04-20T10:31:35.643791+0000","last_active":"2024-04-20T10:34:01.279576+0000","last_peered":"2024-04-20T10:34:01.279576+0000","last_clean":"2024-04-20T10:34:01.279576+0000","last_became_active":"2024-04-20T10:31:35.643390+0000","last_became_peered":"2024-04-20T10:31:35.643390+0000","last_unstale":"2024-04-20T10:34:01.279576+0000","last_undegraded":"2024-04-20T10:34:01.279576+0000","last_fullsized":"2024-04-20T10:34:01.279576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:10.149979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650525+0000","last_change":"2024-04-20T10:31:34.606178+0000","last_active":"2024-04-20T10:34:00.650525+0000","last_peered":"2024-04-20T10:34:00.650525+0000","last_clean":"2024-04-20T10:34:00.650525+0000","last_became_active":"2024-04-20T10:31:34.605817+0000","last_became_peered":"2024-04-20T10:31:34.605817+0000","last_unstale":"2024-04-20T10:34:00.650525+0000","last_undegraded":"2024-04-20T10:34:00.650525+0000","last_fullsized":"2024-04-20T10:34:00.650525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:15.929905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439824+0000","last_change":"2024-04-20T10:31:35.649487+0000","last_active":"2024-04-20T10:34:00.439824+0000","last_peered":"2024-04-20T10:34:00.439824+0000","last_clean":"2024-04-20T10:34:00.439824+0000","last_became_active":"2024-04-20T10:31:35.648884+0000","last_became_peered":"2024-04-20T10:31:35.648884+0000","last_unstale":"2024-04-20T10:34:00.439824+0000","last_undegraded":"2024-04-20T10:34:00.439824+0000","last_fullsized":"2024-04-20T10:34:00.439824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:07.389100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279544+0000","last_change":"2024-04-20T10:32:31.943005+0000","last_active":"2024-04-20T10:34:01.279544+0000","last_peered":"2024-04-20T10:34:01.279544+0000","last_clean":"2024-04-20T10:34:01.279544+0000","last_became_active":"2024-04-20T10:32:31.940063+0000","last_became_peered":"2024-04-20T10:32:31.940063+0000","last_unstale":"2024-04-20T10:34:01.279544+0000","last_undegraded":"2024-04-20T10:34:01.279544+0000","last_fullsized":"2024-04-20T10:34:01.279544+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:11:06.308028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272443+0000","last_change":"2024-04-20T10:31:35.630256+0000","last_active":"2024-04-20T10:34:01.272443+0000","last_peered":"2024-04-20T10:34:01.272443+0000","last_clean":"2024-04-20T10:34:01.272443+0000","last_became_active":"2024-04-20T10:31:35.629879+0000","last_became_peered":"2024-04-20T10:31:35.629879+0000","last_unstale":"2024-04-20T10:34:01.272443+0000","last_undegraded":"2024-04-20T10:34:01.272443+0000","last_fullsized":"2024-04-20T10:34:01.272443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:51.601923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650706+0000","last_change":"2024-04-20T10:31:34.595149+0000","last_active":"2024-04-20T10:34:00.650706+0000","last_peered":"2024-04-20T10:34:00.650706+0000","last_clean":"2024-04-20T10:34:00.650706+0000","last_became_active":"2024-04-20T10:31:34.594808+0000","last_became_peered":"2024-04-20T10:31:34.594808+0000","last_unstale":"2024-04-20T10:34:00.650706+0000","last_undegraded":"2024-04-20T10:34:00.650706+0000","last_fullsized":"2024-04-20T10:34:00.650706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:51.707710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272938+0000","last_change":"2024-04-20T10:31:35.620612+0000","last_active":"2024-04-20T10:34:01.272938+0000","last_peered":"2024-04-20T10:34:01.272938+0000","last_clean":"2024-04-20T10:34:01.272938+0000","last_became_active":"2024-04-20T10:31:35.620312+0000","last_became_peered":"2024-04-20T10:31:35.620312+0000","last_unstale":"2024-04-20T10:34:01.272938+0000","last_undegraded":"2024-04-20T10:34:01.272938+0000","last_fullsized":"2024-04-20T10:34:01.272938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:37:50.756950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'39","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.604966+0000","last_change":"2024-04-20T10:31:34.604613+0000","last_active":"2024-04-20T10:34:00.604966+0000","last_peered":"2024-04-20T10:34:00.604966+0000","last_clean":"2024-04-20T10:34:00.604966+0000","last_became_active":"2024-04-20T10:31:34.604246+0000","last_became_peered":"2024-04-20T10:31:34.604246+0000","last_unstale":"2024-04-20T10:34:00.604966+0000","last_undegraded":"2024-04-20T10:34:00.604966+0000","last_fullsized":"2024-04-20T10:34:00.604966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:26.521769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272437+0000","last_change":"2024-04-20T10:31:35.639396+0000","last_active":"2024-04-20T10:34:01.272437+0000","last_peered":"2024-04-20T10:34:01.272437+0000","last_clean":"2024-04-20T10:34:01.272437+0000","last_became_active":"2024-04-20T10:31:35.638979+0000","last_became_peered":"2024-04-20T10:31:35.638979+0000","last_unstale":"2024-04-20T10:34:01.272437+0000","last_undegraded":"2024-04-20T10:34:01.272437+0000","last_fullsized":"2024-04-20T10:34:01.272437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:47.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.308292+0000","last_change":"2024-04-20T10:31:34.607420+0000","last_active":"2024-04-20T10:34:00.308292+0000","last_peered":"2024-04-20T10:34:00.308292+0000","last_clean":"2024-04-20T10:34:00.308292+0000","last_became_active":"2024-04-20T10:31:34.602587+0000","last_became_peered":"2024-04-20T10:31:34.602587+0000","last_unstale":"2024-04-20T10:34:00.308292+0000","last_undegraded":"2024-04-20T10:34:00.308292+0000","last_fullsized":"2024-04-20T10:34:00.308292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:56.447944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.273183+0000","last_change":"2024-04-20T10:31:35.624903+0000","last_active":"2024-04-20T10:34:01.273183+0000","last_peered":"2024-04-20T10:34:01.273183+0000","last_clean":"2024-04-20T10:34:01.273183+0000","last_became_active":"2024-04-20T10:31:35.619479+0000","last_became_peered":"2024-04-20T10:31:35.619479+0000","last_unstale":"2024-04-20T10:34:01.273183+0000","last_undegraded":"2024-04-20T10:34:01.273183+0000","last_fullsized":"2024-04-20T10:34:01.273183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:25:40.601983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279614+0000","last_change":"2024-04-20T10:31:35.620351+0000","last_active":"2024-04-20T10:34:01.279614+0000","last_peered":"2024-04-20T10:34:01.279614+0000","last_clean":"2024-04-20T10:34:01.279614+0000","last_became_active":"2024-04-20T10:31:35.620028+0000","last_became_peered":"2024-04-20T10:31:35.620028+0000","last_unstale":"2024-04-20T10:34:01.279614+0000","last_undegraded":"2024-04-20T10:34:01.279614+0000","last_fullsized":"2024-04-20T10:34:01.279614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:51:53.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650618+0000","last_change":"2024-04-20T10:31:35.615917+0000","last_active":"2024-04-20T10:34:00.650618+0000","last_peered":"2024-04-20T10:34:00.650618+0000","last_clean":"2024-04-20T10:34:00.650618+0000","last_became_active":"2024-04-20T10:31:35.615582+0000","last_became_peered":"2024-04-20T10:31:35.615582+0000","last_unstale":"2024-04-20T10:34:00.650618+0000","last_undegraded":"2024-04-20T10:34:00.650618+0000","last_fullsized":"2024-04-20T10:34:00.650618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:54:59.754819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650460+0000","last_change":"2024-04-20T10:31:35.650006+0000","last_active":"2024-04-20T10:34:00.650460+0000","last_peered":"2024-04-20T10:34:00.650460+0000","last_clean":"2024-04-20T10:34:00.650460+0000","last_became_active":"2024-04-20T10:31:35.649683+0000","last_became_peered":"2024-04-20T10:31:35.649683+0000","last_unstale":"2024-04-20T10:34:00.650460+0000","last_undegraded":"2024-04-20T10:34:00.650460+0000","last_fullsized":"2024-04-20T10:34:00.650460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:10:01.484809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313550+0000","last_change":"2024-04-20T10:31:35.646591+0000","last_active":"2024-04-20T10:34:00.313550+0000","last_peered":"2024-04-20T10:34:00.313550+0000","last_clean":"2024-04-20T10:34:00.313550+0000","last_became_active":"2024-04-20T10:31:35.646188+0000","last_became_peered":"2024-04-20T10:31:35.646188+0000","last_unstale":"2024-04-20T10:34:00.313550+0000","last_undegraded":"2024-04-20T10:34:00.313550+0000","last_fullsized":"2024-04-20T10:34:00.313550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:23.412022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307919+0000","last_change":"2024-04-20T10:31:35.650879+0000","last_active":"2024-04-20T10:34:00.307919+0000","last_peered":"2024-04-20T10:34:00.307919+0000","last_clean":"2024-04-20T10:34:00.307919+0000","last_became_active":"2024-04-20T10:31:35.650486+0000","last_became_peered":"2024-04-20T10:31:35.650486+0000","last_unstale":"2024-04-20T10:34:00.307919+0000","last_undegraded":"2024-04-20T10:34:00.307919+0000","last_fullsized":"2024-04-20T10:34:00.307919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:51.893446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279735+0000","last_change":"2024-04-20T10:31:35.619883+0000","last_active":"2024-04-20T10:34:01.279735+0000","last_peered":"2024-04-20T10:34:01.279735+0000","last_clean":"2024-04-20T10:34:01.279735+0000","last_became_active":"2024-04-20T10:31:35.619628+0000","last_became_peered":"2024-04-20T10:31:35.619628+0000","last_unstale":"2024-04-20T10:34:01.279735+0000","last_undegraded":"2024-04-20T10:34:01.279735+0000","last_fullsized":"2024-04-20T10:34:01.279735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:45.018417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650973+0000","last_change":"2024-04-20T10:31:35.620773+0000","last_active":"2024-04-20T10:34:00.650973+0000","last_peered":"2024-04-20T10:34:00.650973+0000","last_clean":"2024-04-20T10:34:00.650973+0000","last_became_active":"2024-04-20T10:31:35.620467+0000","last_became_peered":"2024-04-20T10:31:35.620467+0000","last_unstale":"2024-04-20T10:34:00.650973+0000","last_undegraded":"2024-04-20T10:34:00.650973+0000","last_fullsized":"2024-04-20T10:34:00.650973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:31:48.009168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439094+0000","last_change":"2024-04-20T10:32:31.953580+0000","last_active":"2024-04-20T10:34:00.439094+0000","last_peered":"2024-04-20T10:34:00.439094+0000","last_clean":"2024-04-20T10:34:00.439094+0000","last_became_active":"2024-04-20T10:32:31.952985+0000","last_became_peered":"2024-04-20T10:32:31.952985+0000","last_unstale":"2024-04-20T10:34:00.439094+0000","last_undegraded":"2024-04-20T10:34:00.439094+0000","last_fullsized":"2024-04-20T10:34:00.439094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:13:03.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456576+0000","last_change":"2024-04-20T10:31:35.637588+0000","last_active":"2024-04-20T10:34:00.456576+0000","last_peered":"2024-04-20T10:34:00.456576+0000","last_clean":"2024-04-20T10:34:00.456576+0000","last_became_active":"2024-04-20T10:31:35.637319+0000","last_became_peered":"2024-04-20T10:31:35.637319+0000","last_unstale":"2024-04-20T10:34:00.456576+0000","last_undegraded":"2024-04-20T10:34:00.456576+0000","last_fullsized":"2024-04-20T10:34:00.456576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:54.070894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439287+0000","last_change":"2024-04-20T10:31:35.629551+0000","last_active":"2024-04-20T10:34:00.439287+0000","last_peered":"2024-04-20T10:34:00.439287+0000","last_clean":"2024-04-20T10:34:00.439287+0000","last_became_active":"2024-04-20T10:31:35.628710+0000","last_became_peered":"2024-04-20T10:31:35.628710+0000","last_unstale":"2024-04-20T10:34:00.439287+0000","last_undegraded":"2024-04-20T10:34:00.439287+0000","last_fullsized":"2024-04-20T10:34:00.439287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:31.883788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.651244+0000","last_change":"2024-04-20T10:31:35.629978+0000","last_active":"2024-04-20T10:34:00.651244+0000","last_peered":"2024-04-20T10:34:00.651244+0000","last_clean":"2024-04-20T10:34:00.651244+0000","last_became_active":"2024-04-20T10:31:35.629596+0000","last_became_peered":"2024-04-20T10:31:35.629596+0000","last_unstale":"2024-04-20T10:34:00.651244+0000","last_undegraded":"2024-04-20T10:34:00.651244+0000","last_fullsized":"2024-04-20T10:34:00.651244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:02.523099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.308137+0000","last_change":"2024-04-20T10:31:35.652481+0000","last_active":"2024-04-20T10:34:00.308137+0000","last_peered":"2024-04-20T10:34:00.308137+0000","last_clean":"2024-04-20T10:34:00.308137+0000","last_became_active":"2024-04-20T10:31:35.651837+0000","last_became_peered":"2024-04-20T10:31:35.651837+0000","last_unstale":"2024-04-20T10:34:00.308137+0000","last_undegraded":"2024-04-20T10:34:00.308137+0000","last_fullsized":"2024-04-20T10:34:00.308137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:53.206954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279636+0000","last_change":"2024-04-20T10:31:35.622228+0000","last_active":"2024-04-20T10:34:01.279636+0000","last_peered":"2024-04-20T10:34:01.279636+0000","last_clean":"2024-04-20T10:34:01.279636+0000","last_became_active":"2024-04-20T10:31:35.621910+0000","last_became_peered":"2024-04-20T10:31:35.621910+0000","last_unstale":"2024-04-20T10:34:01.279636+0000","last_undegraded":"2024-04-20T10:34:01.279636+0000","last_fullsized":"2024-04-20T10:34:01.279636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:47:34.319839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107397,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":593374,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":393,"num_read_kb":298,"num_write":201,"num_write_kb":1405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":1780122,"data_compressed":81447,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":455821,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1110016,"data_stored":1254063,"data_compressed":52532,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":858604,"data_compressed":27876,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":858626,"data_compressed":27876,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332161,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":1046888,"data_compressed":41606,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":464923,"data_compressed":8730,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":532566,"data_compressed":10577,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":464896,"data_compressed":8730,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411422,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":522833,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":989960,"data_compressed":34032,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":455843,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":1187073,"data_compressed":50685,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":67760,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67794,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":524920,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":131782,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":199554,"data_compressed":8003,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:34:04.033 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:34:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:34:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-20T10:34:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1007185576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:34:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/875804641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:34:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:04.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:04.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:03 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:34:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1007185576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/875804641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:03 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.268+0000 7faab7600640 1 -- 172.21.15.42:0/373932848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab01031a0 msgr2=0x7faab0103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:04.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.268+0000 7faab7600640 1 --2- 172.21.15.42:0/373932848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab01031a0 0x7faab0103620 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009900 tx=0x7faa9c02bc30 comp rx=0 tx=0).stop 2024-04-20T10:34:04.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.268+0000 7faab7600640 1 -- 172.21.15.42:0/373932848 shutdown_connections 2024-04-20T10:34:04.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.268+0000 7faab7600640 1 --2- 172.21.15.42:0/373932848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faab0103b60 0x7faab010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.268+0000 7faab7600640 1 --2- 172.21.15.42:0/373932848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab01031a0 0x7faab0103620 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.268+0000 7faab7600640 1 --2- 172.21.15.42:0/373932848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab0101fb0 0x7faab01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.269+0000 7faab7600640 1 -- 172.21.15.42:0/373932848 >> 172.21.15.42:0/373932848 conn(0x7faab00fd540 msgr2=0x7faab00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:04.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.269+0000 7faab7600640 1 -- 172.21.15.42:0/373932848 shutdown_connections 2024-04-20T10:34:04.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.269+0000 7faab7600640 1 -- 172.21.15.42:0/373932848 wait complete. 2024-04-20T10:34:04.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.270+0000 7faab7600640 1 Processor -- start 2024-04-20T10:34:04.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.270+0000 7faab7600640 1 -- start start 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.270+0000 7faab7600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab0101fb0 0x7faab019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.270+0000 7faab7600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faab01031a0 0x7faab019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.270+0000 7faab7600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab0103b60 0x7faab01a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.270+0000 7faab7600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faab010dc90 con 0x7faab0103b60 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.270+0000 7faab7600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faab010db10 con 0x7faab0101fb0 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.270+0000 7faab7600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7faab010de10 con 0x7faab01031a0 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.270+0000 7faab4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab0101fb0 0x7faab019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.270+0000 7faab4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab0101fb0 0x7faab019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52056/0 (socket says 172.21.15.42:52056) 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.270+0000 7faab4c00640 1 -- 172.21.15.42:0/2870629586 learned_addr learned my addr 172.21.15.42:0/2870629586 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.271+0000 7faaafe00640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faab01031a0 0x7faab019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.271+0000 7faab5600640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab0103b60 0x7faab01a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.271+0000 7faab4c00640 1 -- 172.21.15.42:0/2870629586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faab01031a0 msgr2=0x7faab019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.271+0000 7faab4c00640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faab01031a0 0x7faab019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.271+0000 7faab4c00640 1 -- 172.21.15.42:0/2870629586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab0103b60 msgr2=0x7faab01a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:04.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.271+0000 7faab4c00640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab0103b60 0x7faab01a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.271+0000 7faab4c00640 1 -- 172.21.15.42:0/2870629586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab01a20e0 con 0x7faab0101fb0 2024-04-20T10:34:04.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.271+0000 7faab4c00640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab0101fb0 0x7faab019a4c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faaa400e9c0 tx=0x7faaa400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:04.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.272+0000 7faaad600640 1 -- 172.21.15.42:0/2870629586 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa400cde0 con 0x7faab0101fb0 2024-04-20T10:34:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.272+0000 7faaad600640 1 -- 172.21.15.42:0/2870629586 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa4010910 con 0x7faab0101fb0 2024-04-20T10:34:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.272+0000 7faaad600640 1 -- 172.21.15.42:0/2870629586 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4018a00 con 0x7faab0101fb0 2024-04-20T10:34:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.272+0000 7faab7600640 1 -- 172.21.15.42:0/2870629586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab01a23d0 con 0x7faab0101fb0 2024-04-20T10:34:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.272+0000 7faab7600640 1 -- 172.21.15.42:0/2870629586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab01a28a0 con 0x7faab0101fb0 2024-04-20T10:34:04.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.273+0000 7faab7600640 1 -- 172.21.15.42:0/2870629586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab01a2b00 con 0x7faab0101fb0 2024-04-20T10:34:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.276+0000 7faaad600640 1 -- 172.21.15.42:0/2870629586 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7faaa401f020 con 0x7faab0101fb0 2024-04-20T10:34:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.277+0000 7faaad600640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7faa980a3da0 0x7faa980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:04.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.277+0000 7faaad600640 1 -- 172.21.15.42:0/2870629586 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7faaa40d09b0 con 0x7faab0101fb0 2024-04-20T10:34:04.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.278+0000 7faaafe00640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7faa980a3da0 0x7faa980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:04.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.279+0000 7faaafe00640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7faa980a3da0 0x7faa980a6260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faa9c02c240 tx=0x7faa9c02df90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:04.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.281+0000 7faaad600640 1 -- 172.21.15.42:0/2870629586 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7faaa409cff0 con 0x7faab0101fb0 2024-04-20T10:34:04.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.492+0000 7faab7600640 1 -- 172.21.15.42:0/2870629586 --> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faab00758b0 con 0x7faa980a3da0 2024-04-20T10:34:04.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.516+0000 7faaad600640 1 -- 172.21.15.42:0/2870629586 <== mgr.15024 v2:172.21.15.42:6800/170978916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257971 (secure 0 0 0) 0x7faab00758b0 con 0x7faa980a3da0 2024-04-20T10:34:04.520 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:34:04.522 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-04-20T10:34:04.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.521+0000 7faab7600640 1 -- 172.21.15.42:0/2870629586 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7faa980a3da0 msgr2=0x7faa980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:04.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.522+0000 7faab7600640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7faa980a3da0 0x7faa980a6260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faa9c02c240 tx=0x7faa9c02df90 comp rx=0 tx=0).stop 2024-04-20T10:34:04.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.522+0000 7faab7600640 1 -- 172.21.15.42:0/2870629586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab0101fb0 msgr2=0x7faab019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:04.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.522+0000 7faab7600640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab0101fb0 0x7faab019a4c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faaa400e9c0 tx=0x7faaa400ee80 comp rx=0 tx=0).stop 2024-04-20T10:34:04.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.522+0000 7faab7600640 1 -- 172.21.15.42:0/2870629586 shutdown_connections 2024-04-20T10:34:04.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.522+0000 7faab7600640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7faa980a3da0 0x7faa980a6260 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.522+0000 7faab7600640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faab0103b60 0x7faab01a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.522+0000 7faab7600640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faab01031a0 0x7faab019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.522+0000 7faab7600640 1 --2- 172.21.15.42:0/2870629586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faab0101fb0 0x7faab019a4c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.522+0000 7faab7600640 1 -- 172.21.15.42:0/2870629586 >> 172.21.15.42:0/2870629586 conn(0x7faab00fd540 msgr2=0x7faab0074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:04.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.522+0000 7faab7600640 1 -- 172.21.15.42:0/2870629586 shutdown_connections 2024-04-20T10:34:04.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.522+0000 7faab7600640 1 -- 172.21.15.42:0/2870629586 wait complete. 2024-04-20T10:34:04.546 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-20T10:34:04.327766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1183976,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":529,"num_read_kb":417,"num_write":384,"num_write_kb":4345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369180,"kb_used_data":11664,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572644,"statfs":{"total":1151940427776,"available":1151562387456,"internally_reserved":0,"allocated":11943936,"data_stored":9092096,"data_compressed":281912,"data_compressed_allocated":376832,"data_compressed_original":3604480,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.984344"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279951+0000","last_change":"2024-04-20T10:31:35.649585+0000","last_active":"2024-04-20T10:34:01.279951+0000","last_peered":"2024-04-20T10:34:01.279951+0000","last_clean":"2024-04-20T10:34:01.279951+0000","last_became_active":"2024-04-20T10:31:35.649282+0000","last_became_peered":"2024-04-20T10:31:35.649282+0000","last_unstale":"2024-04-20T10:34:01.279951+0000","last_undegraded":"2024-04-20T10:34:01.279951+0000","last_fullsized":"2024-04-20T10:34:01.279951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:01:00.517383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313417+0000","last_change":"2024-04-20T10:31:35.648299+0000","last_active":"2024-04-20T10:34:00.313417+0000","last_peered":"2024-04-20T10:34:00.313417+0000","last_clean":"2024-04-20T10:34:00.313417+0000","last_became_active":"2024-04-20T10:31:35.647283+0000","last_became_peered":"2024-04-20T10:31:35.647283+0000","last_unstale":"2024-04-20T10:34:00.313417+0000","last_undegraded":"2024-04-20T10:34:00.313417+0000","last_fullsized":"2024-04-20T10:34:00.313417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:59.006234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.314500+0000","last_change":"2024-04-20T10:31:35.647029+0000","last_active":"2024-04-20T10:34:00.314500+0000","last_peered":"2024-04-20T10:34:00.314500+0000","last_clean":"2024-04-20T10:34:00.314500+0000","last_became_active":"2024-04-20T10:31:35.646649+0000","last_became_peered":"2024-04-20T10:31:35.646649+0000","last_unstale":"2024-04-20T10:34:00.314500+0000","last_undegraded":"2024-04-20T10:34:00.314500+0000","last_fullsized":"2024-04-20T10:34:00.314500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:03:23.420078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272555+0000","last_change":"2024-04-20T10:31:35.624628+0000","last_active":"2024-04-20T10:34:01.272555+0000","last_peered":"2024-04-20T10:34:01.272555+0000","last_clean":"2024-04-20T10:34:01.272555+0000","last_became_active":"2024-04-20T10:31:35.623607+0000","last_became_peered":"2024-04-20T10:31:35.623607+0000","last_unstale":"2024-04-20T10:34:01.272555+0000","last_undegraded":"2024-04-20T10:34:01.272555+0000","last_fullsized":"2024-04-20T10:34:01.272555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:44:46.768584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.445865+0000","last_change":"2024-04-20T10:31:35.638293+0000","last_active":"2024-04-20T10:34:00.445865+0000","last_peered":"2024-04-20T10:34:00.445865+0000","last_clean":"2024-04-20T10:34:00.445865+0000","last_became_active":"2024-04-20T10:31:35.637967+0000","last_became_peered":"2024-04-20T10:31:35.637967+0000","last_unstale":"2024-04-20T10:34:00.445865+0000","last_undegraded":"2024-04-20T10:34:00.445865+0000","last_fullsized":"2024-04-20T10:34:00.445865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:58:00.765015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.435951+0000","last_change":"2024-04-20T10:31:35.634565+0000","last_active":"2024-04-20T10:34:00.435951+0000","last_peered":"2024-04-20T10:34:00.435951+0000","last_clean":"2024-04-20T10:34:00.435951+0000","last_became_active":"2024-04-20T10:31:35.634200+0000","last_became_peered":"2024-04-20T10:31:35.634200+0000","last_unstale":"2024-04-20T10:34:00.435951+0000","last_undegraded":"2024-04-20T10:34:00.435951+0000","last_fullsized":"2024-04-20T10:34:00.435951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:14.298732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272189+0000","last_change":"2024-04-20T10:32:31.949166+0000","last_active":"2024-04-20T10:34:01.272189+0000","last_peered":"2024-04-20T10:34:01.272189+0000","last_clean":"2024-04-20T10:34:01.272189+0000","last_became_active":"2024-04-20T10:32:31.948825+0000","last_became_peered":"2024-04-20T10:32:31.948825+0000","last_unstale":"2024-04-20T10:34:01.272189+0000","last_undegraded":"2024-04-20T10:34:01.272189+0000","last_fullsized":"2024-04-20T10:34:01.272189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:05.337941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.438962+0000","last_change":"2024-04-20T10:31:35.631996+0000","last_active":"2024-04-20T10:34:00.438962+0000","last_peered":"2024-04-20T10:34:00.438962+0000","last_clean":"2024-04-20T10:34:00.438962+0000","last_became_active":"2024-04-20T10:31:35.631576+0000","last_became_peered":"2024-04-20T10:31:35.631576+0000","last_unstale":"2024-04-20T10:34:00.438962+0000","last_undegraded":"2024-04-20T10:34:00.438962+0000","last_fullsized":"2024-04-20T10:34:00.438962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:51.448025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.652006+0000","last_change":"2024-04-20T10:31:35.624430+0000","last_active":"2024-04-20T10:34:00.652006+0000","last_peered":"2024-04-20T10:34:00.652006+0000","last_clean":"2024-04-20T10:34:00.652006+0000","last_became_active":"2024-04-20T10:31:35.623018+0000","last_became_peered":"2024-04-20T10:31:35.623018+0000","last_unstale":"2024-04-20T10:34:00.652006+0000","last_undegraded":"2024-04-20T10:34:00.652006+0000","last_fullsized":"2024-04-20T10:34:00.652006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:43.100925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.436193+0000","last_change":"2024-04-20T10:31:35.619038+0000","last_active":"2024-04-20T10:34:00.436193+0000","last_peered":"2024-04-20T10:34:00.436193+0000","last_clean":"2024-04-20T10:34:00.436193+0000","last_became_active":"2024-04-20T10:31:35.618699+0000","last_became_peered":"2024-04-20T10:31:35.618699+0000","last_unstale":"2024-04-20T10:34:00.436193+0000","last_undegraded":"2024-04-20T10:34:00.436193+0000","last_fullsized":"2024-04-20T10:34:00.436193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:27.205634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272793+0000","last_change":"2024-04-20T10:31:35.619908+0000","last_active":"2024-04-20T10:34:01.272793+0000","last_peered":"2024-04-20T10:34:01.272793+0000","last_clean":"2024-04-20T10:34:01.272793+0000","last_became_active":"2024-04-20T10:31:35.619580+0000","last_became_peered":"2024-04-20T10:31:35.619580+0000","last_unstale":"2024-04-20T10:34:01.272793+0000","last_undegraded":"2024-04-20T10:34:01.272793+0000","last_fullsized":"2024-04-20T10:34:01.272793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:54:48.999925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.309895+0000","last_change":"2024-04-20T10:32:31.952701+0000","last_active":"2024-04-20T10:34:00.309895+0000","last_peered":"2024-04-20T10:34:00.309895+0000","last_clean":"2024-04-20T10:34:00.309895+0000","last_became_active":"2024-04-20T10:32:31.951329+0000","last_became_peered":"2024-04-20T10:32:31.951329+0000","last_unstale":"2024-04-20T10:34:00.309895+0000","last_undegraded":"2024-04-20T10:34:00.309895+0000","last_fullsized":"2024-04-20T10:34:00.309895+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:29.442878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307970+0000","last_change":"2024-04-20T10:31:35.619594+0000","last_active":"2024-04-20T10:34:00.307970+0000","last_peered":"2024-04-20T10:34:00.307970+0000","last_clean":"2024-04-20T10:34:00.307970+0000","last_became_active":"2024-04-20T10:31:35.618831+0000","last_became_peered":"2024-04-20T10:31:35.618831+0000","last_unstale":"2024-04-20T10:34:00.307970+0000","last_undegraded":"2024-04-20T10:34:00.307970+0000","last_fullsized":"2024-04-20T10:34:00.307970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:07.625225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313280+0000","last_change":"2024-04-20T10:31:35.647476+0000","last_active":"2024-04-20T10:34:00.313280+0000","last_peered":"2024-04-20T10:34:00.313280+0000","last_clean":"2024-04-20T10:34:00.313280+0000","last_became_active":"2024-04-20T10:31:35.647009+0000","last_became_peered":"2024-04-20T10:31:35.647009+0000","last_unstale":"2024-04-20T10:34:00.313280+0000","last_undegraded":"2024-04-20T10:34:00.313280+0000","last_fullsized":"2024-04-20T10:34:00.313280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:11:37.526115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313153+0000","last_change":"2024-04-20T10:32:31.958321+0000","last_active":"2024-04-20T10:34:00.313153+0000","last_peered":"2024-04-20T10:34:00.313153+0000","last_clean":"2024-04-20T10:34:00.313153+0000","last_became_active":"2024-04-20T10:32:31.957681+0000","last_became_peered":"2024-04-20T10:32:31.957681+0000","last_unstale":"2024-04-20T10:34:00.313153+0000","last_undegraded":"2024-04-20T10:34:00.313153+0000","last_fullsized":"2024-04-20T10:34:00.313153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:49:31.154909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650987+0000","last_change":"2024-04-20T10:31:35.643142+0000","last_active":"2024-04-20T10:34:00.650987+0000","last_peered":"2024-04-20T10:34:00.650987+0000","last_clean":"2024-04-20T10:34:00.650987+0000","last_became_active":"2024-04-20T10:31:35.642742+0000","last_became_peered":"2024-04-20T10:31:35.642742+0000","last_unstale":"2024-04-20T10:34:00.650987+0000","last_undegraded":"2024-04-20T10:34:00.650987+0000","last_fullsized":"2024-04-20T10:34:00.650987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:40.818808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.280039+0000","last_change":"2024-04-20T10:31:35.650333+0000","last_active":"2024-04-20T10:34:01.280039+0000","last_peered":"2024-04-20T10:34:01.280039+0000","last_clean":"2024-04-20T10:34:01.280039+0000","last_became_active":"2024-04-20T10:31:35.649933+0000","last_became_peered":"2024-04-20T10:31:35.649933+0000","last_unstale":"2024-04-20T10:34:01.280039+0000","last_undegraded":"2024-04-20T10:34:01.280039+0000","last_fullsized":"2024-04-20T10:34:01.280039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:34.513990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.315466+0000","last_change":"2024-04-20T10:31:35.637315+0000","last_active":"2024-04-20T10:34:00.315466+0000","last_peered":"2024-04-20T10:34:00.315466+0000","last_clean":"2024-04-20T10:34:00.315466+0000","last_became_active":"2024-04-20T10:31:35.635909+0000","last_became_peered":"2024-04-20T10:31:35.635909+0000","last_unstale":"2024-04-20T10:34:00.315466+0000","last_undegraded":"2024-04-20T10:34:00.315466+0000","last_fullsized":"2024-04-20T10:34:00.315466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:24:07.400032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650759+0000","last_change":"2024-04-20T10:31:35.638888+0000","last_active":"2024-04-20T10:34:00.650759+0000","last_peered":"2024-04-20T10:34:00.650759+0000","last_clean":"2024-04-20T10:34:00.650759+0000","last_became_active":"2024-04-20T10:31:35.638568+0000","last_became_peered":"2024-04-20T10:31:35.638568+0000","last_unstale":"2024-04-20T10:34:00.650759+0000","last_undegraded":"2024-04-20T10:34:00.650759+0000","last_fullsized":"2024-04-20T10:34:00.650759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:48.363874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.315517+0000","last_change":"2024-04-20T10:31:34.597814+0000","last_active":"2024-04-20T10:34:00.315517+0000","last_peered":"2024-04-20T10:34:00.315517+0000","last_clean":"2024-04-20T10:34:00.315517+0000","last_became_active":"2024-04-20T10:31:34.597480+0000","last_became_peered":"2024-04-20T10:31:34.597480+0000","last_unstale":"2024-04-20T10:34:00.315517+0000","last_undegraded":"2024-04-20T10:34:00.315517+0000","last_fullsized":"2024-04-20T10:34:00.315517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:48:26.102787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.309886+0000","last_change":"2024-04-20T10:32:31.956771+0000","last_active":"2024-04-20T10:34:00.309886+0000","last_peered":"2024-04-20T10:34:00.309886+0000","last_clean":"2024-04-20T10:34:00.309886+0000","last_became_active":"2024-04-20T10:32:31.955413+0000","last_became_peered":"2024-04-20T10:32:31.955413+0000","last_unstale":"2024-04-20T10:34:00.309886+0000","last_undegraded":"2024-04-20T10:34:00.309886+0000","last_fullsized":"2024-04-20T10:34:00.309886+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:37.199764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.438965+0000","last_change":"2024-04-20T10:31:34.609942+0000","last_active":"2024-04-20T10:34:00.438965+0000","last_peered":"2024-04-20T10:34:00.438965+0000","last_clean":"2024-04-20T10:34:00.438965+0000","last_became_active":"2024-04-20T10:31:34.608994+0000","last_became_peered":"2024-04-20T10:31:34.608994+0000","last_unstale":"2024-04-20T10:34:00.438965+0000","last_undegraded":"2024-04-20T10:34:00.438965+0000","last_fullsized":"2024-04-20T10:34:00.438965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:26:53.488091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.314873+0000","last_change":"2024-04-20T10:31:35.647710+0000","last_active":"2024-04-20T10:34:00.314873+0000","last_peered":"2024-04-20T10:34:00.314873+0000","last_clean":"2024-04-20T10:34:00.314873+0000","last_became_active":"2024-04-20T10:31:35.646037+0000","last_became_peered":"2024-04-20T10:31:35.646037+0000","last_unstale":"2024-04-20T10:34:00.314873+0000","last_undegraded":"2024-04-20T10:34:00.314873+0000","last_fullsized":"2024-04-20T10:34:00.314873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:50:21.092892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446278+0000","last_change":"2024-04-20T10:31:34.593529+0000","last_active":"2024-04-20T10:34:00.446278+0000","last_peered":"2024-04-20T10:34:00.446278+0000","last_clean":"2024-04-20T10:34:00.446278+0000","last_became_active":"2024-04-20T10:31:34.592583+0000","last_became_peered":"2024-04-20T10:31:34.592583+0000","last_unstale":"2024-04-20T10:34:00.446278+0000","last_undegraded":"2024-04-20T10:34:00.446278+0000","last_fullsized":"2024-04-20T10:34:00.446278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:51:20.570499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.651292+0000","last_change":"2024-04-20T10:31:35.630739+0000","last_active":"2024-04-20T10:34:00.651292+0000","last_peered":"2024-04-20T10:34:00.651292+0000","last_clean":"2024-04-20T10:34:00.651292+0000","last_became_active":"2024-04-20T10:31:35.630424+0000","last_became_peered":"2024-04-20T10:31:35.630424+0000","last_unstale":"2024-04-20T10:34:00.651292+0000","last_undegraded":"2024-04-20T10:34:00.651292+0000","last_fullsized":"2024-04-20T10:34:00.651292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:48:17.485677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446129+0000","last_change":"2024-04-20T10:31:34.591652+0000","last_active":"2024-04-20T10:34:00.446129+0000","last_peered":"2024-04-20T10:34:00.446129+0000","last_clean":"2024-04-20T10:34:00.446129+0000","last_became_active":"2024-04-20T10:31:34.588945+0000","last_became_peered":"2024-04-20T10:31:34.588945+0000","last_unstale":"2024-04-20T10:34:00.446129+0000","last_undegraded":"2024-04-20T10:34:00.446129+0000","last_fullsized":"2024-04-20T10:34:00.446129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:15.137046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456803+0000","last_change":"2024-04-20T10:31:35.622462+0000","last_active":"2024-04-20T10:34:00.456803+0000","last_peered":"2024-04-20T10:34:00.456803+0000","last_clean":"2024-04-20T10:34:00.456803+0000","last_became_active":"2024-04-20T10:31:35.622125+0000","last_became_peered":"2024-04-20T10:31:35.622125+0000","last_unstale":"2024-04-20T10:34:00.456803+0000","last_undegraded":"2024-04-20T10:34:00.456803+0000","last_fullsized":"2024-04-20T10:34:00.456803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:27.894564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456509+0000","last_change":"2024-04-20T10:32:32.043629+0000","last_active":"2024-04-20T10:34:00.456509+0000","last_peered":"2024-04-20T10:34:00.456509+0000","last_clean":"2024-04-20T10:34:00.456509+0000","last_became_active":"2024-04-20T10:32:32.042849+0000","last_became_peered":"2024-04-20T10:32:32.042849+0000","last_unstale":"2024-04-20T10:34:00.456509+0000","last_undegraded":"2024-04-20T10:34:00.456509+0000","last_fullsized":"2024-04-20T10:34:00.456509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:52.646201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.445985+0000","last_change":"2024-04-20T10:32:31.951658+0000","last_active":"2024-04-20T10:34:00.445985+0000","last_peered":"2024-04-20T10:34:00.445985+0000","last_clean":"2024-04-20T10:34:00.445985+0000","last_became_active":"2024-04-20T10:32:31.951307+0000","last_became_peered":"2024-04-20T10:32:31.951307+0000","last_unstale":"2024-04-20T10:34:00.445985+0000","last_undegraded":"2024-04-20T10:34:00.445985+0000","last_fullsized":"2024-04-20T10:34:00.445985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:42.743969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.435759+0000","last_change":"2024-04-20T10:31:34.612038+0000","last_active":"2024-04-20T10:34:00.435759+0000","last_peered":"2024-04-20T10:34:00.435759+0000","last_clean":"2024-04-20T10:34:00.435759+0000","last_became_active":"2024-04-20T10:31:34.611671+0000","last_became_peered":"2024-04-20T10:31:34.611671+0000","last_unstale":"2024-04-20T10:34:00.435759+0000","last_undegraded":"2024-04-20T10:34:00.435759+0000","last_fullsized":"2024-04-20T10:34:00.435759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:47.862702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456602+0000","last_change":"2024-04-20T10:31:35.643930+0000","last_active":"2024-04-20T10:34:00.456602+0000","last_peered":"2024-04-20T10:34:00.456602+0000","last_clean":"2024-04-20T10:34:00.456602+0000","last_became_active":"2024-04-20T10:31:35.643470+0000","last_became_peered":"2024-04-20T10:31:35.643470+0000","last_unstale":"2024-04-20T10:34:00.456602+0000","last_undegraded":"2024-04-20T10:34:00.456602+0000","last_fullsized":"2024-04-20T10:34:00.456602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:55.570176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307827+0000","last_change":"2024-04-20T10:31:34.613224+0000","last_active":"2024-04-20T10:34:00.307827+0000","last_peered":"2024-04-20T10:34:00.307827+0000","last_clean":"2024-04-20T10:34:00.307827+0000","last_became_active":"2024-04-20T10:31:34.612871+0000","last_became_peered":"2024-04-20T10:31:34.612871+0000","last_unstale":"2024-04-20T10:34:00.307827+0000","last_undegraded":"2024-04-20T10:34:00.307827+0000","last_fullsized":"2024-04-20T10:34:00.307827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:05:57.284870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.438784+0000","last_change":"2024-04-20T10:31:35.643824+0000","last_active":"2024-04-20T10:34:00.438784+0000","last_peered":"2024-04-20T10:34:00.438784+0000","last_clean":"2024-04-20T10:34:00.438784+0000","last_became_active":"2024-04-20T10:31:35.643425+0000","last_became_peered":"2024-04-20T10:31:35.643425+0000","last_unstale":"2024-04-20T10:34:00.438784+0000","last_undegraded":"2024-04-20T10:34:00.438784+0000","last_fullsized":"2024-04-20T10:34:00.438784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:34.008374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456641+0000","last_change":"2024-04-20T10:31:34.593071+0000","last_active":"2024-04-20T10:34:00.456641+0000","last_peered":"2024-04-20T10:34:00.456641+0000","last_clean":"2024-04-20T10:34:00.456641+0000","last_became_active":"2024-04-20T10:31:34.592721+0000","last_became_peered":"2024-04-20T10:31:34.592721+0000","last_unstale":"2024-04-20T10:34:00.456641+0000","last_undegraded":"2024-04-20T10:34:00.456641+0000","last_fullsized":"2024-04-20T10:34:00.456641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:23:43.749586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.310079+0000","last_change":"2024-04-20T10:31:35.647524+0000","last_active":"2024-04-20T10:34:00.310079+0000","last_peered":"2024-04-20T10:34:00.310079+0000","last_clean":"2024-04-20T10:34:00.310079+0000","last_became_active":"2024-04-20T10:31:35.647122+0000","last_became_peered":"2024-04-20T10:31:35.647122+0000","last_unstale":"2024-04-20T10:34:00.310079+0000","last_undegraded":"2024-04-20T10:34:00.310079+0000","last_fullsized":"2024-04-20T10:34:00.310079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.436118+0000","last_change":"2024-04-20T10:31:34.604500+0000","last_active":"2024-04-20T10:34:00.436118+0000","last_peered":"2024-04-20T10:34:00.436118+0000","last_clean":"2024-04-20T10:34:00.436118+0000","last_became_active":"2024-04-20T10:31:34.604148+0000","last_became_peered":"2024-04-20T10:31:34.604148+0000","last_unstale":"2024-04-20T10:34:00.436118+0000","last_undegraded":"2024-04-20T10:34:00.436118+0000","last_fullsized":"2024-04-20T10:34:00.436118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:49:15.174414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.652037+0000","last_change":"2024-04-20T10:31:35.636751+0000","last_active":"2024-04-20T10:34:00.652037+0000","last_peered":"2024-04-20T10:34:00.652037+0000","last_clean":"2024-04-20T10:34:00.652037+0000","last_became_active":"2024-04-20T10:31:35.636103+0000","last_became_peered":"2024-04-20T10:31:35.636103+0000","last_unstale":"2024-04-20T10:34:00.652037+0000","last_undegraded":"2024-04-20T10:34:00.652037+0000","last_fullsized":"2024-04-20T10:34:00.652037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:22:40.580512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.273088+0000","last_change":"2024-04-20T10:31:34.595207+0000","last_active":"2024-04-20T10:34:01.273088+0000","last_peered":"2024-04-20T10:34:01.273088+0000","last_clean":"2024-04-20T10:34:01.273088+0000","last_became_active":"2024-04-20T10:31:34.594860+0000","last_became_peered":"2024-04-20T10:31:34.594860+0000","last_unstale":"2024-04-20T10:34:01.273088+0000","last_undegraded":"2024-04-20T10:34:01.273088+0000","last_fullsized":"2024-04-20T10:34:01.273088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:50.054454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.314770+0000","last_change":"2024-04-20T10:31:35.648086+0000","last_active":"2024-04-20T10:34:00.314770+0000","last_peered":"2024-04-20T10:34:00.314770+0000","last_clean":"2024-04-20T10:34:00.314770+0000","last_became_active":"2024-04-20T10:31:35.647275+0000","last_became_peered":"2024-04-20T10:31:35.647275+0000","last_unstale":"2024-04-20T10:34:00.314770+0000","last_undegraded":"2024-04-20T10:34:00.314770+0000","last_fullsized":"2024-04-20T10:34:00.314770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:27.669008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.457023+0000","last_change":"2024-04-20T10:31:34.601674+0000","last_active":"2024-04-20T10:34:00.457023+0000","last_peered":"2024-04-20T10:34:00.457023+0000","last_clean":"2024-04-20T10:34:00.457023+0000","last_became_active":"2024-04-20T10:31:34.601255+0000","last_became_peered":"2024-04-20T10:31:34.601255+0000","last_unstale":"2024-04-20T10:34:00.457023+0000","last_undegraded":"2024-04-20T10:34:00.457023+0000","last_fullsized":"2024-04-20T10:34:00.457023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:57.821635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.651600+0000","last_change":"2024-04-20T10:31:35.621888+0000","last_active":"2024-04-20T10:34:00.651600+0000","last_peered":"2024-04-20T10:34:00.651600+0000","last_clean":"2024-04-20T10:34:00.651600+0000","last_became_active":"2024-04-20T10:31:35.621503+0000","last_became_peered":"2024-04-20T10:31:35.621503+0000","last_unstale":"2024-04-20T10:34:00.651600+0000","last_undegraded":"2024-04-20T10:34:00.651600+0000","last_fullsized":"2024-04-20T10:34:00.651600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:15:21.118638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.185784+0000","last_change":"2024-04-20T10:31:34.590558+0000","last_active":"2024-04-20T10:34:01.185784+0000","last_peered":"2024-04-20T10:34:01.185784+0000","last_clean":"2024-04-20T10:34:01.185784+0000","last_became_active":"2024-04-20T10:31:34.590140+0000","last_became_peered":"2024-04-20T10:31:34.590140+0000","last_unstale":"2024-04-20T10:34:01.185784+0000","last_undegraded":"2024-04-20T10:34:01.185784+0000","last_fullsized":"2024-04-20T10:34:01.185784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:15:18.591748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67760,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":297,"num_read_kb":215,"num_write":34,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446214+0000","last_change":"2024-04-20T10:31:35.620780+0000","last_active":"2024-04-20T10:34:00.446214+0000","last_peered":"2024-04-20T10:34:00.446214+0000","last_clean":"2024-04-20T10:34:00.446214+0000","last_became_active":"2024-04-20T10:31:35.620429+0000","last_became_peered":"2024-04-20T10:31:35.620429+0000","last_unstale":"2024-04-20T10:34:00.446214+0000","last_undegraded":"2024-04-20T10:34:00.446214+0000","last_fullsized":"2024-04-20T10:34:00.446214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:47:45.587920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650597+0000","last_change":"2024-04-20T10:31:34.602135+0000","last_active":"2024-04-20T10:34:00.650597+0000","last_peered":"2024-04-20T10:34:00.650597+0000","last_clean":"2024-04-20T10:34:00.650597+0000","last_became_active":"2024-04-20T10:31:34.601460+0000","last_became_peered":"2024-04-20T10:31:34.601460+0000","last_unstale":"2024-04-20T10:34:00.650597+0000","last_undegraded":"2024-04-20T10:34:00.650597+0000","last_fullsized":"2024-04-20T10:34:00.650597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:00:00.832702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.308031+0000","last_change":"2024-04-20T10:31:35.635526+0000","last_active":"2024-04-20T10:34:00.308031+0000","last_peered":"2024-04-20T10:34:00.308031+0000","last_clean":"2024-04-20T10:34:00.308031+0000","last_became_active":"2024-04-20T10:31:35.635143+0000","last_became_peered":"2024-04-20T10:31:35.635143+0000","last_unstale":"2024-04-20T10:34:00.308031+0000","last_undegraded":"2024-04-20T10:34:00.308031+0000","last_fullsized":"2024-04-20T10:34:00.308031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:57:24.509318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456493+0000","last_change":"2024-04-20T10:31:34.604561+0000","last_active":"2024-04-20T10:34:00.456493+0000","last_peered":"2024-04-20T10:34:00.456493+0000","last_clean":"2024-04-20T10:34:00.456493+0000","last_became_active":"2024-04-20T10:31:34.604232+0000","last_became_peered":"2024-04-20T10:31:34.604232+0000","last_unstale":"2024-04-20T10:34:00.456493+0000","last_undegraded":"2024-04-20T10:34:00.456493+0000","last_fullsized":"2024-04-20T10:34:00.456493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:39.317473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272869+0000","last_change":"2024-04-20T10:31:35.632840+0000","last_active":"2024-04-20T10:34:01.272869+0000","last_peered":"2024-04-20T10:34:01.272869+0000","last_clean":"2024-04-20T10:34:01.272869+0000","last_became_active":"2024-04-20T10:31:35.632471+0000","last_became_peered":"2024-04-20T10:31:35.632471+0000","last_unstale":"2024-04-20T10:34:01.272869+0000","last_undegraded":"2024-04-20T10:34:01.272869+0000","last_fullsized":"2024-04-20T10:34:01.272869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:06.679967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272715+0000","last_change":"2024-04-20T10:31:34.591696+0000","last_active":"2024-04-20T10:34:01.272715+0000","last_peered":"2024-04-20T10:34:01.272715+0000","last_clean":"2024-04-20T10:34:01.272715+0000","last_became_active":"2024-04-20T10:31:34.591330+0000","last_became_peered":"2024-04-20T10:31:34.591330+0000","last_unstale":"2024-04-20T10:34:01.272715+0000","last_undegraded":"2024-04-20T10:34:01.272715+0000","last_fullsized":"2024-04-20T10:34:01.272715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:44:36.385963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272119+0000","last_change":"2024-04-20T10:31:35.627201+0000","last_active":"2024-04-20T10:34:01.272119+0000","last_peered":"2024-04-20T10:34:01.272119+0000","last_clean":"2024-04-20T10:34:01.272119+0000","last_became_active":"2024-04-20T10:31:35.626880+0000","last_became_peered":"2024-04-20T10:31:35.626880+0000","last_unstale":"2024-04-20T10:34:01.272119+0000","last_undegraded":"2024-04-20T10:34:01.272119+0000","last_fullsized":"2024-04-20T10:34:01.272119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:44.562723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.445834+0000","last_change":"2024-04-20T10:31:34.600683+0000","last_active":"2024-04-20T10:34:00.445834+0000","last_peered":"2024-04-20T10:34:00.445834+0000","last_clean":"2024-04-20T10:34:00.445834+0000","last_became_active":"2024-04-20T10:31:34.600305+0000","last_became_peered":"2024-04-20T10:31:34.600305+0000","last_unstale":"2024-04-20T10:34:00.445834+0000","last_undegraded":"2024-04-20T10:34:00.445834+0000","last_fullsized":"2024-04-20T10:34:00.445834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:05.441158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456458+0000","last_change":"2024-04-20T10:31:35.619835+0000","last_active":"2024-04-20T10:34:00.456458+0000","last_peered":"2024-04-20T10:34:00.456458+0000","last_clean":"2024-04-20T10:34:00.456458+0000","last_became_active":"2024-04-20T10:31:35.619554+0000","last_became_peered":"2024-04-20T10:31:35.619554+0000","last_unstale":"2024-04-20T10:34:00.456458+0000","last_undegraded":"2024-04-20T10:34:00.456458+0000","last_fullsized":"2024-04-20T10:34:00.456458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:33.161563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446313+0000","last_change":"2024-04-20T10:31:34.591128+0000","last_active":"2024-04-20T10:34:00.446313+0000","last_peered":"2024-04-20T10:34:00.446313+0000","last_clean":"2024-04-20T10:34:00.446313+0000","last_became_active":"2024-04-20T10:31:34.587772+0000","last_became_peered":"2024-04-20T10:31:34.587772+0000","last_unstale":"2024-04-20T10:34:00.446313+0000","last_undegraded":"2024-04-20T10:34:00.446313+0000","last_fullsized":"2024-04-20T10:34:00.446313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:12:42.541111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279821+0000","last_change":"2024-04-20T10:31:35.616743+0000","last_active":"2024-04-20T10:34:01.279821+0000","last_peered":"2024-04-20T10:34:01.279821+0000","last_clean":"2024-04-20T10:34:01.279821+0000","last_became_active":"2024-04-20T10:31:35.616419+0000","last_became_peered":"2024-04-20T10:31:35.616419+0000","last_unstale":"2024-04-20T10:34:01.279821+0000","last_undegraded":"2024-04-20T10:34:01.279821+0000","last_fullsized":"2024-04-20T10:34:01.279821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:37.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.310071+0000","last_change":"2024-04-20T10:31:34.603734+0000","last_active":"2024-04-20T10:34:00.310071+0000","last_peered":"2024-04-20T10:34:00.310071+0000","last_clean":"2024-04-20T10:34:00.310071+0000","last_became_active":"2024-04-20T10:31:34.603388+0000","last_became_peered":"2024-04-20T10:31:34.603388+0000","last_unstale":"2024-04-20T10:34:00.310071+0000","last_undegraded":"2024-04-20T10:34:00.310071+0000","last_fullsized":"2024-04-20T10:34:00.310071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:58:57.199173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.436119+0000","last_change":"2024-04-20T10:31:35.638996+0000","last_active":"2024-04-20T10:34:00.436119+0000","last_peered":"2024-04-20T10:34:00.436119+0000","last_clean":"2024-04-20T10:34:00.436119+0000","last_became_active":"2024-04-20T10:31:35.638571+0000","last_became_peered":"2024-04-20T10:31:35.638571+0000","last_unstale":"2024-04-20T10:34:00.436119+0000","last_undegraded":"2024-04-20T10:34:00.436119+0000","last_fullsized":"2024-04-20T10:34:00.436119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:59.541738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.273244+0000","last_change":"2024-04-20T10:31:34.612501+0000","last_active":"2024-04-20T10:34:01.273244+0000","last_peered":"2024-04-20T10:34:01.273244+0000","last_clean":"2024-04-20T10:34:01.273244+0000","last_became_active":"2024-04-20T10:31:34.612150+0000","last_became_peered":"2024-04-20T10:31:34.612150+0000","last_unstale":"2024-04-20T10:34:01.273244+0000","last_undegraded":"2024-04-20T10:34:01.273244+0000","last_fullsized":"2024-04-20T10:34:01.273244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:28.172851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.435952+0000","last_change":"2024-04-20T10:32:31.957767+0000","last_active":"2024-04-20T10:34:00.435952+0000","last_peered":"2024-04-20T10:34:00.435952+0000","last_clean":"2024-04-20T10:34:00.435952+0000","last_became_active":"2024-04-20T10:32:31.957319+0000","last_became_peered":"2024-04-20T10:32:31.957319+0000","last_unstale":"2024-04-20T10:34:00.435952+0000","last_undegraded":"2024-04-20T10:34:00.435952+0000","last_fullsized":"2024-04-20T10:34:00.435952+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:56.375705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456794+0000","last_change":"2024-04-20T10:31:35.617775+0000","last_active":"2024-04-20T10:34:00.456794+0000","last_peered":"2024-04-20T10:34:00.456794+0000","last_clean":"2024-04-20T10:34:00.456794+0000","last_became_active":"2024-04-20T10:31:35.617465+0000","last_became_peered":"2024-04-20T10:31:35.617465+0000","last_unstale":"2024-04-20T10:34:00.456794+0000","last_undegraded":"2024-04-20T10:34:00.456794+0000","last_fullsized":"2024-04-20T10:34:00.456794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:37.830953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307653+0000","last_change":"2024-04-20T10:31:34.615044+0000","last_active":"2024-04-20T10:34:00.307653+0000","last_peered":"2024-04-20T10:34:00.307653+0000","last_clean":"2024-04-20T10:34:00.307653+0000","last_became_active":"2024-04-20T10:31:34.612875+0000","last_became_peered":"2024-04-20T10:31:34.612875+0000","last_unstale":"2024-04-20T10:34:00.307653+0000","last_undegraded":"2024-04-20T10:34:00.307653+0000","last_fullsized":"2024-04-20T10:34:00.307653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:59.867501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439537+0000","last_change":"2024-04-20T10:31:35.630909+0000","last_active":"2024-04-20T10:34:00.439537+0000","last_peered":"2024-04-20T10:34:00.439537+0000","last_clean":"2024-04-20T10:34:00.439537+0000","last_became_active":"2024-04-20T10:31:35.630455+0000","last_became_peered":"2024-04-20T10:31:35.630455+0000","last_unstale":"2024-04-20T10:34:00.439537+0000","last_undegraded":"2024-04-20T10:34:00.439537+0000","last_fullsized":"2024-04-20T10:34:00.439537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:02.522485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272616+0000","last_change":"2024-04-20T10:31:34.602466+0000","last_active":"2024-04-20T10:34:01.272616+0000","last_peered":"2024-04-20T10:34:01.272616+0000","last_clean":"2024-04-20T10:34:01.272616+0000","last_became_active":"2024-04-20T10:31:34.602089+0000","last_became_peered":"2024-04-20T10:31:34.602089+0000","last_unstale":"2024-04-20T10:34:01.272616+0000","last_undegraded":"2024-04-20T10:34:01.272616+0000","last_fullsized":"2024-04-20T10:34:01.272616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:19:09.849840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272126+0000","last_change":"2024-04-20T10:30:14.724347+0000","last_active":"2024-04-20T10:34:01.272126+0000","last_peered":"2024-04-20T10:34:01.272126+0000","last_clean":"2024-04-20T10:34:01.272126+0000","last_became_active":"2024-04-20T10:30:14.723877+0000","last_became_peered":"2024-04-20T10:30:14.723877+0000","last_unstale":"2024-04-20T10:34:01.272126+0000","last_undegraded":"2024-04-20T10:34:01.272126+0000","last_fullsized":"2024-04-20T10:34:01.272126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_clean_scrub_stamp":"2024-04-20T10:26:16.803986+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:30.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307565+0000","last_change":"2024-04-20T10:31:35.626505+0000","last_active":"2024-04-20T10:34:00.307565+0000","last_peered":"2024-04-20T10:34:00.307565+0000","last_clean":"2024-04-20T10:34:00.307565+0000","last_became_active":"2024-04-20T10:31:35.626141+0000","last_became_peered":"2024-04-20T10:31:35.626141+0000","last_unstale":"2024-04-20T10:34:00.307565+0000","last_undegraded":"2024-04-20T10:34:00.307565+0000","last_fullsized":"2024-04-20T10:34:00.307565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:29.827225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272242+0000","last_change":"2024-04-20T10:31:34.596936+0000","last_active":"2024-04-20T10:34:01.272242+0000","last_peered":"2024-04-20T10:34:01.272242+0000","last_clean":"2024-04-20T10:34:01.272242+0000","last_became_active":"2024-04-20T10:31:34.596692+0000","last_became_peered":"2024-04-20T10:31:34.596692+0000","last_unstale":"2024-04-20T10:34:01.272242+0000","last_undegraded":"2024-04-20T10:34:01.272242+0000","last_fullsized":"2024-04-20T10:34:01.272242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:02:39.819096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446098+0000","last_change":"2024-04-20T10:31:34.601407+0000","last_active":"2024-04-20T10:34:00.446098+0000","last_peered":"2024-04-20T10:34:00.446098+0000","last_clean":"2024-04-20T10:34:00.446098+0000","last_became_active":"2024-04-20T10:31:34.601007+0000","last_became_peered":"2024-04-20T10:31:34.601007+0000","last_unstale":"2024-04-20T10:34:00.446098+0000","last_undegraded":"2024-04-20T10:34:00.446098+0000","last_fullsized":"2024-04-20T10:34:00.446098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:05:03.037841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.308057+0000","last_change":"2024-04-20T10:31:35.641046+0000","last_active":"2024-04-20T10:34:00.308057+0000","last_peered":"2024-04-20T10:34:00.308057+0000","last_clean":"2024-04-20T10:34:00.308057+0000","last_became_active":"2024-04-20T10:31:35.640666+0000","last_became_peered":"2024-04-20T10:31:35.640666+0000","last_unstale":"2024-04-20T10:34:00.308057+0000","last_undegraded":"2024-04-20T10:34:00.308057+0000","last_fullsized":"2024-04-20T10:34:00.308057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:42.496037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'45","reported_seq":90,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.606523+0000","last_change":"2024-04-20T10:31:34.604643+0000","last_active":"2024-04-20T10:34:00.606523+0000","last_peered":"2024-04-20T10:34:00.606523+0000","last_clean":"2024-04-20T10:34:00.606523+0000","last_became_active":"2024-04-20T10:31:34.604210+0000","last_became_peered":"2024-04-20T10:31:34.604210+0000","last_unstale":"2024-04-20T10:34:00.606523+0000","last_undegraded":"2024-04-20T10:34:00.606523+0000","last_fullsized":"2024-04-20T10:34:00.606523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:24:09.748012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":81,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439226+0000","last_change":"2024-04-20T10:31:35.641981+0000","last_active":"2024-04-20T10:34:00.439226+0000","last_peered":"2024-04-20T10:34:00.439226+0000","last_clean":"2024-04-20T10:34:00.439226+0000","last_became_active":"2024-04-20T10:31:35.641607+0000","last_became_peered":"2024-04-20T10:31:35.641607+0000","last_unstale":"2024-04-20T10:34:00.439226+0000","last_undegraded":"2024-04-20T10:34:00.439226+0000","last_fullsized":"2024-04-20T10:34:00.439226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:48.520813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272757+0000","last_change":"2024-04-20T10:31:34.611218+0000","last_active":"2024-04-20T10:34:01.272757+0000","last_peered":"2024-04-20T10:34:01.272757+0000","last_clean":"2024-04-20T10:34:01.272757+0000","last_became_active":"2024-04-20T10:31:34.610950+0000","last_became_peered":"2024-04-20T10:31:34.610950+0000","last_unstale":"2024-04-20T10:34:01.272757+0000","last_undegraded":"2024-04-20T10:34:01.272757+0000","last_fullsized":"2024-04-20T10:34:01.272757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:14:11.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313611+0000","last_change":"2024-04-20T10:31:35.647531+0000","last_active":"2024-04-20T10:34:00.313611+0000","last_peered":"2024-04-20T10:34:00.313611+0000","last_clean":"2024-04-20T10:34:00.313611+0000","last_became_active":"2024-04-20T10:31:35.646389+0000","last_became_peered":"2024-04-20T10:31:35.646389+0000","last_unstale":"2024-04-20T10:34:00.313611+0000","last_undegraded":"2024-04-20T10:34:00.313611+0000","last_fullsized":"2024-04-20T10:34:00.313611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:50.141385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.651038+0000","last_change":"2024-04-20T10:32:31.957675+0000","last_active":"2024-04-20T10:34:00.651038+0000","last_peered":"2024-04-20T10:34:00.651038+0000","last_clean":"2024-04-20T10:34:00.651038+0000","last_became_active":"2024-04-20T10:32:31.957202+0000","last_became_peered":"2024-04-20T10:32:31.957202+0000","last_unstale":"2024-04-20T10:34:00.651038+0000","last_undegraded":"2024-04-20T10:34:00.651038+0000","last_fullsized":"2024-04-20T10:34:00.651038+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:28.001088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.445870+0000","last_change":"2024-04-20T10:31:35.616751+0000","last_active":"2024-04-20T10:34:00.445870+0000","last_peered":"2024-04-20T10:34:00.445870+0000","last_clean":"2024-04-20T10:34:00.445870+0000","last_became_active":"2024-04-20T10:31:35.616432+0000","last_became_peered":"2024-04-20T10:31:35.616432+0000","last_unstale":"2024-04-20T10:34:00.445870+0000","last_undegraded":"2024-04-20T10:34:00.445870+0000","last_fullsized":"2024-04-20T10:34:00.445870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:25.052140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439413+0000","last_change":"2024-04-20T10:31:34.609139+0000","last_active":"2024-04-20T10:34:00.439413+0000","last_peered":"2024-04-20T10:34:00.439413+0000","last_clean":"2024-04-20T10:34:00.439413+0000","last_became_active":"2024-04-20T10:31:34.608818+0000","last_became_peered":"2024-04-20T10:31:34.608818+0000","last_unstale":"2024-04-20T10:34:00.439413+0000","last_undegraded":"2024-04-20T10:34:00.439413+0000","last_fullsized":"2024-04-20T10:34:00.439413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:42:39.581059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279576+0000","last_change":"2024-04-20T10:31:35.643791+0000","last_active":"2024-04-20T10:34:01.279576+0000","last_peered":"2024-04-20T10:34:01.279576+0000","last_clean":"2024-04-20T10:34:01.279576+0000","last_became_active":"2024-04-20T10:31:35.643390+0000","last_became_peered":"2024-04-20T10:31:35.643390+0000","last_unstale":"2024-04-20T10:34:01.279576+0000","last_undegraded":"2024-04-20T10:34:01.279576+0000","last_fullsized":"2024-04-20T10:34:01.279576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:10.149979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650525+0000","last_change":"2024-04-20T10:31:34.606178+0000","last_active":"2024-04-20T10:34:00.650525+0000","last_peered":"2024-04-20T10:34:00.650525+0000","last_clean":"2024-04-20T10:34:00.650525+0000","last_became_active":"2024-04-20T10:31:34.605817+0000","last_became_peered":"2024-04-20T10:31:34.605817+0000","last_unstale":"2024-04-20T10:34:00.650525+0000","last_undegraded":"2024-04-20T10:34:00.650525+0000","last_fullsized":"2024-04-20T10:34:00.650525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:15.929905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439824+0000","last_change":"2024-04-20T10:31:35.649487+0000","last_active":"2024-04-20T10:34:00.439824+0000","last_peered":"2024-04-20T10:34:00.439824+0000","last_clean":"2024-04-20T10:34:00.439824+0000","last_became_active":"2024-04-20T10:31:35.648884+0000","last_became_peered":"2024-04-20T10:31:35.648884+0000","last_unstale":"2024-04-20T10:34:00.439824+0000","last_undegraded":"2024-04-20T10:34:00.439824+0000","last_fullsized":"2024-04-20T10:34:00.439824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:07.389100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279544+0000","last_change":"2024-04-20T10:32:31.943005+0000","last_active":"2024-04-20T10:34:01.279544+0000","last_peered":"2024-04-20T10:34:01.279544+0000","last_clean":"2024-04-20T10:34:01.279544+0000","last_became_active":"2024-04-20T10:32:31.940063+0000","last_became_peered":"2024-04-20T10:32:31.940063+0000","last_unstale":"2024-04-20T10:34:01.279544+0000","last_undegraded":"2024-04-20T10:34:01.279544+0000","last_fullsized":"2024-04-20T10:34:01.279544+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:11:06.308028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272443+0000","last_change":"2024-04-20T10:31:35.630256+0000","last_active":"2024-04-20T10:34:01.272443+0000","last_peered":"2024-04-20T10:34:01.272443+0000","last_clean":"2024-04-20T10:34:01.272443+0000","last_became_active":"2024-04-20T10:31:35.629879+0000","last_became_peered":"2024-04-20T10:31:35.629879+0000","last_unstale":"2024-04-20T10:34:01.272443+0000","last_undegraded":"2024-04-20T10:34:01.272443+0000","last_fullsized":"2024-04-20T10:34:01.272443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:51.601923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650706+0000","last_change":"2024-04-20T10:31:34.595149+0000","last_active":"2024-04-20T10:34:00.650706+0000","last_peered":"2024-04-20T10:34:00.650706+0000","last_clean":"2024-04-20T10:34:00.650706+0000","last_became_active":"2024-04-20T10:31:34.594808+0000","last_became_peered":"2024-04-20T10:31:34.594808+0000","last_unstale":"2024-04-20T10:34:00.650706+0000","last_undegraded":"2024-04-20T10:34:00.650706+0000","last_fullsized":"2024-04-20T10:34:00.650706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:51.707710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272938+0000","last_change":"2024-04-20T10:31:35.620612+0000","last_active":"2024-04-20T10:34:01.272938+0000","last_peered":"2024-04-20T10:34:01.272938+0000","last_clean":"2024-04-20T10:34:01.272938+0000","last_became_active":"2024-04-20T10:31:35.620312+0000","last_became_peered":"2024-04-20T10:31:35.620312+0000","last_unstale":"2024-04-20T10:34:01.272938+0000","last_undegraded":"2024-04-20T10:34:01.272938+0000","last_fullsized":"2024-04-20T10:34:01.272938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:37:50.756950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'39","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.604966+0000","last_change":"2024-04-20T10:31:34.604613+0000","last_active":"2024-04-20T10:34:00.604966+0000","last_peered":"2024-04-20T10:34:00.604966+0000","last_clean":"2024-04-20T10:34:00.604966+0000","last_became_active":"2024-04-20T10:31:34.604246+0000","last_became_peered":"2024-04-20T10:31:34.604246+0000","last_unstale":"2024-04-20T10:34:00.604966+0000","last_undegraded":"2024-04-20T10:34:00.604966+0000","last_fullsized":"2024-04-20T10:34:00.604966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:26.521769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272437+0000","last_change":"2024-04-20T10:31:35.639396+0000","last_active":"2024-04-20T10:34:01.272437+0000","last_peered":"2024-04-20T10:34:01.272437+0000","last_clean":"2024-04-20T10:34:01.272437+0000","last_became_active":"2024-04-20T10:31:35.638979+0000","last_became_peered":"2024-04-20T10:31:35.638979+0000","last_unstale":"2024-04-20T10:34:01.272437+0000","last_undegraded":"2024-04-20T10:34:01.272437+0000","last_fullsized":"2024-04-20T10:34:01.272437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:47.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.308292+0000","last_change":"2024-04-20T10:31:34.607420+0000","last_active":"2024-04-20T10:34:00.308292+0000","last_peered":"2024-04-20T10:34:00.308292+0000","last_clean":"2024-04-20T10:34:00.308292+0000","last_became_active":"2024-04-20T10:31:34.602587+0000","last_became_peered":"2024-04-20T10:31:34.602587+0000","last_unstale":"2024-04-20T10:34:00.308292+0000","last_undegraded":"2024-04-20T10:34:00.308292+0000","last_fullsized":"2024-04-20T10:34:00.308292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:56.447944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.273183+0000","last_change":"2024-04-20T10:31:35.624903+0000","last_active":"2024-04-20T10:34:01.273183+0000","last_peered":"2024-04-20T10:34:01.273183+0000","last_clean":"2024-04-20T10:34:01.273183+0000","last_became_active":"2024-04-20T10:31:35.619479+0000","last_became_peered":"2024-04-20T10:31:35.619479+0000","last_unstale":"2024-04-20T10:34:01.273183+0000","last_undegraded":"2024-04-20T10:34:01.273183+0000","last_fullsized":"2024-04-20T10:34:01.273183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:25:40.601983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279614+0000","last_change":"2024-04-20T10:31:35.620351+0000","last_active":"2024-04-20T10:34:01.279614+0000","last_peered":"2024-04-20T10:34:01.279614+0000","last_clean":"2024-04-20T10:34:01.279614+0000","last_became_active":"2024-04-20T10:31:35.620028+0000","last_became_peered":"2024-04-20T10:31:35.620028+0000","last_unstale":"2024-04-20T10:34:01.279614+0000","last_undegraded":"2024-04-20T10:34:01.279614+0000","last_fullsized":"2024-04-20T10:34:01.279614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:51:53.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650618+0000","last_change":"2024-04-20T10:31:35.615917+0000","last_active":"2024-04-20T10:34:00.650618+0000","last_peered":"2024-04-20T10:34:00.650618+0000","last_clean":"2024-04-20T10:34:00.650618+0000","last_became_active":"2024-04-20T10:31:35.615582+0000","last_became_peered":"2024-04-20T10:31:35.615582+0000","last_unstale":"2024-04-20T10:34:00.650618+0000","last_undegraded":"2024-04-20T10:34:00.650618+0000","last_fullsized":"2024-04-20T10:34:00.650618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:54:59.754819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650460+0000","last_change":"2024-04-20T10:31:35.650006+0000","last_active":"2024-04-20T10:34:00.650460+0000","last_peered":"2024-04-20T10:34:00.650460+0000","last_clean":"2024-04-20T10:34:00.650460+0000","last_became_active":"2024-04-20T10:31:35.649683+0000","last_became_peered":"2024-04-20T10:31:35.649683+0000","last_unstale":"2024-04-20T10:34:00.650460+0000","last_undegraded":"2024-04-20T10:34:00.650460+0000","last_fullsized":"2024-04-20T10:34:00.650460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:10:01.484809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313550+0000","last_change":"2024-04-20T10:31:35.646591+0000","last_active":"2024-04-20T10:34:00.313550+0000","last_peered":"2024-04-20T10:34:00.313550+0000","last_clean":"2024-04-20T10:34:00.313550+0000","last_became_active":"2024-04-20T10:31:35.646188+0000","last_became_peered":"2024-04-20T10:31:35.646188+0000","last_unstale":"2024-04-20T10:34:00.313550+0000","last_undegraded":"2024-04-20T10:34:00.313550+0000","last_fullsized":"2024-04-20T10:34:00.313550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:23.412022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307919+0000","last_change":"2024-04-20T10:31:35.650879+0000","last_active":"2024-04-20T10:34:00.307919+0000","last_peered":"2024-04-20T10:34:00.307919+0000","last_clean":"2024-04-20T10:34:00.307919+0000","last_became_active":"2024-04-20T10:31:35.650486+0000","last_became_peered":"2024-04-20T10:31:35.650486+0000","last_unstale":"2024-04-20T10:34:00.307919+0000","last_undegraded":"2024-04-20T10:34:00.307919+0000","last_fullsized":"2024-04-20T10:34:00.307919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:51.893446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279735+0000","last_change":"2024-04-20T10:31:35.619883+0000","last_active":"2024-04-20T10:34:01.279735+0000","last_peered":"2024-04-20T10:34:01.279735+0000","last_clean":"2024-04-20T10:34:01.279735+0000","last_became_active":"2024-04-20T10:31:35.619628+0000","last_became_peered":"2024-04-20T10:31:35.619628+0000","last_unstale":"2024-04-20T10:34:01.279735+0000","last_undegraded":"2024-04-20T10:34:01.279735+0000","last_fullsized":"2024-04-20T10:34:01.279735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:45.018417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650973+0000","last_change":"2024-04-20T10:31:35.620773+0000","last_active":"2024-04-20T10:34:00.650973+0000","last_peered":"2024-04-20T10:34:00.650973+0000","last_clean":"2024-04-20T10:34:00.650973+0000","last_became_active":"2024-04-20T10:31:35.620467+0000","last_became_peered":"2024-04-20T10:31:35.620467+0000","last_unstale":"2024-04-20T10:34:00.650973+0000","last_undegraded":"2024-04-20T10:34:00.650973+0000","last_fullsized":"2024-04-20T10:34:00.650973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:31:48.009168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439094+0000","last_change":"2024-04-20T10:32:31.953580+0000","last_active":"2024-04-20T10:34:00.439094+0000","last_peered":"2024-04-20T10:34:00.439094+0000","last_clean":"2024-04-20T10:34:00.439094+0000","last_became_active":"2024-04-20T10:32:31.952985+0000","last_became_peered":"2024-04-20T10:32:31.952985+0000","last_unstale":"2024-04-20T10:34:00.439094+0000","last_undegraded":"2024-04-20T10:34:00.439094+0000","last_fullsized":"2024-04-20T10:34:00.439094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:13:03.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456576+0000","last_change":"2024-04-20T10:31:35.637588+0000","last_active":"2024-04-20T10:34:00.456576+0000","last_peered":"2024-04-20T10:34:00.456576+0000","last_clean":"2024-04-20T10:34:00.456576+0000","last_became_active":"2024-04-20T10:31:35.637319+0000","last_became_peered":"2024-04-20T10:31:35.637319+0000","last_unstale":"2024-04-20T10:34:00.456576+0000","last_undegraded":"2024-04-20T10:34:00.456576+0000","last_fullsized":"2024-04-20T10:34:00.456576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:54.070894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439287+0000","last_change":"2024-04-20T10:31:35.629551+0000","last_active":"2024-04-20T10:34:00.439287+0000","last_peered":"2024-04-20T10:34:00.439287+0000","last_clean":"2024-04-20T10:34:00.439287+0000","last_became_active":"2024-04-20T10:31:35.628710+0000","last_became_peered":"2024-04-20T10:31:35.628710+0000","last_unstale":"2024-04-20T10:34:00.439287+0000","last_undegraded":"2024-04-20T10:34:00.439287+0000","last_fullsized":"2024-04-20T10:34:00.439287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:31.883788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.651244+0000","last_change":"2024-04-20T10:31:35.629978+0000","last_active":"2024-04-20T10:34:00.651244+0000","last_peered":"2024-04-20T10:34:00.651244+0000","last_clean":"2024-04-20T10:34:00.651244+0000","last_became_active":"2024-04-20T10:31:35.629596+0000","last_became_peered":"2024-04-20T10:31:35.629596+0000","last_unstale":"2024-04-20T10:34:00.651244+0000","last_undegraded":"2024-04-20T10:34:00.651244+0000","last_fullsized":"2024-04-20T10:34:00.651244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:02.523099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.308137+0000","last_change":"2024-04-20T10:31:35.652481+0000","last_active":"2024-04-20T10:34:00.308137+0000","last_peered":"2024-04-20T10:34:00.308137+0000","last_clean":"2024-04-20T10:34:00.308137+0000","last_became_active":"2024-04-20T10:31:35.651837+0000","last_became_peered":"2024-04-20T10:31:35.651837+0000","last_unstale":"2024-04-20T10:34:00.308137+0000","last_undegraded":"2024-04-20T10:34:00.308137+0000","last_fullsized":"2024-04-20T10:34:00.308137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:53.206954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279636+0000","last_change":"2024-04-20T10:31:35.622228+0000","last_active":"2024-04-20T10:34:01.279636+0000","last_peered":"2024-04-20T10:34:01.279636+0000","last_clean":"2024-04-20T10:34:01.279636+0000","last_became_active":"2024-04-20T10:31:35.621910+0000","last_became_peered":"2024-04-20T10:31:35.621910+0000","last_unstale":"2024-04-20T10:34:01.279636+0000","last_undegraded":"2024-04-20T10:34:01.279636+0000","last_fullsized":"2024-04-20T10:34:01.279636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:47:34.319839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107397,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":593374,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":393,"num_read_kb":298,"num_write":201,"num_write_kb":1405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":1780122,"data_compressed":81447,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":455821,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1110016,"data_stored":1254063,"data_compressed":52532,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":858604,"data_compressed":27876,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":858626,"data_compressed":27876,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332161,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":1046888,"data_compressed":41606,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":464923,"data_compressed":8730,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":532566,"data_compressed":10577,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":464896,"data_compressed":8730,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411422,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":522833,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":989960,"data_compressed":34032,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":455843,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":1187073,"data_compressed":50685,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":67760,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67794,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":524920,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":131782,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":199554,"data_compressed":8003,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:34:04.550 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-20T10:34:04.550 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:34:04.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.781+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2218858121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac109cc0 msgr2=0x7f6bac10c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:04.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.781+0000 7f6bb0a00640 1 --2- 172.21.15.42:0/2218858121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac109cc0 0x7f6bac10c150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009be0 tx=0x7f6ba402bef0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.782+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2218858121 shutdown_connections 2024-04-20T10:34:04.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.782+0000 7f6bb0a00640 1 --2- 172.21.15.42:0/2218858121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac109cc0 0x7f6bac10c150 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.782+0000 7f6bb0a00640 1 --2- 172.21.15.42:0/2218858121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bac0749c0 0x7f6bac074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.782+0000 7f6bb0a00640 1 --2- 172.21.15.42:0/2218858121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bac075fa0 0x7f6bac0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.782+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2218858121 >> 172.21.15.42:0/2218858121 conn(0x7f6bac0fd370 msgr2=0x7f6bac0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:04.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.782+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2218858121 shutdown_connections 2024-04-20T10:34:04.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.782+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2218858121 wait complete. 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.783+0000 7f6bb0a00640 1 Processor -- start 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.783+0000 7f6bb0a00640 1 -- start start 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.783+0000 7f6bb0a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac0749c0 0x7f6bac19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.783+0000 7f6bb0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bac075fa0 0x7f6bac19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.783+0000 7f6bb0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bac109cc0 0x7f6bac1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.783+0000 7f6bb0a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac10fcd0 con 0x7f6bac109cc0 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.783+0000 7f6bb0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac10fb50 con 0x7f6bac075fa0 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.783+0000 7f6bb0a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac10fe50 con 0x7f6bac0749c0 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.784+0000 7f6bab200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bac109cc0 0x7f6bac1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.784+0000 7f6ba9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bac075fa0 0x7f6bac19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.784+0000 7f6ba9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bac075fa0 0x7f6bac19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52064/0 (socket says 172.21.15.42:52064) 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.784+0000 7f6ba9e00640 1 -- 172.21.15.42:0/2339020277 learned_addr learned my addr 172.21.15.42:0/2339020277 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.784+0000 7f6baa800640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac0749c0 0x7f6bac19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:04.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.784+0000 7f6ba9e00640 1 -- 172.21.15.42:0/2339020277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac0749c0 msgr2=0x7f6bac19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:04.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.784+0000 7f6ba9e00640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac0749c0 0x7f6bac19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.784+0000 7f6ba9e00640 1 -- 172.21.15.42:0/2339020277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bac109cc0 msgr2=0x7f6bac1a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:04.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.784+0000 7f6ba9e00640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bac109cc0 0x7f6bac1a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:04.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.784+0000 7f6ba9e00640 1 -- 172.21.15.42:0/2339020277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bac1a1e60 con 0x7f6bac075fa0 2024-04-20T10:34:04.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.784+0000 7f6ba9e00640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bac075fa0 0x7f6bac19a780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800e9c0 tx=0x7f6b9800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:04.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.785+0000 7f6ba3400640 1 -- 172.21.15.42:0/2339020277 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9800cde0 con 0x7f6bac075fa0 2024-04-20T10:34:04.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.785+0000 7f6ba3400640 1 -- 172.21.15.42:0/2339020277 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b980035f0 con 0x7f6bac075fa0 2024-04-20T10:34:04.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.785+0000 7f6ba3400640 1 -- 172.21.15.42:0/2339020277 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98010690 con 0x7f6bac075fa0 2024-04-20T10:34:04.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.785+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2339020277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bac1a2150 con 0x7f6bac075fa0 2024-04-20T10:34:04.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.785+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2339020277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bac1a26f0 con 0x7f6bac075fa0 2024-04-20T10:34:04.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.786+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2339020277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bac1089f0 con 0x7f6bac075fa0 2024-04-20T10:34:04.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.789+0000 7f6ba3400640 1 -- 172.21.15.42:0/2339020277 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7f6b980037a0 con 0x7f6bac075fa0 2024-04-20T10:34:04.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.790+0000 7f6ba3400640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f6b900a3da0 0x7f6b900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:04.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.790+0000 7f6ba3400640 1 -- 172.21.15.42:0/2339020277 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6b980d0aa0 con 0x7f6bac075fa0 2024-04-20T10:34:04.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.790+0000 7f6baa800640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f6b900a3da0 0x7f6b900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:04.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.791+0000 7f6baa800640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f6b900a3da0 0x7f6b900a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009bd0 tx=0x7f6b9c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:04.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:04.794+0000 7f6ba3400640 1 -- 172.21.15.42:0/2339020277 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6b9809d0e0 con 0x7f6bac075fa0 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi162 to 176.2M 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: Unable to set osd_memory_target on smithi162 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:04.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:04.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:04 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:05.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.000+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2339020277 --> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6bac0623a0 con 0x7f6b900a3da0 2024-04-20T10:34:05.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.018+0000 7f6ba3400640 1 -- 172.21.15.42:0/2339020277 <== mgr.15024 v2:172.21.15.42:6800/170978916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257971 (secure 0 0 0) 0x7f6bac0623a0 con 0x7f6b900a3da0 2024-04-20T10:34:05.022 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:34:05.023 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-04-20T10:34:05.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.022+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2339020277 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f6b900a3da0 msgr2=0x7f6b900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:05.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.022+0000 7f6bb0a00640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f6b900a3da0 0x7f6b900a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009bd0 tx=0x7f6b9c005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:05.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.022+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2339020277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bac075fa0 msgr2=0x7f6bac19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:05.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.022+0000 7f6bb0a00640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bac075fa0 0x7f6bac19a780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800e9c0 tx=0x7f6b9800ee80 comp rx=0 tx=0).stop 2024-04-20T10:34:05.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.023+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2339020277 shutdown_connections 2024-04-20T10:34:05.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.023+0000 7f6bb0a00640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f6b900a3da0 0x7f6b900a6260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.023+0000 7f6bb0a00640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bac109cc0 0x7f6bac1a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.023+0000 7f6bb0a00640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bac075fa0 0x7f6bac19a780 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.023+0000 7f6bb0a00640 1 --2- 172.21.15.42:0/2339020277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac0749c0 0x7f6bac19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.023+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2339020277 >> 172.21.15.42:0/2339020277 conn(0x7f6bac0fd370 msgr2=0x7f6bac105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.023+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2339020277 shutdown_connections 2024-04-20T10:34:05.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.023+0000 7f6bb0a00640 1 -- 172.21.15.42:0/2339020277 wait complete. 2024-04-20T10:34:05.043 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-20T10:34:04.327766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1183976,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":529,"num_read_kb":417,"num_write":384,"num_write_kb":4345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369180,"kb_used_data":11664,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572644,"statfs":{"total":1151940427776,"available":1151562387456,"internally_reserved":0,"allocated":11943936,"data_stored":9092096,"data_compressed":281912,"data_compressed_allocated":376832,"data_compressed_original":3604480,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.984344"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279951+0000","last_change":"2024-04-20T10:31:35.649585+0000","last_active":"2024-04-20T10:34:01.279951+0000","last_peered":"2024-04-20T10:34:01.279951+0000","last_clean":"2024-04-20T10:34:01.279951+0000","last_became_active":"2024-04-20T10:31:35.649282+0000","last_became_peered":"2024-04-20T10:31:35.649282+0000","last_unstale":"2024-04-20T10:34:01.279951+0000","last_undegraded":"2024-04-20T10:34:01.279951+0000","last_fullsized":"2024-04-20T10:34:01.279951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:01:00.517383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313417+0000","last_change":"2024-04-20T10:31:35.648299+0000","last_active":"2024-04-20T10:34:00.313417+0000","last_peered":"2024-04-20T10:34:00.313417+0000","last_clean":"2024-04-20T10:34:00.313417+0000","last_became_active":"2024-04-20T10:31:35.647283+0000","last_became_peered":"2024-04-20T10:31:35.647283+0000","last_unstale":"2024-04-20T10:34:00.313417+0000","last_undegraded":"2024-04-20T10:34:00.313417+0000","last_fullsized":"2024-04-20T10:34:00.313417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:28:59.006234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.314500+0000","last_change":"2024-04-20T10:31:35.647029+0000","last_active":"2024-04-20T10:34:00.314500+0000","last_peered":"2024-04-20T10:34:00.314500+0000","last_clean":"2024-04-20T10:34:00.314500+0000","last_became_active":"2024-04-20T10:31:35.646649+0000","last_became_peered":"2024-04-20T10:31:35.646649+0000","last_unstale":"2024-04-20T10:34:00.314500+0000","last_undegraded":"2024-04-20T10:34:00.314500+0000","last_fullsized":"2024-04-20T10:34:00.314500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:03:23.420078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272555+0000","last_change":"2024-04-20T10:31:35.624628+0000","last_active":"2024-04-20T10:34:01.272555+0000","last_peered":"2024-04-20T10:34:01.272555+0000","last_clean":"2024-04-20T10:34:01.272555+0000","last_became_active":"2024-04-20T10:31:35.623607+0000","last_became_peered":"2024-04-20T10:31:35.623607+0000","last_unstale":"2024-04-20T10:34:01.272555+0000","last_undegraded":"2024-04-20T10:34:01.272555+0000","last_fullsized":"2024-04-20T10:34:01.272555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:44:46.768584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.445865+0000","last_change":"2024-04-20T10:31:35.638293+0000","last_active":"2024-04-20T10:34:00.445865+0000","last_peered":"2024-04-20T10:34:00.445865+0000","last_clean":"2024-04-20T10:34:00.445865+0000","last_became_active":"2024-04-20T10:31:35.637967+0000","last_became_peered":"2024-04-20T10:31:35.637967+0000","last_unstale":"2024-04-20T10:34:00.445865+0000","last_undegraded":"2024-04-20T10:34:00.445865+0000","last_fullsized":"2024-04-20T10:34:00.445865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:58:00.765015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.435951+0000","last_change":"2024-04-20T10:31:35.634565+0000","last_active":"2024-04-20T10:34:00.435951+0000","last_peered":"2024-04-20T10:34:00.435951+0000","last_clean":"2024-04-20T10:34:00.435951+0000","last_became_active":"2024-04-20T10:31:35.634200+0000","last_became_peered":"2024-04-20T10:31:35.634200+0000","last_unstale":"2024-04-20T10:34:00.435951+0000","last_undegraded":"2024-04-20T10:34:00.435951+0000","last_fullsized":"2024-04-20T10:34:00.435951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:14.298732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272189+0000","last_change":"2024-04-20T10:32:31.949166+0000","last_active":"2024-04-20T10:34:01.272189+0000","last_peered":"2024-04-20T10:34:01.272189+0000","last_clean":"2024-04-20T10:34:01.272189+0000","last_became_active":"2024-04-20T10:32:31.948825+0000","last_became_peered":"2024-04-20T10:32:31.948825+0000","last_unstale":"2024-04-20T10:34:01.272189+0000","last_undegraded":"2024-04-20T10:34:01.272189+0000","last_fullsized":"2024-04-20T10:34:01.272189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:05.337941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.438962+0000","last_change":"2024-04-20T10:31:35.631996+0000","last_active":"2024-04-20T10:34:00.438962+0000","last_peered":"2024-04-20T10:34:00.438962+0000","last_clean":"2024-04-20T10:34:00.438962+0000","last_became_active":"2024-04-20T10:31:35.631576+0000","last_became_peered":"2024-04-20T10:31:35.631576+0000","last_unstale":"2024-04-20T10:34:00.438962+0000","last_undegraded":"2024-04-20T10:34:00.438962+0000","last_fullsized":"2024-04-20T10:34:00.438962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:51.448025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.652006+0000","last_change":"2024-04-20T10:31:35.624430+0000","last_active":"2024-04-20T10:34:00.652006+0000","last_peered":"2024-04-20T10:34:00.652006+0000","last_clean":"2024-04-20T10:34:00.652006+0000","last_became_active":"2024-04-20T10:31:35.623018+0000","last_became_peered":"2024-04-20T10:31:35.623018+0000","last_unstale":"2024-04-20T10:34:00.652006+0000","last_undegraded":"2024-04-20T10:34:00.652006+0000","last_fullsized":"2024-04-20T10:34:00.652006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:43.100925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.436193+0000","last_change":"2024-04-20T10:31:35.619038+0000","last_active":"2024-04-20T10:34:00.436193+0000","last_peered":"2024-04-20T10:34:00.436193+0000","last_clean":"2024-04-20T10:34:00.436193+0000","last_became_active":"2024-04-20T10:31:35.618699+0000","last_became_peered":"2024-04-20T10:31:35.618699+0000","last_unstale":"2024-04-20T10:34:00.436193+0000","last_undegraded":"2024-04-20T10:34:00.436193+0000","last_fullsized":"2024-04-20T10:34:00.436193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:27.205634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272793+0000","last_change":"2024-04-20T10:31:35.619908+0000","last_active":"2024-04-20T10:34:01.272793+0000","last_peered":"2024-04-20T10:34:01.272793+0000","last_clean":"2024-04-20T10:34:01.272793+0000","last_became_active":"2024-04-20T10:31:35.619580+0000","last_became_peered":"2024-04-20T10:31:35.619580+0000","last_unstale":"2024-04-20T10:34:01.272793+0000","last_undegraded":"2024-04-20T10:34:01.272793+0000","last_fullsized":"2024-04-20T10:34:01.272793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:54:48.999925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.309895+0000","last_change":"2024-04-20T10:32:31.952701+0000","last_active":"2024-04-20T10:34:00.309895+0000","last_peered":"2024-04-20T10:34:00.309895+0000","last_clean":"2024-04-20T10:34:00.309895+0000","last_became_active":"2024-04-20T10:32:31.951329+0000","last_became_peered":"2024-04-20T10:32:31.951329+0000","last_unstale":"2024-04-20T10:34:00.309895+0000","last_undegraded":"2024-04-20T10:34:00.309895+0000","last_fullsized":"2024-04-20T10:34:00.309895+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:29.442878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307970+0000","last_change":"2024-04-20T10:31:35.619594+0000","last_active":"2024-04-20T10:34:00.307970+0000","last_peered":"2024-04-20T10:34:00.307970+0000","last_clean":"2024-04-20T10:34:00.307970+0000","last_became_active":"2024-04-20T10:31:35.618831+0000","last_became_peered":"2024-04-20T10:31:35.618831+0000","last_unstale":"2024-04-20T10:34:00.307970+0000","last_undegraded":"2024-04-20T10:34:00.307970+0000","last_fullsized":"2024-04-20T10:34:00.307970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:07.625225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313280+0000","last_change":"2024-04-20T10:31:35.647476+0000","last_active":"2024-04-20T10:34:00.313280+0000","last_peered":"2024-04-20T10:34:00.313280+0000","last_clean":"2024-04-20T10:34:00.313280+0000","last_became_active":"2024-04-20T10:31:35.647009+0000","last_became_peered":"2024-04-20T10:31:35.647009+0000","last_unstale":"2024-04-20T10:34:00.313280+0000","last_undegraded":"2024-04-20T10:34:00.313280+0000","last_fullsized":"2024-04-20T10:34:00.313280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:11:37.526115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313153+0000","last_change":"2024-04-20T10:32:31.958321+0000","last_active":"2024-04-20T10:34:00.313153+0000","last_peered":"2024-04-20T10:34:00.313153+0000","last_clean":"2024-04-20T10:34:00.313153+0000","last_became_active":"2024-04-20T10:32:31.957681+0000","last_became_peered":"2024-04-20T10:32:31.957681+0000","last_unstale":"2024-04-20T10:34:00.313153+0000","last_undegraded":"2024-04-20T10:34:00.313153+0000","last_fullsized":"2024-04-20T10:34:00.313153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:49:31.154909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650987+0000","last_change":"2024-04-20T10:31:35.643142+0000","last_active":"2024-04-20T10:34:00.650987+0000","last_peered":"2024-04-20T10:34:00.650987+0000","last_clean":"2024-04-20T10:34:00.650987+0000","last_became_active":"2024-04-20T10:31:35.642742+0000","last_became_peered":"2024-04-20T10:31:35.642742+0000","last_unstale":"2024-04-20T10:34:00.650987+0000","last_undegraded":"2024-04-20T10:34:00.650987+0000","last_fullsized":"2024-04-20T10:34:00.650987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:40.818808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.280039+0000","last_change":"2024-04-20T10:31:35.650333+0000","last_active":"2024-04-20T10:34:01.280039+0000","last_peered":"2024-04-20T10:34:01.280039+0000","last_clean":"2024-04-20T10:34:01.280039+0000","last_became_active":"2024-04-20T10:31:35.649933+0000","last_became_peered":"2024-04-20T10:31:35.649933+0000","last_unstale":"2024-04-20T10:34:01.280039+0000","last_undegraded":"2024-04-20T10:34:01.280039+0000","last_fullsized":"2024-04-20T10:34:01.280039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:34.513990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.315466+0000","last_change":"2024-04-20T10:31:35.637315+0000","last_active":"2024-04-20T10:34:00.315466+0000","last_peered":"2024-04-20T10:34:00.315466+0000","last_clean":"2024-04-20T10:34:00.315466+0000","last_became_active":"2024-04-20T10:31:35.635909+0000","last_became_peered":"2024-04-20T10:31:35.635909+0000","last_unstale":"2024-04-20T10:34:00.315466+0000","last_undegraded":"2024-04-20T10:34:00.315466+0000","last_fullsized":"2024-04-20T10:34:00.315466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:24:07.400032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650759+0000","last_change":"2024-04-20T10:31:35.638888+0000","last_active":"2024-04-20T10:34:00.650759+0000","last_peered":"2024-04-20T10:34:00.650759+0000","last_clean":"2024-04-20T10:34:00.650759+0000","last_became_active":"2024-04-20T10:31:35.638568+0000","last_became_peered":"2024-04-20T10:31:35.638568+0000","last_unstale":"2024-04-20T10:34:00.650759+0000","last_undegraded":"2024-04-20T10:34:00.650759+0000","last_fullsized":"2024-04-20T10:34:00.650759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:48.363874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.315517+0000","last_change":"2024-04-20T10:31:34.597814+0000","last_active":"2024-04-20T10:34:00.315517+0000","last_peered":"2024-04-20T10:34:00.315517+0000","last_clean":"2024-04-20T10:34:00.315517+0000","last_became_active":"2024-04-20T10:31:34.597480+0000","last_became_peered":"2024-04-20T10:31:34.597480+0000","last_unstale":"2024-04-20T10:34:00.315517+0000","last_undegraded":"2024-04-20T10:34:00.315517+0000","last_fullsized":"2024-04-20T10:34:00.315517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:48:26.102787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.309886+0000","last_change":"2024-04-20T10:32:31.956771+0000","last_active":"2024-04-20T10:34:00.309886+0000","last_peered":"2024-04-20T10:34:00.309886+0000","last_clean":"2024-04-20T10:34:00.309886+0000","last_became_active":"2024-04-20T10:32:31.955413+0000","last_became_peered":"2024-04-20T10:32:31.955413+0000","last_unstale":"2024-04-20T10:34:00.309886+0000","last_undegraded":"2024-04-20T10:34:00.309886+0000","last_fullsized":"2024-04-20T10:34:00.309886+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:37.199764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.438965+0000","last_change":"2024-04-20T10:31:34.609942+0000","last_active":"2024-04-20T10:34:00.438965+0000","last_peered":"2024-04-20T10:34:00.438965+0000","last_clean":"2024-04-20T10:34:00.438965+0000","last_became_active":"2024-04-20T10:31:34.608994+0000","last_became_peered":"2024-04-20T10:31:34.608994+0000","last_unstale":"2024-04-20T10:34:00.438965+0000","last_undegraded":"2024-04-20T10:34:00.438965+0000","last_fullsized":"2024-04-20T10:34:00.438965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:26:53.488091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.314873+0000","last_change":"2024-04-20T10:31:35.647710+0000","last_active":"2024-04-20T10:34:00.314873+0000","last_peered":"2024-04-20T10:34:00.314873+0000","last_clean":"2024-04-20T10:34:00.314873+0000","last_became_active":"2024-04-20T10:31:35.646037+0000","last_became_peered":"2024-04-20T10:31:35.646037+0000","last_unstale":"2024-04-20T10:34:00.314873+0000","last_undegraded":"2024-04-20T10:34:00.314873+0000","last_fullsized":"2024-04-20T10:34:00.314873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:50:21.092892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446278+0000","last_change":"2024-04-20T10:31:34.593529+0000","last_active":"2024-04-20T10:34:00.446278+0000","last_peered":"2024-04-20T10:34:00.446278+0000","last_clean":"2024-04-20T10:34:00.446278+0000","last_became_active":"2024-04-20T10:31:34.592583+0000","last_became_peered":"2024-04-20T10:31:34.592583+0000","last_unstale":"2024-04-20T10:34:00.446278+0000","last_undegraded":"2024-04-20T10:34:00.446278+0000","last_fullsized":"2024-04-20T10:34:00.446278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:51:20.570499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.651292+0000","last_change":"2024-04-20T10:31:35.630739+0000","last_active":"2024-04-20T10:34:00.651292+0000","last_peered":"2024-04-20T10:34:00.651292+0000","last_clean":"2024-04-20T10:34:00.651292+0000","last_became_active":"2024-04-20T10:31:35.630424+0000","last_became_peered":"2024-04-20T10:31:35.630424+0000","last_unstale":"2024-04-20T10:34:00.651292+0000","last_undegraded":"2024-04-20T10:34:00.651292+0000","last_fullsized":"2024-04-20T10:34:00.651292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:48:17.485677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446129+0000","last_change":"2024-04-20T10:31:34.591652+0000","last_active":"2024-04-20T10:34:00.446129+0000","last_peered":"2024-04-20T10:34:00.446129+0000","last_clean":"2024-04-20T10:34:00.446129+0000","last_became_active":"2024-04-20T10:31:34.588945+0000","last_became_peered":"2024-04-20T10:31:34.588945+0000","last_unstale":"2024-04-20T10:34:00.446129+0000","last_undegraded":"2024-04-20T10:34:00.446129+0000","last_fullsized":"2024-04-20T10:34:00.446129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:15.137046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456803+0000","last_change":"2024-04-20T10:31:35.622462+0000","last_active":"2024-04-20T10:34:00.456803+0000","last_peered":"2024-04-20T10:34:00.456803+0000","last_clean":"2024-04-20T10:34:00.456803+0000","last_became_active":"2024-04-20T10:31:35.622125+0000","last_became_peered":"2024-04-20T10:31:35.622125+0000","last_unstale":"2024-04-20T10:34:00.456803+0000","last_undegraded":"2024-04-20T10:34:00.456803+0000","last_fullsized":"2024-04-20T10:34:00.456803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:06:27.894564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456509+0000","last_change":"2024-04-20T10:32:32.043629+0000","last_active":"2024-04-20T10:34:00.456509+0000","last_peered":"2024-04-20T10:34:00.456509+0000","last_clean":"2024-04-20T10:34:00.456509+0000","last_became_active":"2024-04-20T10:32:32.042849+0000","last_became_peered":"2024-04-20T10:32:32.042849+0000","last_unstale":"2024-04-20T10:34:00.456509+0000","last_undegraded":"2024-04-20T10:34:00.456509+0000","last_fullsized":"2024-04-20T10:34:00.456509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:52.646201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.445985+0000","last_change":"2024-04-20T10:32:31.951658+0000","last_active":"2024-04-20T10:34:00.445985+0000","last_peered":"2024-04-20T10:34:00.445985+0000","last_clean":"2024-04-20T10:34:00.445985+0000","last_became_active":"2024-04-20T10:32:31.951307+0000","last_became_peered":"2024-04-20T10:32:31.951307+0000","last_unstale":"2024-04-20T10:34:00.445985+0000","last_undegraded":"2024-04-20T10:34:00.445985+0000","last_fullsized":"2024-04-20T10:34:00.445985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:42.743969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.435759+0000","last_change":"2024-04-20T10:31:34.612038+0000","last_active":"2024-04-20T10:34:00.435759+0000","last_peered":"2024-04-20T10:34:00.435759+0000","last_clean":"2024-04-20T10:34:00.435759+0000","last_became_active":"2024-04-20T10:31:34.611671+0000","last_became_peered":"2024-04-20T10:31:34.611671+0000","last_unstale":"2024-04-20T10:34:00.435759+0000","last_undegraded":"2024-04-20T10:34:00.435759+0000","last_fullsized":"2024-04-20T10:34:00.435759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:47.862702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456602+0000","last_change":"2024-04-20T10:31:35.643930+0000","last_active":"2024-04-20T10:34:00.456602+0000","last_peered":"2024-04-20T10:34:00.456602+0000","last_clean":"2024-04-20T10:34:00.456602+0000","last_became_active":"2024-04-20T10:31:35.643470+0000","last_became_peered":"2024-04-20T10:31:35.643470+0000","last_unstale":"2024-04-20T10:34:00.456602+0000","last_undegraded":"2024-04-20T10:34:00.456602+0000","last_fullsized":"2024-04-20T10:34:00.456602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:52:55.570176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307827+0000","last_change":"2024-04-20T10:31:34.613224+0000","last_active":"2024-04-20T10:34:00.307827+0000","last_peered":"2024-04-20T10:34:00.307827+0000","last_clean":"2024-04-20T10:34:00.307827+0000","last_became_active":"2024-04-20T10:31:34.612871+0000","last_became_peered":"2024-04-20T10:31:34.612871+0000","last_unstale":"2024-04-20T10:34:00.307827+0000","last_undegraded":"2024-04-20T10:34:00.307827+0000","last_fullsized":"2024-04-20T10:34:00.307827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:05:57.284870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.438784+0000","last_change":"2024-04-20T10:31:35.643824+0000","last_active":"2024-04-20T10:34:00.438784+0000","last_peered":"2024-04-20T10:34:00.438784+0000","last_clean":"2024-04-20T10:34:00.438784+0000","last_became_active":"2024-04-20T10:31:35.643425+0000","last_became_peered":"2024-04-20T10:31:35.643425+0000","last_unstale":"2024-04-20T10:34:00.438784+0000","last_undegraded":"2024-04-20T10:34:00.438784+0000","last_fullsized":"2024-04-20T10:34:00.438784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:31:34.008374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456641+0000","last_change":"2024-04-20T10:31:34.593071+0000","last_active":"2024-04-20T10:34:00.456641+0000","last_peered":"2024-04-20T10:34:00.456641+0000","last_clean":"2024-04-20T10:34:00.456641+0000","last_became_active":"2024-04-20T10:31:34.592721+0000","last_became_peered":"2024-04-20T10:31:34.592721+0000","last_unstale":"2024-04-20T10:34:00.456641+0000","last_undegraded":"2024-04-20T10:34:00.456641+0000","last_fullsized":"2024-04-20T10:34:00.456641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:23:43.749586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.310079+0000","last_change":"2024-04-20T10:31:35.647524+0000","last_active":"2024-04-20T10:34:00.310079+0000","last_peered":"2024-04-20T10:34:00.310079+0000","last_clean":"2024-04-20T10:34:00.310079+0000","last_became_active":"2024-04-20T10:31:35.647122+0000","last_became_peered":"2024-04-20T10:31:35.647122+0000","last_unstale":"2024-04-20T10:34:00.310079+0000","last_undegraded":"2024-04-20T10:34:00.310079+0000","last_fullsized":"2024-04-20T10:34:00.310079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.931211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.436118+0000","last_change":"2024-04-20T10:31:34.604500+0000","last_active":"2024-04-20T10:34:00.436118+0000","last_peered":"2024-04-20T10:34:00.436118+0000","last_clean":"2024-04-20T10:34:00.436118+0000","last_became_active":"2024-04-20T10:31:34.604148+0000","last_became_peered":"2024-04-20T10:31:34.604148+0000","last_unstale":"2024-04-20T10:34:00.436118+0000","last_undegraded":"2024-04-20T10:34:00.436118+0000","last_fullsized":"2024-04-20T10:34:00.436118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:49:15.174414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.652037+0000","last_change":"2024-04-20T10:31:35.636751+0000","last_active":"2024-04-20T10:34:00.652037+0000","last_peered":"2024-04-20T10:34:00.652037+0000","last_clean":"2024-04-20T10:34:00.652037+0000","last_became_active":"2024-04-20T10:31:35.636103+0000","last_became_peered":"2024-04-20T10:31:35.636103+0000","last_unstale":"2024-04-20T10:34:00.652037+0000","last_undegraded":"2024-04-20T10:34:00.652037+0000","last_fullsized":"2024-04-20T10:34:00.652037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:22:40.580512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.273088+0000","last_change":"2024-04-20T10:31:34.595207+0000","last_active":"2024-04-20T10:34:01.273088+0000","last_peered":"2024-04-20T10:34:01.273088+0000","last_clean":"2024-04-20T10:34:01.273088+0000","last_became_active":"2024-04-20T10:31:34.594860+0000","last_became_peered":"2024-04-20T10:31:34.594860+0000","last_unstale":"2024-04-20T10:34:01.273088+0000","last_undegraded":"2024-04-20T10:34:01.273088+0000","last_fullsized":"2024-04-20T10:34:01.273088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:29:50.054454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.314770+0000","last_change":"2024-04-20T10:31:35.648086+0000","last_active":"2024-04-20T10:34:00.314770+0000","last_peered":"2024-04-20T10:34:00.314770+0000","last_clean":"2024-04-20T10:34:00.314770+0000","last_became_active":"2024-04-20T10:31:35.647275+0000","last_became_peered":"2024-04-20T10:31:35.647275+0000","last_unstale":"2024-04-20T10:34:00.314770+0000","last_undegraded":"2024-04-20T10:34:00.314770+0000","last_fullsized":"2024-04-20T10:34:00.314770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:27.669008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.457023+0000","last_change":"2024-04-20T10:31:34.601674+0000","last_active":"2024-04-20T10:34:00.457023+0000","last_peered":"2024-04-20T10:34:00.457023+0000","last_clean":"2024-04-20T10:34:00.457023+0000","last_became_active":"2024-04-20T10:31:34.601255+0000","last_became_peered":"2024-04-20T10:31:34.601255+0000","last_unstale":"2024-04-20T10:34:00.457023+0000","last_undegraded":"2024-04-20T10:34:00.457023+0000","last_fullsized":"2024-04-20T10:34:00.457023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:57.821635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.651600+0000","last_change":"2024-04-20T10:31:35.621888+0000","last_active":"2024-04-20T10:34:00.651600+0000","last_peered":"2024-04-20T10:34:00.651600+0000","last_clean":"2024-04-20T10:34:00.651600+0000","last_became_active":"2024-04-20T10:31:35.621503+0000","last_became_peered":"2024-04-20T10:31:35.621503+0000","last_unstale":"2024-04-20T10:34:00.651600+0000","last_undegraded":"2024-04-20T10:34:00.651600+0000","last_fullsized":"2024-04-20T10:34:00.651600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:15:21.118638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.185784+0000","last_change":"2024-04-20T10:31:34.590558+0000","last_active":"2024-04-20T10:34:01.185784+0000","last_peered":"2024-04-20T10:34:01.185784+0000","last_clean":"2024-04-20T10:34:01.185784+0000","last_became_active":"2024-04-20T10:31:34.590140+0000","last_became_peered":"2024-04-20T10:31:34.590140+0000","last_unstale":"2024-04-20T10:34:01.185784+0000","last_undegraded":"2024-04-20T10:34:01.185784+0000","last_fullsized":"2024-04-20T10:34:01.185784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:15:18.591748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67760,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":297,"num_read_kb":215,"num_write":34,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446214+0000","last_change":"2024-04-20T10:31:35.620780+0000","last_active":"2024-04-20T10:34:00.446214+0000","last_peered":"2024-04-20T10:34:00.446214+0000","last_clean":"2024-04-20T10:34:00.446214+0000","last_became_active":"2024-04-20T10:31:35.620429+0000","last_became_peered":"2024-04-20T10:31:35.620429+0000","last_unstale":"2024-04-20T10:34:00.446214+0000","last_undegraded":"2024-04-20T10:34:00.446214+0000","last_fullsized":"2024-04-20T10:34:00.446214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:47:45.587920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650597+0000","last_change":"2024-04-20T10:31:34.602135+0000","last_active":"2024-04-20T10:34:00.650597+0000","last_peered":"2024-04-20T10:34:00.650597+0000","last_clean":"2024-04-20T10:34:00.650597+0000","last_became_active":"2024-04-20T10:31:34.601460+0000","last_became_peered":"2024-04-20T10:31:34.601460+0000","last_unstale":"2024-04-20T10:34:00.650597+0000","last_undegraded":"2024-04-20T10:34:00.650597+0000","last_fullsized":"2024-04-20T10:34:00.650597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:00:00.832702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.308031+0000","last_change":"2024-04-20T10:31:35.635526+0000","last_active":"2024-04-20T10:34:00.308031+0000","last_peered":"2024-04-20T10:34:00.308031+0000","last_clean":"2024-04-20T10:34:00.308031+0000","last_became_active":"2024-04-20T10:31:35.635143+0000","last_became_peered":"2024-04-20T10:31:35.635143+0000","last_unstale":"2024-04-20T10:34:00.308031+0000","last_undegraded":"2024-04-20T10:34:00.308031+0000","last_fullsized":"2024-04-20T10:34:00.308031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:57:24.509318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456493+0000","last_change":"2024-04-20T10:31:34.604561+0000","last_active":"2024-04-20T10:34:00.456493+0000","last_peered":"2024-04-20T10:34:00.456493+0000","last_clean":"2024-04-20T10:34:00.456493+0000","last_became_active":"2024-04-20T10:31:34.604232+0000","last_became_peered":"2024-04-20T10:31:34.604232+0000","last_unstale":"2024-04-20T10:34:00.456493+0000","last_undegraded":"2024-04-20T10:34:00.456493+0000","last_fullsized":"2024-04-20T10:34:00.456493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:26:39.317473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272869+0000","last_change":"2024-04-20T10:31:35.632840+0000","last_active":"2024-04-20T10:34:01.272869+0000","last_peered":"2024-04-20T10:34:01.272869+0000","last_clean":"2024-04-20T10:34:01.272869+0000","last_became_active":"2024-04-20T10:31:35.632471+0000","last_became_peered":"2024-04-20T10:31:35.632471+0000","last_unstale":"2024-04-20T10:34:01.272869+0000","last_undegraded":"2024-04-20T10:34:01.272869+0000","last_fullsized":"2024-04-20T10:34:01.272869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:06.679967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272715+0000","last_change":"2024-04-20T10:31:34.591696+0000","last_active":"2024-04-20T10:34:01.272715+0000","last_peered":"2024-04-20T10:34:01.272715+0000","last_clean":"2024-04-20T10:34:01.272715+0000","last_became_active":"2024-04-20T10:31:34.591330+0000","last_became_peered":"2024-04-20T10:31:34.591330+0000","last_unstale":"2024-04-20T10:34:01.272715+0000","last_undegraded":"2024-04-20T10:34:01.272715+0000","last_fullsized":"2024-04-20T10:34:01.272715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:44:36.385963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272119+0000","last_change":"2024-04-20T10:31:35.627201+0000","last_active":"2024-04-20T10:34:01.272119+0000","last_peered":"2024-04-20T10:34:01.272119+0000","last_clean":"2024-04-20T10:34:01.272119+0000","last_became_active":"2024-04-20T10:31:35.626880+0000","last_became_peered":"2024-04-20T10:31:35.626880+0000","last_unstale":"2024-04-20T10:34:01.272119+0000","last_undegraded":"2024-04-20T10:34:01.272119+0000","last_fullsized":"2024-04-20T10:34:01.272119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:45:44.562723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.445834+0000","last_change":"2024-04-20T10:31:34.600683+0000","last_active":"2024-04-20T10:34:00.445834+0000","last_peered":"2024-04-20T10:34:00.445834+0000","last_clean":"2024-04-20T10:34:00.445834+0000","last_became_active":"2024-04-20T10:31:34.600305+0000","last_became_peered":"2024-04-20T10:31:34.600305+0000","last_unstale":"2024-04-20T10:34:00.445834+0000","last_undegraded":"2024-04-20T10:34:00.445834+0000","last_fullsized":"2024-04-20T10:34:00.445834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:05.441158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456458+0000","last_change":"2024-04-20T10:31:35.619835+0000","last_active":"2024-04-20T10:34:00.456458+0000","last_peered":"2024-04-20T10:34:00.456458+0000","last_clean":"2024-04-20T10:34:00.456458+0000","last_became_active":"2024-04-20T10:31:35.619554+0000","last_became_peered":"2024-04-20T10:31:35.619554+0000","last_unstale":"2024-04-20T10:34:00.456458+0000","last_undegraded":"2024-04-20T10:34:00.456458+0000","last_fullsized":"2024-04-20T10:34:00.456458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:33.161563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446313+0000","last_change":"2024-04-20T10:31:34.591128+0000","last_active":"2024-04-20T10:34:00.446313+0000","last_peered":"2024-04-20T10:34:00.446313+0000","last_clean":"2024-04-20T10:34:00.446313+0000","last_became_active":"2024-04-20T10:31:34.587772+0000","last_became_peered":"2024-04-20T10:31:34.587772+0000","last_unstale":"2024-04-20T10:34:00.446313+0000","last_undegraded":"2024-04-20T10:34:00.446313+0000","last_fullsized":"2024-04-20T10:34:00.446313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:12:42.541111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279821+0000","last_change":"2024-04-20T10:31:35.616743+0000","last_active":"2024-04-20T10:34:01.279821+0000","last_peered":"2024-04-20T10:34:01.279821+0000","last_clean":"2024-04-20T10:34:01.279821+0000","last_became_active":"2024-04-20T10:31:35.616419+0000","last_became_peered":"2024-04-20T10:31:35.616419+0000","last_unstale":"2024-04-20T10:34:01.279821+0000","last_undegraded":"2024-04-20T10:34:01.279821+0000","last_fullsized":"2024-04-20T10:34:01.279821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:59:37.781545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.310071+0000","last_change":"2024-04-20T10:31:34.603734+0000","last_active":"2024-04-20T10:34:00.310071+0000","last_peered":"2024-04-20T10:34:00.310071+0000","last_clean":"2024-04-20T10:34:00.310071+0000","last_became_active":"2024-04-20T10:31:34.603388+0000","last_became_peered":"2024-04-20T10:31:34.603388+0000","last_unstale":"2024-04-20T10:34:00.310071+0000","last_undegraded":"2024-04-20T10:34:00.310071+0000","last_fullsized":"2024-04-20T10:34:00.310071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:58:57.199173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.436119+0000","last_change":"2024-04-20T10:31:35.638996+0000","last_active":"2024-04-20T10:34:00.436119+0000","last_peered":"2024-04-20T10:34:00.436119+0000","last_clean":"2024-04-20T10:34:00.436119+0000","last_became_active":"2024-04-20T10:31:35.638571+0000","last_became_peered":"2024-04-20T10:31:35.638571+0000","last_unstale":"2024-04-20T10:34:00.436119+0000","last_undegraded":"2024-04-20T10:34:00.436119+0000","last_fullsized":"2024-04-20T10:34:00.436119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:59.541738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.273244+0000","last_change":"2024-04-20T10:31:34.612501+0000","last_active":"2024-04-20T10:34:01.273244+0000","last_peered":"2024-04-20T10:34:01.273244+0000","last_clean":"2024-04-20T10:34:01.273244+0000","last_became_active":"2024-04-20T10:31:34.612150+0000","last_became_peered":"2024-04-20T10:31:34.612150+0000","last_unstale":"2024-04-20T10:34:01.273244+0000","last_undegraded":"2024-04-20T10:34:01.273244+0000","last_fullsized":"2024-04-20T10:34:01.273244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:28.172851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.435952+0000","last_change":"2024-04-20T10:32:31.957767+0000","last_active":"2024-04-20T10:34:00.435952+0000","last_peered":"2024-04-20T10:34:00.435952+0000","last_clean":"2024-04-20T10:34:00.435952+0000","last_became_active":"2024-04-20T10:32:31.957319+0000","last_became_peered":"2024-04-20T10:32:31.957319+0000","last_unstale":"2024-04-20T10:34:00.435952+0000","last_undegraded":"2024-04-20T10:34:00.435952+0000","last_fullsized":"2024-04-20T10:34:00.435952+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:56.375705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456794+0000","last_change":"2024-04-20T10:31:35.617775+0000","last_active":"2024-04-20T10:34:00.456794+0000","last_peered":"2024-04-20T10:34:00.456794+0000","last_clean":"2024-04-20T10:34:00.456794+0000","last_became_active":"2024-04-20T10:31:35.617465+0000","last_became_peered":"2024-04-20T10:31:35.617465+0000","last_unstale":"2024-04-20T10:34:00.456794+0000","last_undegraded":"2024-04-20T10:34:00.456794+0000","last_fullsized":"2024-04-20T10:34:00.456794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:37.830953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307653+0000","last_change":"2024-04-20T10:31:34.615044+0000","last_active":"2024-04-20T10:34:00.307653+0000","last_peered":"2024-04-20T10:34:00.307653+0000","last_clean":"2024-04-20T10:34:00.307653+0000","last_became_active":"2024-04-20T10:31:34.612875+0000","last_became_peered":"2024-04-20T10:31:34.612875+0000","last_unstale":"2024-04-20T10:34:00.307653+0000","last_undegraded":"2024-04-20T10:34:00.307653+0000","last_fullsized":"2024-04-20T10:34:00.307653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:59.867501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439537+0000","last_change":"2024-04-20T10:31:35.630909+0000","last_active":"2024-04-20T10:34:00.439537+0000","last_peered":"2024-04-20T10:34:00.439537+0000","last_clean":"2024-04-20T10:34:00.439537+0000","last_became_active":"2024-04-20T10:31:35.630455+0000","last_became_peered":"2024-04-20T10:31:35.630455+0000","last_unstale":"2024-04-20T10:34:00.439537+0000","last_undegraded":"2024-04-20T10:34:00.439537+0000","last_fullsized":"2024-04-20T10:34:00.439537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:02.522485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272616+0000","last_change":"2024-04-20T10:31:34.602466+0000","last_active":"2024-04-20T10:34:01.272616+0000","last_peered":"2024-04-20T10:34:01.272616+0000","last_clean":"2024-04-20T10:34:01.272616+0000","last_became_active":"2024-04-20T10:31:34.602089+0000","last_became_peered":"2024-04-20T10:31:34.602089+0000","last_unstale":"2024-04-20T10:34:01.272616+0000","last_undegraded":"2024-04-20T10:34:01.272616+0000","last_fullsized":"2024-04-20T10:34:01.272616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:19:09.849840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272126+0000","last_change":"2024-04-20T10:30:14.724347+0000","last_active":"2024-04-20T10:34:01.272126+0000","last_peered":"2024-04-20T10:34:01.272126+0000","last_clean":"2024-04-20T10:34:01.272126+0000","last_became_active":"2024-04-20T10:30:14.723877+0000","last_became_peered":"2024-04-20T10:30:14.723877+0000","last_unstale":"2024-04-20T10:34:01.272126+0000","last_undegraded":"2024-04-20T10:34:01.272126+0000","last_fullsized":"2024-04-20T10:34:01.272126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:26:16.803986+0000","last_clean_scrub_stamp":"2024-04-20T10:26:16.803986+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:30.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307565+0000","last_change":"2024-04-20T10:31:35.626505+0000","last_active":"2024-04-20T10:34:00.307565+0000","last_peered":"2024-04-20T10:34:00.307565+0000","last_clean":"2024-04-20T10:34:00.307565+0000","last_became_active":"2024-04-20T10:31:35.626141+0000","last_became_peered":"2024-04-20T10:31:35.626141+0000","last_unstale":"2024-04-20T10:34:00.307565+0000","last_undegraded":"2024-04-20T10:34:00.307565+0000","last_fullsized":"2024-04-20T10:34:00.307565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:29.827225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272242+0000","last_change":"2024-04-20T10:31:34.596936+0000","last_active":"2024-04-20T10:34:01.272242+0000","last_peered":"2024-04-20T10:34:01.272242+0000","last_clean":"2024-04-20T10:34:01.272242+0000","last_became_active":"2024-04-20T10:31:34.596692+0000","last_became_peered":"2024-04-20T10:31:34.596692+0000","last_unstale":"2024-04-20T10:34:01.272242+0000","last_undegraded":"2024-04-20T10:34:01.272242+0000","last_fullsized":"2024-04-20T10:34:01.272242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:02:39.819096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.446098+0000","last_change":"2024-04-20T10:31:34.601407+0000","last_active":"2024-04-20T10:34:00.446098+0000","last_peered":"2024-04-20T10:34:00.446098+0000","last_clean":"2024-04-20T10:34:00.446098+0000","last_became_active":"2024-04-20T10:31:34.601007+0000","last_became_peered":"2024-04-20T10:31:34.601007+0000","last_unstale":"2024-04-20T10:34:00.446098+0000","last_undegraded":"2024-04-20T10:34:00.446098+0000","last_fullsized":"2024-04-20T10:34:00.446098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:05:03.037841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.308057+0000","last_change":"2024-04-20T10:31:35.641046+0000","last_active":"2024-04-20T10:34:00.308057+0000","last_peered":"2024-04-20T10:34:00.308057+0000","last_clean":"2024-04-20T10:34:00.308057+0000","last_became_active":"2024-04-20T10:31:35.640666+0000","last_became_peered":"2024-04-20T10:31:35.640666+0000","last_unstale":"2024-04-20T10:34:00.308057+0000","last_undegraded":"2024-04-20T10:34:00.308057+0000","last_fullsized":"2024-04-20T10:34:00.308057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:58:42.496037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'45","reported_seq":90,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.606523+0000","last_change":"2024-04-20T10:31:34.604643+0000","last_active":"2024-04-20T10:34:00.606523+0000","last_peered":"2024-04-20T10:34:00.606523+0000","last_clean":"2024-04-20T10:34:00.606523+0000","last_became_active":"2024-04-20T10:31:34.604210+0000","last_became_peered":"2024-04-20T10:31:34.604210+0000","last_unstale":"2024-04-20T10:34:00.606523+0000","last_undegraded":"2024-04-20T10:34:00.606523+0000","last_fullsized":"2024-04-20T10:34:00.606523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:24:09.748012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":81,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439226+0000","last_change":"2024-04-20T10:31:35.641981+0000","last_active":"2024-04-20T10:34:00.439226+0000","last_peered":"2024-04-20T10:34:00.439226+0000","last_clean":"2024-04-20T10:34:00.439226+0000","last_became_active":"2024-04-20T10:31:35.641607+0000","last_became_peered":"2024-04-20T10:31:35.641607+0000","last_unstale":"2024-04-20T10:34:00.439226+0000","last_undegraded":"2024-04-20T10:34:00.439226+0000","last_fullsized":"2024-04-20T10:34:00.439226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:48.520813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272757+0000","last_change":"2024-04-20T10:31:34.611218+0000","last_active":"2024-04-20T10:34:01.272757+0000","last_peered":"2024-04-20T10:34:01.272757+0000","last_clean":"2024-04-20T10:34:01.272757+0000","last_became_active":"2024-04-20T10:31:34.610950+0000","last_became_peered":"2024-04-20T10:31:34.610950+0000","last_unstale":"2024-04-20T10:34:01.272757+0000","last_undegraded":"2024-04-20T10:34:01.272757+0000","last_fullsized":"2024-04-20T10:34:01.272757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:14:11.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313611+0000","last_change":"2024-04-20T10:31:35.647531+0000","last_active":"2024-04-20T10:34:00.313611+0000","last_peered":"2024-04-20T10:34:00.313611+0000","last_clean":"2024-04-20T10:34:00.313611+0000","last_became_active":"2024-04-20T10:31:35.646389+0000","last_became_peered":"2024-04-20T10:31:35.646389+0000","last_unstale":"2024-04-20T10:34:00.313611+0000","last_undegraded":"2024-04-20T10:34:00.313611+0000","last_fullsized":"2024-04-20T10:34:00.313611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:40:50.141385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.651038+0000","last_change":"2024-04-20T10:32:31.957675+0000","last_active":"2024-04-20T10:34:00.651038+0000","last_peered":"2024-04-20T10:34:00.651038+0000","last_clean":"2024-04-20T10:34:00.651038+0000","last_became_active":"2024-04-20T10:32:31.957202+0000","last_became_peered":"2024-04-20T10:32:31.957202+0000","last_unstale":"2024-04-20T10:34:00.651038+0000","last_undegraded":"2024-04-20T10:34:00.651038+0000","last_fullsized":"2024-04-20T10:34:00.651038+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:57:28.001088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.445870+0000","last_change":"2024-04-20T10:31:35.616751+0000","last_active":"2024-04-20T10:34:00.445870+0000","last_peered":"2024-04-20T10:34:00.445870+0000","last_clean":"2024-04-20T10:34:00.445870+0000","last_became_active":"2024-04-20T10:31:35.616432+0000","last_became_peered":"2024-04-20T10:31:35.616432+0000","last_unstale":"2024-04-20T10:34:00.445870+0000","last_undegraded":"2024-04-20T10:34:00.445870+0000","last_fullsized":"2024-04-20T10:34:00.445870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:25.052140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439413+0000","last_change":"2024-04-20T10:31:34.609139+0000","last_active":"2024-04-20T10:34:00.439413+0000","last_peered":"2024-04-20T10:34:00.439413+0000","last_clean":"2024-04-20T10:34:00.439413+0000","last_became_active":"2024-04-20T10:31:34.608818+0000","last_became_peered":"2024-04-20T10:31:34.608818+0000","last_unstale":"2024-04-20T10:34:00.439413+0000","last_undegraded":"2024-04-20T10:34:00.439413+0000","last_fullsized":"2024-04-20T10:34:00.439413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:42:39.581059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279576+0000","last_change":"2024-04-20T10:31:35.643791+0000","last_active":"2024-04-20T10:34:01.279576+0000","last_peered":"2024-04-20T10:34:01.279576+0000","last_clean":"2024-04-20T10:34:01.279576+0000","last_became_active":"2024-04-20T10:31:35.643390+0000","last_became_peered":"2024-04-20T10:31:35.643390+0000","last_unstale":"2024-04-20T10:34:01.279576+0000","last_undegraded":"2024-04-20T10:34:01.279576+0000","last_fullsized":"2024-04-20T10:34:01.279576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:10.149979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650525+0000","last_change":"2024-04-20T10:31:34.606178+0000","last_active":"2024-04-20T10:34:00.650525+0000","last_peered":"2024-04-20T10:34:00.650525+0000","last_clean":"2024-04-20T10:34:00.650525+0000","last_became_active":"2024-04-20T10:31:34.605817+0000","last_became_peered":"2024-04-20T10:31:34.605817+0000","last_unstale":"2024-04-20T10:34:00.650525+0000","last_undegraded":"2024-04-20T10:34:00.650525+0000","last_fullsized":"2024-04-20T10:34:00.650525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:49:15.929905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439824+0000","last_change":"2024-04-20T10:31:35.649487+0000","last_active":"2024-04-20T10:34:00.439824+0000","last_peered":"2024-04-20T10:34:00.439824+0000","last_clean":"2024-04-20T10:34:00.439824+0000","last_became_active":"2024-04-20T10:31:35.648884+0000","last_became_peered":"2024-04-20T10:31:35.648884+0000","last_unstale":"2024-04-20T10:34:00.439824+0000","last_undegraded":"2024-04-20T10:34:00.439824+0000","last_fullsized":"2024-04-20T10:34:00.439824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:45:07.389100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279544+0000","last_change":"2024-04-20T10:32:31.943005+0000","last_active":"2024-04-20T10:34:01.279544+0000","last_peered":"2024-04-20T10:34:01.279544+0000","last_clean":"2024-04-20T10:34:01.279544+0000","last_became_active":"2024-04-20T10:32:31.940063+0000","last_became_peered":"2024-04-20T10:32:31.940063+0000","last_unstale":"2024-04-20T10:34:01.279544+0000","last_undegraded":"2024-04-20T10:34:01.279544+0000","last_fullsized":"2024-04-20T10:34:01.279544+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:11:06.308028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272443+0000","last_change":"2024-04-20T10:31:35.630256+0000","last_active":"2024-04-20T10:34:01.272443+0000","last_peered":"2024-04-20T10:34:01.272443+0000","last_clean":"2024-04-20T10:34:01.272443+0000","last_became_active":"2024-04-20T10:31:35.629879+0000","last_became_peered":"2024-04-20T10:31:35.629879+0000","last_unstale":"2024-04-20T10:34:01.272443+0000","last_undegraded":"2024-04-20T10:34:01.272443+0000","last_fullsized":"2024-04-20T10:34:01.272443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:51.601923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650706+0000","last_change":"2024-04-20T10:31:34.595149+0000","last_active":"2024-04-20T10:34:00.650706+0000","last_peered":"2024-04-20T10:34:00.650706+0000","last_clean":"2024-04-20T10:34:00.650706+0000","last_became_active":"2024-04-20T10:31:34.594808+0000","last_became_peered":"2024-04-20T10:31:34.594808+0000","last_unstale":"2024-04-20T10:34:00.650706+0000","last_undegraded":"2024-04-20T10:34:00.650706+0000","last_fullsized":"2024-04-20T10:34:00.650706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:51.707710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272938+0000","last_change":"2024-04-20T10:31:35.620612+0000","last_active":"2024-04-20T10:34:01.272938+0000","last_peered":"2024-04-20T10:34:01.272938+0000","last_clean":"2024-04-20T10:34:01.272938+0000","last_became_active":"2024-04-20T10:31:35.620312+0000","last_became_peered":"2024-04-20T10:31:35.620312+0000","last_unstale":"2024-04-20T10:34:01.272938+0000","last_undegraded":"2024-04-20T10:34:01.272938+0000","last_fullsized":"2024-04-20T10:34:01.272938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:37:50.756950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'39","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.604966+0000","last_change":"2024-04-20T10:31:34.604613+0000","last_active":"2024-04-20T10:34:00.604966+0000","last_peered":"2024-04-20T10:34:00.604966+0000","last_clean":"2024-04-20T10:34:00.604966+0000","last_became_active":"2024-04-20T10:31:34.604246+0000","last_became_peered":"2024-04-20T10:31:34.604246+0000","last_unstale":"2024-04-20T10:34:00.604966+0000","last_undegraded":"2024-04-20T10:34:00.604966+0000","last_fullsized":"2024-04-20T10:34:00.604966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:26.521769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.272437+0000","last_change":"2024-04-20T10:31:35.639396+0000","last_active":"2024-04-20T10:34:01.272437+0000","last_peered":"2024-04-20T10:34:01.272437+0000","last_clean":"2024-04-20T10:34:01.272437+0000","last_became_active":"2024-04-20T10:31:35.638979+0000","last_became_peered":"2024-04-20T10:31:35.638979+0000","last_unstale":"2024-04-20T10:34:01.272437+0000","last_undegraded":"2024-04-20T10:34:01.272437+0000","last_fullsized":"2024-04-20T10:34:01.272437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:12:47.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.308292+0000","last_change":"2024-04-20T10:31:34.607420+0000","last_active":"2024-04-20T10:34:00.308292+0000","last_peered":"2024-04-20T10:34:00.308292+0000","last_clean":"2024-04-20T10:34:00.308292+0000","last_became_active":"2024-04-20T10:31:34.602587+0000","last_became_peered":"2024-04-20T10:31:34.602587+0000","last_unstale":"2024-04-20T10:34:00.308292+0000","last_undegraded":"2024-04-20T10:34:00.308292+0000","last_fullsized":"2024-04-20T10:34:00.308292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:33.569746+0000","last_clean_scrub_stamp":"2024-04-20T10:31:33.569746+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:54:56.447944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.273183+0000","last_change":"2024-04-20T10:31:35.624903+0000","last_active":"2024-04-20T10:34:01.273183+0000","last_peered":"2024-04-20T10:34:01.273183+0000","last_clean":"2024-04-20T10:34:01.273183+0000","last_became_active":"2024-04-20T10:31:35.619479+0000","last_became_peered":"2024-04-20T10:31:35.619479+0000","last_unstale":"2024-04-20T10:34:01.273183+0000","last_undegraded":"2024-04-20T10:34:01.273183+0000","last_fullsized":"2024-04-20T10:34:01.273183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:25:40.601983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279614+0000","last_change":"2024-04-20T10:31:35.620351+0000","last_active":"2024-04-20T10:34:01.279614+0000","last_peered":"2024-04-20T10:34:01.279614+0000","last_clean":"2024-04-20T10:34:01.279614+0000","last_became_active":"2024-04-20T10:31:35.620028+0000","last_became_peered":"2024-04-20T10:31:35.620028+0000","last_unstale":"2024-04-20T10:34:01.279614+0000","last_undegraded":"2024-04-20T10:34:01.279614+0000","last_fullsized":"2024-04-20T10:34:01.279614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:51:53.970894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650618+0000","last_change":"2024-04-20T10:31:35.615917+0000","last_active":"2024-04-20T10:34:00.650618+0000","last_peered":"2024-04-20T10:34:00.650618+0000","last_clean":"2024-04-20T10:34:00.650618+0000","last_became_active":"2024-04-20T10:31:35.615582+0000","last_became_peered":"2024-04-20T10:31:35.615582+0000","last_unstale":"2024-04-20T10:34:00.650618+0000","last_undegraded":"2024-04-20T10:34:00.650618+0000","last_fullsized":"2024-04-20T10:34:00.650618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:54:59.754819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650460+0000","last_change":"2024-04-20T10:31:35.650006+0000","last_active":"2024-04-20T10:34:00.650460+0000","last_peered":"2024-04-20T10:34:00.650460+0000","last_clean":"2024-04-20T10:34:00.650460+0000","last_became_active":"2024-04-20T10:31:35.649683+0000","last_became_peered":"2024-04-20T10:31:35.649683+0000","last_unstale":"2024-04-20T10:34:00.650460+0000","last_undegraded":"2024-04-20T10:34:00.650460+0000","last_fullsized":"2024-04-20T10:34:00.650460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:10:01.484809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.313550+0000","last_change":"2024-04-20T10:31:35.646591+0000","last_active":"2024-04-20T10:34:00.313550+0000","last_peered":"2024-04-20T10:34:00.313550+0000","last_clean":"2024-04-20T10:34:00.313550+0000","last_became_active":"2024-04-20T10:31:35.646188+0000","last_became_peered":"2024-04-20T10:31:35.646188+0000","last_unstale":"2024-04-20T10:34:00.313550+0000","last_undegraded":"2024-04-20T10:34:00.313550+0000","last_fullsized":"2024-04-20T10:34:00.313550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:23.412022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.307919+0000","last_change":"2024-04-20T10:31:35.650879+0000","last_active":"2024-04-20T10:34:00.307919+0000","last_peered":"2024-04-20T10:34:00.307919+0000","last_clean":"2024-04-20T10:34:00.307919+0000","last_became_active":"2024-04-20T10:31:35.650486+0000","last_became_peered":"2024-04-20T10:31:35.650486+0000","last_unstale":"2024-04-20T10:34:00.307919+0000","last_undegraded":"2024-04-20T10:34:00.307919+0000","last_fullsized":"2024-04-20T10:34:00.307919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:51.893446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279735+0000","last_change":"2024-04-20T10:31:35.619883+0000","last_active":"2024-04-20T10:34:01.279735+0000","last_peered":"2024-04-20T10:34:01.279735+0000","last_clean":"2024-04-20T10:34:01.279735+0000","last_became_active":"2024-04-20T10:31:35.619628+0000","last_became_peered":"2024-04-20T10:31:35.619628+0000","last_unstale":"2024-04-20T10:34:01.279735+0000","last_undegraded":"2024-04-20T10:34:01.279735+0000","last_fullsized":"2024-04-20T10:34:01.279735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:45.018417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.650973+0000","last_change":"2024-04-20T10:31:35.620773+0000","last_active":"2024-04-20T10:34:00.650973+0000","last_peered":"2024-04-20T10:34:00.650973+0000","last_clean":"2024-04-20T10:34:00.650973+0000","last_became_active":"2024-04-20T10:31:35.620467+0000","last_became_peered":"2024-04-20T10:31:35.620467+0000","last_unstale":"2024-04-20T10:34:00.650973+0000","last_undegraded":"2024-04-20T10:34:00.650973+0000","last_fullsized":"2024-04-20T10:34:00.650973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:31:48.009168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439094+0000","last_change":"2024-04-20T10:32:31.953580+0000","last_active":"2024-04-20T10:34:00.439094+0000","last_peered":"2024-04-20T10:34:00.439094+0000","last_clean":"2024-04-20T10:34:00.439094+0000","last_became_active":"2024-04-20T10:32:31.952985+0000","last_became_peered":"2024-04-20T10:32:31.952985+0000","last_unstale":"2024-04-20T10:34:00.439094+0000","last_undegraded":"2024-04-20T10:34:00.439094+0000","last_fullsized":"2024-04-20T10:34:00.439094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:13:03.994581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.456576+0000","last_change":"2024-04-20T10:31:35.637588+0000","last_active":"2024-04-20T10:34:00.456576+0000","last_peered":"2024-04-20T10:34:00.456576+0000","last_clean":"2024-04-20T10:34:00.456576+0000","last_became_active":"2024-04-20T10:31:35.637319+0000","last_became_peered":"2024-04-20T10:31:35.637319+0000","last_unstale":"2024-04-20T10:34:00.456576+0000","last_undegraded":"2024-04-20T10:34:00.456576+0000","last_fullsized":"2024-04-20T10:34:00.456576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:54.070894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.439287+0000","last_change":"2024-04-20T10:31:35.629551+0000","last_active":"2024-04-20T10:34:00.439287+0000","last_peered":"2024-04-20T10:34:00.439287+0000","last_clean":"2024-04-20T10:34:00.439287+0000","last_became_active":"2024-04-20T10:31:35.628710+0000","last_became_peered":"2024-04-20T10:31:35.628710+0000","last_unstale":"2024-04-20T10:34:00.439287+0000","last_undegraded":"2024-04-20T10:34:00.439287+0000","last_fullsized":"2024-04-20T10:34:00.439287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:31.883788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.651244+0000","last_change":"2024-04-20T10:31:35.629978+0000","last_active":"2024-04-20T10:34:00.651244+0000","last_peered":"2024-04-20T10:34:00.651244+0000","last_clean":"2024-04-20T10:34:00.651244+0000","last_became_active":"2024-04-20T10:31:35.629596+0000","last_became_peered":"2024-04-20T10:31:35.629596+0000","last_unstale":"2024-04-20T10:34:00.651244+0000","last_undegraded":"2024-04-20T10:34:00.651244+0000","last_fullsized":"2024-04-20T10:34:00.651244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:43:02.523099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:00.308137+0000","last_change":"2024-04-20T10:31:35.652481+0000","last_active":"2024-04-20T10:34:00.308137+0000","last_peered":"2024-04-20T10:34:00.308137+0000","last_clean":"2024-04-20T10:34:00.308137+0000","last_became_active":"2024-04-20T10:31:35.651837+0000","last_became_peered":"2024-04-20T10:31:35.651837+0000","last_unstale":"2024-04-20T10:34:00.308137+0000","last_undegraded":"2024-04-20T10:34:00.308137+0000","last_fullsized":"2024-04-20T10:34:00.308137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:53.206954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T10:34:01.279636+0000","last_change":"2024-04-20T10:31:35.622228+0000","last_active":"2024-04-20T10:34:01.279636+0000","last_peered":"2024-04-20T10:34:01.279636+0000","last_clean":"2024-04-20T10:34:01.279636+0000","last_became_active":"2024-04-20T10:31:35.621910+0000","last_became_peered":"2024-04-20T10:31:35.621910+0000","last_unstale":"2024-04-20T10:34:01.279636+0000","last_undegraded":"2024-04-20T10:34:01.279636+0000","last_fullsized":"2024-04-20T10:34:01.279636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:31:34.571945+0000","last_clean_scrub_stamp":"2024-04-20T10:31:34.571945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:47:34.319839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107397,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":593374,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":393,"num_read_kb":298,"num_write":201,"num_write_kb":1405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":1780122,"data_compressed":81447,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":455821,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1110016,"data_stored":1254063,"data_compressed":52532,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808886,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":858604,"data_compressed":27876,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070523,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":858626,"data_compressed":27876,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332161,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":1046888,"data_compressed":41606,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":464923,"data_compressed":8730,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":532566,"data_compressed":10577,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":464896,"data_compressed":8730,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411422,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":522833,"data_compressed":7654,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":989960,"data_compressed":34032,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":455843,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":1187073,"data_compressed":50685,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35799,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":67760,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67794,"data_compressed":1847,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":524920,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":131782,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":199554,"data_compressed":8003,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:34:05.047 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:34:05.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi162 to 176.2M 2024-04-20T10:34:05.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: Unable to set osd_memory_target on smithi162 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-20T10:34:05.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:05.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:05.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.055 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.055 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:05.055 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:04 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:05.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi162 to 176.2M 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: Unable to set osd_memory_target on smithi162 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:05.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:04 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:05.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.298+0000 7f3e79e00640 1 -- 172.21.15.42:0/1332104987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e74102ff0 msgr2=0x7f3e74103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:05.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.298+0000 7f3e79e00640 1 --2- 172.21.15.42:0/1332104987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e74102ff0 0x7f3e74103470 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009900 tx=0x7f3e6002bc40 comp rx=0 tx=0).stop 2024-04-20T10:34:05.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.299+0000 7f3e79e00640 1 -- 172.21.15.42:0/1332104987 shutdown_connections 2024-04-20T10:34:05.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.299+0000 7f3e79e00640 1 --2- 172.21.15.42:0/1332104987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e741039b0 0x7f3e7410a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.299+0000 7f3e79e00640 1 --2- 172.21.15.42:0/1332104987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e74102ff0 0x7f3e74103470 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.299+0000 7f3e79e00640 1 --2- 172.21.15.42:0/1332104987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e74101e00 0x7f3e74102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.299+0000 7f3e79e00640 1 -- 172.21.15.42:0/1332104987 >> 172.21.15.42:0/1332104987 conn(0x7f3e740fd370 msgr2=0x7f3e740ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:05.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.299+0000 7f3e79e00640 1 -- 172.21.15.42:0/1332104987 shutdown_connections 2024-04-20T10:34:05.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.299+0000 7f3e79e00640 1 -- 172.21.15.42:0/1332104987 wait complete. 2024-04-20T10:34:05.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.300+0000 7f3e79e00640 1 Processor -- start 2024-04-20T10:34:05.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.300+0000 7f3e79e00640 1 -- start start 2024-04-20T10:34:05.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.300+0000 7f3e79e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e74101e00 0x7f3e7419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:05.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.300+0000 7f3e79e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741039b0 0x7f3e7419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:05.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.301+0000 7f3e79e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e7419faf0 0x7f3e741a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:05.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.301+0000 7f3e79e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7410dab0 con 0x7f3e7419faf0 2024-04-20T10:34:05.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.301+0000 7f3e79e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7410d930 con 0x7f3e741039b0 2024-04-20T10:34:05.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.301+0000 7f3e79e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7410dc30 con 0x7f3e74101e00 2024-04-20T10:34:05.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.301+0000 7f3e79400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e7419faf0 0x7f3e741a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:05.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.301+0000 7f3e73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741039b0 0x7f3e7419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:05.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.301+0000 7f3e73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741039b0 0x7f3e7419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52080/0 (socket says 172.21.15.42:52080) 2024-04-20T10:34:05.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.301+0000 7f3e73e00640 1 -- 172.21.15.42:0/2837407961 learned_addr learned my addr 172.21.15.42:0/2837407961 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:05.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.301+0000 7f3e78a00640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e74101e00 0x7f3e7419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:05.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.301+0000 7f3e73e00640 1 -- 172.21.15.42:0/2837407961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e74101e00 msgr2=0x7f3e7419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:05.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.301+0000 7f3e73e00640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e74101e00 0x7f3e7419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.302+0000 7f3e73e00640 1 -- 172.21.15.42:0/2837407961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e7419faf0 msgr2=0x7f3e741a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:05.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.302+0000 7f3e73e00640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e7419faf0 0x7f3e741a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.302+0000 7f3e73e00640 1 -- 172.21.15.42:0/2837407961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e741a2550 con 0x7f3e741039b0 2024-04-20T10:34:05.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.302+0000 7f3e73e00640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741039b0 0x7f3e7419aa90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e6002c250 tx=0x7f3e6002c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:05.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.312+0000 7f3e71600640 1 -- 172.21.15.42:0/2837407961 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60041070 con 0x7f3e741039b0 2024-04-20T10:34:05.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.312+0000 7f3e71600640 1 -- 172.21.15.42:0/2837407961 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e60004450 con 0x7f3e741039b0 2024-04-20T10:34:05.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.312+0000 7f3e71600640 1 -- 172.21.15.42:0/2837407961 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60003e50 con 0x7f3e741039b0 2024-04-20T10:34:05.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.313+0000 7f3e79e00640 1 -- 172.21.15.42:0/2837407961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e741a27e0 con 0x7f3e741039b0 2024-04-20T10:34:05.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.313+0000 7f3e79e00640 1 -- 172.21.15.42:0/2837407961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e741a2c50 con 0x7f3e741039b0 2024-04-20T10:34:05.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.315+0000 7f3e79e00640 1 -- 172.21.15.42:0/2837407961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e741a2eb0 con 0x7f3e741039b0 2024-04-20T10:34:05.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.317+0000 7f3e71600640 1 -- 172.21.15.42:0/2837407961 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f3e60044070 con 0x7f3e741039b0 2024-04-20T10:34:05.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.320+0000 7f3e71600640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f3e5c0a3fd0 0x7f3e5c0a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:05.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.320+0000 7f3e71600640 1 -- 172.21.15.42:0/2837407961 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3e600f23e0 con 0x7f3e741039b0 2024-04-20T10:34:05.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.321+0000 7f3e78a00640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f3e5c0a3fd0 0x7f3e5c0a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:05.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.321+0000 7f3e78a00640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f3e5c0a3fd0 0x7f3e5c0a6490 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3e74102e50 tx=0x7f3e68005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:05.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.322+0000 7f3e71600640 1 -- 172.21.15.42:0/2837407961 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3e600be3d0 con 0x7f3e741039b0 2024-04-20T10:34:05.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.535+0000 7f3e79e00640 1 -- 172.21.15.42:0/2837407961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e741a3080 con 0x7f3e741039b0 2024-04-20T10:34:05.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.537+0000 7f3e71600640 1 -- 172.21.15.42:0/2837407961 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23100 (secure 0 0 0) 0x7f3e600c26d0 con 0x7f3e741039b0 2024-04-20T10:34:05.538 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:34:05.538 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":88,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:34:00.299452+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/230562472":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/1321607472":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/3804122415":"2024-04-21T10:34:00.299359+0000","172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/3610568749":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/869471821":"2024-04-21T10:34:00.299359+0000","172.21.15.42:6800/3610568749":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:34:05.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.540+0000 7f3e79e00640 1 -- 172.21.15.42:0/2837407961 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f3e5c0a3fd0 msgr2=0x7f3e5c0a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:05.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.540+0000 7f3e79e00640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f3e5c0a3fd0 0x7f3e5c0a6490 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3e74102e50 tx=0x7f3e68005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:05.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.540+0000 7f3e79e00640 1 -- 172.21.15.42:0/2837407961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741039b0 msgr2=0x7f3e7419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:05.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.540+0000 7f3e79e00640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741039b0 0x7f3e7419aa90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e6002c250 tx=0x7f3e6002c6e0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.540+0000 7f3e79e00640 1 -- 172.21.15.42:0/2837407961 shutdown_connections 2024-04-20T10:34:05.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.540+0000 7f3e79e00640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f3e5c0a3fd0 0x7f3e5c0a6490 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.540+0000 7f3e79e00640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e7419faf0 0x7f3e741a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.540+0000 7f3e79e00640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e741039b0 0x7f3e7419aa90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.540+0000 7f3e79e00640 1 --2- 172.21.15.42:0/2837407961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e74101e00 0x7f3e7419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.540+0000 7f3e79e00640 1 -- 172.21.15.42:0/2837407961 >> 172.21.15.42:0/2837407961 conn(0x7f3e740fd370 msgr2=0x7f3e74074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:05.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.540+0000 7f3e79e00640 1 -- 172.21.15.42:0/2837407961 shutdown_connections 2024-04-20T10:34:05.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.540+0000 7f3e79e00640 1 -- 172.21.15.42:0/2837407961 wait complete. 2024-04-20T10:34:05.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:26:16.537072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:31:33.523692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:31:34.019519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:34:05.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:34:05.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.757+0000 7f4838000640 1 -- 172.21.15.42:0/2069710496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4830105d70 msgr2=0x7f483010c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:05.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.757+0000 7f4838000640 1 --2- 172.21.15.42:0/2069710496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4830105d70 0x7f483010c600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4828009be0 tx=0x7f482802bef0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.757+0000 7f4838000640 1 -- 172.21.15.42:0/2069710496 shutdown_connections 2024-04-20T10:34:05.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.757+0000 7f4838000640 1 --2- 172.21.15.42:0/2069710496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4830105d70 0x7f483010c600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.757+0000 7f4838000640 1 --2- 172.21.15.42:0/2069710496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48301053b0 0x7f4830105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.757+0000 7f4838000640 1 --2- 172.21.15.42:0/2069710496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48301041c0 0x7f48301045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.757+0000 7f4838000640 1 -- 172.21.15.42:0/2069710496 >> 172.21.15.42:0/2069710496 conn(0x7f48300ff750 msgr2=0x7f4830101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:05.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.758+0000 7f4838000640 1 -- 172.21.15.42:0/2069710496 shutdown_connections 2024-04-20T10:34:05.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.758+0000 7f4838000640 1 -- 172.21.15.42:0/2069710496 wait complete. 2024-04-20T10:34:05.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.759+0000 7f4838000640 1 Processor -- start 2024-04-20T10:34:05.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.759+0000 7f4838000640 1 -- start start 2024-04-20T10:34:05.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.760+0000 7f4838000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48301041c0 0x7f483019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:05.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.760+0000 7f4838000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48301053b0 0x7f483019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:05.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.760+0000 7f4838000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4830105d70 0x7f48301a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:05.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.760+0000 7f4838000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f483010fea0 con 0x7f4830105d70 2024-04-20T10:34:05.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.760+0000 7f4838000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f483010fd20 con 0x7f48301053b0 2024-04-20T10:34:05.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.760+0000 7f4838000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4830110020 con 0x7f48301041c0 2024-04-20T10:34:05.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.760+0000 7f4836000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4830105d70 0x7f48301a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:05.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.760+0000 7f4836000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4830105d70 0x7f48301a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34336/0 (socket says 172.21.15.42:34336) 2024-04-20T10:34:05.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.760+0000 7f4836000640 1 -- 172.21.15.42:0/3965204434 learned_addr learned my addr 172.21.15.42:0/3965204434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:05.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.760+0000 7f4834c00640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48301053b0 0x7f483019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:05.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.760+0000 7f4835600640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48301041c0 0x7f483019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:05.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.761+0000 7f4836000640 1 -- 172.21.15.42:0/3965204434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48301041c0 msgr2=0x7f483019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:05.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.761+0000 7f4836000640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48301041c0 0x7f483019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.761+0000 7f4836000640 1 -- 172.21.15.42:0/3965204434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48301053b0 msgr2=0x7f483019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:05.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.761+0000 7f4836000640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48301053b0 0x7f483019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.761+0000 7f4836000640 1 -- 172.21.15.42:0/3965204434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48301a2050 con 0x7f4830105d70 2024-04-20T10:34:05.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.761+0000 7f4836000640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4830105d70 0x7f48301a19e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4828036040 tx=0x7f48280337b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.761+0000 7f482e000640 1 -- 172.21.15.42:0/3965204434 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828043020 con 0x7f4830105d70 2024-04-20T10:34:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.761+0000 7f4838000640 1 -- 172.21.15.42:0/3965204434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48301a22e0 con 0x7f4830105d70 2024-04-20T10:34:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.761+0000 7f482e000640 1 -- 172.21.15.42:0/3965204434 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f482802ca60 con 0x7f4830105d70 2024-04-20T10:34:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.761+0000 7f482e000640 1 -- 172.21.15.42:0/3965204434 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828034690 con 0x7f4830105d70 2024-04-20T10:34:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.761+0000 7f4838000640 1 -- 172.21.15.42:0/3965204434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48301a2880 con 0x7f4830105d70 2024-04-20T10:34:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.763+0000 7f4838000640 1 -- 172.21.15.42:0/3965204434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483010b480 con 0x7f4830105d70 2024-04-20T10:34:05.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.765+0000 7f482e000640 1 -- 172.21.15.42:0/3965204434 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f4828046020 con 0x7f4830105d70 2024-04-20T10:34:05.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.766+0000 7f482e000640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f48180a3e30 0x7f48180a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.766+0000 7f482e000640 1 -- 172.21.15.42:0/3965204434 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f48280f2f80 con 0x7f4830105d70 2024-04-20T10:34:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.767+0000 7f4835600640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f48180a3e30 0x7f48180a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:05.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.768+0000 7f4835600640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f48180a3e30 0x7f48180a62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4824009530 tx=0x7f4824009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:05.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.771+0000 7f482e000640 1 -- 172.21.15.42:0/3965204434 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f482804d030 con 0x7f4830105d70 2024-04-20T10:34:05.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:05 smithi042 ceph-mon[17078]: from='client.25267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:34:05.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:05 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-04-20T10:34:05.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:05 smithi042 ceph-mon[17078]: Unable to set osd_memory_target on smithi046 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-20T10:34:05.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:05 smithi042 ceph-mon[17078]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:34:05.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:05 smithi042 ceph-mon[17078]: from='client.25259 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:34:05.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:05 smithi042 ceph-mon[17078]: mgrmap e25: x(active, since 5s), standbys: y, z 2024-04-20T10:34:05.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:05 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2837407961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:34:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.979+0000 7f4838000640 1 -- 172.21.15.42:0/3965204434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f483002d670 con 0x7f4830105d70 2024-04-20T10:34:05.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.980+0000 7f482e000640 1 -- 172.21.15.42:0/3965204434 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f48280bf560 con 0x7f4830105d70 2024-04-20T10:34:05.980 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-20T10:34:05.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.982+0000 7f4838000640 1 -- 172.21.15.42:0/3965204434 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f48180a3e30 msgr2=0x7f48180a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:05.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.982+0000 7f4838000640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f48180a3e30 0x7f48180a62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4824009530 tx=0x7f4824009260 comp rx=0 tx=0).stop 2024-04-20T10:34:05.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.982+0000 7f4838000640 1 -- 172.21.15.42:0/3965204434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4830105d70 msgr2=0x7f48301a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:05.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.982+0000 7f4838000640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4830105d70 0x7f48301a19e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4828036040 tx=0x7f48280337b0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.983+0000 7f4838000640 1 -- 172.21.15.42:0/3965204434 shutdown_connections 2024-04-20T10:34:05.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.983+0000 7f4838000640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f48180a3e30 0x7f48180a62f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.983+0000 7f4838000640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4830105d70 0x7f48301a19e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.983+0000 7f4838000640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48301053b0 0x7f483019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.983+0000 7f4838000640 1 --2- 172.21.15.42:0/3965204434 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48301041c0 0x7f483019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:05.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.983+0000 7f4838000640 1 -- 172.21.15.42:0/3965204434 >> 172.21.15.42:0/3965204434 conn(0x7f48300ff750 msgr2=0x7f4830108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:05.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.983+0000 7f4838000640 1 -- 172.21.15.42:0/3965204434 shutdown_connections 2024-04-20T10:34:05.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:05.983+0000 7f4838000640 1 -- 172.21.15.42:0/3965204434 wait complete. 2024-04-20T10:34:06.002 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:34:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:05 smithi046 ceph-mon[21574]: from='client.25267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:34:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:05 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-04-20T10:34:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:05 smithi046 ceph-mon[21574]: Unable to set osd_memory_target on smithi046 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-20T10:34:06.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:05 smithi046 ceph-mon[21574]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:34:06.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:05 smithi046 ceph-mon[21574]: from='client.25259 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:34:06.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:05 smithi046 ceph-mon[21574]: mgrmap e25: x(active, since 5s), standbys: y, z 2024-04-20T10:34:06.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:05 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2837407961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:34:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:05 smithi162 ceph-mon[21854]: from='client.25267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:34:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:05 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-04-20T10:34:06.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:05 smithi162 ceph-mon[21854]: Unable to set osd_memory_target on smithi046 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-20T10:34:06.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:05 smithi162 ceph-mon[21854]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:34:06.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:05 smithi162 ceph-mon[21854]: from='client.25259 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:34:06.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:05 smithi162 ceph-mon[21854]: mgrmap e25: x(active, since 5s), standbys: y, z 2024-04-20T10:34:06.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:05 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2837407961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:34:06.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.206+0000 7ffabfe00640 1 -- 172.21.15.42:0/3991287977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab81041c0 msgr2=0x7ffab81045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:06.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.206+0000 7ffabfe00640 1 --2- 172.21.15.42:0/3991287977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab81041c0 0x7ffab81045e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009ca0 tx=0x7ffaa802c0f0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.206+0000 7ffabfe00640 1 -- 172.21.15.42:0/3991287977 shutdown_connections 2024-04-20T10:34:06.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.206+0000 7ffabfe00640 1 --2- 172.21.15.42:0/3991287977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab8105d70 0x7ffab810c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.206+0000 7ffabfe00640 1 --2- 172.21.15.42:0/3991287977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffab81053b0 0x7ffab8105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.206+0000 7ffabfe00640 1 --2- 172.21.15.42:0/3991287977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab81041c0 0x7ffab81045e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.206+0000 7ffabfe00640 1 -- 172.21.15.42:0/3991287977 >> 172.21.15.42:0/3991287977 conn(0x7ffab80ff710 msgr2=0x7ffab8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:06.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.206+0000 7ffabfe00640 1 -- 172.21.15.42:0/3991287977 shutdown_connections 2024-04-20T10:34:06.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.207+0000 7ffabfe00640 1 -- 172.21.15.42:0/3991287977 wait complete. 2024-04-20T10:34:06.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.208+0000 7ffabfe00640 1 Processor -- start 2024-04-20T10:34:06.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.208+0000 7ffabfe00640 1 -- start start 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.208+0000 7ffabfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab81041c0 0x7ffab819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.208+0000 7ffabfe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffab81053b0 0x7ffab819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.208+0000 7ffabfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab8105d70 0x7ffab81a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.208+0000 7ffabfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffab810fe50 con 0x7ffab81041c0 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.208+0000 7ffabfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffab810fcd0 con 0x7ffab8105d70 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.208+0000 7ffabfe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ffab810ffd0 con 0x7ffab81053b0 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.209+0000 7ffabd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab81041c0 0x7ffab819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.209+0000 7ffabd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab81041c0 0x7ffab819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34344/0 (socket says 172.21.15.42:34344) 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.209+0000 7ffabd400640 1 -- 172.21.15.42:0/3221585259 learned_addr learned my addr 172.21.15.42:0/3221585259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.209+0000 7ffabca00640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffab81053b0 0x7ffab819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.209+0000 7ffabde00640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab8105d70 0x7ffab81a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.209+0000 7ffabd400640 1 -- 172.21.15.42:0/3221585259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffab81053b0 msgr2=0x7ffab819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.209+0000 7ffabd400640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffab81053b0 0x7ffab819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.209+0000 7ffabd400640 1 -- 172.21.15.42:0/3221585259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab8105d70 msgr2=0x7ffab81a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:06.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.209+0000 7ffabd400640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab8105d70 0x7ffab81a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.209+0000 7ffabd400640 1 -- 172.21.15.42:0/3221585259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab81a2100 con 0x7ffab81041c0 2024-04-20T10:34:06.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.209+0000 7ffabde00640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab8105d70 0x7ffab81a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:34:06.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.210+0000 7ffabd400640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab81041c0 0x7ffab819a4e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8000f80 tx=0x7ffaa8032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:06.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.210+0000 7ffab6000640 1 -- 172.21.15.42:0/3221585259 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8041020 con 0x7ffab81041c0 2024-04-20T10:34:06.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.210+0000 7ffabfe00640 1 -- 172.21.15.42:0/3221585259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab81a2390 con 0x7ffab81041c0 2024-04-20T10:34:06.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.210+0000 7ffab6000640 1 -- 172.21.15.42:0/3221585259 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa8034cd0 con 0x7ffab81041c0 2024-04-20T10:34:06.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.210+0000 7ffab6000640 1 -- 172.21.15.42:0/3221585259 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8039040 con 0x7ffab81041c0 2024-04-20T10:34:06.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.210+0000 7ffabfe00640 1 -- 172.21.15.42:0/3221585259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab81a28a0 con 0x7ffab81041c0 2024-04-20T10:34:06.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.211+0000 7ffabfe00640 1 -- 172.21.15.42:0/3221585259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab810b530 con 0x7ffab81041c0 2024-04-20T10:34:06.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.213+0000 7ffab6000640 1 -- 172.21.15.42:0/3221585259 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7ffaa8044020 con 0x7ffab81041c0 2024-04-20T10:34:06.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.215+0000 7ffab6000640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ffa9c0a3db0 0x7ffa9c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:06.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.215+0000 7ffab6000640 1 -- 172.21.15.42:0/3221585259 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ffaa80f26f0 con 0x7ffab81041c0 2024-04-20T10:34:06.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.215+0000 7ffabca00640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ffa9c0a3db0 0x7ffa9c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:06.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.216+0000 7ffabca00640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ffa9c0a3db0 0x7ffa9c0a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffab819b9f0 tx=0x7ffaa4005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:06.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.220+0000 7ffab6000640 1 -- 172.21.15.42:0/3221585259 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffaa804b020 con 0x7ffab81041c0 2024-04-20T10:34:06.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.428+0000 7ffabfe00640 1 -- 172.21.15.42:0/3221585259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffab8062b60 con 0x7ffab81041c0 2024-04-20T10:34:06.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.429+0000 7ffab6000640 1 -- 172.21.15.42:0/3221585259 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7ffaa80348f0 con 0x7ffab81041c0 2024-04-20T10:34:06.430 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-20T10:34:06.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.431+0000 7ffabfe00640 1 -- 172.21.15.42:0/3221585259 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ffa9c0a3db0 msgr2=0x7ffa9c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:06.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.431+0000 7ffabfe00640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ffa9c0a3db0 0x7ffa9c0a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffab819b9f0 tx=0x7ffaa4005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:06.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.431+0000 7ffabfe00640 1 -- 172.21.15.42:0/3221585259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab81041c0 msgr2=0x7ffab819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:06.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.431+0000 7ffabfe00640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab81041c0 0x7ffab819a4e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8000f80 tx=0x7ffaa8032ec0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.432+0000 7ffabfe00640 1 -- 172.21.15.42:0/3221585259 shutdown_connections 2024-04-20T10:34:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.432+0000 7ffabfe00640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ffa9c0a3db0 0x7ffa9c0a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.432+0000 7ffabfe00640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab8105d70 0x7ffab81a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.432+0000 7ffabfe00640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffab81053b0 0x7ffab819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.432+0000 7ffabfe00640 1 --2- 172.21.15.42:0/3221585259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab81041c0 0x7ffab819a4e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.432+0000 7ffabfe00640 1 -- 172.21.15.42:0/3221585259 >> 172.21.15.42:0/3221585259 conn(0x7ffab80ff710 msgr2=0x7ffab8108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.432+0000 7ffabfe00640 1 -- 172.21.15.42:0/3221585259 shutdown_connections 2024-04-20T10:34:06.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.432+0000 7ffabfe00640 1 -- 172.21.15.42:0/3221585259 wait complete. 2024-04-20T10:34:06.452 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:34:06.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.638+0000 7f0cdd600640 1 -- 172.21.15.42:0/3641455109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd8076920 msgr2=0x7f0cd8076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:06.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.638+0000 7f0cdd600640 1 --2- 172.21.15.42:0/3641455109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd8076920 0x7f0cd8076d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009900 tx=0x7f0cc802bc40 comp rx=0 tx=0).stop 2024-04-20T10:34:06.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.638+0000 7f0cdd600640 1 -- 172.21.15.42:0/3641455109 shutdown_connections 2024-04-20T10:34:06.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.638+0000 7f0cdd600640 1 --2- 172.21.15.42:0/3641455109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cd8078500 0x7f0cd8078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.638+0000 7f0cdd600640 1 --2- 172.21.15.42:0/3641455109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cd8077b60 0x7f0cd8077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.638+0000 7f0cdd600640 1 --2- 172.21.15.42:0/3641455109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd8076920 0x7f0cd8076d40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.638+0000 7f0cdd600640 1 -- 172.21.15.42:0/3641455109 >> 172.21.15.42:0/3641455109 conn(0x7f0cd80ff750 msgr2=0x7f0cd8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:06.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.638+0000 7f0cdd600640 1 -- 172.21.15.42:0/3641455109 shutdown_connections 2024-04-20T10:34:06.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.638+0000 7f0cdd600640 1 -- 172.21.15.42:0/3641455109 wait complete. 2024-04-20T10:34:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.639+0000 7f0cdd600640 1 Processor -- start 2024-04-20T10:34:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.639+0000 7f0cdd600640 1 -- start start 2024-04-20T10:34:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cdd600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cd8076920 0x7f0cd819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cdd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd8077b60 0x7f0cd819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cdd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cd8078500 0x7f0cd81a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cdd600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd8114270 con 0x7f0cd8077b60 2024-04-20T10:34:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cdd600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd81140f0 con 0x7f0cd8078500 2024-04-20T10:34:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cdd600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd81143f0 con 0x7f0cd8076920 2024-04-20T10:34:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cd6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd8077b60 0x7f0cd819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cd6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd8077b60 0x7f0cd819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34360/0 (socket says 172.21.15.42:34360) 2024-04-20T10:34:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cd6000640 1 -- 172.21.15.42:0/2922860338 learned_addr learned my addr 172.21.15.42:0/2922860338 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cd7400640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cd8078500 0x7f0cd81a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cd6a00640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cd8076920 0x7f0cd819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cd6000640 1 -- 172.21.15.42:0/2922860338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cd8076920 msgr2=0x7f0cd819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cd6000640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cd8076920 0x7f0cd819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cd6000640 1 -- 172.21.15.42:0/2922860338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cd8078500 msgr2=0x7f0cd81a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cd6000640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cd8078500 0x7f0cd81a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.640+0000 7f0cd6000640 1 -- 172.21.15.42:0/2922860338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd81a6490 con 0x7f0cd8077b60 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.641+0000 7f0cd6000640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd8077b60 0x7f0cd819edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000e9f0 tx=0x7f0cc000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.641+0000 7f0ccf400640 1 -- 172.21.15.42:0/2922860338 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc000cd80 con 0x7f0cd8077b60 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.641+0000 7f0ccf400640 1 -- 172.21.15.42:0/2922860338 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc0005500 con 0x7f0cd8077b60 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.641+0000 7f0cdd600640 1 -- 172.21.15.42:0/2922860338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd81a6780 con 0x7f0cd8077b60 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.641+0000 7f0ccf400640 1 -- 172.21.15.42:0/2922860338 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0010690 con 0x7f0cd8077b60 2024-04-20T10:34:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.641+0000 7f0cdd600640 1 -- 172.21.15.42:0/2922860338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd81a6c80 con 0x7f0cd8077b60 2024-04-20T10:34:06.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.642+0000 7f0cdd600640 1 -- 172.21.15.42:0/2922860338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd8110b10 con 0x7f0cd8077b60 2024-04-20T10:34:06.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.645+0000 7f0ccf400640 1 -- 172.21.15.42:0/2922860338 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f0cc00046c0 con 0x7f0cd8077b60 2024-04-20T10:34:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.647+0000 7f0ccf400640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0cbc0a3db0 0x7f0cbc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:06.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.647+0000 7f0ccf400640 1 -- 172.21.15.42:0/2922860338 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0cc00d0ec0 con 0x7f0cd8077b60 2024-04-20T10:34:06.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.647+0000 7f0cd6a00640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0cbc0a3db0 0x7f0cbc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:06.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.648+0000 7f0cd6a00640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0cbc0a3db0 0x7f0cbc0a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0cc802c250 tx=0x7f0cc8005fd0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:06.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.651+0000 7f0ccf400640 1 -- 172.21.15.42:0/2922860338 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0cc009d390 con 0x7f0cd8077b60 2024-04-20T10:34:06.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:34:06.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3965204434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:34:06.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3221585259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:34:06.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:06.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:06.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:06.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:06.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:06.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:06.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:06.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:06.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:06.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:06.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:34:06.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:06 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:34:06.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.864+0000 7f0cdd600640 1 -- 172.21.15.42:0/2922860338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0cd802d820 con 0x7f0cd8077b60 2024-04-20T10:34:06.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.864+0000 7f0ccf400640 1 -- 172.21.15.42:0/2922860338 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f0cc009cc90 con 0x7f0cd8077b60 2024-04-20T10:34:06.865 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-20T10:34:06.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.867+0000 7f0cdd600640 1 -- 172.21.15.42:0/2922860338 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0cbc0a3db0 msgr2=0x7f0cbc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.867+0000 7f0cdd600640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0cbc0a3db0 0x7f0cbc0a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0cc802c250 tx=0x7f0cc8005fd0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.867+0000 7f0cdd600640 1 -- 172.21.15.42:0/2922860338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd8077b60 msgr2=0x7f0cd819edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.867+0000 7f0cdd600640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd8077b60 0x7f0cd819edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000e9f0 tx=0x7f0cc000edf0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.867+0000 7f0cdd600640 1 -- 172.21.15.42:0/2922860338 shutdown_connections 2024-04-20T10:34:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.867+0000 7f0cdd600640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0cbc0a3db0 0x7f0cbc0a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.867+0000 7f0cdd600640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cd8078500 0x7f0cd81a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.867+0000 7f0cdd600640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0cd8077b60 0x7f0cd819edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.867+0000 7f0cdd600640 1 --2- 172.21.15.42:0/2922860338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0cd8076920 0x7f0cd819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.867+0000 7f0cdd600640 1 -- 172.21.15.42:0/2922860338 >> 172.21.15.42:0/2922860338 conn(0x7f0cd80ff750 msgr2=0x7f0cd8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.867+0000 7f0cdd600640 1 -- 172.21.15.42:0/2922860338 shutdown_connections 2024-04-20T10:34:06.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:06.867+0000 7f0cdd600640 1 -- 172.21.15.42:0/2922860338 wait complete. 2024-04-20T10:34:06.885 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:34:07.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:34:07.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3965204434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:34:07.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3221585259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:34:07.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:07.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:07.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:34:07.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:06 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:34:07.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.063+0000 7f333d800640 1 -- 172.21.15.42:0/3538244147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33381039b0 msgr2=0x7f333810a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.063+0000 7f333d800640 1 --2- 172.21.15.42:0/3538244147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33381039b0 0x7f333810a240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3330009900 tx=0x7f333002bc30 comp rx=0 tx=0).stop 2024-04-20T10:34:07.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.063+0000 7f333d800640 1 -- 172.21.15.42:0/3538244147 shutdown_connections 2024-04-20T10:34:07.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.064+0000 7f333d800640 1 --2- 172.21.15.42:0/3538244147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33381039b0 0x7f333810a240 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.064+0000 7f333d800640 1 --2- 172.21.15.42:0/3538244147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3338102ff0 0x7f3338103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.064+0000 7f333d800640 1 --2- 172.21.15.42:0/3538244147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3338101e00 0x7f3338102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.064+0000 7f333d800640 1 -- 172.21.15.42:0/3538244147 >> 172.21.15.42:0/3538244147 conn(0x7f33380fd370 msgr2=0x7f33380ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:07.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.064+0000 7f333d800640 1 -- 172.21.15.42:0/3538244147 shutdown_connections 2024-04-20T10:34:07.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.064+0000 7f333d800640 1 -- 172.21.15.42:0/3538244147 wait complete. 2024-04-20T10:34:07.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.065+0000 7f333d800640 1 Processor -- start 2024-04-20T10:34:07.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.065+0000 7f333d800640 1 -- start start 2024-04-20T10:34:07.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.066+0000 7f333d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3338101e00 0x7f3338075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.066+0000 7f333d800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3338102ff0 0x7f3338074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.066+0000 7f333d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33381039b0 0x7f3338074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.066+0000 7f333d800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f333810dad0 con 0x7f3338101e00 2024-04-20T10:34:07.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.066+0000 7f333d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f333810d950 con 0x7f33381039b0 2024-04-20T10:34:07.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.066+0000 7f333d800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f333810dc50 con 0x7f3338102ff0 2024-04-20T10:34:07.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.066+0000 7f3337e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3338101e00 0x7f3338075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.066+0000 7f3337400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3338102ff0 0x7f3338074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.066+0000 7f3337e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3338101e00 0x7f3338075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34376/0 (socket says 172.21.15.42:34376) 2024-04-20T10:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.066+0000 7f3337e00640 1 -- 172.21.15.42:0/2501804659 learned_addr learned my addr 172.21.15.42:0/2501804659 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.066+0000 7f333ce00640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33381039b0 0x7f3338074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.067+0000 7f3337e00640 1 -- 172.21.15.42:0/2501804659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3338102ff0 msgr2=0x7f3338074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.067+0000 7f3337e00640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3338102ff0 0x7f3338074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.067+0000 7f3337e00640 1 -- 172.21.15.42:0/2501804659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33381039b0 msgr2=0x7f3338074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.067+0000 7f3337e00640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33381039b0 0x7f3338074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.067+0000 7f3337e00640 1 -- 172.21.15.42:0/2501804659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3338075120 con 0x7f3338101e00 2024-04-20T10:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.067+0000 7f3337400640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3338102ff0 0x7f3338074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.067+0000 7f333ce00640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33381039b0 0x7f3338074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.067+0000 7f3337e00640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3338101e00 0x7f3338075d10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f332800e9c0 tx=0x7f332800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.067+0000 7f3334c00640 1 -- 172.21.15.42:0/2501804659 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332800cde0 con 0x7f3338101e00 2024-04-20T10:34:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.067+0000 7f333d800640 1 -- 172.21.15.42:0/2501804659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33381a21d0 con 0x7f3338101e00 2024-04-20T10:34:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.067+0000 7f3334c00640 1 -- 172.21.15.42:0/2501804659 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3328010910 con 0x7f3338101e00 2024-04-20T10:34:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.067+0000 7f3334c00640 1 -- 172.21.15.42:0/2501804659 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33280189d0 con 0x7f3338101e00 2024-04-20T10:34:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.068+0000 7f333d800640 1 -- 172.21.15.42:0/2501804659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33381a2770 con 0x7f3338101e00 2024-04-20T10:34:07.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.069+0000 7f333d800640 1 -- 172.21.15.42:0/2501804659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33381090c0 con 0x7f3338101e00 2024-04-20T10:34:07.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.072+0000 7f3334c00640 1 -- 172.21.15.42:0/2501804659 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f3328010ac0 con 0x7f3338101e00 2024-04-20T10:34:07.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.074+0000 7f3334c00640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f33180a3db0 0x7f33180a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.074+0000 7f3334c00640 1 -- 172.21.15.42:0/2501804659 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f33280d16b0 con 0x7f3338101e00 2024-04-20T10:34:07.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.075+0000 7f3337400640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f33180a3db0 0x7f33180a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.075+0000 7f3337400640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f33180a3db0 0x7f33180a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3338075a80 tx=0x7f3320009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.077+0000 7f3334c00640 1 -- 172.21.15.42:0/2501804659 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3328026070 con 0x7f3338101e00 2024-04-20T10:34:07.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T10:34:07.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3965204434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:34:07.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3221585259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:34:07.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:07.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:07.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:34:07.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:34:07.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:06 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:34:07.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.320+0000 7f333d800640 1 -- 172.21.15.42:0/2501804659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f333802d5d0 con 0x7f3338101e00 2024-04-20T10:34:07.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.321+0000 7f3334c00640 1 -- 172.21.15.42:0/2501804659 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f332809db80 con 0x7f3338101e00 2024-04-20T10:34:07.327 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:34:07.327 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:34:07.327 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:34:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.324+0000 7f333d800640 1 -- 172.21.15.42:0/2501804659 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f33180a3db0 msgr2=0x7f33180a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.324+0000 7f333d800640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f33180a3db0 0x7f33180a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3338075a80 tx=0x7f3320009500 comp rx=0 tx=0).stop 2024-04-20T10:34:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.324+0000 7f333d800640 1 -- 172.21.15.42:0/2501804659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3338101e00 msgr2=0x7f3338075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.324+0000 7f333d800640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3338101e00 0x7f3338075d10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f332800e9c0 tx=0x7f332800ee80 comp rx=0 tx=0).stop 2024-04-20T10:34:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.324+0000 7f333d800640 1 -- 172.21.15.42:0/2501804659 shutdown_connections 2024-04-20T10:34:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.324+0000 7f333d800640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f33180a3db0 0x7f33180a6270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.324+0000 7f333d800640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33381039b0 0x7f3338074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.324+0000 7f333d800640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3338102ff0 0x7f3338074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.324+0000 7f333d800640 1 --2- 172.21.15.42:0/2501804659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3338101e00 0x7f3338075d10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.324+0000 7f333d800640 1 -- 172.21.15.42:0/2501804659 >> 172.21.15.42:0/2501804659 conn(0x7f33380fd370 msgr2=0x7f3338106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.324+0000 7f333d800640 1 -- 172.21.15.42:0/2501804659 shutdown_connections 2024-04-20T10:34:07.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.324+0000 7f333d800640 1 -- 172.21.15.42:0/2501804659 wait complete. 2024-04-20T10:34:07.342 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:34:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.520+0000 7fde4d600640 1 -- 172.21.15.42:0/1744353730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde48100820 msgr2=0x7fde48100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.520+0000 7fde4d600640 1 --2- 172.21.15.42:0/1744353730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde48100820 0x7fde48100c40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fde38009900 tx=0x7fde3802bc40 comp rx=0 tx=0).stop 2024-04-20T10:34:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.520+0000 7fde4d600640 1 -- 172.21.15.42:0/1744353730 shutdown_connections 2024-04-20T10:34:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.520+0000 7fde4d600640 1 --2- 172.21.15.42:0/1744353730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde48101b20 0x7fde48101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.520+0000 7fde4d600640 1 --2- 172.21.15.42:0/1744353730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde48101180 0x7fde481015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.520+0000 7fde4d600640 1 --2- 172.21.15.42:0/1744353730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde48100820 0x7fde48100c40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.520+0000 7fde4d600640 1 -- 172.21.15.42:0/1744353730 >> 172.21.15.42:0/1744353730 conn(0x7fde480fc440 msgr2=0x7fde480fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.520+0000 7fde4d600640 1 -- 172.21.15.42:0/1744353730 shutdown_connections 2024-04-20T10:34:07.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.520+0000 7fde4d600640 1 -- 172.21.15.42:0/1744353730 wait complete. 2024-04-20T10:34:07.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.521+0000 7fde4d600640 1 Processor -- start 2024-04-20T10:34:07.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.521+0000 7fde4d600640 1 -- start start 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.521+0000 7fde4d600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde48100820 0x7fde4819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde4d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde48101180 0x7fde4819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde4d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde48101b20 0x7fde481a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde4d600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fde4810fea0 con 0x7fde48101180 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde4d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde4810fd20 con 0x7fde48101b20 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde4d600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fde48110020 con 0x7fde48100820 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde46a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde48100820 0x7fde4819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde46a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde48100820 0x7fde4819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:55734/0 (socket says 172.21.15.42:55734) 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde46a00640 1 -- 172.21.15.42:0/3484731971 learned_addr learned my addr 172.21.15.42:0/3484731971 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde47400640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde48101b20 0x7fde481a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde46000640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde48101180 0x7fde4819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde46a00640 1 -- 172.21.15.42:0/3484731971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde48101b20 msgr2=0x7fde481a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde46a00640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde48101b20 0x7fde481a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde46a00640 1 -- 172.21.15.42:0/3484731971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde48101180 msgr2=0x7fde4819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde46a00640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde48101180 0x7fde4819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde46a00640 1 -- 172.21.15.42:0/3484731971 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde481a2090 con 0x7fde48100820 2024-04-20T10:34:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.522+0000 7fde46000640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde48101180 0x7fde4819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:34:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.523+0000 7fde46a00640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde48100820 0x7fde4819a470 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde3802c8f0 tx=0x7fde3802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.523+0000 7fde3f400640 1 -- 172.21.15.42:0/3484731971 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38041070 con 0x7fde48100820 2024-04-20T10:34:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.523+0000 7fde3f400640 1 -- 172.21.15.42:0/3484731971 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde38004450 con 0x7fde48100820 2024-04-20T10:34:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.523+0000 7fde4d600640 1 -- 172.21.15.42:0/3484731971 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde481a2320 con 0x7fde48100820 2024-04-20T10:34:07.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.523+0000 7fde3f400640 1 -- 172.21.15.42:0/3484731971 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38034440 con 0x7fde48100820 2024-04-20T10:34:07.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.523+0000 7fde4d600640 1 -- 172.21.15.42:0/3484731971 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde481a2860 con 0x7fde48100820 2024-04-20T10:34:07.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.524+0000 7fde4d600640 1 -- 172.21.15.42:0/3484731971 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4810b6b0 con 0x7fde48100820 2024-04-20T10:34:07.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.527+0000 7fde3f400640 1 -- 172.21.15.42:0/3484731971 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7fde38044070 con 0x7fde48100820 2024-04-20T10:34:07.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.528+0000 7fde3f400640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fde2c0a3e30 0x7fde2c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.529+0000 7fde3f400640 1 -- 172.21.15.42:0/3484731971 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fde380f23e0 con 0x7fde48100820 2024-04-20T10:34:07.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.529+0000 7fde46000640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fde2c0a3e30 0x7fde2c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.530+0000 7fde46000640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fde2c0a3e30 0x7fde2c0a62f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fde4819b980 tx=0x7fde34005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:07.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.533+0000 7fde3f400640 1 -- 172.21.15.42:0/3484731971 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fde380be9c0 con 0x7fde48100820 2024-04-20T10:34:07.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.757+0000 7fde4d600640 1 -- 172.21.15.42:0/3484731971 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde48062b60 con 0x7fde48100820 2024-04-20T10:34:07.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.759+0000 7fde3f400640 1 -- 172.21.15.42:0/3484731971 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23100 (secure 0 0 0) 0x7fde380be2c0 con 0x7fde48100820 2024-04-20T10:34:07.760 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:34:07.760 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":88,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:34:00.299452+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/230562472":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/1321607472":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/3804122415":"2024-04-21T10:34:00.299359+0000","172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/3610568749":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/869471821":"2024-04-21T10:34:00.299359+0000","172.21.15.42:6800/3610568749":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:34:07.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.761+0000 7fde4d600640 1 -- 172.21.15.42:0/3484731971 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fde2c0a3e30 msgr2=0x7fde2c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.761+0000 7fde4d600640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fde2c0a3e30 0x7fde2c0a62f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fde4819b980 tx=0x7fde34005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:07.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.761+0000 7fde4d600640 1 -- 172.21.15.42:0/3484731971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde48100820 msgr2=0x7fde4819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.761+0000 7fde4d600640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde48100820 0x7fde4819a470 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde3802c8f0 tx=0x7fde3802c660 comp rx=0 tx=0).stop 2024-04-20T10:34:07.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.761+0000 7fde4d600640 1 -- 172.21.15.42:0/3484731971 shutdown_connections 2024-04-20T10:34:07.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.762+0000 7fde4d600640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fde2c0a3e30 0x7fde2c0a62f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.762+0000 7fde4d600640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde48101b20 0x7fde481a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.762+0000 7fde4d600640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde48101180 0x7fde4819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.762+0000 7fde4d600640 1 --2- 172.21.15.42:0/3484731971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde48100820 0x7fde4819a470 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.762+0000 7fde4d600640 1 -- 172.21.15.42:0/3484731971 >> 172.21.15.42:0/3484731971 conn(0x7fde480fc440 msgr2=0x7fde480fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:07.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.762+0000 7fde4d600640 1 -- 172.21.15.42:0/3484731971 shutdown_connections 2024-04-20T10:34:07.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.762+0000 7fde4d600640 1 -- 172.21.15.42:0/3484731971 wait complete. 2024-04-20T10:34:07.780 DEBUG:tasks.fwd_scrub:Yielding 2024-04-20T10:34:07.780 INFO:teuthology.run_tasks:Running task exec... 2024-04-20T10:34:07.790 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T10:34:07.790 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-04-20T10:34:07.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-20T10:34:07.791 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-20T10:34:07.791 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:34:07.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:34:07.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:34:07.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.969+0000 7f842e400640 1 -- 172.21.15.42:0/983358443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8428102f70 msgr2=0x7f84281033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.969+0000 7f842e400640 1 --2- 172.21.15.42:0/983358443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8428102f70 0x7f84281033f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8414009860 tx=0x7f841402bb70 comp rx=0 tx=0).stop 2024-04-20T10:34:07.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.969+0000 7f842e400640 1 -- 172.21.15.42:0/983358443 shutdown_connections 2024-04-20T10:34:07.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.969+0000 7f842e400640 1 --2- 172.21.15.42:0/983358443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8428103930 0x7f842810a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.969+0000 7f842e400640 1 --2- 172.21.15.42:0/983358443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8428102f70 0x7f84281033f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.969+0000 7f842e400640 1 --2- 172.21.15.42:0/983358443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8428101e20 0x7f8428102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.969+0000 7f842e400640 1 -- 172.21.15.42:0/983358443 >> 172.21.15.42:0/983358443 conn(0x7f84280fd390 msgr2=0x7f84280ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:07.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.970+0000 7f842e400640 1 -- 172.21.15.42:0/983358443 shutdown_connections 2024-04-20T10:34:07.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.970+0000 7f842e400640 1 -- 172.21.15.42:0/983358443 wait complete. 2024-04-20T10:34:07.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.971+0000 7f842e400640 1 Processor -- start 2024-04-20T10:34:07.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.971+0000 7f842e400640 1 -- start start 2024-04-20T10:34:07.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.971+0000 7f842e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8428101e20 0x7f8428075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.971+0000 7f842e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8428102f70 0x7f8428074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.971+0000 7f842e400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8428103930 0x7f8428074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.971+0000 7f842e400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f842810daa0 con 0x7f8428102f70 2024-04-20T10:34:07.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.971+0000 7f842e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f842810d920 con 0x7f8428101e20 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.971+0000 7f842e400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f842810dc20 con 0x7f8428103930 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.972+0000 7f842d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8428101e20 0x7f8428075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.972+0000 7f8427e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8428102f70 0x7f8428074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.972+0000 7f842d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8428101e20 0x7f8428075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52174/0 (socket says 172.21.15.42:52174) 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.972+0000 7f842d000640 1 -- 172.21.15.42:0/1240701613 learned_addr learned my addr 172.21.15.42:0/1240701613 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.972+0000 7f842da00640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8428103930 0x7f8428074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.972+0000 7f842d000640 1 -- 172.21.15.42:0/1240701613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8428103930 msgr2=0x7f8428074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.972+0000 7f842d000640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8428103930 0x7f8428074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.972+0000 7f842d000640 1 -- 172.21.15.42:0/1240701613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8428102f70 msgr2=0x7f8428074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.972+0000 7f842d000640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8428102f70 0x7f8428074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.972+0000 7f842d000640 1 -- 172.21.15.42:0/1240701613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8428075120 con 0x7f8428101e20 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.972+0000 7f842d000640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8428101e20 0x7f8428075d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f841c00ea60 tx=0x7f841c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.973+0000 7f8425600640 1 -- 172.21.15.42:0/1240701613 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c00cde0 con 0x7f8428101e20 2024-04-20T10:34:07.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.973+0000 7f8425600640 1 -- 172.21.15.42:0/1240701613 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841c0035f0 con 0x7f8428101e20 2024-04-20T10:34:07.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.973+0000 7f842e400640 1 -- 172.21.15.42:0/1240701613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84281a2230 con 0x7f8428101e20 2024-04-20T10:34:07.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.973+0000 7f8425600640 1 -- 172.21.15.42:0/1240701613 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c010670 con 0x7f8428101e20 2024-04-20T10:34:07.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.973+0000 7f842e400640 1 -- 172.21.15.42:0/1240701613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84281a2750 con 0x7f8428101e20 2024-04-20T10:34:07.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.974+0000 7f842e400640 1 -- 172.21.15.42:0/1240701613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84080052c0 con 0x7f8428101e20 2024-04-20T10:34:07.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.976+0000 7f8425600640 1 -- 172.21.15.42:0/1240701613 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f841c010810 con 0x7f8428101e20 2024-04-20T10:34:07.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.977+0000 7f8425600640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f84100a3da0 0x7f84100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.977+0000 7f8425600640 1 -- 172.21.15.42:0/1240701613 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f841c0d0cc0 con 0x7f8428101e20 2024-04-20T10:34:07.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.977+0000 7f8427e00640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f84100a3da0 0x7f84100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.978+0000 7f8427e00640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f84100a3da0 0x7f84100a6260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f84140349e0 tx=0x7f8414034ab0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:07.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.981+0000 7f8425600640 1 -- 172.21.15.42:0/1240701613 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f841c0d7080 con 0x7f8428101e20 2024-04-20T10:34:07.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.986+0000 7f67b9600640 1 -- 172.21.15.42:0/812719734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4102fe0 msgr2=0x7f67b4103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.986+0000 7f67b9600640 1 --2- 172.21.15.42:0/812719734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4102fe0 0x7f67b4103460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009860 tx=0x7f67a002bb70 comp rx=0 tx=0).stop 2024-04-20T10:34:07.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.986+0000 7f67b9600640 1 -- 172.21.15.42:0/812719734 shutdown_connections 2024-04-20T10:34:07.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.986+0000 7f67b9600640 1 --2- 172.21.15.42:0/812719734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b41039a0 0x7f67b410a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.986+0000 7f67b9600640 1 --2- 172.21.15.42:0/812719734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4102fe0 0x7f67b4103460 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.986+0000 7f67b9600640 1 --2- 172.21.15.42:0/812719734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67b4101e90 0x7f67b41022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.986+0000 7f67b9600640 1 -- 172.21.15.42:0/812719734 >> 172.21.15.42:0/812719734 conn(0x7f67b40fd400 msgr2=0x7f67b40ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:07.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.986+0000 7f67b9600640 1 -- 172.21.15.42:0/812719734 shutdown_connections 2024-04-20T10:34:07.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.986+0000 7f67b9600640 1 -- 172.21.15.42:0/812719734 wait complete. 2024-04-20T10:34:07.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.987+0000 7f67b9600640 1 Processor -- start 2024-04-20T10:34:07.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.987+0000 7f67b9600640 1 -- start start 2024-04-20T10:34:07.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b9600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67b4101e90 0x7f67b4075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4102fe0 0x7f67b4074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b41039a0 0x7f67b4074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b9600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67b410db20 con 0x7f67b41039a0 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b9600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67b410d9a0 con 0x7f67b4102fe0 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b9600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f67b410dca0 con 0x7f67b4101e90 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4102fe0 0x7f67b4074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b41039a0 0x7f67b4074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4102fe0 0x7f67b4074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52196/0 (socket says 172.21.15.42:52196) 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b3400640 1 -- 172.21.15.42:0/4192148057 learned_addr learned my addr 172.21.15.42:0/4192148057 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b3e00640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67b4101e90 0x7f67b4075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b3400640 1 -- 172.21.15.42:0/4192148057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67b4101e90 msgr2=0x7f67b4075cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b3400640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67b4101e90 0x7f67b4075cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b3400640 1 -- 172.21.15.42:0/4192148057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b41039a0 msgr2=0x7f67b4074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b3400640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b41039a0 0x7f67b4074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.988+0000 7f67b3400640 1 -- 172.21.15.42:0/4192148057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b4075150 con 0x7f67b4102fe0 2024-04-20T10:34:07.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.989+0000 7f67b3400640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4102fe0 0x7f67b4074350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f67a00349e0 tx=0x7f67a0032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:07.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.989+0000 7f67b0c00640 1 -- 172.21.15.42:0/4192148057 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0041020 con 0x7f67b4102fe0 2024-04-20T10:34:07.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.989+0000 7f67b0c00640 1 -- 172.21.15.42:0/4192148057 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a002c9f0 con 0x7f67b4102fe0 2024-04-20T10:34:07.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.989+0000 7f67b0c00640 1 -- 172.21.15.42:0/4192148057 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0039040 con 0x7f67b4102fe0 2024-04-20T10:34:07.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.989+0000 7f67b9600640 1 -- 172.21.15.42:0/4192148057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b41a2280 con 0x7f67b4102fe0 2024-04-20T10:34:07.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.989+0000 7f67b9600640 1 -- 172.21.15.42:0/4192148057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b41a26f0 con 0x7f67b4102fe0 2024-04-20T10:34:07.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.992+0000 7f67b0c00640 1 -- 172.21.15.42:0/4192148057 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f67a0044020 con 0x7f67b4102fe0 2024-04-20T10:34:07.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.992+0000 7f67bd9f1740 1 Processor -- start 2024-04-20T10:34:07.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.992+0000 7f67bd9f1740 1 -- start start 2024-04-20T10:34:07.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.993+0000 7f67bd9f1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94081110 0x562f94081530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.993+0000 7f67bd9f1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081b00 0x562f94086600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.993+0000 7f67bd9f1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f94089100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.993+0000 7f67bd9f1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562f93fe6d20 con 0x562f94081110 2024-04-20T10:34:07.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.993+0000 7f67bd9f1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562f93f8d3e0 con 0x562f94081b00 2024-04-20T10:34:07.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.993+0000 7f67bd9f1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562f93fe2ce0 con 0x562f94086c70 2024-04-20T10:34:07.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.993+0000 7f67b8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94081110 0x562f94081530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.993+0000 7f67b3400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f94089100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.993+0000 7f67b3400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f94089100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:55794/0 (socket says 172.21.15.42:55794) 2024-04-20T10:34:07.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.993+0000 7f67b3400640 1 -- 172.21.15.42:0/1951255739 learned_addr learned my addr 172.21.15.42:0/1951255739 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:07.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.993+0000 7f67b3e00640 1 --2- 172.21.15.42:0/1951255739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081b00 0x562f94086600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.994+0000 7f67b3400640 1 -- 172.21.15.42:0/1951255739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081b00 msgr2=0x562f94086600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:07.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.994+0000 7f67b3400640 1 --2- 172.21.15.42:0/1951255739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081b00 0x562f94086600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.994+0000 7f67b3400640 1 -- 172.21.15.42:0/1951255739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94081110 msgr2=0x562f94081530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:07.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.994+0000 7f67b3400640 1 --2- 172.21.15.42:0/1951255739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94081110 0x562f94081530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.994+0000 7f67b3400640 1 -- 172.21.15.42:0/1951255739 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f93fac980 con 0x562f94086c70 2024-04-20T10:34:07.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.994+0000 7f67b3400640 1 --2- 172.21.15.42:0/1951255739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f94089100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f67a00ef810 tx=0x7f67a00fa1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dba098baaa9748d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:07.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.994+0000 7f67b0c00640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f67980a3e30 0x7f67980a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:07.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.995+0000 7f67b0c00640 1 -- 172.21.15.42:0/4192148057 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f67a00f2160 con 0x7f67b4102fe0 2024-04-20T10:34:07.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.995+0000 7f67b8c00640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f67980a3e30 0x7f67980a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:07.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.995+0000 7f67a9200640 1 -- 172.21.15.42:0/1951255739 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a00ba2e0 con 0x562f94086c70 2024-04-20T10:34:07.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.995+0000 7f67a9200640 1 -- 172.21.15.42:0/1951255739 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a00ff030 con 0x562f94086c70 2024-04-20T10:34:07.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.995+0000 7f67a9200640 1 -- 172.21.15.42:0/1951255739 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a00f5c50 con 0x562f94086c70 2024-04-20T10:34:07.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.996+0000 7f67bd9f1740 1 -- 172.21.15.42:0/1951255739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 msgr2=0x562f94089100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:07.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.996+0000 7f67b8c00640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f67980a3e30 0x7f67980a62f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x562f94082360 tx=0x7f67ac009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:07.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.996+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/1951255739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f94089100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f67a00ef810 tx=0x7f67a00fa1a0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.996+0000 7f67bd9f1740 1 -- 172.21.15.42:0/1951255739 shutdown_connections 2024-04-20T10:34:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.996+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/1951255739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f94089100 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.996+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/1951255739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081b00 0x562f94086600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.996+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/1951255739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94081110 0x562f94081530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.996+0000 7f67bd9f1740 1 -- 172.21.15.42:0/1951255739 >> 172.21.15.42:0/1951255739 conn(0x562f9407c680 msgr2=0x562f9407ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.996+0000 7f67bd9f1740 1 -- 172.21.15.42:0/1951255739 shutdown_connections 2024-04-20T10:34:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.996+0000 7f67bd9f1740 1 -- 172.21.15.42:0/1951255739 wait complete. 2024-04-20T10:34:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.996+0000 7f67bd9f1740 1 -- start start 2024-04-20T10:34:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.997+0000 7f67bd9f1740 1 Processor -- start 2024-04-20T10:34:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.997+0000 7f67bd9f1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94081110 0x562f9408b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.997+0000 7f67bd9f1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081b00 0x562f9408b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.997+0000 7f67bd9f1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f94092920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.997+0000 7f67bd9f1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562f940899e0 con 0x562f94081110 2024-04-20T10:34:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.997+0000 7f67bd9f1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562f93fa07a0 con 0x562f94081b00 2024-04-20T10:34:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.997+0000 7f67bd9f1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562f94089b60 con 0x562f94086c70 2024-04-20T10:34:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.997+0000 7f67bd9f1740 10 client.0 _ll_register_callbacks cb 0x562f93fa2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:34:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.997+0000 7f67b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081b00 0x562f9408b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.997+0000 7f67b8c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f94092920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.997+0000 7f67b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081b00 0x562f9408b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52214/0 (socket says 172.21.15.42:52214) 2024-04-20T10:34:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.997+0000 7f67b3400640 1 -- 172.21.15.42:0/38488995 learned_addr learned my addr 172.21.15.42:0/38488995 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f67b3e00640 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94081110 0x562f9408b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f67b3400640 1 -- 172.21.15.42:0/38488995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 msgr2=0x562f94092920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f67b3400640 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f94092920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f67b3400640 1 -- 172.21.15.42:0/38488995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94081110 msgr2=0x562f9408b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f67b3400640 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94081110 0x562f9408b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f67b3400640 1 -- 172.21.15.42:0/38488995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f93dc17e0 con 0x562f94081b00 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f67b3e00640 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94081110 0x562f9408b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f67b3400640 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081b00 0x562f9408b8e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f67a00ef7e0 tx=0x7f67a00be740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c21ba556419fbb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f679d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f679d600640 1 -- 172.21.15.42:0/38488995 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a00bd9a0 con 0x562f94081b00 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f679d600640 1 -- 172.21.15.42:0/38488995 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f67a00bc340 con 0x562f94081b00 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f67bd9f1740 1 -- 172.21.15.42:0/38488995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a00ff030 con 0x562f94081b00 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.998+0000 7f679d600640 1 -- 172.21.15.42:0/38488995 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f67a00bdb80 con 0x562f94081b00 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.999+0000 7f67bd9f1740 10 client.25319 fetch_fsmap learned FSMap version 19 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.999+0000 7f67bd9f1740 1 -- 172.21.15.42:0/38488995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562f93e21f70 con 0x562f94081b00 2024-04-20T10:34:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.999+0000 7f679d600640 1 -- 172.21.15.42:0/38488995 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a00ff030 con 0x562f94081b00 2024-04-20T10:34:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.999+0000 7f679d600640 1 -- 172.21.15.42:0/38488995 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f67a003c030 con 0x562f94081b00 2024-04-20T10:34:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.999+0000 7f67bd9f1740 10 client.25319 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:34:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.999+0000 7f67bd9f1740 10 client.25319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:34:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.999+0000 7f67bd9f1740 20 client.25319 populate_metadata read hostname 'smithi042' 2024-04-20T10:34:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.999+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562f94096eb0 0x562f94099300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.999+0000 7f67bd9f1740 4 client.25319 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:34:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.999+0000 7f67bd9f1740 1 -- 172.21.15.42:0/38488995 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f93ff4740 con 0x562f94096eb0 2024-04-20T10:34:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:07.999+0000 7f67b3e00640 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562f94096eb0 0x562f94099300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.000+0000 7f67b3e00640 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562f94096eb0 0x562f94099300 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d15aaef26bc010d5 server_cookie=5dcdcb9b6caed444 in_seq=0 out_seq=0 2024-04-20T10:34:08.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.000+0000 7f679d600640 10 client.25319 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:08.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f679d600640 1 -- 172.21.15.42:0/38488995 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f93ff4740 con 0x562f94096eb0 2024-04-20T10:34:08.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f679d600640 10 client.25319 handle_command_reply: tid=0 2024-04-20T10:34:08.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f67bd9f1740 1 client.25319 shutdown 2024-04-20T10:34:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f67bd9f1740 1 -- 172.21.15.42:0/38488995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081b00 msgr2=0x562f9408b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081b00 0x562f9408b8e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f67a00ef7e0 tx=0x7f67a00be740 comp rx=0 tx=0).stop 2024-04-20T10:34:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f67bd9f1740 1 -- 172.21.15.42:0/38488995 shutdown_connections 2024-04-20T10:34:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562f94096eb0 0x562f94099300 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f94092920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081b00 0x562f9408b8e0 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f67a00ef7e0 tx=0x7f67a00be740 comp rx=0 tx=0).stop 2024-04-20T10:34:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/38488995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94081110 0x562f9408b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f67bd9f1740 1 -- 172.21.15.42:0/38488995 >> 172.21.15.42:0/38488995 conn(0x562f9407c680 msgr2=0x562f9407ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f679d600640 0 client.25319 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f67bd9f1740 1 -- 172.21.15.42:0/38488995 shutdown_connections 2024-04-20T10:34:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.001+0000 7f67bd9f1740 1 -- 172.21.15.42:0/38488995 wait complete. 2024-04-20T10:34:08.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.002+0000 7f67bd9f1740 20 client.25319 trim_cache size 0 max 16384 2024-04-20T10:34:08.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.020+0000 7f67bd9f1740 1 Processor -- start 2024-04-20T10:34:08.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.020+0000 7f67bd9f1740 1 -- start start 2024-04-20T10:34:08.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.021+0000 7f67bd9f1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94081110 0x562f9407b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.021+0000 7f67bd9f1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94086c70 0x562f940928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.021+0000 7f67bd9f1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94083710 0x562f9407b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.021+0000 7f67bd9f1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41a53e0 con 0x562f94086c70 2024-04-20T10:34:08.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.021+0000 7f67bd9f1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41a7c60 con 0x562f94083710 2024-04-20T10:34:08.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.021+0000 7f67bd9f1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41a5a50 con 0x562f94081110 2024-04-20T10:34:08.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.021+0000 7f67b3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94086c70 0x562f940928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.021+0000 7f67b3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94081110 0x562f9407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.021+0000 7f67b3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94081110 0x562f9407b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:55808/0 (socket says 172.21.15.42:55808) 2024-04-20T10:34:08.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.021+0000 7f67b3e00640 1 -- 172.21.15.42:0/3532758426 learned_addr learned my addr 172.21.15.42:0/3532758426 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:08.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.021+0000 7f67b8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94083710 0x562f9407b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.022+0000 7f67b3e00640 1 -- 172.21.15.42:0/3532758426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94083710 msgr2=0x562f9407b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.022+0000 7f67b3e00640 1 --2- 172.21.15.42:0/3532758426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94083710 0x562f9407b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.022+0000 7f67b3e00640 1 -- 172.21.15.42:0/3532758426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94086c70 msgr2=0x562f940928e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.022+0000 7f67b3e00640 1 --2- 172.21.15.42:0/3532758426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94086c70 0x562f940928e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.022+0000 7f67b3e00640 1 -- 172.21.15.42:0/3532758426 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f93cb9170 con 0x562f94081110 2024-04-20T10:34:08.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.022+0000 7f67b3e00640 1 --2- 172.21.15.42:0/3532758426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94081110 0x562f9407b170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x562f94081f00 tx=0x7f67a4012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfbe0ee72e93974a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:08.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.022+0000 7f679fe00640 1 -- 172.21.15.42:0/3532758426 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4005020 con 0x562f94081110 2024-04-20T10:34:08.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.022+0000 7f679fe00640 1 -- 172.21.15.42:0/3532758426 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a40047a0 con 0x562f94081110 2024-04-20T10:34:08.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.022+0000 7f679fe00640 1 -- 172.21.15.42:0/3532758426 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a401f350 con 0x562f94081110 2024-04-20T10:34:08.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.023+0000 7f67bd9f1740 1 -- 172.21.15.42:0/3532758426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94081110 msgr2=0x562f9407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.023+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/3532758426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94081110 0x562f9407b170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x562f94081f00 tx=0x7f67a4012820 comp rx=0 tx=0).stop 2024-04-20T10:34:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.023+0000 7f67bd9f1740 1 -- 172.21.15.42:0/3532758426 shutdown_connections 2024-04-20T10:34:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.023+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/3532758426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94083710 0x562f9407b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.023+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/3532758426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94086c70 0x562f940928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.023+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/3532758426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94081110 0x562f9407b170 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.023+0000 7f67bd9f1740 1 -- 172.21.15.42:0/3532758426 >> 172.21.15.42:0/3532758426 conn(0x562f9407c680 msgr2=0x562f94083000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.024+0000 7f67bd9f1740 1 -- 172.21.15.42:0/3532758426 shutdown_connections 2024-04-20T10:34:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.024+0000 7f67bd9f1740 1 -- 172.21.15.42:0/3532758426 wait complete. 2024-04-20T10:34:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.024+0000 7f67bd9f1740 1 -- start start 2024-04-20T10:34:08.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.024+0000 7f67bd9f1740 1 Processor -- start 2024-04-20T10:34:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.024+0000 7f67bd9f1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081110 0x562f94092230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.024+0000 7f67bd9f1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94083710 0x562f94092770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.024+0000 7f67bd9f1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f940908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67bd9f1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41a6120 con 0x562f94083710 2024-04-20T10:34:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67bd9f1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41a7590 con 0x562f94081110 2024-04-20T10:34:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67bd9f1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41a67f0 con 0x562f94086c70 2024-04-20T10:34:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67bd9f1740 10 client.0 _ll_register_callbacks cb 0x7f67ac0098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:34:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67b3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94083710 0x562f94092770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67b3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94083710 0x562f94092770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34464/0 (socket says 172.21.15.42:34464) 2024-04-20T10:34:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67b3400640 1 -- 172.21.15.42:0/2536857910 learned_addr learned my addr 172.21.15.42:0/2536857910 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67b3e00640 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081110 0x562f94092230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67b8c00640 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f940908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67b3400640 1 -- 172.21.15.42:0/2536857910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 msgr2=0x562f940908e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67b3400640 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f940908e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67b3400640 1 -- 172.21.15.42:0/2536857910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081110 msgr2=0x562f94092230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67b3400640 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081110 0x562f94092230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.025+0000 7f67b3400640 1 -- 172.21.15.42:0/2536857910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f94091050 con 0x562f94083710 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.026+0000 7f67b3400640 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94083710 0x562f94092770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f67a00be200 tx=0x7f67a00ef8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed793b2fae92248 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.026+0000 7f679d600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.026+0000 7f679d600640 1 -- 172.21.15.42:0/2536857910 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a00bde10 con 0x562f94083710 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.026+0000 7f679d600640 1 -- 172.21.15.42:0/2536857910 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f67a00f5350 con 0x562f94083710 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.026+0000 7f67bd9f1740 1 -- 172.21.15.42:0/2536857910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a40047a0 con 0x562f94083710 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.026+0000 7f679d600640 1 -- 172.21.15.42:0/2536857910 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f67a009ce60 con 0x562f94083710 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.026+0000 7f67bd9f1740 10 client.15210 fetch_fsmap learned FSMap version 19 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.026+0000 7f67bd9f1740 1 -- 172.21.15.42:0/2536857910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562f94091050 con 0x562f94083710 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.026+0000 7f679d600640 1 -- 172.21.15.42:0/2536857910 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a40047a0 con 0x562f94083710 2024-04-20T10:34:08.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.027+0000 7f679d600640 1 -- 172.21.15.42:0/2536857910 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f67a004d300 con 0x562f94083710 2024-04-20T10:34:08.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.027+0000 7f67bd9f1740 10 client.15210 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:34:08.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.027+0000 7f67bd9f1740 10 client.15210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:34:08.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.027+0000 7f67bd9f1740 20 client.15210 populate_metadata read hostname 'smithi042' 2024-04-20T10:34:08.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.027+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562f940fe4b0 0x562f941008a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.027+0000 7f67bd9f1740 4 client.15210 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:34:08.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.027+0000 7f67bd9f1740 1 -- 172.21.15.42:0/2536857910 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562f93fee5b0 con 0x562f940fe4b0 2024-04-20T10:34:08.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.027+0000 7f67b3e00640 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562f940fe4b0 0x562f941008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.028+0000 7f67b3e00640 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562f940fe4b0 0x562f941008a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ab10ae26fbfbc10 server_cookie=2cb6de1b8084f79d in_seq=0 out_seq=0 2024-04-20T10:34:08.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.028+0000 7f679d600640 10 client.15210 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:08.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.029+0000 7f679d600640 1 -- 172.21.15.42:0/2536857910 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562f93fee5b0 con 0x562f940fe4b0 2024-04-20T10:34:08.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.029+0000 7f679d600640 10 client.15210 handle_command_reply: tid=0 2024-04-20T10:34:08.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.029+0000 7f67bd9f1740 1 client.15210 shutdown 2024-04-20T10:34:08.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.029+0000 7f67bd9f1740 1 -- 172.21.15.42:0/2536857910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94083710 msgr2=0x562f94092770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.029+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94083710 0x562f94092770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f67a00be200 tx=0x7f67a00ef8c0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.029+0000 7f67bd9f1740 1 -- 172.21.15.42:0/2536857910 shutdown_connections 2024-04-20T10:34:08.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.029+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562f940fe4b0 0x562f941008a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.029+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562f94086c70 0x562f940908e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.029+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562f94083710 0x562f94092770 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.029+0000 7f67bd9f1740 1 --2- 172.21.15.42:0/2536857910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f94081110 0x562f94092230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.029+0000 7f67bd9f1740 1 -- 172.21.15.42:0/2536857910 >> 172.21.15.42:0/2536857910 conn(0x562f9407c680 msgr2=0x562f9407f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.029+0000 7f679d600640 0 client.15210 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:08.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.030+0000 7f67bd9f1740 1 -- 172.21.15.42:0/2536857910 shutdown_connections 2024-04-20T10:34:08.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.030+0000 7f67bd9f1740 1 -- 172.21.15.42:0/2536857910 wait complete. 2024-04-20T10:34:08.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.030+0000 7f67bd9f1740 20 client.15210 trim_cache size 0 max 16384 2024-04-20T10:34:08.033 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:34:08.033 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:34:08.033 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "a9684aae-4787-42a9-b99f-0d00ebbf6d38", 2024-04-20T10:34:08.033 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:34:08.033 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:34:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.030+0000 7f67b9600640 1 -- 172.21.15.42:0/4192148057 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f67980a3e30 msgr2=0x7f67980a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.030+0000 7f67b9600640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f67980a3e30 0x7f67980a62f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x562f94082360 tx=0x7f67ac009500 comp rx=0 tx=0).stop 2024-04-20T10:34:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.030+0000 7f67b9600640 1 -- 172.21.15.42:0/4192148057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4102fe0 msgr2=0x7f67b4074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.030+0000 7f67b9600640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4102fe0 0x7f67b4074350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f67a00349e0 tx=0x7f67a0032430 comp rx=0 tx=0).stop 2024-04-20T10:34:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.031+0000 7f67b9600640 1 -- 172.21.15.42:0/4192148057 shutdown_connections 2024-04-20T10:34:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.031+0000 7f67b9600640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f67980a3e30 0x7f67980a62f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.031+0000 7f67b9600640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b41039a0 0x7f67b4074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.031+0000 7f67b9600640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b4102fe0 0x7f67b4074350 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.031+0000 7f67b9600640 1 --2- 172.21.15.42:0/4192148057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67b4101e90 0x7f67b4075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.031+0000 7f67b9600640 1 -- 172.21.15.42:0/4192148057 >> 172.21.15.42:0/4192148057 conn(0x7f67b40fd400 msgr2=0x7f67b41063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.031+0000 7f67b9600640 1 -- 172.21.15.42:0/4192148057 shutdown_connections 2024-04-20T10:34:08.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.031+0000 7f67b9600640 1 -- 172.21.15.42:0/4192148057 wait complete. 2024-04-20T10:34:08.045 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:07 smithi042 ceph-mon[17078]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:34:08.046 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:07 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-04-20T10:34:08.046 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:07 smithi042 ceph-mon[17078]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T10:34:08.046 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:07 smithi042 ceph-mon[17078]: Updating smithi042:/etc/ceph/ceph.conf 2024-04-20T10:34:08.046 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:07 smithi042 ceph-mon[17078]: Updating smithi046:/etc/ceph/ceph.conf 2024-04-20T10:34:08.046 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:07 smithi042 ceph-mon[17078]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-20T10:34:08.046 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:07 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2922860338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:34:08.046 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:07 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2501804659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:34:08.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:34:08.048 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:34:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:07 smithi046 ceph-mon[21574]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:34:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:07 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-04-20T10:34:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:07 smithi046 ceph-mon[21574]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T10:34:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:07 smithi046 ceph-mon[21574]: Updating smithi042:/etc/ceph/ceph.conf 2024-04-20T10:34:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:07 smithi046 ceph-mon[21574]: Updating smithi046:/etc/ceph/ceph.conf 2024-04-20T10:34:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:07 smithi046 ceph-mon[21574]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-20T10:34:08.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:07 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2922860338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:34:08.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:07 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2501804659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:34:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:07 smithi162 ceph-mon[21854]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T10:34:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:07 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-04-20T10:34:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:07 smithi162 ceph-mon[21854]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T10:34:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:07 smithi162 ceph-mon[21854]: Updating smithi042:/etc/ceph/ceph.conf 2024-04-20T10:34:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:07 smithi162 ceph-mon[21854]: Updating smithi046:/etc/ceph/ceph.conf 2024-04-20T10:34:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:07 smithi162 ceph-mon[21854]: Updating smithi162:/etc/ceph/ceph.conf 2024-04-20T10:34:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:07 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2922860338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:34:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:07 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2501804659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:34:08.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.219+0000 7f9aea400640 1 -- 172.21.15.42:0/253522869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ae4105220 msgr2=0x7f9ae4107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.219+0000 7f9aea400640 1 --2- 172.21.15.42:0/253522869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ae4105220 0x7f9ae4107610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009900 tx=0x7f9acc02bc40 comp rx=0 tx=0).stop 2024-04-20T10:34:08.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.219+0000 7f9aea400640 1 -- 172.21.15.42:0/253522869 shutdown_connections 2024-04-20T10:34:08.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.219+0000 7f9aea400640 1 --2- 172.21.15.42:0/253522869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae4107b50 0x7f9ae4109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.219+0000 7f9aea400640 1 --2- 172.21.15.42:0/253522869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ae4105220 0x7f9ae4107610 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.219+0000 7f9aea400640 1 --2- 172.21.15.42:0/253522869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae41028f0 0x7f9ae4104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.219+0000 7f9aea400640 1 -- 172.21.15.42:0/253522869 >> 172.21.15.42:0/253522869 conn(0x7f9ae40fc300 msgr2=0x7f9ae40fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.219+0000 7f9aea400640 1 -- 172.21.15.42:0/253522869 shutdown_connections 2024-04-20T10:34:08.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.219+0000 7f9aea400640 1 -- 172.21.15.42:0/253522869 wait complete. 2024-04-20T10:34:08.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.220+0000 7f9aea400640 1 Processor -- start 2024-04-20T10:34:08.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.220+0000 7f9aea400640 1 -- start start 2024-04-20T10:34:08.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.221+0000 7f9aea400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae41028f0 0x7f9ae4102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.221+0000 7f9aea400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae4105220 0x7f9ae41008a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.221+0000 7f9aea400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ae4107b50 0x7f9ae4100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.221+0000 7f9aea400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae410dae0 con 0x7f9ae4107b50 2024-04-20T10:34:08.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.221+0000 7f9aea400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae410d960 con 0x7f9ae41028f0 2024-04-20T10:34:08.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.221+0000 7f9aea400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae410dc60 con 0x7f9ae4105220 2024-04-20T10:34:08.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.221+0000 7f9ae9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ae4107b50 0x7f9ae4100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.221+0000 7f9ae9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae41028f0 0x7f9ae4102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.221+0000 7f9ae9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ae4107b50 0x7f9ae4100de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34494/0 (socket says 172.21.15.42:34494) 2024-04-20T10:34:08.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.221+0000 7f9ae9a00640 1 -- 172.21.15.42:0/626388808 learned_addr learned my addr 172.21.15.42:0/626388808 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:08.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.221+0000 7f9adbe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae4105220 0x7f9ae41008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.222+0000 7f9adbe00640 1 -- 172.21.15.42:0/626388808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae41028f0 msgr2=0x7f9ae4102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.222+0000 7f9adbe00640 1 --2- 172.21.15.42:0/626388808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae41028f0 0x7f9ae4102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.222+0000 7f9adbe00640 1 -- 172.21.15.42:0/626388808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ae4107b50 msgr2=0x7f9ae4100de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.222+0000 7f9adbe00640 1 --2- 172.21.15.42:0/626388808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ae4107b50 0x7f9ae4100de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.222+0000 7f9adbe00640 1 -- 172.21.15.42:0/626388808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae4101670 con 0x7f9ae4105220 2024-04-20T10:34:08.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.222+0000 7f9adbe00640 1 --2- 172.21.15.42:0/626388808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae4105220 0x7f9ae41008a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9acc02c180 tx=0x7f9acc02c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:08.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.222+0000 7f9ae2000640 1 -- 172.21.15.42:0/626388808 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc041070 con 0x7f9ae4105220 2024-04-20T10:34:08.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.222+0000 7f9aea400640 1 -- 172.21.15.42:0/626388808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae41a2240 con 0x7f9ae4105220 2024-04-20T10:34:08.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.222+0000 7f9ae2000640 1 -- 172.21.15.42:0/626388808 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9acc004450 con 0x7f9ae4105220 2024-04-20T10:34:08.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.222+0000 7f9ae2000640 1 -- 172.21.15.42:0/626388808 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc034440 con 0x7f9ae4105220 2024-04-20T10:34:08.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.223+0000 7f9aea400640 1 -- 172.21.15.42:0/626388808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae41a27b0 con 0x7f9ae4105220 2024-04-20T10:34:08.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9aee627740 1 Processor -- start 2024-04-20T10:34:08.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9aee627740 1 -- start start 2024-04-20T10:34:08.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9aee627740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 0x5626628e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9aee627740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 0x5626628e8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9aee627740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e8f00 0x5626628eb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9aee627740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56266284a150 con 0x5626628e8f00 2024-04-20T10:34:08.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9aee627740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5626627f0370 con 0x5626628e33a0 2024-04-20T10:34:08.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9aee627740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562662846110 con 0x5626628e3d90 2024-04-20T10:34:08.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9ae9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 0x5626628e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9ae9000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 0x5626628e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9ae9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 0x5626628e37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52250/0 (socket says 172.21.15.42:52250) 2024-04-20T10:34:08.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9ae9a00640 1 -- 172.21.15.42:0/2964119175 learned_addr learned my addr 172.21.15.42:0/2964119175 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:08.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.226+0000 7f9ae9000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 0x5626628e8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:55848/0 (socket says 172.21.15.42:55848) 2024-04-20T10:34:08.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.227+0000 7f9ae9a00640 1 -- 172.21.15.42:0/2964119175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 msgr2=0x5626628e8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.227+0000 7f9ae9a00640 1 --2- 172.21.15.42:0/2964119175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 0x5626628e8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.227+0000 7f9ae9a00640 1 -- 172.21.15.42:0/2964119175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e8f00 msgr2=0x5626628eb390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:34:08.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.227+0000 7f9ae9a00640 1 --2- 172.21.15.42:0/2964119175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e8f00 0x5626628eb390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.227+0000 7f9ae9a00640 1 -- 172.21.15.42:0/2964119175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626626267d0 con 0x5626628e33a0 2024-04-20T10:34:08.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.227+0000 7f9ae9a00640 1 --2- 172.21.15.42:0/2964119175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 0x5626628e37c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9adc011570 tx=0x7f9adc011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3417f2c55065405 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.227+0000 7f9ae2000640 1 -- 172.21.15.42:0/626388808 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f9acc044070 con 0x7f9ae4105220 2024-04-20T10:34:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.227+0000 7f9ada000640 1 -- 172.21.15.42:0/2964119175 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc004b20 con 0x5626628e33a0 2024-04-20T10:34:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.227+0000 7f9ada000640 1 -- 172.21.15.42:0/2964119175 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc004cc0 con 0x5626628e33a0 2024-04-20T10:34:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.227+0000 7f9ada000640 1 -- 172.21.15.42:0/2964119175 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc005540 con 0x5626628e33a0 2024-04-20T10:34:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.228+0000 7f9aee627740 1 -- 172.21.15.42:0/2964119175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 msgr2=0x5626628e37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.228+0000 7f9aee627740 1 --2- 172.21.15.42:0/2964119175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 0x5626628e37c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9adc011570 tx=0x7f9adc011a30 comp rx=0 tx=0).stop 2024-04-20T10:34:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.229+0000 7f9aee627740 1 -- 172.21.15.42:0/2964119175 shutdown_connections 2024-04-20T10:34:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.229+0000 7f9aee627740 1 --2- 172.21.15.42:0/2964119175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e8f00 0x5626628eb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.229+0000 7f9aee627740 1 --2- 172.21.15.42:0/2964119175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 0x5626628e8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.229+0000 7f9aee627740 1 --2- 172.21.15.42:0/2964119175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 0x5626628e37c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.229+0000 7f9aee627740 1 -- 172.21.15.42:0/2964119175 >> 172.21.15.42:0/2964119175 conn(0x5626628de910 msgr2=0x5626628ded10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.229+0000 7f9aee627740 1 -- 172.21.15.42:0/2964119175 shutdown_connections 2024-04-20T10:34:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.229+0000 7f9aee627740 1 -- 172.21.15.42:0/2964119175 wait complete. 2024-04-20T10:34:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.229+0000 7f9ae2000640 1 --2- 172.21.15.42:0/626388808 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9ac80a3d40 0x7f9ac80a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.229+0000 7f9ae2000640 1 -- 172.21.15.42:0/626388808 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9acc0f22c0 con 0x7f9ae4105220 2024-04-20T10:34:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.229+0000 7f9aee627740 1 -- start start 2024-04-20T10:34:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.229+0000 7f9ae9000640 1 --2- 172.21.15.42:0/626388808 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9ac80a3d40 0x7f9ac80a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.230+0000 7f9aee627740 1 Processor -- start 2024-04-20T10:34:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.230+0000 7f9aee627740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e33a0 0x5626628ed860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.230+0000 7f9aee627740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e3d90 0x5626628edda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.230+0000 7f9aee627740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e8f00 0x5626628f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.230+0000 7f9aee627740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5626628ebeb0 con 0x5626628e33a0 2024-04-20T10:34:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.230+0000 7f9aee627740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562662803740 con 0x5626628e3d90 2024-04-20T10:34:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.230+0000 7f9aee627740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5626628ec030 con 0x5626628e8f00 2024-04-20T10:34:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.230+0000 7f9aee627740 10 client.0 _ll_register_callbacks cb 0x5626627feb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:34:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.230+0000 7f9adbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e3d90 0x5626628edda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9adbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e3d90 0x5626628edda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52262/0 (socket says 172.21.15.42:52262) 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9adbe00640 1 -- 172.21.15.42:0/4211459881 learned_addr learned my addr 172.21.15.42:0/4211459881 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9ae9a00640 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e8f00 0x5626628f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9ae9000640 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e33a0 0x5626628ed860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9adbe00640 1 -- 172.21.15.42:0/4211459881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e8f00 msgr2=0x5626628f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9adbe00640 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e8f00 0x5626628f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9adbe00640 1 -- 172.21.15.42:0/4211459881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e33a0 msgr2=0x5626628ed860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9adbe00640 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e33a0 0x5626628ed860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9adbe00640 1 -- 172.21.15.42:0/4211459881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56266255c3f0 con 0x5626628e3d90 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9ae9000640 1 --2- 172.21.15.42:0/626388808 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9ac80a3d40 0x7f9ac80a6200 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x5626628df880 tx=0x7f9ad4005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9adbe00640 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e3d90 0x5626628edda0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9acc04c000 tx=0x7f9acc0be020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e456e9067a616bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9ad2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9ad2a00640 1 -- 172.21.15.42:0/4211459881 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc04b040 con 0x5626628e3d90 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.231+0000 7f9ad2a00640 1 -- 172.21.15.42:0/4211459881 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9acc0efba0 con 0x5626628e3d90 2024-04-20T10:34:08.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.232+0000 7f9aee627740 1 -- 172.21.15.42:0/4211459881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9adc004cc0 con 0x5626628e3d90 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.232+0000 7f9ad2a00640 1 -- 172.21.15.42:0/4211459881 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9acc004c80 con 0x5626628e3d90 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.232+0000 7f9aee627740 10 client.25337 fetch_fsmap learned FSMap version 19 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.232+0000 7f9aee627740 1 -- 172.21.15.42:0/4211459881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56266251c0f0 con 0x5626628e3d90 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.232+0000 7f9ad2a00640 1 -- 172.21.15.42:0/4211459881 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc004cc0 con 0x5626628e3d90 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.232+0000 7f9ad2a00640 1 -- 172.21.15.42:0/4211459881 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f9acc0c6980 con 0x5626628e3d90 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.232+0000 7f9aee627740 10 client.25337 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.232+0000 7f9aee627740 10 client.25337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.232+0000 7f9aee627740 20 client.25337 populate_metadata read hostname 'smithi042' 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.232+0000 7f9aee627740 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5626628f9610 0x5626628fba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.233+0000 7f9aee627740 4 client.25337 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.233+0000 7f9aee627740 1 -- 172.21.15.42:0/4211459881 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562662857b70 con 0x5626628f9610 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.233+0000 7f9ae9000640 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5626628f9610 0x5626628fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.233+0000 7f9ae9000640 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5626628f9610 0x5626628fba50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ec44a6fb6d9814 server_cookie=1c663c90868bd20d in_seq=0 out_seq=0 2024-04-20T10:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.233+0000 7f9ad2a00640 10 client.25337 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:08.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.234+0000 7f9ad2a00640 1 -- 172.21.15.42:0/4211459881 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562662857b70 con 0x5626628f9610 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.234+0000 7f9ad2a00640 10 client.25337 handle_command_reply: tid=0 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.234+0000 7f9aee627740 1 client.25337 shutdown 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.234+0000 7f9aee627740 1 -- 172.21.15.42:0/4211459881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e3d90 msgr2=0x5626628edda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.234+0000 7f9aee627740 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e3d90 0x5626628edda0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9acc04c000 tx=0x7f9acc0be020 comp rx=0 tx=0).stop 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.235+0000 7f9aee627740 1 -- 172.21.15.42:0/4211459881 shutdown_connections 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.235+0000 7f9aee627740 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5626628f9610 0x5626628fba50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.235+0000 7f9aee627740 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e8f00 0x5626628f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.235+0000 7f9aee627740 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e3d90 0x5626628edda0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.235+0000 7f9ad2a00640 0 client.25337 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.235+0000 7f9aee627740 1 --2- 172.21.15.42:0/4211459881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e33a0 0x5626628ed860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.235+0000 7f9aee627740 1 -- 172.21.15.42:0/4211459881 >> 172.21.15.42:0/4211459881 conn(0x5626628de910 msgr2=0x5626628ddb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.235+0000 7f9aee627740 1 -- 172.21.15.42:0/4211459881 shutdown_connections 2024-04-20T10:34:08.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.235+0000 7f9aee627740 1 -- 172.21.15.42:0/4211459881 wait complete. 2024-04-20T10:34:08.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.235+0000 7f9aee627740 20 client.25337 trim_cache size 0 max 16384 2024-04-20T10:34:08.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.275+0000 7f9aee627740 1 Processor -- start 2024-04-20T10:34:08.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.275+0000 7f9aee627740 1 -- start start 2024-04-20T10:34:08.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.275+0000 7f9aee627740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e33a0 0x5626628decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.275+0000 7f9aee627740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 0x5626628f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.275+0000 7f9aee627740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e8f00 0x5626628e2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.275+0000 7f9aee627740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae41a9720 con 0x5626628e33a0 2024-04-20T10:34:08.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.275+0000 7f9aee627740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae41abfa0 con 0x5626628e8f00 2024-04-20T10:34:08.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.275+0000 7f9aee627740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae41a9d90 con 0x5626628e3d90 2024-04-20T10:34:08.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.275+0000 7f9adbe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 0x5626628f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.275+0000 7f9ae9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e33a0 0x5626628decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.275+0000 7f9adbe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 0x5626628f4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:55864/0 (socket says 172.21.15.42:55864) 2024-04-20T10:34:08.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.275+0000 7f9adbe00640 1 -- 172.21.15.42:0/439753411 learned_addr learned my addr 172.21.15.42:0/439753411 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:08.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.276+0000 7f9ae9a00640 1 --2- 172.21.15.42:0/439753411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e8f00 0x5626628e2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.276+0000 7f9adbe00640 1 -- 172.21.15.42:0/439753411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e8f00 msgr2=0x5626628e2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.276+0000 7f9adbe00640 1 --2- 172.21.15.42:0/439753411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e8f00 0x5626628e2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.276+0000 7f9adbe00640 1 -- 172.21.15.42:0/439753411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e33a0 msgr2=0x5626628decf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.276+0000 7f9adbe00640 1 --2- 172.21.15.42:0/439753411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e33a0 0x5626628decf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.276+0000 7f9adbe00640 1 -- 172.21.15.42:0/439753411 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626628e5910 con 0x5626628e3d90 2024-04-20T10:34:08.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.276+0000 7f9adbe00640 1 --2- 172.21.15.42:0/439753411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 0x5626628f4da0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0be1e0 tx=0x7f9acc0be150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ea186b202a073b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:08.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.276+0000 7f9ad9600640 1 -- 172.21.15.42:0/439753411 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc004c80 con 0x5626628e3d90 2024-04-20T10:34:08.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.276+0000 7f9ad9600640 1 -- 172.21.15.42:0/439753411 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9acc0be250 con 0x5626628e3d90 2024-04-20T10:34:08.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.276+0000 7f9ad9600640 1 -- 172.21.15.42:0/439753411 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc0efd70 con 0x5626628e3d90 2024-04-20T10:34:08.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.277+0000 7f842e400640 1 -- 172.21.15.42:0/1240701613 --> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f8408000bf0 con 0x7f84100a3da0 2024-04-20T10:34:08.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.277+0000 7f9aee627740 1 -- 172.21.15.42:0/439753411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 msgr2=0x5626628f4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.277+0000 7f9aee627740 1 --2- 172.21.15.42:0/439753411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 0x5626628f4da0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0be1e0 tx=0x7f9acc0be150 comp rx=0 tx=0).stop 2024-04-20T10:34:08.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.277+0000 7f9aee627740 1 -- 172.21.15.42:0/439753411 shutdown_connections 2024-04-20T10:34:08.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.277+0000 7f9aee627740 1 --2- 172.21.15.42:0/439753411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e8f00 0x5626628e2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.277+0000 7f9aee627740 1 --2- 172.21.15.42:0/439753411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e3d90 0x5626628f4da0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.277+0000 7f9aee627740 1 --2- 172.21.15.42:0/439753411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e33a0 0x5626628decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.277+0000 7f9aee627740 1 -- 172.21.15.42:0/439753411 >> 172.21.15.42:0/439753411 conn(0x5626628de910 msgr2=0x5626628e1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.277+0000 7f9aee627740 1 -- 172.21.15.42:0/439753411 shutdown_connections 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.277+0000 7f9aee627740 1 -- 172.21.15.42:0/439753411 wait complete. 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.278+0000 7f9aee627740 1 -- start start 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.278+0000 7f9aee627740 1 Processor -- start 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.278+0000 7f9aee627740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 0x5626628f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.278+0000 7f9aee627740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e3d90 0x5626628f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9aee627740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e8f00 0x5626628f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9aee627740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae41aa460 con 0x5626628e3d90 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9aee627740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae41ab8d0 con 0x5626628e33a0 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9aee627740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae41aab30 con 0x5626628e8f00 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9aee627740 10 client.0 _ll_register_callbacks cb 0x7f9acc0b9dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9adbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e3d90 0x5626628f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9adbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e3d90 0x5626628f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34520/0 (socket says 172.21.15.42:34520) 2024-04-20T10:34:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9adbe00640 1 -- 172.21.15.42:0/2778434158 learned_addr learned my addr 172.21.15.42:0/2778434158 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9ae9000640 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 0x5626628f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9ae9a00640 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e8f00 0x5626628f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9ae9000640 1 -- 172.21.15.42:0/2778434158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e8f00 msgr2=0x5626628f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9ae9000640 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e8f00 0x5626628f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9ae9000640 1 -- 172.21.15.42:0/2778434158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e3d90 msgr2=0x5626628f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9ae9000640 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e3d90 0x5626628f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.279+0000 7f9ae9000640 1 -- 172.21.15.42:0/2778434158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626628f46b0 con 0x5626628e33a0 2024-04-20T10:34:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.280+0000 7f9ae9000640 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 0x5626628f2b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4013010 tx=0x7f9ad40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7eea474e950d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.280+0000 7f9ad2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:34:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.280+0000 7f9ad2a00640 1 -- 172.21.15.42:0/2778434158 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad402e030 con 0x5626628e33a0 2024-04-20T10:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.280+0000 7f9ad2a00640 1 -- 172.21.15.42:0/2778434158 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9ad4018030 con 0x5626628e33a0 2024-04-20T10:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.280+0000 7f9aee627740 1 -- 172.21.15.42:0/2778434158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9acc0be250 con 0x5626628e33a0 2024-04-20T10:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.281+0000 7f9ad2a00640 1 -- 172.21.15.42:0/2778434158 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9ad401dd40 con 0x5626628e33a0 2024-04-20T10:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.281+0000 7f9aee627740 10 client.25343 fetch_fsmap learned FSMap version 19 2024-04-20T10:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.281+0000 7f9aee627740 1 -- 172.21.15.42:0/2778434158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562662960330 con 0x5626628e33a0 2024-04-20T10:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.281+0000 7f9ad2a00640 1 -- 172.21.15.42:0/2778434158 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9acc0be250 con 0x5626628e33a0 2024-04-20T10:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.281+0000 7f9ad2a00640 1 -- 172.21.15.42:0/2778434158 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f9ad403caa0 con 0x5626628e33a0 2024-04-20T10:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.281+0000 7f9aee627740 10 client.25343 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.281+0000 7f9aee627740 10 client.25343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.281+0000 7f9aee627740 20 client.25343 populate_metadata read hostname 'smithi042' 2024-04-20T10:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.281+0000 7f9aee627740 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562662960ce0 0x562662963100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.281+0000 7f9aee627740 4 client.25343 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.281+0000 7f9aee627740 1 -- 172.21.15.42:0/2778434158 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5626628519e0 con 0x562662960ce0 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.282+0000 7f9adbe00640 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562662960ce0 0x562662963100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.282+0000 7f9adbe00640 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562662960ce0 0x562662963100 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbbbca16dcb60400 server_cookie=847730d5c873b1a2 in_seq=0 out_seq=0 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.282+0000 7f9ad2a00640 10 client.25343 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.282+0000 7f9ad2a00640 1 -- 172.21.15.42:0/2778434158 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5626628519e0 con 0x562662960ce0 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.282+0000 7f9ad2a00640 10 client.25343 handle_command_reply: tid=0 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.282+0000 7f9aee627740 1 client.25343 shutdown 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.283+0000 7f9aee627740 1 -- 172.21.15.42:0/2778434158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 msgr2=0x5626628f2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.283+0000 7f9aee627740 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 0x5626628f2b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4013010 tx=0x7f9ad40046c0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.283+0000 7f9aee627740 1 -- 172.21.15.42:0/2778434158 shutdown_connections 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.283+0000 7f9aee627740 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562662960ce0 0x562662963100 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.283+0000 7f9ad2a00640 0 client.25343 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.283+0000 7f9aee627740 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5626628e8f00 0x5626628f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.283+0000 7f9aee627740 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5626628e3d90 0x5626628f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.283+0000 7f9aee627740 1 --2- 172.21.15.42:0/2778434158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626628e33a0 0x5626628f2b70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.283+0000 7f9aee627740 1 -- 172.21.15.42:0/2778434158 >> 172.21.15.42:0/2778434158 conn(0x5626628de910 msgr2=0x5626628e2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.283+0000 7f9aee627740 1 -- 172.21.15.42:0/2778434158 shutdown_connections 2024-04-20T10:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.283+0000 7f9aee627740 1 -- 172.21.15.42:0/2778434158 wait complete. 2024-04-20T10:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.283+0000 7f9aee627740 20 client.25343 trim_cache size 0 max 16384 2024-04-20T10:34:08.287 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:34:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T10:34:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:34:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "a9684aae-4787-42a9-b99f-0d00ebbf6d38": { 2024-04-20T10:34:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:34:08.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "a9684aae-4787-42a9-b99f-0d00ebbf6d38", 2024-04-20T10:34:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:34:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:34:08.288 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:34:08.288 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:34:08.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.284+0000 7f9aea400640 1 -- 172.21.15.42:0/626388808 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9ac80a3d40 msgr2=0x7f9ac80a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.284+0000 7f9aea400640 1 --2- 172.21.15.42:0/626388808 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9ac80a3d40 0x7f9ac80a6200 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x5626628df880 tx=0x7f9ad4005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.284+0000 7f9aea400640 1 -- 172.21.15.42:0/626388808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae4105220 msgr2=0x7f9ae41008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.284+0000 7f9aea400640 1 --2- 172.21.15.42:0/626388808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae4105220 0x7f9ae41008a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9acc02c180 tx=0x7f9acc02c660 comp rx=0 tx=0).stop 2024-04-20T10:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.284+0000 7f9aea400640 1 -- 172.21.15.42:0/626388808 shutdown_connections 2024-04-20T10:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.284+0000 7f9aea400640 1 --2- 172.21.15.42:0/626388808 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9ac80a3d40 0x7f9ac80a6200 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.284+0000 7f9aea400640 1 --2- 172.21.15.42:0/626388808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ae4107b50 0x7f9ae4100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.284+0000 7f9aea400640 1 --2- 172.21.15.42:0/626388808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ae4105220 0x7f9ae41008a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.284+0000 7f9aea400640 1 --2- 172.21.15.42:0/626388808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae41028f0 0x7f9ae4102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.284+0000 7f9aea400640 1 -- 172.21.15.42:0/626388808 >> 172.21.15.42:0/626388808 conn(0x7f9ae40fc300 msgr2=0x7f9ae4103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.284+0000 7f9aea400640 1 -- 172.21.15.42:0/626388808 shutdown_connections 2024-04-20T10:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.284+0000 7f9aea400640 1 -- 172.21.15.42:0/626388808 wait complete. 2024-04-20T10:34:08.307 INFO:tasks.cephfs.filesystem:scrub status for tag:a9684aae-4787-42a9-b99f-0d00ebbf6d38 - {'path': '/', 'tag': 'a9684aae-4787-42a9-b99f-0d00ebbf6d38', 'options': 'recursive,force'} 2024-04-20T10:34:08.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.359+0000 7f8425600640 1 -- 172.21.15.42:0/1240701613 <== mgr.15024 v2:172.21.15.42:6800/170978916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8408000bf0 con 0x7f84100a3da0 2024-04-20T10:34:08.359 INFO:teuthology.orchestra.run.smithi042.stdout:Schedule set for path / 2024-04-20T10:34:08.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.361+0000 7f842e400640 1 -- 172.21.15.42:0/1240701613 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f84100a3da0 msgr2=0x7f84100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.361+0000 7f842e400640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f84100a3da0 0x7f84100a6260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f84140349e0 tx=0x7f8414034ab0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.361+0000 7f842e400640 1 -- 172.21.15.42:0/1240701613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8428101e20 msgr2=0x7f8428075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.361+0000 7f842e400640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8428101e20 0x7f8428075d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f841c00ea60 tx=0x7f841c00ef20 comp rx=0 tx=0).stop 2024-04-20T10:34:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.361+0000 7f842e400640 1 -- 172.21.15.42:0/1240701613 shutdown_connections 2024-04-20T10:34:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.361+0000 7f842e400640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f84100a3da0 0x7f84100a6260 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.361+0000 7f842e400640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8428103930 0x7f8428074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.361+0000 7f842e400640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8428102f70 0x7f8428074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.361+0000 7f842e400640 1 --2- 172.21.15.42:0/1240701613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8428101e20 0x7f8428075d10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.361+0000 7f842e400640 1 -- 172.21.15.42:0/1240701613 >> 172.21.15.42:0/1240701613 conn(0x7f84280fd390 msgr2=0x7f8428106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.361+0000 7f842e400640 1 -- 172.21.15.42:0/1240701613 shutdown_connections 2024-04-20T10:34:08.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.361+0000 7f842e400640 1 -- 172.21.15.42:0/1240701613 wait complete. 2024-04-20T10:34:08.378 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-20T10:34:08.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.545+0000 7fa335400640 1 -- 172.21.15.42:0/3604435704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa330103840 msgr2=0x7fa33010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.545+0000 7fa335400640 1 --2- 172.21.15.42:0/3604435704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa330103840 0x7fa33010bd20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009860 tx=0x7fa31c02bc00 comp rx=0 tx=0).stop 2024-04-20T10:34:08.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.545+0000 7fa335400640 1 -- 172.21.15.42:0/3604435704 shutdown_connections 2024-04-20T10:34:08.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.545+0000 7fa335400640 1 --2- 172.21.15.42:0/3604435704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa33010c340 0x7fa33010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.545+0000 7fa335400640 1 --2- 172.21.15.42:0/3604435704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa330103840 0x7fa33010bd20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.545+0000 7fa335400640 1 --2- 172.21.15.42:0/3604435704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa330102f20 0x7fa330103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.545+0000 7fa335400640 1 -- 172.21.15.42:0/3604435704 >> 172.21.15.42:0/3604435704 conn(0x7fa3300fc780 msgr2=0x7fa3300feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.545+0000 7fa335400640 1 -- 172.21.15.42:0/3604435704 shutdown_connections 2024-04-20T10:34:08.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.545+0000 7fa335400640 1 -- 172.21.15.42:0/3604435704 wait complete. 2024-04-20T10:34:08.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.546+0000 7fa335400640 1 Processor -- start 2024-04-20T10:34:08.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.546+0000 7fa335400640 1 -- start start 2024-04-20T10:34:08.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa335400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa330102f20 0x7fa3301a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa335400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa330103840 0x7fa3301a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa335400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33010c340 0x7fa33019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa335400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa330112100 con 0x7fa330102f20 2024-04-20T10:34:08.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa335400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa330111f80 con 0x7fa33010c340 2024-04-20T10:34:08.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa335400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa330112280 con 0x7fa330103840 2024-04-20T10:34:08.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa32fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa330102f20 0x7fa3301a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa32fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa330102f20 0x7fa3301a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34530/0 (socket says 172.21.15.42:34530) 2024-04-20T10:34:08.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa32fe00640 1 -- 172.21.15.42:0/1987692622 learned_addr learned my addr 172.21.15.42:0/1987692622 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:08.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa32f400640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa330103840 0x7fa3301a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa334a00640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33010c340 0x7fa33019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa32fe00640 1 -- 172.21.15.42:0/1987692622 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa330103840 msgr2=0x7fa3301a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa32fe00640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa330103840 0x7fa3301a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa32fe00640 1 -- 172.21.15.42:0/1987692622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33010c340 msgr2=0x7fa33019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa32fe00640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33010c340 0x7fa33019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.547+0000 7fa32fe00640 1 -- 172.21.15.42:0/1987692622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33019af80 con 0x7fa330102f20 2024-04-20T10:34:08.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.548+0000 7fa32fe00640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa330102f20 0x7fa3301a05b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa32000e9f0 tx=0x7fa32000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:08.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.548+0000 7fa32cc00640 1 -- 172.21.15.42:0/1987692622 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32000cd80 con 0x7fa330102f20 2024-04-20T10:34:08.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.548+0000 7fa335400640 1 -- 172.21.15.42:0/1987692622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33019b270 con 0x7fa330102f20 2024-04-20T10:34:08.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.548+0000 7fa32cc00640 1 -- 172.21.15.42:0/1987692622 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa320010910 con 0x7fa330102f20 2024-04-20T10:34:08.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.548+0000 7fa32cc00640 1 -- 172.21.15.42:0/1987692622 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200188f0 con 0x7fa330102f20 2024-04-20T10:34:08.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.548+0000 7fa335400640 1 -- 172.21.15.42:0/1987692622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3301a7530 con 0x7fa330102f20 2024-04-20T10:34:08.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.550+0000 7fa335400640 1 -- 172.21.15.42:0/1987692622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33019b5b0 con 0x7fa330102f20 2024-04-20T10:34:08.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.551+0000 7fa32cc00640 1 -- 172.21.15.42:0/1987692622 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7fa320010ac0 con 0x7fa330102f20 2024-04-20T10:34:08.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.553+0000 7fa32cc00640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa3140a3da0 0x7fa3140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:08.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.553+0000 7fa32cc00640 1 -- 172.21.15.42:0/1987692622 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fa3200d0da0 con 0x7fa330102f20 2024-04-20T10:34:08.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.553+0000 7fa32f400640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa3140a3da0 0x7fa3140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:08.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.554+0000 7fa32f400640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa3140a3da0 0x7fa3140a6260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa31c02c210 tx=0x7fa31c02e040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:08.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.557+0000 7fa32cc00640 1 -- 172.21.15.42:0/1987692622 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa32009cc20 con 0x7fa330102f20 2024-04-20T10:34:08.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.783+0000 7fa335400640 1 -- 172.21.15.42:0/1987692622 --> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fa3301022d0 con 0x7fa3140a3da0 2024-04-20T10:34:08.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.862+0000 7fa32cc00640 1 -- 172.21.15.42:0/1987692622 <== mgr.15024 v2:172.21.15.42:6800/170978916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fa3301022d0 con 0x7fa3140a3da0 2024-04-20T10:34:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:Retention added to path / 2024-04-20T10:34:08.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.865+0000 7fa335400640 1 -- 172.21.15.42:0/1987692622 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa3140a3da0 msgr2=0x7fa3140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.865+0000 7fa335400640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa3140a3da0 0x7fa3140a6260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa31c02c210 tx=0x7fa31c02e040 comp rx=0 tx=0).stop 2024-04-20T10:34:08.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.865+0000 7fa335400640 1 -- 172.21.15.42:0/1987692622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa330102f20 msgr2=0x7fa3301a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:08.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.865+0000 7fa335400640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa330102f20 0x7fa3301a05b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa32000e9f0 tx=0x7fa32000eeb0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.865+0000 7fa335400640 1 -- 172.21.15.42:0/1987692622 shutdown_connections 2024-04-20T10:34:08.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.865+0000 7fa335400640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa3140a3da0 0x7fa3140a6260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.865+0000 7fa335400640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa33010c340 0x7fa33019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.865+0000 7fa335400640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa330103840 0x7fa3301a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.865+0000 7fa335400640 1 --2- 172.21.15.42:0/1987692622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa330102f20 0x7fa3301a05b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:08.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.865+0000 7fa335400640 1 -- 172.21.15.42:0/1987692622 >> 172.21.15.42:0/1987692622 conn(0x7fa3300fc780 msgr2=0x7fa330101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:08.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.865+0000 7fa335400640 1 -- 172.21.15.42:0/1987692622 shutdown_connections 2024-04-20T10:34:08.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:08.865+0000 7fa335400640 1 -- 172.21.15.42:0/1987692622 wait complete. 2024-04-20T10:34:08.883 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-20T10:34:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: Updating smithi162:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:34:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: Updating smithi046:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:34:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:34:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:34:09.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: Updating smithi042:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:34:09.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: Updating smithi162:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:34:09.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: Updating smithi046:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:34:09.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:09.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:09.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:09.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:09.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3484731971' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:34:09.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:34:09.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:34:09.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:34:09.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:08 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.053+0000 7f862b000640 1 -- 172.21.15.42:0/3113798592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8624074310 msgr2=0x7f86240746f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.053+0000 7f862b000640 1 --2- 172.21.15.42:0/3113798592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8624074310 0x7f86240746f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8618009900 tx=0x7f861802bcc0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.053+0000 7f862b000640 1 -- 172.21.15.42:0/3113798592 shutdown_connections 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.054+0000 7f862b000640 1 --2- 172.21.15.42:0/3113798592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f862410e630 0x7f8624110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.054+0000 7f862b000640 1 --2- 172.21.15.42:0/3113798592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8624074c30 0x7f862410e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.054+0000 7f862b000640 1 --2- 172.21.15.42:0/3113798592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8624074310 0x7f86240746f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.054+0000 7f862b000640 1 -- 172.21.15.42:0/3113798592 >> 172.21.15.42:0/3113798592 conn(0x7f86240fd8a0 msgr2=0x7f86240ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.054+0000 7f862b000640 1 -- 172.21.15.42:0/3113798592 shutdown_connections 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.054+0000 7f862b000640 1 -- 172.21.15.42:0/3113798592 wait complete. 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.055+0000 7f862b000640 1 Processor -- start 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.055+0000 7f862b000640 1 -- start start 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.055+0000 7f862b000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8624074310 0x7f86241a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:09.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.055+0000 7f862b000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8624074c30 0x7f86241a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.055+0000 7f862b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f862410e630 0x7f862419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.055+0000 7f862b000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8624114380 con 0x7f8624074c30 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.055+0000 7f862b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8624114200 con 0x7f862410e630 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.055+0000 7f862b000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8624114500 con 0x7f8624074310 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.056+0000 7f8629c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8624074310 0x7f86241a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.056+0000 7f8629c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8624074310 0x7f86241a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:57782/0 (socket says 172.21.15.42:57782) 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.056+0000 7f8629c00640 1 -- 172.21.15.42:0/625641333 learned_addr learned my addr 172.21.15.42:0/625641333 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.056+0000 7f8629200640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8624074c30 0x7f86241a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.056+0000 7f862a600640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f862410e630 0x7f862419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.056+0000 7f8629c00640 1 -- 172.21.15.42:0/625641333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f862410e630 msgr2=0x7f862419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.056+0000 7f8629c00640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f862410e630 0x7f862419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.056+0000 7f8629c00640 1 -- 172.21.15.42:0/625641333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8624074c30 msgr2=0x7f86241a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.056+0000 7f8629c00640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8624074c30 0x7f86241a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.056+0000 7f8629c00640 1 -- 172.21.15.42:0/625641333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862419afc0 con 0x7f8624074310 2024-04-20T10:34:09.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.056+0000 7f862a600640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f862410e630 0x7f862419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:34:09.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.057+0000 7f8629c00640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8624074310 0x7f86241a05f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8618000c00 tx=0x7f8618004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:09.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.057+0000 7f8622000640 1 -- 172.21.15.42:0/625641333 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618042020 con 0x7f8624074310 2024-04-20T10:34:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.057+0000 7f8622000640 1 -- 172.21.15.42:0/625641333 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8618005000 con 0x7f8624074310 2024-04-20T10:34:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.057+0000 7f8622000640 1 -- 172.21.15.42:0/625641333 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618003b20 con 0x7f8624074310 2024-04-20T10:34:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.057+0000 7f862b000640 1 -- 172.21.15.42:0/625641333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862419b250 con 0x7f8624074310 2024-04-20T10:34:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.057+0000 7f862b000640 1 -- 172.21.15.42:0/625641333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86241a7390 con 0x7f8624074310 2024-04-20T10:34:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.058+0000 7f862b000640 1 -- 172.21.15.42:0/625641333 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86240759c0 con 0x7f8624074310 2024-04-20T10:34:09.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.060+0000 7f8622000640 1 -- 172.21.15.42:0/625641333 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f8618045020 con 0x7f8624074310 2024-04-20T10:34:09.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.062+0000 7f8622000640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f860c0a3e20 0x7f860c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:09.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.062+0000 7f8622000640 1 -- 172.21.15.42:0/625641333 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f86180f2dd0 con 0x7f8624074310 2024-04-20T10:34:09.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.062+0000 7f8629200640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f860c0a3e20 0x7f860c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:09.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.063+0000 7f8629200640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f860c0a3e20 0x7f860c0a62e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f86100095f0 tx=0x7f8610005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:09.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.064+0000 7f8622000640 1 -- 172.21.15.42:0/625641333 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f861804c020 con 0x7f8624074310 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: Updating smithi162:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: Updating smithi046:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: Updating smithi042:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: Updating smithi162:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: Updating smithi046:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3484731971' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:34:09.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:34:09.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:34:09.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:08 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:34:09.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: Updating smithi162:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:34:09.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: Updating smithi046:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: Updating smithi042:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.conf 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: Updating smithi162:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: Updating smithi046:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3484731971' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:34:09.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:08 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:34:09.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.320+0000 7f862b000640 1 -- 172.21.15.42:0/625641333 --> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f86240623a0 con 0x7f860c0a3e20 2024-04-20T10:34:09.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.338+0000 7f8622000640 1 -- 172.21.15.42:0/625641333 <== mgr.15024 v2:172.21.15.42:6800/170978916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f86240623a0 con 0x7f860c0a3e20 2024-04-20T10:34:09.338 INFO:teuthology.orchestra.run.smithi042.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-20T00:00:00", "created": "2024-04-20T10:34:08", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-20T10:34:09.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.342+0000 7f862b000640 1 -- 172.21.15.42:0/625641333 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f860c0a3e20 msgr2=0x7f860c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:09.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.342+0000 7f862b000640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f860c0a3e20 0x7f860c0a62e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f86100095f0 tx=0x7f8610005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:09.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.342+0000 7f862b000640 1 -- 172.21.15.42:0/625641333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8624074310 msgr2=0x7f86241a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:09.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.342+0000 7f862b000640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8624074310 0x7f86241a05f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8618000c00 tx=0x7f8618004360 comp rx=0 tx=0).stop 2024-04-20T10:34:09.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.342+0000 7f862b000640 1 -- 172.21.15.42:0/625641333 shutdown_connections 2024-04-20T10:34:09.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.342+0000 7f862b000640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f860c0a3e20 0x7f860c0a62e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.342+0000 7f862b000640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f862410e630 0x7f862419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.342+0000 7f862b000640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8624074c30 0x7f86241a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.342+0000 7f862b000640 1 --2- 172.21.15.42:0/625641333 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8624074310 0x7f86241a05f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.342+0000 7f862b000640 1 -- 172.21.15.42:0/625641333 >> 172.21.15.42:0/625641333 conn(0x7f86240fd8a0 msgr2=0x7f86240ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:09.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.342+0000 7f862b000640 1 -- 172.21.15.42:0/625641333 shutdown_connections 2024-04-20T10:34:09.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.343+0000 7f862b000640 1 -- 172.21.15.42:0/625641333 wait complete. 2024-04-20T10:34:09.361 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-20T10:34:09.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.556+0000 7f4d5ca00640 1 -- 172.21.15.42:0/1782988195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d58104bb0 msgr2=0x7f4d58108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.556+0000 7f4d5ca00640 1 --2- 172.21.15.42:0/1782988195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d58104bb0 0x7f4d58108bf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009860 tx=0x7f4d4002bb70 comp rx=0 tx=0).stop 2024-04-20T10:34:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.556+0000 7f4d5ca00640 1 -- 172.21.15.42:0/1782988195 shutdown_connections 2024-04-20T10:34:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.556+0000 7f4d5ca00640 1 --2- 172.21.15.42:0/1782988195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d581092d0 0x7f4d58110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.556+0000 7f4d5ca00640 1 --2- 172.21.15.42:0/1782988195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d58104bb0 0x7f4d58108bf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.556+0000 7f4d5ca00640 1 --2- 172.21.15.42:0/1782988195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d58104200 0x7f4d581045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.556+0000 7f4d5ca00640 1 -- 172.21.15.42:0/1782988195 >> 172.21.15.42:0/1782988195 conn(0x7f4d580ffa40 msgr2=0x7f4d58101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.556+0000 7f4d5ca00640 1 -- 172.21.15.42:0/1782988195 shutdown_connections 2024-04-20T10:34:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.556+0000 7f4d5ca00640 1 -- 172.21.15.42:0/1782988195 wait complete. 2024-04-20T10:34:09.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.557+0000 7f4d5ca00640 1 Processor -- start 2024-04-20T10:34:09.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.558+0000 7f4d5ca00640 1 -- start start 2024-04-20T10:34:09.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.558+0000 7f4d5ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d58104200 0x7f4d581a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:09.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.558+0000 7f4d5ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d581092d0 0x7f4d581a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:09.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.558+0000 7f4d5ca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5819a6c0 0x7f4d5819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:09.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.558+0000 7f4d5ca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4d58114360 con 0x7f4d58104200 2024-04-20T10:34:09.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.558+0000 7f4d5ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d581141e0 con 0x7f4d581092d0 2024-04-20T10:34:09.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.558+0000 7f4d5ca00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4d581144e0 con 0x7f4d5819a6c0 2024-04-20T10:34:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.558+0000 7f4d57400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d58104200 0x7f4d581a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.558+0000 7f4d57400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d58104200 0x7f4d581a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52872/0 (socket says 172.21.15.42:52872) 2024-04-20T10:34:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.558+0000 7f4d57400640 1 -- 172.21.15.42:0/89560999 learned_addr learned my addr 172.21.15.42:0/89560999 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.558+0000 7f4d57e00640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5819a6c0 0x7f4d5819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.558+0000 7f4d4fe00640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d581092d0 0x7f4d581a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.559+0000 7f4d4fe00640 1 -- 172.21.15.42:0/89560999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5819a6c0 msgr2=0x7f4d5819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.559+0000 7f4d4fe00640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5819a6c0 0x7f4d5819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.559+0000 7f4d4fe00640 1 -- 172.21.15.42:0/89560999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d58104200 msgr2=0x7f4d581a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.559+0000 7f4d4fe00640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d58104200 0x7f4d581a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.559+0000 7f4d4fe00640 1 -- 172.21.15.42:0/89560999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5819b2c0 con 0x7f4d581092d0 2024-04-20T10:34:09.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.559+0000 7f4d57400640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d58104200 0x7f4d581a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.559+0000 7f4d4fe00640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d581092d0 0x7f4d581a0b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4d40037040 tx=0x7f4d400045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.559+0000 7f4d54c00640 1 -- 172.21.15.42:0/89560999 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40041020 con 0x7f4d581092d0 2024-04-20T10:34:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.560+0000 7f4d54c00640 1 -- 172.21.15.42:0/89560999 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d4002c9f0 con 0x7f4d581092d0 2024-04-20T10:34:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.560+0000 7f4d5ca00640 1 -- 172.21.15.42:0/89560999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5819b550 con 0x7f4d581092d0 2024-04-20T10:34:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.560+0000 7f4d54c00640 1 -- 172.21.15.42:0/89560999 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40039040 con 0x7f4d581092d0 2024-04-20T10:34:09.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.560+0000 7f4d5ca00640 1 -- 172.21.15.42:0/89560999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d581a7690 con 0x7f4d581092d0 2024-04-20T10:34:09.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.561+0000 7f4d5ca00640 1 -- 172.21.15.42:0/89560999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d581056d0 con 0x7f4d581092d0 2024-04-20T10:34:09.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.563+0000 7f4d54c00640 1 -- 172.21.15.42:0/89560999 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f4d40044020 con 0x7f4d581092d0 2024-04-20T10:34:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.564+0000 7f4d54c00640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4d3c0a3da0 0x7f4d3c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.565+0000 7f4d54c00640 1 -- 172.21.15.42:0/89560999 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4d400f1fe0 con 0x7f4d581092d0 2024-04-20T10:34:09.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.565+0000 7f4d57400640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4d3c0a3da0 0x7f4d3c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:09.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.565+0000 7f4d57400640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4d3c0a3da0 0x7f4d3c0a6260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4d480058c0 tx=0x7f4d48009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:09.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.569+0000 7f4d54c00640 1 -- 172.21.15.42:0/89560999 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4d4004b020 con 0x7f4d581092d0 2024-04-20T10:34:09.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.804+0000 7f4d5ca00640 1 -- 172.21.15.42:0/89560999 --> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f4d58104bb0 con 0x7f4d3c0a3da0 2024-04-20T10:34:09.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.819+0000 7f4d54c00640 1 -- 172.21.15.42:0/89560999 <== mgr.15024 v2:172.21.15.42:6800/170978916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f4d58104bb0 con 0x7f4d3c0a3da0 2024-04-20T10:34:09.819 INFO:teuthology.orchestra.run.smithi042.stdout:/ 1m 6m3h 2024-04-20T10:34:09.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.821+0000 7f4d5ca00640 1 -- 172.21.15.42:0/89560999 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4d3c0a3da0 msgr2=0x7f4d3c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:09.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.821+0000 7f4d5ca00640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4d3c0a3da0 0x7f4d3c0a6260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4d480058c0 tx=0x7f4d48009260 comp rx=0 tx=0).stop 2024-04-20T10:34:09.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.821+0000 7f4d5ca00640 1 -- 172.21.15.42:0/89560999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d581092d0 msgr2=0x7f4d581a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:09.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.821+0000 7f4d5ca00640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d581092d0 0x7f4d581a0b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4d40037040 tx=0x7f4d400045b0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.821+0000 7f4d5ca00640 1 -- 172.21.15.42:0/89560999 shutdown_connections 2024-04-20T10:34:09.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.821+0000 7f4d5ca00640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4d3c0a3da0 0x7f4d3c0a6260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.821+0000 7f4d5ca00640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5819a6c0 0x7f4d5819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.821+0000 7f4d5ca00640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d581092d0 0x7f4d581a0b40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.821+0000 7f4d5ca00640 1 --2- 172.21.15.42:0/89560999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d58104200 0x7f4d581a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:09.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.821+0000 7f4d5ca00640 1 -- 172.21.15.42:0/89560999 >> 172.21.15.42:0/89560999 conn(0x7f4d580ffa40 msgr2=0x7f4d58101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:09.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.822+0000 7f4d5ca00640 1 -- 172.21.15.42:0/89560999 shutdown_connections 2024-04-20T10:34:09.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:09.822+0000 7f4d5ca00640 1 -- 172.21.15.42:0/89560999 wait complete. 2024-04-20T10:34:09.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-20T10:34:09.929 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-20T10:34:09.938 INFO:teuthology.run_tasks:Running task workunit... 2024-04-20T10:34:09.988 INFO:tasks.workunit:Pulling workunits from ref a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:34:09.988 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-20T10:34:09.988 INFO:tasks.workunit:timeout=3h 2024-04-20T10:34:09.988 INFO:tasks.workunit:cleanup=True 2024-04-20T10:34:09.989 DEBUG:teuthology.orchestra.run.smithi042:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-20T10:34:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-20T10:34:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-20T10:34:10.050 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-20T10:34:10.050 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:34:10.050 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-20T10:34:10.050 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-20 10:31:41.815925697 +0000 2024-04-20T10:34:10.050 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-20 10:31:41.815925697 +0000 2024-04-20T10:34:10.050 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-20 10:33:58.791585678 +0000 2024-04-20T10:34:10.050 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-20 10:31:41.815925697 +0000 2024-04-20T10:34:10.051 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-20T10:34:10.051 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-20T10:34:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:09 smithi046 ceph-mon[21574]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:34:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:09 smithi046 ceph-mon[21574]: from='client.25307 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:34:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:09 smithi046 ceph-mon[21574]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-20T10:34:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:09 smithi046 ceph-mon[21574]: Updating smithi042:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:34:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:09 smithi046 ceph-mon[21574]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:34:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:09 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:09 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:09 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:10.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:09 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:34:10.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:09 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:34:10.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:09 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:34:10.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:09 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:09 smithi042 ceph-mon[17078]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:34:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:09 smithi042 ceph-mon[17078]: from='client.25307 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:34:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:09 smithi042 ceph-mon[17078]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-20T10:34:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:09 smithi042 ceph-mon[17078]: Updating smithi042:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:34:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:09 smithi042 ceph-mon[17078]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:34:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:09 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:09 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:09 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:09 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:34:10.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:09 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:34:10.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:09 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:34:10.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:09 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:09 smithi162 ceph-mon[21854]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-04-20T10:34:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:09 smithi162 ceph-mon[21854]: from='client.25307 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:34:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:09 smithi162 ceph-mon[21854]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-20T10:34:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:09 smithi162 ceph-mon[21854]: Updating smithi042:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T10:34:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:09 smithi162 ceph-mon[21854]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:34:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:09 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:09 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:09 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:09 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:34:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:09 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:34:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:09 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:34:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:09 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:34:10.156 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:34:10.220 INFO:tasks.workunit.client.0.smithi042.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-20T10:34:11.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:10 smithi046 ceph-mon[21574]: from='client.25369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:34:11.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:10 smithi046 ceph-mon[21574]: from='client.25358 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:34:11.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:10 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:34:11.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:10 smithi042 ceph-mon[17078]: from='client.25369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:34:11.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:10 smithi042 ceph-mon[17078]: from='client.25358 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:34:11.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:10 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:34:11.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:10 smithi162 ceph-mon[21854]: from='client.25369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:34:11.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:10 smithi162 ceph-mon[21854]: from='client.25358 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:34:11.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:10 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:34:12.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:11 smithi046 ceph-mon[21574]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-20T10:34:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:11 smithi042 ceph-mon[17078]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-20T10:34:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:11 smithi162 ceph-mon[21854]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-20T10:34:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:13 smithi046 ceph-mon[21574]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 47 op/s 2024-04-20T10:34:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:13 smithi042 ceph-mon[17078]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 47 op/s 2024-04-20T10:34:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:13 smithi162 ceph-mon[21854]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 47 op/s 2024-04-20T10:34:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:15 smithi046 ceph-mon[21574]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-20T10:34:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:15 smithi042 ceph-mon[17078]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-20T10:34:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:15 smithi162 ceph-mon[21854]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-20T10:34:18.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:17 smithi046 ceph-mon[21574]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-20T10:34:18.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:17 smithi042 ceph-mon[17078]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-20T10:34:18.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:17 smithi162 ceph-mon[21854]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-04-20T10:34:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:19 smithi046 ceph-mon[21574]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-20T10:34:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:19 smithi042 ceph-mon[17078]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-20T10:34:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:19 smithi162 ceph-mon[21854]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-20T10:34:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:21 smithi046 ceph-mon[21574]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 33 op/s 2024-04-20T10:34:22.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:21 smithi042 ceph-mon[17078]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 33 op/s 2024-04-20T10:34:22.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:21 smithi162 ceph-mon[21854]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 33 op/s 2024-04-20T10:34:24.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:23 smithi046 ceph-mon[21574]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 34 op/s 2024-04-20T10:34:24.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:23 smithi042 ceph-mon[17078]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 34 op/s 2024-04-20T10:34:24.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:23 smithi162 ceph-mon[21854]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 34 op/s 2024-04-20T10:34:26.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:25 smithi042 ceph-mon[17078]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:26.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:25 smithi162 ceph-mon[21854]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:26.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:25 smithi046 ceph-mon[21574]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:28.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:27 smithi042 ceph-mon[17078]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:28.105 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:27 smithi046 ceph-mon[21574]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:28.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:27 smithi162 ceph-mon[21854]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:30.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:29 smithi042 ceph-mon[17078]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:34:30.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:29 smithi162 ceph-mon[21854]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:34:30.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:29 smithi046 ceph-mon[21574]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:34:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:31 smithi042 ceph-mon[17078]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:32.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:31 smithi162 ceph-mon[21854]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:32.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:31 smithi046 ceph-mon[21574]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:34.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:33 smithi042 ceph-mon[17078]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:34:34.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:33 smithi162 ceph-mon[21854]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:34:34.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:33 smithi046 ceph-mon[21574]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:34:35.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:34 smithi042 ceph-mon[17078]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:35.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:34 smithi162 ceph-mon[21854]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:35.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:34 smithi046 ceph-mon[21574]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:37.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:37 smithi046 ceph-mon[21574]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:37.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:37 smithi042 ceph-mon[17078]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:37 smithi162 ceph-mon[21854]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:38.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:34:38.308 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:34:38.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.538+0000 7f349dc00640 1 -- 172.21.15.42:0/3844368972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3498076db0 msgr2=0x7f3498112900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:38.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.538+0000 7f349dc00640 1 --2- 172.21.15.42:0/3844368972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3498076db0 0x7f3498112900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3480009930 tx=0x7f348002bc20 comp rx=0 tx=0).stop 2024-04-20T10:34:38.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.538+0000 7f349dc00640 1 -- 172.21.15.42:0/3844368972 shutdown_connections 2024-04-20T10:34:38.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.538+0000 7f349dc00640 1 --2- 172.21.15.42:0/3844368972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3498112e40 0x7f3498115230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.539+0000 7f349dc00640 1 --2- 172.21.15.42:0/3844368972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3498076db0 0x7f3498112900 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.539+0000 7f349dc00640 1 --2- 172.21.15.42:0/3844368972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498076490 0x7f3498076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.539+0000 7f349dc00640 1 -- 172.21.15.42:0/3844368972 >> 172.21.15.42:0/3844368972 conn(0x7f34980ffcf0 msgr2=0x7f3498102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:38.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.539+0000 7f349dc00640 1 -- 172.21.15.42:0/3844368972 shutdown_connections 2024-04-20T10:34:38.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.539+0000 7f349dc00640 1 -- 172.21.15.42:0/3844368972 wait complete. 2024-04-20T10:34:38.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.540+0000 7f349dc00640 1 Processor -- start 2024-04-20T10:34:38.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.540+0000 7f349dc00640 1 -- start start 2024-04-20T10:34:38.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.541+0000 7f349dc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3498076490 0x7f349819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.541+0000 7f349dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3498076db0 0x7f349819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.541+0000 7f349dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498112e40 0x7f34981a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.541+0000 7f349dc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f34981189f0 con 0x7f3498112e40 2024-04-20T10:34:38.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.541+0000 7f349dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3498118870 con 0x7f3498076db0 2024-04-20T10:34:38.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.541+0000 7f349dc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3498118b70 con 0x7f3498076490 2024-04-20T10:34:38.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.541+0000 7f3497400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498112e40 0x7f34981a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.541+0000 7f3496a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3498076490 0x7f349819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.541+0000 7f3497400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498112e40 0x7f34981a38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37324/0 (socket says 172.21.15.42:37324) 2024-04-20T10:34:38.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.541+0000 7f3497400640 1 -- 172.21.15.42:0/886889545 learned_addr learned my addr 172.21.15.42:0/886889545 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:38.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.541+0000 7f348fe00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3498076db0 0x7f349819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.542+0000 7f348fe00640 1 -- 172.21.15.42:0/886889545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3498076490 msgr2=0x7f349819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:38.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.542+0000 7f348fe00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3498076490 0x7f349819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.542+0000 7f348fe00640 1 -- 172.21.15.42:0/886889545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498112e40 msgr2=0x7f34981a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:38.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.542+0000 7f348fe00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498112e40 0x7f34981a38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.542+0000 7f348fe00640 1 -- 172.21.15.42:0/886889545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34981a3fe0 con 0x7f3498076db0 2024-04-20T10:34:38.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.542+0000 7f3496a00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3498076490 0x7f349819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:38.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.542+0000 7f348fe00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3498076db0 0x7f349819f530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3480036040 tx=0x7f3480004340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:38.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.542+0000 7f348f400640 1 -- 172.21.15.42:0/886889545 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480043020 con 0x7f3498076db0 2024-04-20T10:34:38.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.542+0000 7f348f400640 1 -- 172.21.15.42:0/886889545 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34800054e0 con 0x7f3498076db0 2024-04-20T10:34:38.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.542+0000 7f349dc00640 1 -- 172.21.15.42:0/886889545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34981a4270 con 0x7f3498076db0 2024-04-20T10:34:38.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.542+0000 7f348f400640 1 -- 172.21.15.42:0/886889545 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3480003ab0 con 0x7f3498076db0 2024-04-20T10:34:38.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.543+0000 7f349dc00640 1 -- 172.21.15.42:0/886889545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34981abb60 con 0x7f3498076db0 2024-04-20T10:34:38.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.546+0000 7f34a0981740 1 Processor -- start 2024-04-20T10:34:38.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.546+0000 7f34a0981740 1 -- start start 2024-04-20T10:34:38.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.546+0000 7f348f400640 1 -- 172.21.15.42:0/886889545 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f3480046020 con 0x7f3498076db0 2024-04-20T10:34:38.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f34a0981740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 0x563f9351b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f34a0981740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 0x563f93524160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f34a0981740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f93526a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f34a0981740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563f93481150 con 0x563f9351b1a0 2024-04-20T10:34:38.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f34a0981740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563f93427370 con 0x563f935246a0 2024-04-20T10:34:38.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f34a0981740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563f9347d110 con 0x563f9351bac0 2024-04-20T10:34:38.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f3497400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 0x563f9351b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f3497400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 0x563f9351b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37326/0 (socket says 172.21.15.42:37326) 2024-04-20T10:34:38.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f3497400640 1 -- 172.21.15.42:0/3878506175 learned_addr learned my addr 172.21.15.42:0/3878506175 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:38.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f3496a00640 1 --2- 172.21.15.42:0/3878506175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 0x563f93524160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f348fe00640 1 --2- 172.21.15.42:0/3878506175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f93526a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f3497400640 1 -- 172.21.15.42:0/3878506175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 msgr2=0x563f93524160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:38.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f3497400640 1 --2- 172.21.15.42:0/3878506175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 0x563f93524160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f3497400640 1 -- 172.21.15.42:0/3878506175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 msgr2=0x563f93526a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:38.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f3497400640 1 --2- 172.21.15.42:0/3878506175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f93526a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.547+0000 7f3497400640 1 -- 172.21.15.42:0/3878506175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f9325d7d0 con 0x563f9351b1a0 2024-04-20T10:34:38.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.548+0000 7f348fe00640 1 --2- 172.21.15.42:0/3878506175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f93526a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:34:38.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.548+0000 7f3497400640 1 --2- 172.21.15.42:0/3878506175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 0x563f9351b580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3490010450 tx=0x7f3490010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6845737baedfc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:38.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.548+0000 7f348f400640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f34780a3e30 0x7f34780a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.548+0000 7f3487e00640 1 -- 172.21.15.42:0/3878506175 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490011760 con 0x563f9351b1a0 2024-04-20T10:34:38.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.548+0000 7f3487e00640 1 -- 172.21.15.42:0/3878506175 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3490011de0 con 0x563f9351b1a0 2024-04-20T10:34:38.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.548+0000 7f348f400640 1 -- 172.21.15.42:0/886889545 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f34800f2700 con 0x7f3498076db0 2024-04-20T10:34:38.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.548+0000 7f3487e00640 1 -- 172.21.15.42:0/3878506175 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490019e40 con 0x563f9351b1a0 2024-04-20T10:34:38.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.548+0000 7f3496a00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f34780a3e30 0x7f34780a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.549+0000 7f34a0981740 1 -- 172.21.15.42:0/3878506175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 msgr2=0x563f9351b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.549+0000 7f34a0981740 1 --2- 172.21.15.42:0/3878506175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 0x563f9351b580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3490010450 tx=0x7f3490010910 comp rx=0 tx=0).stop 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.549+0000 7f3496a00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f34780a3e30 0x7f34780a62f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x563f9351ac70 tx=0x7f3488005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.549+0000 7f34a0981740 1 -- 172.21.15.42:0/3878506175 shutdown_connections 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.549+0000 7f34a0981740 1 --2- 172.21.15.42:0/3878506175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f93526a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.549+0000 7f34a0981740 1 --2- 172.21.15.42:0/3878506175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 0x563f93524160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.549+0000 7f34a0981740 1 --2- 172.21.15.42:0/3878506175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 0x563f9351b580 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.549+0000 7f34a0981740 1 -- 172.21.15.42:0/3878506175 >> 172.21.15.42:0/3878506175 conn(0x563f935158d0 msgr2=0x563f93515cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.549+0000 7f34a0981740 1 -- 172.21.15.42:0/3878506175 shutdown_connections 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.549+0000 7f34a0981740 1 -- 172.21.15.42:0/3878506175 wait complete. 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.550+0000 7f34a0981740 1 -- start start 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.550+0000 7f34a0981740 1 Processor -- start 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.550+0000 7f34a0981740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351b1a0 0x563f93521700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.550+0000 7f34a0981740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351bac0 0x563f93521c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.550+0000 7f34a0981740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f93522180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.550+0000 7f34a0981740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563f93527490 con 0x563f9351bac0 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.550+0000 7f34a0981740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563f9343a740 con 0x563f935246a0 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.550+0000 7f34a0981740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563f93527610 con 0x563f9351b1a0 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.551+0000 7f34a0981740 10 client.0 _ll_register_callbacks cb 0x563f93435b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.551+0000 7f3496a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351b1a0 0x563f93521700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.551+0000 7f348fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351bac0 0x563f93521c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.551+0000 7f3496a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351b1a0 0x563f93521700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:46402/0 (socket says 172.21.15.42:46402) 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.551+0000 7f3496a00640 1 -- 172.21.15.42:0/883514898 learned_addr learned my addr 172.21.15.42:0/883514898 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.551+0000 7f3497400640 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f93522180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.551+0000 7f3496a00640 1 -- 172.21.15.42:0/883514898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 msgr2=0x563f93522180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.551+0000 7f3496a00640 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f93522180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.551+0000 7f3496a00640 1 -- 172.21.15.42:0/883514898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351bac0 msgr2=0x563f93521c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.551+0000 7f3496a00640 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351bac0 0x563f93521c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.551+0000 7f3496a00640 1 -- 172.21.15.42:0/883514898 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f931933f0 con 0x563f9351b1a0 2024-04-20T10:34:38.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.551+0000 7f348fe00640 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351bac0 0x563f93521c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:38.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.552+0000 7f3496a00640 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351b1a0 0x563f93521700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f348800fb20 tx=0x7f3488004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b71cbb17220da2aa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:38.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.552+0000 7f3484c00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:34:38.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.552+0000 7f3484c00640 1 -- 172.21.15.42:0/883514898 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488004020 con 0x563f9351b1a0 2024-04-20T10:34:38.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.552+0000 7f34a0981740 1 -- 172.21.15.42:0/883514898 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3490011de0 con 0x563f9351b1a0 2024-04-20T10:34:38.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.552+0000 7f3484c00640 1 -- 172.21.15.42:0/883514898 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f3488018030 con 0x563f9351b1a0 2024-04-20T10:34:38.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.553+0000 7f3484c00640 1 -- 172.21.15.42:0/883514898 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f34880041c0 con 0x563f9351b1a0 2024-04-20T10:34:38.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.553+0000 7f34a0981740 10 client.25387 fetch_fsmap learned FSMap version 19 2024-04-20T10:34:38.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.553+0000 7f34a0981740 1 -- 172.21.15.42:0/883514898 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563f931530f0 con 0x563f9351b1a0 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.553+0000 7f3484c00640 1 -- 172.21.15.42:0/883514898 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3490011de0 con 0x563f9351b1a0 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.553+0000 7f3484c00640 1 -- 172.21.15.42:0/883514898 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f3488003740 con 0x563f9351b1a0 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.554+0000 7f34a0981740 10 client.25387 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.554+0000 7f34a0981740 10 client.25387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.554+0000 7f34a0981740 20 client.25387 populate_metadata read hostname 'smithi042' 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.554+0000 7f34a0981740 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563f93534aa0 0x563f93536f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.554+0000 7f34a0981740 4 client.25387 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.554+0000 7f34a0981740 1 -- 172.21.15.42:0/883514898 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f9348eb70 con 0x563f93534aa0 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.554+0000 7f348fe00640 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563f93534aa0 0x563f93536f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.555+0000 7f348fe00640 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563f93534aa0 0x563f93536f10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f14cb71e0424c8 server_cookie=cd10add64fca6657 in_seq=0 out_seq=0 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.555+0000 7f3484c00640 10 client.25387 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f3484c00640 1 -- 172.21.15.42:0/883514898 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f9348eb70 con 0x563f93534aa0 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f3484c00640 10 client.25387 handle_command_reply: tid=0 2024-04-20T10:34:38.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f34a0981740 1 client.25387 shutdown 2024-04-20T10:34:38.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f34a0981740 1 -- 172.21.15.42:0/883514898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351b1a0 msgr2=0x563f93521700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:38.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f34a0981740 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351b1a0 0x563f93521700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f348800fb20 tx=0x7f3488004730 comp rx=0 tx=0).stop 2024-04-20T10:34:38.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f34a0981740 1 -- 172.21.15.42:0/883514898 shutdown_connections 2024-04-20T10:34:38.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f34a0981740 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563f93534aa0 0x563f93536f10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f34a0981740 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f93522180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f34a0981740 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351bac0 0x563f93521c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f3484c00640 0 client.25387 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:38.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f34a0981740 1 --2- 172.21.15.42:0/883514898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351b1a0 0x563f93521700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f34a0981740 1 -- 172.21.15.42:0/883514898 >> 172.21.15.42:0/883514898 conn(0x563f935158d0 msgr2=0x563f93517620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:38.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f34a0981740 1 -- 172.21.15.42:0/883514898 shutdown_connections 2024-04-20T10:34:38.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f34a0981740 1 -- 172.21.15.42:0/883514898 wait complete. 2024-04-20T10:34:38.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.556+0000 7f34a0981740 20 client.25387 trim_cache size 0 max 16384 2024-04-20T10:34:38.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.576+0000 7f34a0981740 1 Processor -- start 2024-04-20T10:34:38.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.576+0000 7f34a0981740 1 -- start start 2024-04-20T10:34:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.576+0000 7f34a0981740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f9351b1a0 0x563f93523e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.576+0000 7f34a0981740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 0x563f93520640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.576+0000 7f34a0981740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f935246a0 0x563f93530350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.576+0000 7f34a0981740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f34981b23e0 con 0x563f935246a0 2024-04-20T10:34:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.576+0000 7f34a0981740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34981b4c60 con 0x563f9351b1a0 2024-04-20T10:34:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.576+0000 7f34a0981740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f34981b2a50 con 0x563f9351bac0 2024-04-20T10:34:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.577+0000 7f3497400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f935246a0 0x563f93530350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.577+0000 7f3496a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f9351b1a0 0x563f93523e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.577+0000 7f3497400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f935246a0 0x563f93530350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37338/0 (socket says 172.21.15.42:37338) 2024-04-20T10:34:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.577+0000 7f3497400640 1 -- 172.21.15.42:0/198174304 learned_addr learned my addr 172.21.15.42:0/198174304 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.577+0000 7f348fe00640 1 --2- 172.21.15.42:0/198174304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 0x563f93520640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.577+0000 7f3496a00640 1 -- 172.21.15.42:0/198174304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 msgr2=0x563f93520640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.577+0000 7f3496a00640 1 --2- 172.21.15.42:0/198174304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 0x563f93520640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.577+0000 7f3496a00640 1 -- 172.21.15.42:0/198174304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f935246a0 msgr2=0x563f93530350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.577+0000 7f3496a00640 1 --2- 172.21.15.42:0/198174304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f935246a0 0x563f93530350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.577+0000 7f3496a00640 1 -- 172.21.15.42:0/198174304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f935336a0 con 0x563f9351b1a0 2024-04-20T10:34:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.578+0000 7f3496a00640 1 --2- 172.21.15.42:0/198174304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f9351b1a0 0x563f93523e30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3488003be0 tx=0x7f34880092a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84814255725140ad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:38.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.578+0000 7f3487400640 1 -- 172.21.15.42:0/198174304 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348800f910 con 0x563f9351b1a0 2024-04-20T10:34:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.578+0000 7f3487400640 1 -- 172.21.15.42:0/198174304 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3488004360 con 0x563f9351b1a0 2024-04-20T10:34:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.578+0000 7f3487400640 1 -- 172.21.15.42:0/198174304 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488004d70 con 0x563f9351b1a0 2024-04-20T10:34:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.579+0000 7f34a0981740 1 -- 172.21.15.42:0/198174304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f9351b1a0 msgr2=0x563f93523e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.579+0000 7f34a0981740 1 --2- 172.21.15.42:0/198174304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f9351b1a0 0x563f93523e30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3488003be0 tx=0x7f34880092a0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.579+0000 7f34a0981740 1 -- 172.21.15.42:0/198174304 shutdown_connections 2024-04-20T10:34:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.579+0000 7f34a0981740 1 --2- 172.21.15.42:0/198174304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f935246a0 0x563f93530350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.579+0000 7f34a0981740 1 --2- 172.21.15.42:0/198174304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 0x563f93520640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.579+0000 7f34a0981740 1 --2- 172.21.15.42:0/198174304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f9351b1a0 0x563f93523e30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.579+0000 7f34a0981740 1 -- 172.21.15.42:0/198174304 >> 172.21.15.42:0/198174304 conn(0x563f935158d0 msgr2=0x563f935148c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.579+0000 7f34a0981740 1 -- 172.21.15.42:0/198174304 shutdown_connections 2024-04-20T10:34:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.579+0000 7f34a0981740 1 -- 172.21.15.42:0/198174304 wait complete. 2024-04-20T10:34:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.579+0000 7f34a0981740 1 -- start start 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.579+0000 7f34a0981740 1 Processor -- start 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.580+0000 7f34a0981740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 0x563f93528660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.580+0000 7f34a0981740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 0x563f93528ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.580+0000 7f34a0981740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f9359a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.580+0000 7f34a0981740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f34981b3120 con 0x563f9351b1a0 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.580+0000 7f34a0981740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34981b4590 con 0x563f935246a0 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.580+0000 7f34a0981740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f34981b37f0 con 0x563f9351bac0 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.580+0000 7f34a0981740 10 client.0 _ll_register_callbacks cb 0x7f3488009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.580+0000 7f3496a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 0x563f93528660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.580+0000 7f3497400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f9359a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.580+0000 7f3497400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f9359a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:48572/0 (socket says 172.21.15.42:48572) 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.580+0000 7f3497400640 1 -- 172.21.15.42:0/583162015 learned_addr learned my addr 172.21.15.42:0/583162015 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.580+0000 7f348fe00640 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 0x563f93528ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.581+0000 7f3497400640 1 -- 172.21.15.42:0/583162015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 msgr2=0x563f93528ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.581+0000 7f3497400640 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 0x563f93528ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.581+0000 7f3497400640 1 -- 172.21.15.42:0/583162015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 msgr2=0x563f93528660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.581+0000 7f3497400640 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 0x563f93528660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.581+0000 7f3497400640 1 -- 172.21.15.42:0/583162015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f93526a90 con 0x563f935246a0 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.581+0000 7f3496a00640 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 0x563f93528660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.581+0000 7f3497400640 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f9359a050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3490000f80 tx=0x7f3490009dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfad00bb0f654c91 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.581+0000 7f3484c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.581+0000 7f3484c00640 1 -- 172.21.15.42:0/583162015 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490029070 con 0x563f935246a0 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.581+0000 7f34a0981740 1 -- 172.21.15.42:0/583162015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3488004360 con 0x563f935246a0 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.581+0000 7f3484c00640 1 -- 172.21.15.42:0/583162015 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f34900193d0 con 0x563f935246a0 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.582+0000 7f3484c00640 1 -- 172.21.15.42:0/583162015 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3490034050 con 0x563f935246a0 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.582+0000 7f34a0981740 10 client.25376 fetch_fsmap learned FSMap version 19 2024-04-20T10:34:38.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.582+0000 7f34a0981740 1 -- 172.21.15.42:0/583162015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563f9359b5c0 con 0x563f935246a0 2024-04-20T10:34:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.582+0000 7f3484c00640 1 -- 172.21.15.42:0/583162015 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3488004360 con 0x563f935246a0 2024-04-20T10:34:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.582+0000 7f3484c00640 1 -- 172.21.15.42:0/583162015 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f3490011930 con 0x563f935246a0 2024-04-20T10:34:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.582+0000 7f34a0981740 10 client.25376 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:34:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.582+0000 7f34a0981740 10 client.25376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:34:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.582+0000 7f34a0981740 20 client.25376 populate_metadata read hostname 'smithi042' 2024-04-20T10:34:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.582+0000 7f34a0981740 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563f9359ba90 0x563f9359deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.582+0000 7f34a0981740 4 client.25376 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:34:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.582+0000 7f34a0981740 1 -- 172.21.15.42:0/583162015 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f934889e0 con 0x563f9359ba90 2024-04-20T10:34:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.582+0000 7f3496a00640 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563f9359ba90 0x563f9359deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.583+0000 7f3496a00640 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563f9359ba90 0x563f9359deb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e94fe92555857ef server_cookie=d10434ae922fecd4 in_seq=0 out_seq=0 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.583+0000 7f3484c00640 10 client.25376 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.583+0000 7f3484c00640 1 -- 172.21.15.42:0/583162015 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563f934889e0 con 0x563f9359ba90 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.583+0000 7f3484c00640 10 client.25376 handle_command_reply: tid=0 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.583+0000 7f34a0981740 1 client.25376 shutdown 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.583+0000 7f34a0981740 1 -- 172.21.15.42:0/583162015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 msgr2=0x563f9359a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.583+0000 7f34a0981740 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f9359a050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3490000f80 tx=0x7f3490009dc0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.583+0000 7f34a0981740 1 -- 172.21.15.42:0/583162015 shutdown_connections 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.583+0000 7f34a0981740 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563f9359ba90 0x563f9359deb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.584+0000 7f34a0981740 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f935246a0 0x563f9359a050 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.584+0000 7f34a0981740 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563f9351bac0 0x563f93528ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.584+0000 7f34a0981740 1 --2- 172.21.15.42:0/583162015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563f9351b1a0 0x563f93528660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.584+0000 7f3484c00640 0 client.25376 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.584+0000 7f34a0981740 1 -- 172.21.15.42:0/583162015 >> 172.21.15.42:0/583162015 conn(0x563f935158d0 msgr2=0x563f93516bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.584+0000 7f34a0981740 1 -- 172.21.15.42:0/583162015 shutdown_connections 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.584+0000 7f34a0981740 1 -- 172.21.15.42:0/583162015 wait complete. 2024-04-20T10:34:38.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.584+0000 7f34a0981740 20 client.25376 trim_cache size 0 max 16384 2024-04-20T10:34:38.588 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:34:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:34:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:34:38.588 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:34:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.584+0000 7f349dc00640 1 -- 172.21.15.42:0/886889545 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f34780a3e30 msgr2=0x7f34780a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.584+0000 7f349dc00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f34780a3e30 0x7f34780a62f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x563f9351ac70 tx=0x7f3488005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.584+0000 7f349dc00640 1 -- 172.21.15.42:0/886889545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3498076db0 msgr2=0x7f349819f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.584+0000 7f349dc00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3498076db0 0x7f349819f530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3480036040 tx=0x7f3480004340 comp rx=0 tx=0).stop 2024-04-20T10:34:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.585+0000 7f349dc00640 1 -- 172.21.15.42:0/886889545 shutdown_connections 2024-04-20T10:34:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.585+0000 7f349dc00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f34780a3e30 0x7f34780a62f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.585+0000 7f349dc00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498112e40 0x7f34981a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.585+0000 7f349dc00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3498076db0 0x7f349819f530 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.585+0000 7f349dc00640 1 --2- 172.21.15.42:0/886889545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3498076490 0x7f349819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.585+0000 7f349dc00640 1 -- 172.21.15.42:0/886889545 >> 172.21.15.42:0/886889545 conn(0x7f34980ffcf0 msgr2=0x7f3498114610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.585+0000 7f349dc00640 1 -- 172.21.15.42:0/886889545 shutdown_connections 2024-04-20T10:34:38.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.585+0000 7f349dc00640 1 -- 172.21.15.42:0/886889545 wait complete. 2024-04-20T10:34:38.602 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:34:38.602 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:34:38.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.815+0000 7f42b1800640 1 -- 172.21.15.42:0/1696081912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac1092a0 msgr2=0x7f42ac110e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:38.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.815+0000 7f42b1800640 1 --2- 172.21.15.42:0/1696081912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac1092a0 0x7f42ac110e70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f42a4009ca0 tx=0x7f42a402c100 comp rx=0 tx=0).stop 2024-04-20T10:34:38.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.815+0000 7f42b1800640 1 -- 172.21.15.42:0/1696081912 shutdown_connections 2024-04-20T10:34:38.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.815+0000 7f42b1800640 1 --2- 172.21.15.42:0/1696081912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac1092a0 0x7f42ac110e70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.815+0000 7f42b1800640 1 --2- 172.21.15.42:0/1696081912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac104b80 0x7f42ac108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.815+0000 7f42b1800640 1 --2- 172.21.15.42:0/1696081912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42ac1041d0 0x7f42ac1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.815+0000 7f42b1800640 1 -- 172.21.15.42:0/1696081912 >> 172.21.15.42:0/1696081912 conn(0x7f42ac0ffa30 msgr2=0x7f42ac101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:38.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.815+0000 7f42b1800640 1 -- 172.21.15.42:0/1696081912 shutdown_connections 2024-04-20T10:34:38.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.815+0000 7f42b1800640 1 -- 172.21.15.42:0/1696081912 wait complete. 2024-04-20T10:34:38.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.816+0000 7f42b1800640 1 Processor -- start 2024-04-20T10:34:38.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42b1800640 1 -- start start 2024-04-20T10:34:38.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42b1800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42ac1041d0 0x7f42ac1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42b1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac1092a0 0x7f42ac1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42b1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac19a610 0x7f42ac19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42b1800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac114330 con 0x7f42ac19a610 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42b1800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac1141b0 con 0x7f42ac1092a0 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42b1800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac1144b0 con 0x7f42ac1041d0 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42b0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac19a610 0x7f42ac19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac1092a0 0x7f42ac1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac1092a0 0x7f42ac1a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:48598/0 (socket says 172.21.15.42:48598) 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42b0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac19a610 0x7f42ac19aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37360/0 (socket says 172.21.15.42:37360) 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42ab400640 1 -- 172.21.15.42:0/1321680482 learned_addr learned my addr 172.21.15.42:0/1321680482 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.817+0000 7f42abe00640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42ac1041d0 0x7f42ac1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.818+0000 7f42b0e00640 1 -- 172.21.15.42:0/1321680482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42ac1041d0 msgr2=0x7f42ac1a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.818+0000 7f42b0e00640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42ac1041d0 0x7f42ac1a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.818+0000 7f42b0e00640 1 -- 172.21.15.42:0/1321680482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac1092a0 msgr2=0x7f42ac1a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:38.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.818+0000 7f42b0e00640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac1092a0 0x7f42ac1a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:38.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.818+0000 7f42b0e00640 1 -- 172.21.15.42:0/1321680482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac19b130 con 0x7f42ac19a610 2024-04-20T10:34:38.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.818+0000 7f42ab400640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac1092a0 0x7f42ac1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:38.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.818+0000 7f42b0e00640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac19a610 0x7f42ac19aac0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f42a402c710 tx=0x7f42a4035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:38.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.818+0000 7f42a8c00640 1 -- 172.21.15.42:0/1321680482 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4039040 con 0x7f42ac19a610 2024-04-20T10:34:38.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.819+0000 7f42a8c00640 1 -- 172.21.15.42:0/1321680482 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42a4044070 con 0x7f42ac19a610 2024-04-20T10:34:38.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.819+0000 7f42a8c00640 1 -- 172.21.15.42:0/1321680482 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4004380 con 0x7f42ac19a610 2024-04-20T10:34:38.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.819+0000 7f42b1800640 1 -- 172.21.15.42:0/1321680482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ac19b3c0 con 0x7f42ac19a610 2024-04-20T10:34:38.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.819+0000 7f42b1800640 1 -- 172.21.15.42:0/1321680482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ac1a7560 con 0x7f42ac19a610 2024-04-20T10:34:38.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.820+0000 7f42b1800640 1 -- 172.21.15.42:0/1321680482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ac0690f0 con 0x7f42ac19a610 2024-04-20T10:34:38.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.822+0000 7f42a8c00640 1 -- 172.21.15.42:0/1321680482 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7f42a4004520 con 0x7f42ac19a610 2024-04-20T10:34:38.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.824+0000 7f42a8c00640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f42900a3e30 0x7f42900a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:38.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.824+0000 7f42a8c00640 1 -- 172.21.15.42:0/1321680482 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f42a40f2e00 con 0x7f42ac19a610 2024-04-20T10:34:38.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.824+0000 7f42abe00640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f42900a3e30 0x7f42900a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:38.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.825+0000 7f42abe00640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f42900a3e30 0x7f42900a62f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f429c00bbe0 tx=0x7f429c00b390 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:38.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:38.829+0000 7f42a8c00640 1 -- 172.21.15.42:0/1321680482 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f42a40bf3e0 con 0x7f42ac19a610 2024-04-20T10:34:39.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.079+0000 7f42b1800640 1 -- 172.21.15.42:0/1321680482 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42ac104b80 con 0x7f42ac19a610 2024-04-20T10:34:39.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.081+0000 7f42a8c00640 1 -- 172.21.15.42:0/1321680482 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f42a40bece0 con 0x7f42ac19a610 2024-04-20T10:34:39.081 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:34:39.081 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:34:39.081 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:34:39.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.084+0000 7f42b1800640 1 -- 172.21.15.42:0/1321680482 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f42900a3e30 msgr2=0x7f42900a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:39.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.084+0000 7f42b1800640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f42900a3e30 0x7f42900a62f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f429c00bbe0 tx=0x7f429c00b390 comp rx=0 tx=0).stop 2024-04-20T10:34:39.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.084+0000 7f42b1800640 1 -- 172.21.15.42:0/1321680482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac19a610 msgr2=0x7f42ac19aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:39.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.084+0000 7f42b1800640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac19a610 0x7f42ac19aac0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f42a402c710 tx=0x7f42a4035d90 comp rx=0 tx=0).stop 2024-04-20T10:34:39.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.084+0000 7f42b1800640 1 -- 172.21.15.42:0/1321680482 shutdown_connections 2024-04-20T10:34:39.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.084+0000 7f42b1800640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f42900a3e30 0x7f42900a62f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.084+0000 7f42b1800640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac19a610 0x7f42ac19aac0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.084+0000 7f42b1800640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac1092a0 0x7f42ac1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.084+0000 7f42b1800640 1 --2- 172.21.15.42:0/1321680482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42ac1041d0 0x7f42ac1a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.084+0000 7f42b1800640 1 -- 172.21.15.42:0/1321680482 >> 172.21.15.42:0/1321680482 conn(0x7f42ac0ffa30 msgr2=0x7f42ac100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:39.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.084+0000 7f42b1800640 1 -- 172.21.15.42:0/1321680482 shutdown_connections 2024-04-20T10:34:39.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.084+0000 7f42b1800640 1 -- 172.21.15.42:0/1321680482 wait complete. 2024-04-20T10:34:39.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:34:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.312+0000 7fb637200640 1 -- 172.21.15.42:0/1664330912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb638112df0 msgr2=0x7fb6381151e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.312+0000 7fb637200640 1 --2- 172.21.15.42:0/1664330912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb638112df0 0x7fb6381151e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb630009ca0 tx=0x7fb63002c180 comp rx=0 tx=0).stop 2024-04-20T10:34:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.312+0000 7fb637200640 1 -- 172.21.15.42:0/1664330912 shutdown_connections 2024-04-20T10:34:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.312+0000 7fb637200640 1 --2- 172.21.15.42:0/1664330912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb638112df0 0x7fb6381151e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.312+0000 7fb637200640 1 --2- 172.21.15.42:0/1664330912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb638074c20 0x7fb6381128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.312+0000 7fb637200640 1 --2- 172.21.15.42:0/1664330912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb638074300 0x7fb6380746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.312+0000 7fb637200640 1 -- 172.21.15.42:0/1664330912 >> 172.21.15.42:0/1664330912 conn(0x7fb6380fd930 msgr2=0x7fb6380ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:39.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.312+0000 7fb637200640 1 -- 172.21.15.42:0/1664330912 shutdown_connections 2024-04-20T10:34:39.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.312+0000 7fb637200640 1 -- 172.21.15.42:0/1664330912 wait complete. 2024-04-20T10:34:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.313+0000 7fb637200640 1 Processor -- start 2024-04-20T10:34:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.313+0000 7fb637200640 1 -- start start 2024-04-20T10:34:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.314+0000 7fb637200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb638074300 0x7fb6381a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.314+0000 7fb637200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb638074c20 0x7fb6381a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.314+0000 7fb637200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb638112df0 0x7fb63819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.314+0000 7fb637200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6381187b0 con 0x7fb638074c20 2024-04-20T10:34:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.314+0000 7fb637200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb638118630 con 0x7fb638112df0 2024-04-20T10:34:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.314+0000 7fb637200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb638118930 con 0x7fb638074300 2024-04-20T10:34:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.314+0000 7fb635400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb638074c20 0x7fb6381a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.314+0000 7fb635400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb638074c20 0x7fb6381a4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55836/0 (socket says 172.21.15.42:55836) 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.314+0000 7fb635400640 1 -- 172.21.15.42:0/1528313093 learned_addr learned my addr 172.21.15.42:0/1528313093 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.314+0000 7fb635e00640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb638074300 0x7fb6381a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.314+0000 7fb636800640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb638112df0 0x7fb63819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.315+0000 7fb635400640 1 -- 172.21.15.42:0/1528313093 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb638074300 msgr2=0x7fb6381a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.315+0000 7fb635400640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb638074300 0x7fb6381a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.315+0000 7fb635400640 1 -- 172.21.15.42:0/1528313093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb638112df0 msgr2=0x7fb63819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.315+0000 7fb635400640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb638112df0 0x7fb63819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.315+0000 7fb635400640 1 -- 172.21.15.42:0/1528313093 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63819f3d0 con 0x7fb638074c20 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.315+0000 7fb635400640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb638074c20 0x7fb6381a4f90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb62400e9f0 tx=0x7fb62400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.315+0000 7fb62ea00640 1 -- 172.21.15.42:0/1528313093 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62400cd80 con 0x7fb638074c20 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.315+0000 7fb62ea00640 1 -- 172.21.15.42:0/1528313093 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb624010910 con 0x7fb638074c20 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.315+0000 7fb637200640 1 -- 172.21.15.42:0/1528313093 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63819f6c0 con 0x7fb638074c20 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.315+0000 7fb62ea00640 1 -- 172.21.15.42:0/1528313093 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624018990 con 0x7fb638074c20 2024-04-20T10:34:39.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.315+0000 7fb637200640 1 -- 172.21.15.42:0/1528313093 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6381ab9d0 con 0x7fb638074c20 2024-04-20T10:34:39.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.319+0000 7fb63fbf2740 1 Processor -- start 2024-04-20T10:34:39.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.319+0000 7fb63fbf2740 1 -- start start 2024-04-20T10:34:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.319+0000 7fb63fbf2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 0x5555ae23b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.319+0000 7fb63fbf2740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae23ba90 0x5555ae244130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.319+0000 7fb63fbf2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae244670 0x5555ae246a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.319+0000 7fb63fbf2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5555ae1a0050 con 0x5555ae23b170 2024-04-20T10:34:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.319+0000 7fb63fbf2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5555ae146490 con 0x5555ae244670 2024-04-20T10:34:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.319+0000 7fb63fbf2740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5555ae19c010 con 0x5555ae23ba90 2024-04-20T10:34:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.319+0000 7fb62ea00640 1 -- 172.21.15.42:0/1528313093 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7fb624010ac0 con 0x7fb638074c20 2024-04-20T10:34:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.319+0000 7fb636800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 0x5555ae23b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.320+0000 7fb636800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 0x5555ae23b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55838/0 (socket says 172.21.15.42:55838) 2024-04-20T10:34:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.320+0000 7fb636800640 1 -- 172.21.15.42:0/1206592928 learned_addr learned my addr 172.21.15.42:0/1206592928 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.320+0000 7fb635e00640 1 --2- 172.21.15.42:0/1206592928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae23ba90 0x5555ae244130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.320+0000 7fb635400640 1 --2- 172.21.15.42:0/1206592928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae244670 0x5555ae246a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.320+0000 7fb636800640 1 -- 172.21.15.42:0/1206592928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae23ba90 msgr2=0x5555ae244130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.320+0000 7fb636800640 1 --2- 172.21.15.42:0/1206592928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae23ba90 0x5555ae244130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.320+0000 7fb636800640 1 -- 172.21.15.42:0/1206592928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae244670 msgr2=0x5555ae246a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.320+0000 7fb636800640 1 --2- 172.21.15.42:0/1206592928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae244670 0x5555ae246a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.320+0000 7fb636800640 1 -- 172.21.15.42:0/1206592928 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ae14e070 con 0x5555ae23b170 2024-04-20T10:34:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.320+0000 7fb635e00640 1 --2- 172.21.15.42:0/1206592928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae23ba90 0x5555ae244130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.320+0000 7fb636800640 1 --2- 172.21.15.42:0/1206592928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 0x5555ae23b550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb63806b560 tx=0x7fb630009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b582ebd876ae62a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.321+0000 7fb623400640 1 -- 172.21.15.42:0/1206592928 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630043070 con 0x5555ae23b170 2024-04-20T10:34:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.321+0000 7fb623400640 1 -- 172.21.15.42:0/1206592928 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb630003e50 con 0x5555ae23b170 2024-04-20T10:34:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.321+0000 7fb623400640 1 -- 172.21.15.42:0/1206592928 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630039040 con 0x5555ae23b170 2024-04-20T10:34:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.321+0000 7fb62ea00640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb61c0a3e30 0x7fb61c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.321+0000 7fb62ea00640 1 -- 172.21.15.42:0/1528313093 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb6240d0b80 con 0x7fb638074c20 2024-04-20T10:34:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.321+0000 7fb635e00640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb61c0a3e30 0x7fb61c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.322+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1206592928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 msgr2=0x5555ae23b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.322+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/1206592928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 0x5555ae23b550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb63806b560 tx=0x7fb630009900 comp rx=0 tx=0).stop 2024-04-20T10:34:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.322+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1206592928 shutdown_connections 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.322+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/1206592928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae244670 0x5555ae246a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.322+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/1206592928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae23ba90 0x5555ae244130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.322+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/1206592928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 0x5555ae23b550 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.322+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1206592928 >> 172.21.15.42:0/1206592928 conn(0x5555ae2358a0 msgr2=0x5555ae235c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.322+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1206592928 shutdown_connections 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.322+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1206592928 wait complete. 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.322+0000 7fb635e00640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb61c0a3e30 0x7fb61c0a62f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x5555ae23ac40 tx=0x7fb628005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.323+0000 7fb63fbf2740 1 -- start start 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.323+0000 7fb63fbf2740 1 Processor -- start 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.323+0000 7fb63fbf2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23b170 0x5555ae2416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.323+0000 7fb63fbf2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23ba90 0x5555ae241be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.323+0000 7fb63fbf2740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae244670 0x5555ae242120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.323+0000 7fb63fbf2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5555ae247400 con 0x5555ae23ba90 2024-04-20T10:34:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.323+0000 7fb63fbf2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5555ae159860 con 0x5555ae23b170 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.323+0000 7fb63fbf2740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5555ae247580 con 0x5555ae244670 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.323+0000 7fb63fbf2740 10 client.0 _ll_register_callbacks cb 0x5555ae127fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.324+0000 7fb635400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23ba90 0x5555ae241be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.324+0000 7fb635400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23ba90 0x5555ae241be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55854/0 (socket says 172.21.15.42:55854) 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.324+0000 7fb635400640 1 -- 172.21.15.42:0/1258546961 learned_addr learned my addr 172.21.15.42:0/1258546961 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.324+0000 7fb635e00640 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23b170 0x5555ae2416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.324+0000 7fb636800640 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae244670 0x5555ae242120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.324+0000 7fb635400640 1 -- 172.21.15.42:0/1258546961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae244670 msgr2=0x5555ae242120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.324+0000 7fb635400640 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae244670 0x5555ae242120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.324+0000 7fb635400640 1 -- 172.21.15.42:0/1258546961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23b170 msgr2=0x5555ae2416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.324+0000 7fb635400640 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23b170 0x5555ae2416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.324+0000 7fb635400640 1 -- 172.21.15.42:0/1258546961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ae15d950 con 0x5555ae23ba90 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.324+0000 7fb635400640 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23ba90 0x5555ae241be0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb6240ce2a0 tx=0x7fb6240d4200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779b5f3f14a88824 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.324+0000 7fb617e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:34:39.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.325+0000 7fb617e00640 1 -- 172.21.15.42:0/1258546961 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62409c5a0 con 0x5555ae23ba90 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.325+0000 7fb617e00640 1 -- 172.21.15.42:0/1258546961 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb6240bbb70 con 0x5555ae23ba90 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.325+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1258546961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb630003e50 con 0x5555ae23ba90 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.325+0000 7fb617e00640 1 -- 172.21.15.42:0/1258546961 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb62409c3c0 con 0x5555ae23ba90 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.325+0000 7fb63fbf2740 10 client.15279 fetch_fsmap learned FSMap version 19 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.325+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1258546961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5555ae12b820 con 0x5555ae23ba90 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.325+0000 7fb617e00640 1 -- 172.21.15.42:0/1258546961 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb630003e50 con 0x5555ae23ba90 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.325+0000 7fb617e00640 1 -- 172.21.15.42:0/1258546961 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fb62409dce0 con 0x5555ae23ba90 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.325+0000 7fb63fbf2740 10 client.15279 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.326+0000 7fb63fbf2740 10 client.15279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.326+0000 7fb63fbf2740 20 client.15279 populate_metadata read hostname 'smithi042' 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.326+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5555ae254600 0x5555ae256a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.326+0000 7fb63fbf2740 4 client.15279 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:34:39.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.326+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1258546961 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555ae1ada70 con 0x5555ae254600 2024-04-20T10:34:39.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.326+0000 7fb635e00640 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5555ae254600 0x5555ae256a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.327+0000 7fb635e00640 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5555ae254600 0x5555ae256a10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf844c16af65795 server_cookie=9a38dbedacfcc689 in_seq=0 out_seq=0 2024-04-20T10:34:39.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.327+0000 7fb617e00640 10 client.15279 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:39.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb617e00640 1 -- 172.21.15.42:0/1258546961 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555ae1ada70 con 0x5555ae254600 2024-04-20T10:34:39.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb617e00640 10 client.15279 handle_command_reply: tid=0 2024-04-20T10:34:39.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb63fbf2740 1 client.15279 shutdown 2024-04-20T10:34:39.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1258546961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23ba90 msgr2=0x5555ae241be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:39.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23ba90 0x5555ae241be0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb6240ce2a0 tx=0x7fb6240d4200 comp rx=0 tx=0).stop 2024-04-20T10:34:39.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1258546961 shutdown_connections 2024-04-20T10:34:39.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5555ae254600 0x5555ae256a10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae244670 0x5555ae242120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23ba90 0x5555ae241be0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/1258546961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23b170 0x5555ae2416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1258546961 >> 172.21.15.42:0/1258546961 conn(0x5555ae2358a0 msgr2=0x5555ae2375f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:39.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb617e00640 0 client.15279 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:39.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1258546961 shutdown_connections 2024-04-20T10:34:39.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb63fbf2740 1 -- 172.21.15.42:0/1258546961 wait complete. 2024-04-20T10:34:39.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.328+0000 7fb63fbf2740 20 client.15279 trim_cache size 0 max 16384 2024-04-20T10:34:39.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.343+0000 7fb63fbf2740 1 Processor -- start 2024-04-20T10:34:39.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.343+0000 7fb63fbf2740 1 -- start start 2024-04-20T10:34:39.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.343+0000 7fb63fbf2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23b170 0x5555ae243d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.343+0000 7fb63fbf2740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae23ba90 0x5555ae2502c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.343+0000 7fb63fbf2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae244670 0x5555ae25c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb63fbf2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6381b2130 con 0x5555ae244670 2024-04-20T10:34:39.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb63fbf2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb6381b49b0 con 0x5555ae23b170 2024-04-20T10:34:39.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb63fbf2740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb6381b27a0 con 0x5555ae23ba90 2024-04-20T10:34:39.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb636800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae244670 0x5555ae25c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb636800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae244670 0x5555ae25c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55868/0 (socket says 172.21.15.42:55868) 2024-04-20T10:34:39.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb636800640 1 -- 172.21.15.42:0/924176002 learned_addr learned my addr 172.21.15.42:0/924176002 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb635e00640 1 --2- 172.21.15.42:0/924176002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23b170 0x5555ae243d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb635400640 1 --2- 172.21.15.42:0/924176002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae23ba90 0x5555ae2502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb636800640 1 -- 172.21.15.42:0/924176002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae23ba90 msgr2=0x5555ae2502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb636800640 1 --2- 172.21.15.42:0/924176002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae23ba90 0x5555ae2502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb636800640 1 -- 172.21.15.42:0/924176002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23b170 msgr2=0x5555ae243d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb636800640 1 --2- 172.21.15.42:0/924176002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23b170 0x5555ae243d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb636800640 1 -- 172.21.15.42:0/924176002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555adeb23f0 con 0x5555ae244670 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.344+0000 7fb635e00640 1 --2- 172.21.15.42:0/924176002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23b170 0x5555ae243d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.345+0000 7fb636800640 1 --2- 172.21.15.42:0/924176002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae244670 0x5555ae25c2e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb6300046c0 tx=0x7fb630004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c597623b96ed13 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.345+0000 7fb622a00640 1 -- 172.21.15.42:0/924176002 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630039040 con 0x5555ae244670 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.345+0000 7fb622a00640 1 -- 172.21.15.42:0/924176002 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb630032b50 con 0x5555ae244670 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.345+0000 7fb622a00640 1 -- 172.21.15.42:0/924176002 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630043040 con 0x5555ae244670 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.346+0000 7fb63fbf2740 1 -- 172.21.15.42:0/924176002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae244670 msgr2=0x5555ae25c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:39.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.346+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/924176002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae244670 0x5555ae25c2e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb6300046c0 tx=0x7fb630004790 comp rx=0 tx=0).stop 2024-04-20T10:34:39.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.346+0000 7fb63fbf2740 1 -- 172.21.15.42:0/924176002 shutdown_connections 2024-04-20T10:34:39.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.346+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/924176002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae244670 0x5555ae25c2e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.346+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/924176002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae23ba90 0x5555ae2502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.346+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/924176002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23b170 0x5555ae243d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.346+0000 7fb63fbf2740 1 -- 172.21.15.42:0/924176002 >> 172.21.15.42:0/924176002 conn(0x5555ae2358a0 msgr2=0x5555ae234960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:39.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.346+0000 7fb63fbf2740 1 -- 172.21.15.42:0/924176002 shutdown_connections 2024-04-20T10:34:39.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.346+0000 7fb63fbf2740 1 -- 172.21.15.42:0/924176002 wait complete. 2024-04-20T10:34:39.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.346+0000 7fb63fbf2740 1 -- start start 2024-04-20T10:34:39.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.346+0000 7fb63fbf2740 1 Processor -- start 2024-04-20T10:34:39.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.347+0000 7fb63fbf2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 0x5555ae23e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.347+0000 7fb63fbf2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23ba90 0x5555ae23eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.347+0000 7fb63fbf2740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae244670 0x5555ae2b9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.347+0000 7fb63fbf2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6381b2e70 con 0x5555ae23b170 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.347+0000 7fb63fbf2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb6381b42e0 con 0x5555ae23ba90 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.347+0000 7fb63fbf2740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb6381b3540 con 0x5555ae244670 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.347+0000 7fb63fbf2740 10 client.0 _ll_register_callbacks cb 0x7fb62400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.347+0000 7fb635e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 0x5555ae23e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.347+0000 7fb635e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 0x5555ae23e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55876/0 (socket says 172.21.15.42:55876) 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.347+0000 7fb635e00640 1 -- 172.21.15.42:0/477805963 learned_addr learned my addr 172.21.15.42:0/477805963 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.347+0000 7fb636800640 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae244670 0x5555ae2b9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.347+0000 7fb635400640 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23ba90 0x5555ae23eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.348+0000 7fb636800640 1 -- 172.21.15.42:0/477805963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23ba90 msgr2=0x5555ae23eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:39.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.348+0000 7fb636800640 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23ba90 0x5555ae23eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.348+0000 7fb636800640 1 -- 172.21.15.42:0/477805963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 msgr2=0x5555ae23e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.348+0000 7fb636800640 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 0x5555ae23e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.348+0000 7fb636800640 1 -- 172.21.15.42:0/477805963 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555adf7c7d0 con 0x5555ae244670 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.348+0000 7fb636800640 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae244670 0x5555ae2b9b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb630009bf0 tx=0x7fb63002c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3506d5c7535f9bdc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.348+0000 7fb617e00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.348+0000 7fb617e00640 1 -- 172.21.15.42:0/477805963 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630054020 con 0x5555ae244670 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.348+0000 7fb63fbf2740 1 -- 172.21.15.42:0/477805963 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb630032b50 con 0x5555ae244670 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.348+0000 7fb617e00640 1 -- 172.21.15.42:0/477805963 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb630032340 con 0x5555ae244670 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.349+0000 7fb617e00640 1 -- 172.21.15.42:0/477805963 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb630039040 con 0x5555ae244670 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.349+0000 7fb63fbf2740 10 client.25417 fetch_fsmap learned FSMap version 19 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.349+0000 7fb63fbf2740 1 -- 172.21.15.42:0/477805963 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5555ae2baf20 con 0x5555ae244670 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.349+0000 7fb617e00640 1 -- 172.21.15.42:0/477805963 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb630032b50 con 0x5555ae244670 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.349+0000 7fb617e00640 1 -- 172.21.15.42:0/477805963 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fb63003f670 con 0x5555ae244670 2024-04-20T10:34:39.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.349+0000 7fb63fbf2740 10 client.25417 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:34:39.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.349+0000 7fb63fbf2740 10 client.25417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:34:39.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.349+0000 7fb63fbf2740 20 client.25417 populate_metadata read hostname 'smithi042' 2024-04-20T10:34:39.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.349+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5555ae2bb200 0x5555ae2bd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:39.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.349+0000 7fb63fbf2740 4 client.25417 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:34:39.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.349+0000 7fb63fbf2740 1 -- 172.21.15.42:0/477805963 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555ae1a78e0 con 0x5555ae2bb200 2024-04-20T10:34:39.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.350+0000 7fb635e00640 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5555ae2bb200 0x5555ae2bd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:39.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.350+0000 7fb635e00640 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5555ae2bb200 0x5555ae2bd5f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3720909219653c0c server_cookie=6c232434bdb10bd7 in_seq=0 out_seq=0 2024-04-20T10:34:39.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.350+0000 7fb617e00640 10 client.25417 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:39.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb617e00640 1 -- 172.21.15.42:0/477805963 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555ae1a78e0 con 0x5555ae2bb200 2024-04-20T10:34:39.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb617e00640 10 client.25417 handle_command_reply: tid=0 2024-04-20T10:34:39.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb63fbf2740 1 client.25417 shutdown 2024-04-20T10:34:39.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb63fbf2740 1 -- 172.21.15.42:0/477805963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae244670 msgr2=0x5555ae2b9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:39.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae244670 0x5555ae2b9b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb630009bf0 tx=0x7fb63002c800 comp rx=0 tx=0).stop 2024-04-20T10:34:39.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb63fbf2740 1 -- 172.21.15.42:0/477805963 shutdown_connections 2024-04-20T10:34:39.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5555ae2bb200 0x5555ae2bd5f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5555ae244670 0x5555ae2b9b50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ae23ba90 0x5555ae23eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb617e00640 0 client.25417 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:39.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb63fbf2740 1 --2- 172.21.15.42:0/477805963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5555ae23b170 0x5555ae23e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb63fbf2740 1 -- 172.21.15.42:0/477805963 >> 172.21.15.42:0/477805963 conn(0x5555ae2358a0 msgr2=0x5555ae2340d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:39.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb63fbf2740 1 -- 172.21.15.42:0/477805963 shutdown_connections 2024-04-20T10:34:39.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb63fbf2740 1 -- 172.21.15.42:0/477805963 wait complete. 2024-04-20T10:34:39.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.351+0000 7fb63fbf2740 20 client.25417 trim_cache size 0 max 16384 2024-04-20T10:34:39.354 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:34:39.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.352+0000 7fb637200640 1 -- 172.21.15.42:0/1528313093 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb61c0a3e30 msgr2=0x7fb61c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:39.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.352+0000 7fb637200640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb61c0a3e30 0x7fb61c0a62f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x5555ae23ac40 tx=0x7fb628005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:39.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.352+0000 7fb637200640 1 -- 172.21.15.42:0/1528313093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb638074c20 msgr2=0x7fb6381a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:39.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.352+0000 7fb637200640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb638074c20 0x7fb6381a4f90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb62400e9f0 tx=0x7fb62400eeb0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.352+0000 7fb637200640 1 -- 172.21.15.42:0/1528313093 shutdown_connections 2024-04-20T10:34:39.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.352+0000 7fb637200640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb61c0a3e30 0x7fb61c0a62f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.352+0000 7fb637200640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb638112df0 0x7fb63819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.352+0000 7fb637200640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb638074c20 0x7fb6381a4f90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.352+0000 7fb637200640 1 --2- 172.21.15.42:0/1528313093 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb638074300 0x7fb6381a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:39.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.352+0000 7fb637200640 1 -- 172.21.15.42:0/1528313093 >> 172.21.15.42:0/1528313093 conn(0x7fb6380fd930 msgr2=0x7fb6380ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:39.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.352+0000 7fb637200640 1 -- 172.21.15.42:0/1528313093 shutdown_connections 2024-04-20T10:34:39.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:39.352+0000 7fb637200640 1 -- 172.21.15.42:0/1528313093 wait complete. 2024-04-20T10:34:39.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:39 smithi046 ceph-mon[21574]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:34:39.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:39 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1321680482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:34:39.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:39 smithi042 ceph-mon[17078]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:34:39.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:39 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1321680482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:34:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:39 smithi162 ceph-mon[21854]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-20T10:34:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:39 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1321680482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:34:40.369 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:34:40.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:34:40.370 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:34:40.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.591+0000 7fcdcbe00640 1 -- 172.21.15.42:0/4277182137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdcc076db0 msgr2=0x7fcdcc112760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.591+0000 7fcdcbe00640 1 --2- 172.21.15.42:0/4277182137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdcc076db0 0x7fcdcc112760 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009900 tx=0x7fcdb402bc40 comp rx=0 tx=0).stop 2024-04-20T10:34:40.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.591+0000 7fcdcbe00640 1 -- 172.21.15.42:0/4277182137 shutdown_connections 2024-04-20T10:34:40.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.591+0000 7fcdcbe00640 1 --2- 172.21.15.42:0/4277182137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdcc112ca0 0x7fcdcc115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.591+0000 7fcdcbe00640 1 --2- 172.21.15.42:0/4277182137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdcc076db0 0x7fcdcc112760 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.591+0000 7fcdcbe00640 1 --2- 172.21.15.42:0/4277182137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc076490 0x7fcdcc076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.591+0000 7fcdcbe00640 1 -- 172.21.15.42:0/4277182137 >> 172.21.15.42:0/4277182137 conn(0x7fcdcc0ffad0 msgr2=0x7fcdcc101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:40.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.591+0000 7fcdcbe00640 1 -- 172.21.15.42:0/4277182137 shutdown_connections 2024-04-20T10:34:40.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.592+0000 7fcdcbe00640 1 -- 172.21.15.42:0/4277182137 wait complete. 2024-04-20T10:34:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.592+0000 7fcdcbe00640 1 Processor -- start 2024-04-20T10:34:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.593+0000 7fcdcbe00640 1 -- start start 2024-04-20T10:34:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.593+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdcc076490 0x7fcdcc19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.593+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdcc076db0 0x7fcdcc19f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.593+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc112ca0 0x7fcdcc1a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.593+0000 7fcdcbe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc118800 con 0x7fcdcc076db0 2024-04-20T10:34:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.593+0000 7fcdcbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc118680 con 0x7fcdcc112ca0 2024-04-20T10:34:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.593+0000 7fcdcbe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc118980 con 0x7fcdcc076490 2024-04-20T10:34:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.593+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdcc076db0 0x7fcdcc19f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.593+0000 7fcdcaa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdcc076490 0x7fcdcc19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.593+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdcc076db0 0x7fcdcc19f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55886/0 (socket says 172.21.15.42:55886) 2024-04-20T10:34:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.593+0000 7fcdcb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc112ca0 0x7fcdcc1a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.593+0000 7fcdc2000640 1 -- 172.21.15.42:0/1438563170 learned_addr learned my addr 172.21.15.42:0/1438563170 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.594+0000 7fcdcb400640 1 -- 172.21.15.42:0/1438563170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdcc076490 msgr2=0x7fcdcc19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.594+0000 7fcdcb400640 1 --2- 172.21.15.42:0/1438563170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdcc076490 0x7fcdcc19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.594+0000 7fcdcb400640 1 -- 172.21.15.42:0/1438563170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdcc076db0 msgr2=0x7fcdcc19f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.594+0000 7fcdcb400640 1 --2- 172.21.15.42:0/1438563170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdcc076db0 0x7fcdcc19f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.594+0000 7fcdcb400640 1 -- 172.21.15.42:0/1438563170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc1a3da0 con 0x7fcdcc112ca0 2024-04-20T10:34:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.594+0000 7fcdcb400640 1 --2- 172.21.15.42:0/1438563170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc112ca0 0x7fcdcc1a3670 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4005fd0 tx=0x7fcdc400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.594+0000 7fcdc3e00640 1 -- 172.21.15.42:0/1438563170 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc400cdf0 con 0x7fcdcc112ca0 2024-04-20T10:34:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.595+0000 7fcdc3e00640 1 -- 172.21.15.42:0/1438563170 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc4005020 con 0x7fcdcc112ca0 2024-04-20T10:34:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.595+0000 7fcdcbe00640 1 -- 172.21.15.42:0/1438563170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdcc1a4090 con 0x7fcdcc112ca0 2024-04-20T10:34:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.595+0000 7fcdc3e00640 1 -- 172.21.15.42:0/1438563170 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40106d0 con 0x7fcdcc112ca0 2024-04-20T10:34:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.595+0000 7fcdcbe00640 1 -- 172.21.15.42:0/1438563170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdcc1ab920 con 0x7fcdcc112ca0 2024-04-20T10:34:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.598+0000 7fcdd47cc740 1 Processor -- start 2024-04-20T10:34:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.598+0000 7fcdc3e00640 1 -- 172.21.15.42:0/1438563170 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7fcdc40054d0 con 0x7fcdcc112ca0 2024-04-20T10:34:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.598+0000 7fcdd47cc740 1 -- start start 2024-04-20T10:34:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.599+0000 7fcdd47cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30270f0 0x55ada30274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.599+0000 7fcdd47cc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada3027a10 0x55ada30300b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.599+0000 7fcdd47cc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30305f0 0x55ada30329e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.599+0000 7fcdd47cc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ada2f8bd20 con 0x55ada30305f0 2024-04-20T10:34:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.599+0000 7fcdd47cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ada2f323e0 con 0x55ada30270f0 2024-04-20T10:34:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.599+0000 7fcdd47cc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ada2f87ce0 con 0x55ada3027a10 2024-04-20T10:34:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.599+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30305f0 0x55ada30329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.599+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30305f0 0x55ada30329e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55900/0 (socket says 172.21.15.42:55900) 2024-04-20T10:34:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.599+0000 7fcdc2000640 1 -- 172.21.15.42:0/1492763293 learned_addr learned my addr 172.21.15.42:0/1492763293 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:40.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.599+0000 7fcdc2000640 1 --2- 172.21.15.42:0/1492763293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30270f0 0x55ada30274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.599+0000 7fcdcaa00640 1 --2- 172.21.15.42:0/1492763293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada3027a10 0x55ada30300b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.600+0000 7fcdc2000640 1 -- 172.21.15.42:0/1492763293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada3027a10 msgr2=0x55ada30300b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.600+0000 7fcdc2000640 1 --2- 172.21.15.42:0/1492763293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada3027a10 0x55ada30300b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.600+0000 7fcdc2000640 1 -- 172.21.15.42:0/1492763293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30270f0 msgr2=0x55ada30274d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.600+0000 7fcdc2000640 1 --2- 172.21.15.42:0/1492763293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30270f0 0x55ada30274d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.600+0000 7fcdc2000640 1 -- 172.21.15.42:0/1492763293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ada2f51980 con 0x55ada30305f0 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.600+0000 7fcdc3e00640 1 --2- 172.21.15.42:0/1438563170 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fcdb00a3db0 0x7fcdb00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.600+0000 7fcdc2000640 1 --2- 172.21.15.42:0/1492763293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30305f0 0x55ada30329e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4006010 tx=0x7fcdb4005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba749e8d92db09d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.600+0000 7fcdc3e00640 1 -- 172.21.15.42:0/1438563170 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fcdc40d0f90 con 0x7fcdcc112ca0 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.600+0000 7fcdcaa00640 1 --2- 172.21.15.42:0/1438563170 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fcdb00a3db0 0x7fcdb00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.600+0000 7fcdbbe00640 1 -- 172.21.15.42:0/1492763293 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb402c8c0 con 0x55ada30305f0 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.600+0000 7fcdbbe00640 1 -- 172.21.15.42:0/1492763293 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb4004ba0 con 0x55ada30305f0 2024-04-20T10:34:40.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.600+0000 7fcdbbe00640 1 -- 172.21.15.42:0/1492763293 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4005280 con 0x55ada30305f0 2024-04-20T10:34:40.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.601+0000 7fcdd47cc740 1 -- 172.21.15.42:0/1492763293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30305f0 msgr2=0x55ada30329e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.601+0000 7fcdcaa00640 1 --2- 172.21.15.42:0/1438563170 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fcdb00a3db0 0x7fcdb00a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55ada3026bc0 tx=0x7fcdbc00b410 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:40.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.601+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/1492763293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30305f0 0x55ada30329e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4006010 tx=0x7fcdb4005b70 comp rx=0 tx=0).stop 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.601+0000 7fcdd47cc740 1 -- 172.21.15.42:0/1492763293 shutdown_connections 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.601+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/1492763293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30305f0 0x55ada30329e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.601+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/1492763293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada3027a10 0x55ada30300b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.601+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/1492763293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30270f0 0x55ada30274d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.601+0000 7fcdd47cc740 1 -- 172.21.15.42:0/1492763293 >> 172.21.15.42:0/1492763293 conn(0x55ada3021820 msgr2=0x55ada3021c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.601+0000 7fcdd47cc740 1 -- 172.21.15.42:0/1492763293 shutdown_connections 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.601+0000 7fcdd47cc740 1 -- 172.21.15.42:0/1492763293 wait complete. 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.602+0000 7fcdd47cc740 1 -- start start 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.602+0000 7fcdd47cc740 1 Processor -- start 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.602+0000 7fcdd47cc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30270f0 0x55ada302d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.602+0000 7fcdd47cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada3027a10 0x55ada302db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.602+0000 7fcdd47cc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30305f0 0x55ada302e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.602+0000 7fcdd47cc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ada30333d0 con 0x55ada30270f0 2024-04-20T10:34:40.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.602+0000 7fcdd47cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ada2f457a0 con 0x55ada3027a10 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.602+0000 7fcdd47cc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ada3033550 con 0x55ada30305f0 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.603+0000 7fcdd47cc740 10 client.0 _ll_register_callbacks cb 0x55ada2f47b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.603+0000 7fcdcaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30270f0 0x55ada302d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.603+0000 7fcdcaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30270f0 0x55ada302d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55910/0 (socket says 172.21.15.42:55910) 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.603+0000 7fcdcaa00640 1 -- 172.21.15.42:0/2984377955 learned_addr learned my addr 172.21.15.42:0/2984377955 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.603+0000 7fcdcb400640 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30305f0 0x55ada302e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.603+0000 7fcdc2000640 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada3027a10 0x55ada302db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.603+0000 7fcdcaa00640 1 -- 172.21.15.42:0/2984377955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30305f0 msgr2=0x55ada302e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.603+0000 7fcdcaa00640 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30305f0 0x55ada302e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.603+0000 7fcdcaa00640 1 -- 172.21.15.42:0/2984377955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada3027a10 msgr2=0x55ada302db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.603+0000 7fcdcaa00640 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada3027a10 0x55ada302db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.603+0000 7fcdcaa00640 1 -- 172.21.15.42:0/2984377955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ada2d667e0 con 0x55ada30270f0 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.604+0000 7fcdcaa00640 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30270f0 0x55ada302d640 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc01f550 tx=0x7fcdbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249f5dc6230e1619 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:40.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.604+0000 7fcdb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.604+0000 7fcdb8c00640 1 -- 172.21.15.42:0/2984377955 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc02c060 con 0x55ada30270f0 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.604+0000 7fcdb8c00640 1 -- 172.21.15.42:0/2984377955 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fcdbc017030 con 0x55ada30270f0 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.604+0000 7fcdd47cc740 1 -- 172.21.15.42:0/2984377955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb4004ba0 con 0x55ada30270f0 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.604+0000 7fcdb8c00640 1 -- 172.21.15.42:0/2984377955 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fcdbc01c6c0 con 0x55ada30270f0 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.604+0000 7fcdd47cc740 10 client.15297 fetch_fsmap learned FSMap version 19 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.604+0000 7fcdd47cc740 1 -- 172.21.15.42:0/2984377955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ada2dc6f70 con 0x55ada30270f0 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.604+0000 7fcdb8c00640 1 -- 172.21.15.42:0/2984377955 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb4004ba0 con 0x55ada30270f0 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.605+0000 7fcdb8c00640 1 -- 172.21.15.42:0/2984377955 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fcdbc010ad0 con 0x55ada30270f0 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.605+0000 7fcdd47cc740 10 client.15297 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.605+0000 7fcdd47cc740 10 client.15297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.605+0000 7fcdd47cc740 20 client.15297 populate_metadata read hostname 'smithi042' 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.605+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ada3040a10 0x55ada3042eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.605+0000 7fcdd47cc740 4 client.15297 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:34:40.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.605+0000 7fcdd47cc740 1 -- 172.21.15.42:0/2984377955 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ada2f99740 con 0x55ada3040a10 2024-04-20T10:34:40.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.605+0000 7fcdc2000640 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ada3040a10 0x55ada3042eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.606+0000 7fcdc2000640 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ada3040a10 0x55ada3042eb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c460477f086e01a2 server_cookie=1b252e0c5f50129c in_seq=0 out_seq=0 2024-04-20T10:34:40.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.606+0000 7fcdb8c00640 10 client.15297 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.607+0000 7fcdb8c00640 1 -- 172.21.15.42:0/2984377955 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ada2f99740 con 0x55ada3040a10 2024-04-20T10:34:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.607+0000 7fcdb8c00640 10 client.15297 handle_command_reply: tid=0 2024-04-20T10:34:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.607+0000 7fcdd47cc740 1 client.15297 shutdown 2024-04-20T10:34:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.607+0000 7fcdd47cc740 1 -- 172.21.15.42:0/2984377955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30270f0 msgr2=0x55ada302d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.607+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30270f0 0x55ada302d640 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc01f550 tx=0x7fcdbc004730 comp rx=0 tx=0).stop 2024-04-20T10:34:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.607+0000 7fcdd47cc740 1 -- 172.21.15.42:0/2984377955 shutdown_connections 2024-04-20T10:34:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.607+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ada3040a10 0x55ada3042eb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.607+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30305f0 0x55ada302e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.607+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada3027a10 0x55ada302db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.607+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/2984377955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada30270f0 0x55ada302d640 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.607+0000 7fcdb8c00640 0 client.15297 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.607+0000 7fcdd47cc740 1 -- 172.21.15.42:0/2984377955 >> 172.21.15.42:0/2984377955 conn(0x55ada3021820 msgr2=0x55ada3023570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.608+0000 7fcdd47cc740 1 -- 172.21.15.42:0/2984377955 shutdown_connections 2024-04-20T10:34:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.608+0000 7fcdd47cc740 1 -- 172.21.15.42:0/2984377955 wait complete. 2024-04-20T10:34:40.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.608+0000 7fcdd47cc740 20 client.15297 trim_cache size 0 max 16384 2024-04-20T10:34:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.627+0000 7fcdd47cc740 1 Processor -- start 2024-04-20T10:34:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.627+0000 7fcdd47cc740 1 -- start start 2024-04-20T10:34:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.627+0000 7fcdd47cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30270f0 0x55ada302fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.627+0000 7fcdd47cc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 0x55ada303c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdd47cc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30305f0 0x55ada3048780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdd47cc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc1b2120 con 0x55ada3027a10 2024-04-20T10:34:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdd47cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc1b49a0 con 0x55ada30270f0 2024-04-20T10:34:40.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdd47cc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc1b2790 con 0x55ada30305f0 2024-04-20T10:34:40.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 0x55ada303c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdcaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30270f0 0x55ada302fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdcb400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30305f0 0x55ada3048780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 0x55ada303c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55918/0 (socket says 172.21.15.42:55918) 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdc2000640 1 -- 172.21.15.42:0/4188220439 learned_addr learned my addr 172.21.15.42:0/4188220439 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdcaa00640 1 -- 172.21.15.42:0/4188220439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30305f0 msgr2=0x55ada3048780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdcaa00640 1 --2- 172.21.15.42:0/4188220439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30305f0 0x55ada3048780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdcaa00640 1 -- 172.21.15.42:0/4188220439 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 msgr2=0x55ada303c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdcaa00640 1 --2- 172.21.15.42:0/4188220439 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 0x55ada303c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.628+0000 7fcdcaa00640 1 -- 172.21.15.42:0/4188220439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ada3027840 con 0x55ada30270f0 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.629+0000 7fcdcaa00640 1 --2- 172.21.15.42:0/4188220439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30270f0 0x55ada302fd70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc000c00 tx=0x7fcdbc02f6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d10b8c05d600f2a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.629+0000 7fcdbb400640 1 -- 172.21.15.42:0/4188220439 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc01f230 con 0x55ada30270f0 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.629+0000 7fcdbb400640 1 -- 172.21.15.42:0/4188220439 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdbc01cd40 con 0x55ada30270f0 2024-04-20T10:34:40.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.629+0000 7fcdbb400640 1 -- 172.21.15.42:0/4188220439 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc02fcd0 con 0x55ada30270f0 2024-04-20T10:34:40.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.630+0000 7fcdd47cc740 1 -- 172.21.15.42:0/4188220439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30270f0 msgr2=0x55ada302fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.630+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/4188220439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30270f0 0x55ada302fd70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc000c00 tx=0x7fcdbc02f6d0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.630+0000 7fcdd47cc740 1 -- 172.21.15.42:0/4188220439 shutdown_connections 2024-04-20T10:34:40.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.630+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/4188220439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30305f0 0x55ada3048780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.630+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/4188220439 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 0x55ada303c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.630+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/4188220439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30270f0 0x55ada302fd70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.630+0000 7fcdd47cc740 1 -- 172.21.15.42:0/4188220439 >> 172.21.15.42:0/4188220439 conn(0x55ada3021820 msgr2=0x55ada3023500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:40.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.630+0000 7fcdd47cc740 1 -- 172.21.15.42:0/4188220439 shutdown_connections 2024-04-20T10:34:40.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.630+0000 7fcdd47cc740 1 -- 172.21.15.42:0/4188220439 wait complete. 2024-04-20T10:34:40.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.630+0000 7fcdd47cc740 1 -- start start 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.630+0000 7fcdd47cc740 1 Processor -- start 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.631+0000 7fcdd47cc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30270f0 0x55ada302a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.631+0000 7fcdd47cc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 0x55ada302a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.631+0000 7fcdd47cc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30305f0 0x55ada302aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.631+0000 7fcdd47cc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc1b2e60 con 0x55ada3027a10 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.631+0000 7fcdd47cc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc1b42d0 con 0x55ada30305f0 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.631+0000 7fcdd47cc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc1b3530 con 0x55ada30270f0 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.631+0000 7fcdd47cc740 10 client.0 _ll_register_callbacks cb 0x7fcdbc00bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.631+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 0x55ada302a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.631+0000 7fcdcaa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30270f0 0x55ada302a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.631+0000 7fcdc2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 0x55ada302a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55922/0 (socket says 172.21.15.42:55922) 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.631+0000 7fcdc2000640 1 -- 172.21.15.42:0/3644440171 learned_addr learned my addr 172.21.15.42:0/3644440171 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.631+0000 7fcdcb400640 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30305f0 0x55ada302aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.632+0000 7fcdc2000640 1 -- 172.21.15.42:0/3644440171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30270f0 msgr2=0x55ada302a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.632+0000 7fcdc2000640 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30270f0 0x55ada302a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.632+0000 7fcdc2000640 1 -- 172.21.15.42:0/3644440171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30305f0 msgr2=0x55ada302aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.632+0000 7fcdc2000640 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30305f0 0x55ada302aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.632+0000 7fcdc2000640 1 -- 172.21.15.42:0/3644440171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ada2c5e170 con 0x55ada3027a10 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.632+0000 7fcdcb400640 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30305f0 0x55ada302aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.632+0000 7fcdc2000640 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 0x55ada302a960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4006010 tx=0x7fcdb4004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd0e9cfba0a7f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.632+0000 7fcdb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.632+0000 7fcdb8c00640 1 -- 172.21.15.42:0/3644440171 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4053d50 con 0x55ada3027a10 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.632+0000 7fcdb8c00640 1 -- 172.21.15.42:0/3644440171 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fcdb402c8c0 con 0x55ada3027a10 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.632+0000 7fcdd47cc740 1 -- 172.21.15.42:0/3644440171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdbc01cd40 con 0x55ada3027a10 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.633+0000 7fcdb8c00640 1 -- 172.21.15.42:0/3644440171 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fcdb4061050 con 0x55ada3027a10 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.633+0000 7fcdd47cc740 10 client.15303 fetch_fsmap learned FSMap version 19 2024-04-20T10:34:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.633+0000 7fcdd47cc740 1 -- 172.21.15.42:0/3644440171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ada30a7270 con 0x55ada3027a10 2024-04-20T10:34:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.633+0000 7fcdb8c00640 1 -- 172.21.15.42:0/3644440171 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdbc01cd40 con 0x55ada3027a10 2024-04-20T10:34:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.633+0000 7fcdb8c00640 1 -- 172.21.15.42:0/3644440171 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fcdb40532b0 con 0x55ada3027a10 2024-04-20T10:34:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.633+0000 7fcdd47cc740 10 client.15303 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:34:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.633+0000 7fcdd47cc740 10 client.15303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:34:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.633+0000 7fcdd47cc740 20 client.15303 populate_metadata read hostname 'smithi042' 2024-04-20T10:34:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.633+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ada30a7600 0x55ada30a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.633+0000 7fcdd47cc740 4 client.15303 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.633+0000 7fcdd47cc740 1 -- 172.21.15.42:0/3644440171 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ada2f935b0 con 0x55ada30a7600 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.634+0000 7fcdcaa00640 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ada30a7600 0x55ada30a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.634+0000 7fcdcaa00640 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ada30a7600 0x55ada30a9a20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=504e7f10e1b7c6e3 server_cookie=5d429b5c0ac262da in_seq=0 out_seq=0 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.634+0000 7fcdb8c00640 10 client.15303 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.635+0000 7fcdb8c00640 1 -- 172.21.15.42:0/3644440171 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ada2f935b0 con 0x55ada30a7600 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.635+0000 7fcdb8c00640 10 client.15303 handle_command_reply: tid=0 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.635+0000 7fcdd47cc740 1 client.15303 shutdown 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.636+0000 7fcdd47cc740 1 -- 172.21.15.42:0/3644440171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 msgr2=0x55ada302a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.636+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 0x55ada302a960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4006010 tx=0x7fcdb4004ec0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.636+0000 7fcdd47cc740 1 -- 172.21.15.42:0/3644440171 shutdown_connections 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.636+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ada30a7600 0x55ada30a9a20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.636+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ada30305f0 0x55ada302aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.636+0000 7fcdb8c00640 0 client.15303 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.636+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ada3027a10 0x55ada302a960 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.636+0000 7fcdd47cc740 1 --2- 172.21.15.42:0/3644440171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ada30270f0 0x55ada302a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.636+0000 7fcdd47cc740 1 -- 172.21.15.42:0/3644440171 >> 172.21.15.42:0/3644440171 conn(0x55ada3021820 msgr2=0x55ada3020170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:40.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.636+0000 7fcdd47cc740 1 -- 172.21.15.42:0/3644440171 shutdown_connections 2024-04-20T10:34:40.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.636+0000 7fcdd47cc740 1 -- 172.21.15.42:0/3644440171 wait complete. 2024-04-20T10:34:40.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.636+0000 7fcdd47cc740 20 client.15303 trim_cache size 0 max 16384 2024-04-20T10:34:40.638 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:34:40.638 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:34:40.638 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "8f69c087-6c85-4620-bf19-03a77ebec9d7", 2024-04-20T10:34:40.638 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:34:40.638 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:34:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.637+0000 7fcdcbe00640 1 -- 172.21.15.42:0/1438563170 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fcdb00a3db0 msgr2=0x7fcdb00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.637+0000 7fcdcbe00640 1 --2- 172.21.15.42:0/1438563170 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fcdb00a3db0 0x7fcdb00a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55ada3026bc0 tx=0x7fcdbc00b410 comp rx=0 tx=0).stop 2024-04-20T10:34:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.637+0000 7fcdcbe00640 1 -- 172.21.15.42:0/1438563170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc112ca0 msgr2=0x7fcdcc1a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.637+0000 7fcdcbe00640 1 --2- 172.21.15.42:0/1438563170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc112ca0 0x7fcdcc1a3670 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4005fd0 tx=0x7fcdc400ef10 comp rx=0 tx=0).stop 2024-04-20T10:34:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.637+0000 7fcdcbe00640 1 -- 172.21.15.42:0/1438563170 shutdown_connections 2024-04-20T10:34:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.637+0000 7fcdcbe00640 1 --2- 172.21.15.42:0/1438563170 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fcdb00a3db0 0x7fcdb00a6270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.637+0000 7fcdcbe00640 1 --2- 172.21.15.42:0/1438563170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdcc112ca0 0x7fcdcc1a3670 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.637+0000 7fcdcbe00640 1 --2- 172.21.15.42:0/1438563170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdcc076db0 0x7fcdcc19f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.637+0000 7fcdcbe00640 1 --2- 172.21.15.42:0/1438563170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcdcc076490 0x7fcdcc19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.637+0000 7fcdcbe00640 1 -- 172.21.15.42:0/1438563170 >> 172.21.15.42:0/1438563170 conn(0x7fcdcc0ffad0 msgr2=0x7fcdcc101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:40.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.637+0000 7fcdcbe00640 1 -- 172.21.15.42:0/1438563170 shutdown_connections 2024-04-20T10:34:40.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.637+0000 7fcdcbe00640 1 -- 172.21.15.42:0/1438563170 wait complete. 2024-04-20T10:34:40.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:34:40.654 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:34:40.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.858+0000 7ff5be200640 1 -- 172.21.15.42:0/2106334329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b81016d0 msgr2=0x7ff5b8105b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.858+0000 7ff5be200640 1 --2- 172.21.15.42:0/2106334329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b81016d0 0x7ff5b8105b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009900 tx=0x7ff5a002bd10 comp rx=0 tx=0).stop 2024-04-20T10:34:40.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.858+0000 7ff5be200640 1 -- 172.21.15.42:0/2106334329 shutdown_connections 2024-04-20T10:34:40.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.858+0000 7ff5be200640 1 --2- 172.21.15.42:0/2106334329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5b81061b0 0x7ff5b8106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.858+0000 7ff5be200640 1 --2- 172.21.15.42:0/2106334329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b81016d0 0x7ff5b8105b90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.858+0000 7ff5be200640 1 --2- 172.21.15.42:0/2106334329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b8100db0 0x7ff5b8101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.858+0000 7ff5be200640 1 -- 172.21.15.42:0/2106334329 >> 172.21.15.42:0/2106334329 conn(0x7ff5b80fc7e0 msgr2=0x7ff5b80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:40.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.859+0000 7ff5be200640 1 -- 172.21.15.42:0/2106334329 shutdown_connections 2024-04-20T10:34:40.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.859+0000 7ff5be200640 1 -- 172.21.15.42:0/2106334329 wait complete. 2024-04-20T10:34:40.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.860+0000 7ff5be200640 1 Processor -- start 2024-04-20T10:34:40.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.860+0000 7ff5be200640 1 -- start start 2024-04-20T10:34:40.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.860+0000 7ff5be200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b8100db0 0x7ff5b819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.860+0000 7ff5be200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5b81016d0 0x7ff5b819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.860+0000 7ff5be200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b81061b0 0x7ff5b81a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.860+0000 7ff5be200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b8118820 con 0x7ff5b8100db0 2024-04-20T10:34:40.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.860+0000 7ff5be200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81186a0 con 0x7ff5b81016d0 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.860+0000 7ff5be200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81189a0 con 0x7ff5b81061b0 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.860+0000 7ff5bce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b8100db0 0x7ff5b819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.860+0000 7ff5bce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b8100db0 0x7ff5b819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55932/0 (socket says 172.21.15.42:55932) 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.860+0000 7ff5bce00640 1 -- 172.21.15.42:0/4252928777 learned_addr learned my addr 172.21.15.42:0/4252928777 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.861+0000 7ff5bd800640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b81061b0 0x7ff5b81a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.861+0000 7ff5b7e00640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5b81016d0 0x7ff5b819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.861+0000 7ff5bce00640 1 -- 172.21.15.42:0/4252928777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b81061b0 msgr2=0x7ff5b81a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.861+0000 7ff5bce00640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b81061b0 0x7ff5b81a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.861+0000 7ff5bce00640 1 -- 172.21.15.42:0/4252928777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5b81016d0 msgr2=0x7ff5b819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.861+0000 7ff5bce00640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5b81016d0 0x7ff5b819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.861+0000 7ff5bce00640 1 -- 172.21.15.42:0/4252928777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b81a3d50 con 0x7ff5b8100db0 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.861+0000 7ff5b7e00640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5b81016d0 0x7ff5b819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.861+0000 7ff5bce00640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b8100db0 0x7ff5b819ed60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800eb90 tx=0x7ff5a800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:40.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.862+0000 7ff5b5600640 1 -- 172.21.15.42:0/4252928777 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8004b20 con 0x7ff5b8100db0 2024-04-20T10:34:40.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.862+0000 7ff5b5600640 1 -- 172.21.15.42:0/4252928777 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a8004cc0 con 0x7ff5b8100db0 2024-04-20T10:34:40.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.862+0000 7ff5be200640 1 -- 172.21.15.42:0/4252928777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b81a4040 con 0x7ff5b8100db0 2024-04-20T10:34:40.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.862+0000 7ff5b5600640 1 -- 172.21.15.42:0/4252928777 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8010720 con 0x7ff5b8100db0 2024-04-20T10:34:40.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.862+0000 7ff5be200640 1 -- 172.21.15.42:0/4252928777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b81ab8d0 con 0x7ff5b8100db0 2024-04-20T10:34:40.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.865+0000 7ff5c2422740 1 Processor -- start 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.865+0000 7ff5b5600640 1 -- 172.21.15.42:0/4252928777 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 138822+0+0 (secure 0 0 0) 0x7ff5a8005020 con 0x7ff5b8100db0 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.865+0000 7ff5c2422740 1 -- start start 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.865+0000 7ff5c2422740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 0x55bdeed58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.865+0000 7ff5c2422740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 0x55bdeed61340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.865+0000 7ff5c2422740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed61880 0x55bdeed63c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.865+0000 7ff5c2422740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bdeecbe150 con 0x55bdeed61880 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.865+0000 7ff5c2422740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bdeec64370 con 0x55bdeed58380 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.865+0000 7ff5c2422740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55bdeecba110 con 0x55bdeed58ca0 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.866+0000 7ff5b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed61880 0x55bdeed63c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.866+0000 7ff5b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed61880 0x55bdeed63c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55944/0 (socket says 172.21.15.42:55944) 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.866+0000 7ff5b7e00640 1 -- 172.21.15.42:0/2953611930 learned_addr learned my addr 172.21.15.42:0/2953611930 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.866+0000 7ff5bd800640 1 --2- 172.21.15.42:0/2953611930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 0x55bdeed58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.866+0000 7ff5bce00640 1 --2- 172.21.15.42:0/2953611930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 0x55bdeed61340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.866+0000 7ff5b7e00640 1 -- 172.21.15.42:0/2953611930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 msgr2=0x55bdeed61340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.866+0000 7ff5b7e00640 1 --2- 172.21.15.42:0/2953611930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 0x55bdeed61340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.866+0000 7ff5b7e00640 1 -- 172.21.15.42:0/2953611930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 msgr2=0x55bdeed58760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.866+0000 7ff5b7e00640 1 --2- 172.21.15.42:0/2953611930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 0x55bdeed58760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.866+0000 7ff5b7e00640 1 -- 172.21.15.42:0/2953611930 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdeea9a7d0 con 0x55bdeed61880 2024-04-20T10:34:40.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.866+0000 7ff5bce00640 1 --2- 172.21.15.42:0/2953611930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 0x55bdeed61340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:34:40.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.866+0000 7ff5bd800640 1 --2- 172.21.15.42:0/2953611930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 0x55bdeed58760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.867+0000 7ff5b7e00640 1 --2- 172.21.15.42:0/2953611930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed61880 0x55bdeed63c70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009230 tx=0x7ff5a002c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb30b41f8842fcce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.867+0000 7ff5b5600640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff59c0a3db0 0x7ff59c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.867+0000 7ff5b5600640 1 -- 172.21.15.42:0/4252928777 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff5a80d10b0 con 0x7ff5b8100db0 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.867+0000 7ff5ae000640 1 -- 172.21.15.42:0/2953611930 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a00042c0 con 0x55bdeed61880 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.867+0000 7ff5bd800640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff59c0a3db0 0x7ff59c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.867+0000 7ff5ae000640 1 -- 172.21.15.42:0/2953611930 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a0004460 con 0x55bdeed61880 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.867+0000 7ff5ae000640 1 -- 172.21.15.42:0/2953611930 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0004900 con 0x55bdeed61880 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.868+0000 7ff5c2422740 1 -- 172.21.15.42:0/2953611930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed61880 msgr2=0x55bdeed63c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.868+0000 7ff5c2422740 1 --2- 172.21.15.42:0/2953611930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed61880 0x55bdeed63c70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009230 tx=0x7ff5a002c6d0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.868+0000 7ff5bd800640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff59c0a3db0 0x7ff59c0a6270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009f00 tx=0x7ff5b0005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.868+0000 7ff5c2422740 1 -- 172.21.15.42:0/2953611930 shutdown_connections 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.868+0000 7ff5c2422740 1 --2- 172.21.15.42:0/2953611930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed61880 0x55bdeed63c70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.868+0000 7ff5c2422740 1 --2- 172.21.15.42:0/2953611930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 0x55bdeed61340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.868+0000 7ff5c2422740 1 --2- 172.21.15.42:0/2953611930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 0x55bdeed58760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.868+0000 7ff5c2422740 1 -- 172.21.15.42:0/2953611930 >> 172.21.15.42:0/2953611930 conn(0x55bdeed52ab0 msgr2=0x55bdeed52e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.868+0000 7ff5c2422740 1 -- 172.21.15.42:0/2953611930 shutdown_connections 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.868+0000 7ff5c2422740 1 -- 172.21.15.42:0/2953611930 wait complete. 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.869+0000 7ff5c2422740 1 -- start start 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.869+0000 7ff5c2422740 1 Processor -- start 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5c2422740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58380 0x55bdeed5e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5c2422740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58ca0 0x55bdeed5eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5c2422740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed61880 0x55bdeed5f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5c2422740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bdeed64610 con 0x55bdeed58ca0 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5c2422740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bdeec77740 con 0x55bdeed61880 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5c2422740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55bdeed64790 con 0x55bdeed58380 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5c2422740 10 client.0 _ll_register_callbacks cb 0x55bdeec72b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58ca0 0x55bdeed5eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5bce00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58380 0x55bdeed5e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5bd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed61880 0x55bdeed5f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58ca0 0x55bdeed5eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55948/0 (socket says 172.21.15.42:55948) 2024-04-20T10:34:40.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5b7e00640 1 -- 172.21.15.42:0/1185934557 learned_addr learned my addr 172.21.15.42:0/1185934557 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:40.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5bd800640 1 -- 172.21.15.42:0/1185934557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58380 msgr2=0x55bdeed5e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5bd800640 1 --2- 172.21.15.42:0/1185934557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58380 0x55bdeed5e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5bd800640 1 -- 172.21.15.42:0/1185934557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58ca0 msgr2=0x55bdeed5eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5bd800640 1 --2- 172.21.15.42:0/1185934557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58ca0 0x55bdeed5eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.870+0000 7ff5bd800640 1 -- 172.21.15.42:0/1185934557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdee9d03f0 con 0x55bdeed61880 2024-04-20T10:34:40.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.871+0000 7ff5bd800640 1 --2- 172.21.15.42:0/1185934557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed61880 0x55bdeed5f2e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0012cb0 tx=0x7ff5b0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c67c85bf187af71 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:40.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.871+0000 7ff5a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:34:40.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.871+0000 7ff5a6a00640 1 -- 172.21.15.42:0/1185934557 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0009840 con 0x55bdeed61880 2024-04-20T10:34:40.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.871+0000 7ff5c2422740 1 -- 172.21.15.42:0/1185934557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a0004460 con 0x55bdeed61880 2024-04-20T10:34:40.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.871+0000 7ff5a6a00640 1 -- 172.21.15.42:0/1185934557 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff5b001f640 con 0x55bdeed61880 2024-04-20T10:34:40.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.871+0000 7ff5a6a00640 1 -- 172.21.15.42:0/1185934557 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff5b001f840 con 0x55bdeed61880 2024-04-20T10:34:40.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.871+0000 7ff5c2422740 10 client.25430 fetch_fsmap learned FSMap version 19 2024-04-20T10:34:40.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.871+0000 7ff5c2422740 1 -- 172.21.15.42:0/1185934557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bdee9900f0 con 0x55bdeed61880 2024-04-20T10:34:40.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.872+0000 7ff5a6a00640 1 -- 172.21.15.42:0/1185934557 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a0004460 con 0x55bdeed61880 2024-04-20T10:34:40.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.872+0000 7ff5a6a00640 1 -- 172.21.15.42:0/1185934557 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7ff5b0027070 con 0x55bdeed61880 2024-04-20T10:34:40.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.872+0000 7ff5c2422740 10 client.25430 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:34:40.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.872+0000 7ff5c2422740 10 client.25430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:34:40.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.872+0000 7ff5c2422740 20 client.25430 populate_metadata read hostname 'smithi042' 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.872+0000 7ff5c2422740 1 --2- 172.21.15.42:0/1185934557 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bdeed71bf0 0x55bdeed74050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.872+0000 7ff5c2422740 4 client.25430 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.872+0000 7ff5c2422740 1 -- 172.21.15.42:0/1185934557 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdeeccbb70 con 0x55bdeed71bf0 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.873+0000 7ff5bce00640 1 --2- 172.21.15.42:0/1185934557 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bdeed71bf0 0x55bdeed74050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.873+0000 7ff5bce00640 1 --2- 172.21.15.42:0/1185934557 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bdeed71bf0 0x55bdeed74050 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda2d52524269d0d server_cookie=e482a257e6d97052 in_seq=0 out_seq=0 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.873+0000 7ff5a6a00640 10 client.25430 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5a6a00640 1 -- 172.21.15.42:0/1185934557 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bdeeccbb70 con 0x55bdeed71bf0 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5a6a00640 10 client.25430 handle_command_reply: tid=0 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5c2422740 1 client.25430 shutdown 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5c2422740 1 -- 172.21.15.42:0/1185934557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed61880 msgr2=0x55bdeed5f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5c2422740 1 --2- 172.21.15.42:0/1185934557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed61880 0x55bdeed5f2e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0012cb0 tx=0x7ff5b0004730 comp rx=0 tx=0).stop 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5c2422740 1 -- 172.21.15.42:0/1185934557 shutdown_connections 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5c2422740 1 --2- 172.21.15.42:0/1185934557 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bdeed71bf0 0x55bdeed74050 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5c2422740 1 --2- 172.21.15.42:0/1185934557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed61880 0x55bdeed5f2e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5c2422740 1 --2- 172.21.15.42:0/1185934557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58ca0 0x55bdeed5eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5c2422740 1 --2- 172.21.15.42:0/1185934557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58380 0x55bdeed5e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5a6a00640 0 client.25430 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:40.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5c2422740 1 -- 172.21.15.42:0/1185934557 >> 172.21.15.42:0/1185934557 conn(0x55bdeed52ab0 msgr2=0x55bdeed54800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:40.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5c2422740 1 -- 172.21.15.42:0/1185934557 shutdown_connections 2024-04-20T10:34:40.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5c2422740 1 -- 172.21.15.42:0/1185934557 wait complete. 2024-04-20T10:34:40.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.874+0000 7ff5c2422740 20 client.25430 trim_cache size 0 max 16384 2024-04-20T10:34:40.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5c2422740 1 Processor -- start 2024-04-20T10:34:40.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5c2422740 1 -- start start 2024-04-20T10:34:40.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5c2422740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58380 0x55bdeed60f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5c2422740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 0x55bdeed6d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5c2422740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed61880 0x55bdeed5d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5c2422740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81b2170 con 0x55bdeed58380 2024-04-20T10:34:40.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5c2422740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81b49f0 con 0x55bdeed61880 2024-04-20T10:34:40.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5c2422740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81b27e0 con 0x55bdeed58ca0 2024-04-20T10:34:40.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5b7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 0x55bdeed6d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5bce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58380 0x55bdeed60f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5b7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 0x55bdeed6d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:59858/0 (socket says 172.21.15.42:59858) 2024-04-20T10:34:40.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5b7e00640 1 -- 172.21.15.42:0/1329249987 learned_addr learned my addr 172.21.15.42:0/1329249987 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:40.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.894+0000 7ff5bd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed61880 0x55bdeed5d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.895+0000 7ff5b7e00640 1 -- 172.21.15.42:0/1329249987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed61880 msgr2=0x55bdeed5d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.895+0000 7ff5b7e00640 1 --2- 172.21.15.42:0/1329249987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed61880 0x55bdeed5d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.895+0000 7ff5b7e00640 1 -- 172.21.15.42:0/1329249987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58380 msgr2=0x55bdeed60f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.895+0000 7ff5b7e00640 1 --2- 172.21.15.42:0/1329249987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58380 0x55bdeed60f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.895+0000 7ff5b7e00640 1 -- 172.21.15.42:0/1329249987 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdeed528e0 con 0x55bdeed58ca0 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.895+0000 7ff5bd800640 1 --2- 172.21.15.42:0/1329249987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed61880 0x55bdeed5d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.895+0000 7ff5b7e00640 1 --2- 172.21.15.42:0/1329249987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 0x55bdeed6d4d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009900 tx=0x7ff5a002bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18ac86666aa5a817 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.896+0000 7ff5ad600640 1 -- 172.21.15.42:0/1329249987 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0056070 con 0x55bdeed58ca0 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.896+0000 7ff5ad600640 1 -- 172.21.15.42:0/1329249987 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a002cde0 con 0x55bdeed58ca0 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.896+0000 7ff5ad600640 1 -- 172.21.15.42:0/1329249987 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0051ce0 con 0x55bdeed58ca0 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.896+0000 7ff5c2422740 1 -- 172.21.15.42:0/1329249987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 msgr2=0x55bdeed6d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.896+0000 7ff5c2422740 1 --2- 172.21.15.42:0/1329249987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 0x55bdeed6d4d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009900 tx=0x7ff5a002bd10 comp rx=0 tx=0).stop 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.896+0000 7ff5c2422740 1 -- 172.21.15.42:0/1329249987 shutdown_connections 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.896+0000 7ff5c2422740 1 --2- 172.21.15.42:0/1329249987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed61880 0x55bdeed5d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.896+0000 7ff5c2422740 1 --2- 172.21.15.42:0/1329249987 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed58ca0 0x55bdeed6d4d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.896+0000 7ff5c2422740 1 --2- 172.21.15.42:0/1329249987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58380 0x55bdeed60f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.896+0000 7ff5c2422740 1 -- 172.21.15.42:0/1329249987 >> 172.21.15.42:0/1329249987 conn(0x55bdeed52ab0 msgr2=0x55bdeed51b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.896+0000 7ff5c2422740 1 -- 172.21.15.42:0/1329249987 shutdown_connections 2024-04-20T10:34:40.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.896+0000 7ff5c2422740 1 -- 172.21.15.42:0/1329249987 wait complete. 2024-04-20T10:34:40.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.897+0000 7ff5c2422740 1 -- start start 2024-04-20T10:34:40.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.897+0000 7ff5c2422740 1 Processor -- start 2024-04-20T10:34:40.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.897+0000 7ff5c2422740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 0x55bdeed699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.897+0000 7ff5c2422740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58ca0 0x55bdeed69f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.897+0000 7ff5c2422740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed61880 0x55bdeed669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5c2422740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81b2eb0 con 0x55bdeed58ca0 2024-04-20T10:34:40.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5c2422740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81b4320 con 0x55bdeed58380 2024-04-20T10:34:40.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5c2422740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b81b3580 con 0x55bdeed61880 2024-04-20T10:34:40.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5c2422740 10 client.0 _ll_register_callbacks cb 0x7ff5b0009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:34:40.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5bce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 0x55bdeed699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5bce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 0x55bdeed699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:55584/0 (socket says 172.21.15.42:55584) 2024-04-20T10:34:40.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5bce00640 1 -- 172.21.15.42:0/4089734077 learned_addr learned my addr 172.21.15.42:0/4089734077 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:34:40.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5b7e00640 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58ca0 0x55bdeed69f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5bd800640 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed61880 0x55bdeed669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5bce00640 1 -- 172.21.15.42:0/4089734077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed61880 msgr2=0x55bdeed669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5bce00640 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed61880 0x55bdeed669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5bce00640 1 -- 172.21.15.42:0/4089734077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58ca0 msgr2=0x55bdeed69f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5bce00640 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58ca0 0x55bdeed69f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5bce00640 1 -- 172.21.15.42:0/4089734077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdeed707a0 con 0x55bdeed58380 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5bd800640 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed61880 0x55bdeed669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.898+0000 7ff5b7e00640 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58ca0 0x55bdeed69f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.899+0000 7ff5bce00640 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 0x55bdeed699e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5a809ce10 tx=0x7ff5a80d3570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef698ba836fe669 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.899+0000 7ff5a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.899+0000 7ff5a6a00640 1 -- 172.21.15.42:0/4089734077 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a809ce80 con 0x55bdeed58380 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.899+0000 7ff5c2422740 1 -- 172.21.15.42:0/4089734077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a002cde0 con 0x55bdeed58380 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.899+0000 7ff5a6a00640 1 -- 172.21.15.42:0/4089734077 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff5a8016df0 con 0x55bdeed58380 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.899+0000 7ff5a6a00640 1 -- 172.21.15.42:0/4089734077 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff5a80a1230 con 0x55bdeed58380 2024-04-20T10:34:40.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.899+0000 7ff5c2422740 10 client.25436 fetch_fsmap learned FSMap version 19 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.899+0000 7ff5c2422740 1 -- 172.21.15.42:0/4089734077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bdeed522d0 con 0x55bdeed58380 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.899+0000 7ff5a6a00640 1 -- 172.21.15.42:0/4089734077 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a002cde0 con 0x55bdeed58380 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.899+0000 7ff5a6a00640 1 -- 172.21.15.42:0/4089734077 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7ff5a80b6170 con 0x55bdeed58380 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.900+0000 7ff5c2422740 10 client.25436 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.900+0000 7ff5c2422740 10 client.25436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.900+0000 7ff5c2422740 20 client.25436 populate_metadata read hostname 'smithi042' 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.900+0000 7ff5c2422740 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bdeedd8db0 0x55bdeeddb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.900+0000 7ff5c2422740 4 client.25436 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.900+0000 7ff5c2422740 1 -- 172.21.15.42:0/4089734077 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bdeecc59e0 con 0x55bdeedd8db0 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.900+0000 7ff5b7e00640 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bdeedd8db0 0x55bdeeddb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.900+0000 7ff5b7e00640 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bdeedd8db0 0x55bdeeddb1d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ca0a117d8f997f server_cookie=94f6fdee7358d6c9 in_seq=0 out_seq=0 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.900+0000 7ff5a6a00640 10 client.25436 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:40.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5a6a00640 1 -- 172.21.15.42:0/4089734077 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bdeecc59e0 con 0x55bdeedd8db0 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5a6a00640 10 client.25436 handle_command_reply: tid=0 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5c2422740 1 client.25436 shutdown 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5c2422740 1 -- 172.21.15.42:0/4089734077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 msgr2=0x55bdeed699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5c2422740 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 0x55bdeed699e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5a809ce10 tx=0x7ff5a80d3570 comp rx=0 tx=0).stop 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5c2422740 1 -- 172.21.15.42:0/4089734077 shutdown_connections 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5c2422740 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bdeedd8db0 0x55bdeeddb1d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5c2422740 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdeed61880 0x55bdeed669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5c2422740 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bdeed58ca0 0x55bdeed69f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5c2422740 1 --2- 172.21.15.42:0/4089734077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bdeed58380 0x55bdeed699e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5a6a00640 0 client.25436 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5c2422740 1 -- 172.21.15.42:0/4089734077 >> 172.21.15.42:0/4089734077 conn(0x55bdeed52ab0 msgr2=0x55bdeed5e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5c2422740 1 -- 172.21.15.42:0/4089734077 shutdown_connections 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5c2422740 1 -- 172.21.15.42:0/4089734077 wait complete. 2024-04-20T10:34:40.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.901+0000 7ff5c2422740 20 client.25436 trim_cache size 0 max 16384 2024-04-20T10:34:40.906 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:34:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T10:34:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:34:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "8f69c087-6c85-4620-bf19-03a77ebec9d7": { 2024-04-20T10:34:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:34:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "8f69c087-6c85-4620-bf19-03a77ebec9d7", 2024-04-20T10:34:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:34:40.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:34:40.907 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:34:40.907 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:34:40.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.902+0000 7ff5be200640 1 -- 172.21.15.42:0/4252928777 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff59c0a3db0 msgr2=0x7ff59c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.902+0000 7ff5be200640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff59c0a3db0 0x7ff59c0a6270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009f00 tx=0x7ff5b0005f90 comp rx=0 tx=0).stop 2024-04-20T10:34:40.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.902+0000 7ff5be200640 1 -- 172.21.15.42:0/4252928777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b8100db0 msgr2=0x7ff5b819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:34:40.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.902+0000 7ff5be200640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b8100db0 0x7ff5b819ed60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800eb90 tx=0x7ff5a800ef90 comp rx=0 tx=0).stop 2024-04-20T10:34:40.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.902+0000 7ff5be200640 1 -- 172.21.15.42:0/4252928777 shutdown_connections 2024-04-20T10:34:40.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.902+0000 7ff5be200640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff59c0a3db0 0x7ff59c0a6270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.902+0000 7ff5be200640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b81061b0 0x7ff5b81a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.902+0000 7ff5be200640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5b81016d0 0x7ff5b819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.902+0000 7ff5be200640 1 --2- 172.21.15.42:0/4252928777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b8100db0 0x7ff5b819ed60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:34:40.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.902+0000 7ff5be200640 1 -- 172.21.15.42:0/4252928777 >> 172.21.15.42:0/4252928777 conn(0x7ff5b80fc7e0 msgr2=0x7ff5b80fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:34:40.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.902+0000 7ff5be200640 1 -- 172.21.15.42:0/4252928777 shutdown_connections 2024-04-20T10:34:40.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:34:40.902+0000 7ff5be200640 1 -- 172.21.15.42:0/4252928777 wait complete. 2024-04-20T10:34:40.919 INFO:tasks.cephfs.filesystem:scrub status for tag:8f69c087-6c85-4620-bf19-03a77ebec9d7 - {'path': '/', 'tag': '8f69c087-6c85-4620-bf19-03a77ebec9d7', 'options': 'recursive,force'} 2024-04-20T10:34:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:41 smithi046 ceph-mon[21574]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:41 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:34:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:41 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:34:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:41 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:34:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:41 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:34:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:41 smithi042 ceph-mon[17078]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:41 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:34:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:41 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:34:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:41 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:34:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:41 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:34:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:41 smithi162 ceph-mon[21854]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:41 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:34:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:41 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:34:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:41 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:34:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:41 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:34:43.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:43 smithi046 ceph-mon[21574]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T10:34:43.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:43 smithi042 ceph-mon[17078]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T10:34:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:43 smithi162 ceph-mon[21854]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T10:34:45.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:45 smithi046 ceph-mon[21574]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T10:34:45.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:45 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:34:45.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:45 smithi042 ceph-mon[17078]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T10:34:45.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:45 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:34:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:45 smithi162 ceph-mon[21854]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T10:34:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:45 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:34:47.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:47 smithi046 ceph-mon[21574]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-20T10:34:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:47 smithi042 ceph-mon[17078]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-20T10:34:47.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:47 smithi162 ceph-mon[21854]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-20T10:34:49.701 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:49 smithi162 ceph-mon[21854]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-20T10:34:49.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:49 smithi046 ceph-mon[21574]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-20T10:34:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:49 smithi042 ceph-mon[17078]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-20T10:34:51.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:51 smithi046 ceph-mon[21574]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-20T10:34:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:51 smithi042 ceph-mon[17078]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-20T10:34:51.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:51 smithi162 ceph-mon[21854]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-20T10:34:53.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:53 smithi046 ceph-mon[21574]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-20T10:34:53.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:53 smithi042 ceph-mon[17078]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-20T10:34:53.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:53 smithi162 ceph-mon[21854]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-20T10:34:55.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:55 smithi046 ceph-mon[21574]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-20T10:34:55.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:55 smithi042 ceph-mon[17078]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-20T10:34:55.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:55 smithi162 ceph-mon[21854]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-20T10:34:57.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:57 smithi046 ceph-mon[21574]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-20T10:34:57.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:57 smithi042 ceph-mon[17078]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-20T10:34:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:57 smithi162 ceph-mon[21854]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-20T10:34:59.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:34:59 smithi046 ceph-mon[21574]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:59.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:34:59 smithi042 ceph-mon[17078]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:34:59.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:34:59 smithi162 ceph-mon[21854]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:35:01.354 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:35:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:35:00.883+0000 7fd42ac00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:35:01.354 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:35:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:35:00.883+0000 7fd42ac00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:35:01.354 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:35:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:35:00.883+0000 7fd42ac00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:35:01.354 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:35:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:35:00.883+0000 7fd42ac00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:35:01.354 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:35:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x[17316]: 2024-04-20T10:35:00.883+0000 7fd42ac00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T10:35:01.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:01 smithi046 ceph-mon[21574]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:35:01.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:01 smithi042 ceph-mon[17078]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:35:01.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:01 smithi162 ceph-mon[21854]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-20T10:35:03.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:03 smithi046 ceph-mon[21574]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T10:35:03.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:03 smithi046 ceph-mon[21574]: mgrmap e26: x(active, since 62s), standbys: y, z 2024-04-20T10:35:03.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:03 smithi042 ceph-mon[17078]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T10:35:03.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:03 smithi042 ceph-mon[17078]: mgrmap e26: x(active, since 62s), standbys: y, z 2024-04-20T10:35:03.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:03 smithi162 ceph-mon[21854]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T10:35:03.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:03 smithi162 ceph-mon[21854]: mgrmap e26: x(active, since 62s), standbys: y, z 2024-04-20T10:35:05.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:05 smithi046 ceph-mon[21574]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T10:35:05.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:05 smithi042 ceph-mon[17078]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T10:35:05.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:05 smithi162 ceph-mon[21854]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T10:35:07.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:07 smithi046 ceph-mon[21574]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T10:35:07.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:07 smithi042 ceph-mon[17078]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T10:35:07.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:07 smithi162 ceph-mon[21854]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T10:35:09.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:09 smithi046 ceph-mon[21574]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T10:35:09.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:09 smithi042 ceph-mon[17078]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T10:35:09.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:09 smithi162 ceph-mon[21854]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T10:35:10.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:10 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:35:10.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:10 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:35:10.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:10 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:35:10.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:10 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:35:10.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:10 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:35:10.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:10 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:35:10.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:10 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:35:10.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:10 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:35:10.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:10 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:35:10.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:35:10.921 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:35:11.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.153+0000 7f66fd800640 1 -- 172.21.15.42:0/2191758143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66f8104410 msgr2=0x7f66f81047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.153+0000 7f66fd800640 1 --2- 172.21.15.42:0/2191758143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66f8104410 0x7f66f81047f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009860 tx=0x7f66e802bb70 comp rx=0 tx=0).stop 2024-04-20T10:35:11.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.153+0000 7f66fd800640 1 -- 172.21.15.42:0/2191758143 shutdown_connections 2024-04-20T10:35:11.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.153+0000 7f66fd800640 1 --2- 172.21.15.42:0/2191758143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f81094e0 0x7f66f81110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.153+0000 7f66fd800640 1 --2- 172.21.15.42:0/2191758143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f8104dc0 0x7f66f8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.153+0000 7f66fd800640 1 --2- 172.21.15.42:0/2191758143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66f8104410 0x7f66f81047f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.153+0000 7f66fd800640 1 -- 172.21.15.42:0/2191758143 >> 172.21.15.42:0/2191758143 conn(0x7f66f80ffcf0 msgr2=0x7f66f8102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:11.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.153+0000 7f66fd800640 1 -- 172.21.15.42:0/2191758143 shutdown_connections 2024-04-20T10:35:11.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.153+0000 7f66fd800640 1 -- 172.21.15.42:0/2191758143 wait complete. 2024-04-20T10:35:11.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.155+0000 7f66fd800640 1 Processor -- start 2024-04-20T10:35:11.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.155+0000 7f66fd800640 1 -- start start 2024-04-20T10:35:11.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66fd800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66f8104410 0x7f66f81a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66fd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f81094e0 0x7f66f81a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66fd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f819a8d0 0x7f66f819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66fd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f66f8114570 con 0x7f66f81094e0 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66fd800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81143f0 con 0x7f66f819a8d0 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66fd800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81146f0 con 0x7f66f8104410 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66f6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f81094e0 0x7f66f81a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66f6a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66f8104410 0x7f66f81a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66f6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f81094e0 0x7f66f81a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36952/0 (socket says 172.21.15.42:36952) 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66f6000640 1 -- 172.21.15.42:0/1032988764 learned_addr learned my addr 172.21.15.42:0/1032988764 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66f6a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66f8104410 0x7f66f81a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:35200/0 (socket says 172.21.15.42:35200) 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66f7400640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f819a8d0 0x7f66f819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66f6a00640 1 -- 172.21.15.42:0/1032988764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f819a8d0 msgr2=0x7f66f819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66f6a00640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f819a8d0 0x7f66f819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66f6a00640 1 -- 172.21.15.42:0/1032988764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f81094e0 msgr2=0x7f66f81a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66f6a00640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f81094e0 0x7f66f81a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.156+0000 7f66f6a00640 1 -- 172.21.15.42:0/1032988764 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f819b3f0 con 0x7f66f8104410 2024-04-20T10:35:11.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.157+0000 7f66f7400640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f819a8d0 0x7f66f819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:35:11.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.157+0000 7f66f6a00640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66f8104410 0x7f66f81a0720 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f66e802c0b0 tx=0x7f66e80323b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:11.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.157+0000 7f66ef400640 1 -- 172.21.15.42:0/1032988764 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8041020 con 0x7f66f8104410 2024-04-20T10:35:11.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.157+0000 7f66ef400640 1 -- 172.21.15.42:0/1032988764 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e802c9f0 con 0x7f66f8104410 2024-04-20T10:35:11.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.157+0000 7f66fd800640 1 -- 172.21.15.42:0/1032988764 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f819b680 con 0x7f66f8104410 2024-04-20T10:35:11.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.157+0000 7f66ef400640 1 -- 172.21.15.42:0/1032988764 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8039040 con 0x7f66f8104410 2024-04-20T10:35:11.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.157+0000 7f66fd800640 1 -- 172.21.15.42:0/1032988764 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f81a78d0 con 0x7f66f8104410 2024-04-20T10:35:11.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.161+0000 7f67005d3740 1 Processor -- start 2024-04-20T10:35:11.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.161+0000 7f67005d3740 1 -- start start 2024-04-20T10:35:11.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.162+0000 7f67005d3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 0x56288b329580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.162+0000 7f67005d3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b332160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.162+0000 7f67005d3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3326a0 0x56288b334a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.162+0000 7f67005d3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56288b28f150 con 0x56288b329ac0 2024-04-20T10:35:11.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.162+0000 7f67005d3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56288b235370 con 0x56288b3326a0 2024-04-20T10:35:11.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.162+0000 7f67005d3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56288b28b110 con 0x56288b3291a0 2024-04-20T10:35:11.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.162+0000 7f66ef400640 1 -- 172.21.15.42:0/1032988764 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7f66e8044020 con 0x7f66f8104410 2024-04-20T10:35:11.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.162+0000 7f66f7400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 0x56288b329580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.162+0000 7f66f7400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 0x56288b329580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:35216/0 (socket says 172.21.15.42:35216) 2024-04-20T10:35:11.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.162+0000 7f66f7400640 1 -- 172.21.15.42:0/1960272260 learned_addr learned my addr 172.21.15.42:0/1960272260 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:11.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.163+0000 7f66f6a00640 1 --2- 172.21.15.42:0/1960272260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b332160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.163+0000 7f66f7400640 1 -- 172.21.15.42:0/1960272260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3326a0 msgr2=0x56288b334a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:35:11.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.163+0000 7f66f7400640 1 --2- 172.21.15.42:0/1960272260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3326a0 0x56288b334a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.163+0000 7f66f7400640 1 -- 172.21.15.42:0/1960272260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 msgr2=0x56288b332160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:11.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.163+0000 7f66f7400640 1 --2- 172.21.15.42:0/1960272260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b332160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.163+0000 7f66f7400640 1 -- 172.21.15.42:0/1960272260 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56288b06b7d0 con 0x56288b3291a0 2024-04-20T10:35:11.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.163+0000 7f66f6a00640 1 --2- 172.21.15.42:0/1960272260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b332160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:35:11.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.164+0000 7f66f7400640 1 --2- 172.21.15.42:0/1960272260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 0x56288b329580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009fd0 tx=0x7f66f000f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bccc146d0189849 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:11.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.164+0000 7f66e3e00640 1 -- 172.21.15.42:0/1960272260 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0015070 con 0x56288b3291a0 2024-04-20T10:35:11.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.164+0000 7f66e3e00640 1 -- 172.21.15.42:0/1960272260 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f0011be0 con 0x56288b3291a0 2024-04-20T10:35:11.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.164+0000 7f66e3e00640 1 -- 172.21.15.42:0/1960272260 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0010c50 con 0x56288b3291a0 2024-04-20T10:35:11.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.164+0000 7f66ef400640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f66dc0a3f40 0x7f66dc0a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.165+0000 7f66ef400640 1 -- 172.21.15.42:0/1032988764 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f66e80f2180 con 0x7f66f8104410 2024-04-20T10:35:11.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.165+0000 7f67005d3740 1 -- 172.21.15.42:0/1960272260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 msgr2=0x56288b329580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.165+0000 7f67005d3740 1 --2- 172.21.15.42:0/1960272260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 0x56288b329580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009fd0 tx=0x7f66f000f7f0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.165+0000 7f66f6a00640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f66dc0a3f40 0x7f66dc0a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.165+0000 7f67005d3740 1 -- 172.21.15.42:0/1960272260 shutdown_connections 2024-04-20T10:35:11.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.165+0000 7f67005d3740 1 --2- 172.21.15.42:0/1960272260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3326a0 0x56288b334a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.165+0000 7f67005d3740 1 --2- 172.21.15.42:0/1960272260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b332160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.165+0000 7f67005d3740 1 --2- 172.21.15.42:0/1960272260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 0x56288b329580 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.165+0000 7f67005d3740 1 -- 172.21.15.42:0/1960272260 >> 172.21.15.42:0/1960272260 conn(0x56288b3238d0 msgr2=0x56288b323cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:11.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.165+0000 7f67005d3740 1 -- 172.21.15.42:0/1960272260 shutdown_connections 2024-04-20T10:35:11.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.165+0000 7f67005d3740 1 -- 172.21.15.42:0/1960272260 wait complete. 2024-04-20T10:35:11.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.166+0000 7f66f6a00640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f66dc0a3f40 0x7f66dc0a6400 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66e80ef860 tx=0x7f66e8032820 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:11.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.166+0000 7f67005d3740 1 -- start start 2024-04-20T10:35:11.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.166+0000 7f67005d3740 1 Processor -- start 2024-04-20T10:35:11.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.167+0000 7f67005d3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3291a0 0x56288b32f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.167+0000 7f67005d3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b32fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.167+0000 7f67005d3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3326a0 0x56288b3301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.167+0000 7f67005d3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56288b335490 con 0x56288b329ac0 2024-04-20T10:35:11.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.167+0000 7f67005d3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56288b248740 con 0x56288b3291a0 2024-04-20T10:35:11.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.167+0000 7f67005d3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56288b335610 con 0x56288b3326a0 2024-04-20T10:35:11.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.167+0000 7f67005d3740 10 client.0 _ll_register_callbacks cb 0x56288b243b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:11.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.167+0000 7f66f7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3291a0 0x56288b32f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.167+0000 7f66f7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3291a0 0x56288b32f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52344/0 (socket says 172.21.15.42:52344) 2024-04-20T10:35:11.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.167+0000 7f66f7400640 1 -- 172.21.15.42:0/2257505084 learned_addr learned my addr 172.21.15.42:0/2257505084 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:11.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.167+0000 7f66f7400640 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3326a0 0x56288b3301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.167+0000 7f66f6000640 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b32fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.168+0000 7f66f7400640 1 -- 172.21.15.42:0/2257505084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3291a0 msgr2=0x56288b32f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:11.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.168+0000 7f66f7400640 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3291a0 0x56288b32f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.168+0000 7f66f7400640 1 -- 172.21.15.42:0/2257505084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 msgr2=0x56288b32fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:11.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.168+0000 7f66f7400640 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b32fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.168+0000 7f66f7400640 1 -- 172.21.15.42:0/2257505084 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56288afa13f0 con 0x56288b3326a0 2024-04-20T10:35:11.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.168+0000 7f66f6000640 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b32fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:35:11.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.168+0000 7f66f7400640 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3326a0 0x56288b3301c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f66f0000f80 tx=0x7f66f00098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd0b1f0ff65253e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:11.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.168+0000 7f66e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.168+0000 7f66e0c00640 1 -- 172.21.15.42:0/2257505084 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0015040 con 0x56288b3326a0 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.168+0000 7f66e0c00640 1 -- 172.21.15.42:0/2257505084 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f66f0010de0 con 0x56288b3326a0 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.168+0000 7f67005d3740 1 -- 172.21.15.42:0/2257505084 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66f0011be0 con 0x56288b3326a0 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.169+0000 7f66e0c00640 1 -- 172.21.15.42:0/2257505084 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f66f0003970 con 0x56288b3326a0 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.169+0000 7f67005d3740 10 client.25471 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.169+0000 7f67005d3740 1 -- 172.21.15.42:0/2257505084 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56288af610f0 con 0x56288b3326a0 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.169+0000 7f66e0c00640 1 -- 172.21.15.42:0/2257505084 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f0011be0 con 0x56288b3326a0 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.169+0000 7f66e0c00640 1 -- 172.21.15.42:0/2257505084 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f66f0028030 con 0x56288b3326a0 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.170+0000 7f67005d3740 10 client.25471 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.170+0000 7f67005d3740 10 client.25471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.170+0000 7f67005d3740 20 client.25471 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.170+0000 7f67005d3740 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56288b3429d0 0x56288b344dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.170+0000 7f67005d3740 4 client.25471 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:35:11.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.170+0000 7f67005d3740 1 -- 172.21.15.42:0/2257505084 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56288b29cb70 con 0x56288b3429d0 2024-04-20T10:35:11.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.170+0000 7f66f6000640 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56288b3429d0 0x56288b344dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.170+0000 7f66f6000640 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56288b3429d0 0x56288b344dc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cb369979ae5ad08 server_cookie=f990897c1a26459e in_seq=0 out_seq=0 2024-04-20T10:35:11.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.170+0000 7f66e0c00640 10 client.25471 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:11.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.171+0000 7f66e0c00640 1 -- 172.21.15.42:0/2257505084 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56288b29cb70 con 0x56288b3429d0 2024-04-20T10:35:11.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.171+0000 7f66e0c00640 10 client.25471 handle_command_reply: tid=0 2024-04-20T10:35:11.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.171+0000 7f67005d3740 1 client.25471 shutdown 2024-04-20T10:35:11.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.171+0000 7f67005d3740 1 -- 172.21.15.42:0/2257505084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3326a0 msgr2=0x56288b3301c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.171+0000 7f67005d3740 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3326a0 0x56288b3301c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f66f0000f80 tx=0x7f66f00098c0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.172+0000 7f67005d3740 1 -- 172.21.15.42:0/2257505084 shutdown_connections 2024-04-20T10:35:11.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.172+0000 7f67005d3740 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56288b3429d0 0x56288b344dc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.172+0000 7f67005d3740 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3326a0 0x56288b3301c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.172+0000 7f67005d3740 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b32fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.172+0000 7f67005d3740 1 --2- 172.21.15.42:0/2257505084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3291a0 0x56288b32f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.172+0000 7f67005d3740 1 -- 172.21.15.42:0/2257505084 >> 172.21.15.42:0/2257505084 conn(0x56288b3238d0 msgr2=0x56288b325620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:11.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.172+0000 7f66e0c00640 0 client.25471 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:11.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.172+0000 7f67005d3740 1 -- 172.21.15.42:0/2257505084 shutdown_connections 2024-04-20T10:35:11.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.172+0000 7f67005d3740 1 -- 172.21.15.42:0/2257505084 wait complete. 2024-04-20T10:35:11.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.172+0000 7f67005d3740 20 client.25471 trim_cache size 0 max 16384 2024-04-20T10:35:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.193+0000 7f67005d3740 1 Processor -- start 2024-04-20T10:35:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.193+0000 7f67005d3740 1 -- start start 2024-04-20T10:35:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.193+0000 7f67005d3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 0x56288b32ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.193+0000 7f67005d3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b325410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.193+0000 7f67005d3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3326a0 0x56288b32c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.193+0000 7f67005d3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81adf80 con 0x56288b329ac0 2024-04-20T10:35:11.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.193+0000 7f67005d3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81b0800 con 0x56288b3326a0 2024-04-20T10:35:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.193+0000 7f67005d3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81ae5f0 con 0x56288b3291a0 2024-04-20T10:35:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.194+0000 7f66f6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b325410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.194+0000 7f66f7400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 0x56288b32ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.194+0000 7f66f6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b325410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36980/0 (socket says 172.21.15.42:36980) 2024-04-20T10:35:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.194+0000 7f66f6000640 1 -- 172.21.15.42:0/3954666111 learned_addr learned my addr 172.21.15.42:0/3954666111 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.194+0000 7f66f7400640 1 --2- 172.21.15.42:0/3954666111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3326a0 0x56288b32c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.194+0000 7f66f6000640 1 -- 172.21.15.42:0/3954666111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 msgr2=0x56288b32ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:11.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.194+0000 7f66f6000640 1 --2- 172.21.15.42:0/3954666111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 0x56288b32ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.194+0000 7f66f6000640 1 -- 172.21.15.42:0/3954666111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3326a0 msgr2=0x56288b32c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.194+0000 7f66f6000640 1 --2- 172.21.15.42:0/3954666111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3326a0 0x56288b32c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.194+0000 7f66f6000640 1 -- 172.21.15.42:0/3954666111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56288b324210 con 0x56288b329ac0 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.194+0000 7f66f7400640 1 --2- 172.21.15.42:0/3954666111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 0x56288b32ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.194+0000 7f66f6000640 1 --2- 172.21.15.42:0/3954666111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b325410 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x56288b324a60 tx=0x7f66e40054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9f87f5c8b24940 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.195+0000 7f66e3400640 1 -- 172.21.15.42:0/3954666111 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4022070 con 0x56288b329ac0 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.195+0000 7f66e3400640 1 -- 172.21.15.42:0/3954666111 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e4004e00 con 0x56288b329ac0 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.195+0000 7f66e3400640 1 -- 172.21.15.42:0/3954666111 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4003850 con 0x56288b329ac0 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.195+0000 7f67005d3740 1 -- 172.21.15.42:0/3954666111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 msgr2=0x56288b325410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.195+0000 7f67005d3740 1 --2- 172.21.15.42:0/3954666111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b325410 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x56288b324a60 tx=0x7f66e40054d0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.196+0000 7f67005d3740 1 -- 172.21.15.42:0/3954666111 shutdown_connections 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.196+0000 7f67005d3740 1 --2- 172.21.15.42:0/3954666111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3326a0 0x56288b32c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.196+0000 7f67005d3740 1 --2- 172.21.15.42:0/3954666111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b325410 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.196+0000 7f67005d3740 1 --2- 172.21.15.42:0/3954666111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3291a0 0x56288b32ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.196+0000 7f67005d3740 1 -- 172.21.15.42:0/3954666111 >> 172.21.15.42:0/3954666111 conn(0x56288b3238d0 msgr2=0x56288b324d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.196+0000 7f67005d3740 1 -- 172.21.15.42:0/3954666111 shutdown_connections 2024-04-20T10:35:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.196+0000 7f67005d3740 1 -- 172.21.15.42:0/3954666111 wait complete. 2024-04-20T10:35:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.196+0000 7f67005d3740 1 -- start start 2024-04-20T10:35:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.196+0000 7f67005d3740 1 Processor -- start 2024-04-20T10:35:11.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.197+0000 7f67005d3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3291a0 0x56288b345a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.197+0000 7f67005d3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b345f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.197+0000 7f67005d3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3326a0 0x56288b3464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.197+0000 7f67005d3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81aecc0 con 0x56288b329ac0 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.197+0000 7f67005d3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81b0130 con 0x56288b3291a0 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.197+0000 7f67005d3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81af390 con 0x56288b3326a0 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.197+0000 7f67005d3740 10 client.0 _ll_register_callbacks cb 0x7f66f0011d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.197+0000 7f66f6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b345f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.197+0000 7f66f6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b345f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36988/0 (socket says 172.21.15.42:36988) 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.197+0000 7f66f6000640 1 -- 172.21.15.42:0/1903204994 learned_addr learned my addr 172.21.15.42:0/1903204994 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.197+0000 7f66f7400640 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3291a0 0x56288b345a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.197+0000 7f66f7400640 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3326a0 0x56288b3464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.198+0000 7f66f6000640 1 -- 172.21.15.42:0/1903204994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3326a0 msgr2=0x56288b3464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.198+0000 7f66f6000640 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3326a0 0x56288b3464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.198+0000 7f66f6000640 1 -- 172.21.15.42:0/1903204994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3291a0 msgr2=0x56288b345a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.198+0000 7f66f6000640 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3291a0 0x56288b345a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.198+0000 7f66f6000640 1 -- 172.21.15.42:0/1903204994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56288b3b05c0 con 0x56288b329ac0 2024-04-20T10:35:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.198+0000 7f66f6000640 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b345f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f66e401c8b0 tx=0x7f66e4009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10cb2164cd012de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.198+0000 7f66e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:35:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.198+0000 7f66e0c00640 1 -- 172.21.15.42:0/1903204994 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4022070 con 0x56288b329ac0 2024-04-20T10:35:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.198+0000 7f66e0c00640 1 -- 172.21.15.42:0/1903204994 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f66e4004040 con 0x56288b329ac0 2024-04-20T10:35:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.198+0000 7f67005d3740 1 -- 172.21.15.42:0/1903204994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66e4004e00 con 0x56288b329ac0 2024-04-20T10:35:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.199+0000 7f66e0c00640 1 -- 172.21.15.42:0/1903204994 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f66e4036050 con 0x56288b329ac0 2024-04-20T10:35:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.199+0000 7f67005d3740 10 client.15345 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.199+0000 7f67005d3740 1 -- 172.21.15.42:0/1903204994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56288b3b1890 con 0x56288b329ac0 2024-04-20T10:35:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.199+0000 7f66e0c00640 1 -- 172.21.15.42:0/1903204994 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e4004e00 con 0x56288b329ac0 2024-04-20T10:35:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.200+0000 7f66e0c00640 1 -- 172.21.15.42:0/1903204994 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f66e40035f0 con 0x56288b329ac0 2024-04-20T10:35:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.200+0000 7f67005d3740 10 client.15345 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.200+0000 7f67005d3740 10 client.15345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.200+0000 7f67005d3740 20 client.15345 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.200+0000 7f67005d3740 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56288b3b1f20 0x56288b3b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.200+0000 7f67005d3740 4 client.15345 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:35:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.200+0000 7f67005d3740 1 -- 172.21.15.42:0/1903204994 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56288b2969e0 con 0x56288b3b1f20 2024-04-20T10:35:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.200+0000 7f66f7400640 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56288b3b1f20 0x56288b3b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.201+0000 7f66e0c00640 10 client.15345 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.201+0000 7f66f7400640 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56288b3b1f20 0x56288b3b4340 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d41902321b644d6 server_cookie=b0686bf56d70979b in_seq=0 out_seq=0 2024-04-20T10:35:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.201+0000 7f66e0c00640 1 -- 172.21.15.42:0/1903204994 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56288b2969e0 con 0x56288b3b1f20 2024-04-20T10:35:11.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.201+0000 7f66e0c00640 10 client.15345 handle_command_reply: tid=0 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.201+0000 7f67005d3740 1 client.15345 shutdown 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.201+0000 7f67005d3740 1 -- 172.21.15.42:0/1903204994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 msgr2=0x56288b345f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.201+0000 7f67005d3740 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b345f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f66e401c8b0 tx=0x7f66e4009760 comp rx=0 tx=0).stop 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.202+0000 7f67005d3740 1 -- 172.21.15.42:0/1903204994 shutdown_connections 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.202+0000 7f67005d3740 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56288b3b1f20 0x56288b3b4340 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.202+0000 7f67005d3740 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56288b3326a0 0x56288b3464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.202+0000 7f67005d3740 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56288b329ac0 0x56288b345f90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.202+0000 7f67005d3740 1 --2- 172.21.15.42:0/1903204994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56288b3291a0 0x56288b345a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.202+0000 7f67005d3740 1 -- 172.21.15.42:0/1903204994 >> 172.21.15.42:0/1903204994 conn(0x56288b3238d0 msgr2=0x56288b33e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.202+0000 7f66e0c00640 0 client.15345 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.202+0000 7f67005d3740 1 -- 172.21.15.42:0/1903204994 shutdown_connections 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.202+0000 7f67005d3740 1 -- 172.21.15.42:0/1903204994 wait complete. 2024-04-20T10:35:11.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.202+0000 7f67005d3740 20 client.15345 trim_cache size 0 max 16384 2024-04-20T10:35:11.204 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:35:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:35:11.204 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:35:11.204 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:35:11.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.203+0000 7f66fd800640 1 -- 172.21.15.42:0/1032988764 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f66dc0a3f40 msgr2=0x7f66dc0a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.203+0000 7f66fd800640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f66dc0a3f40 0x7f66dc0a6400 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66e80ef860 tx=0x7f66e8032820 comp rx=0 tx=0).stop 2024-04-20T10:35:11.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.203+0000 7f66fd800640 1 -- 172.21.15.42:0/1032988764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66f8104410 msgr2=0x7f66f81a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.203+0000 7f66fd800640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66f8104410 0x7f66f81a0720 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f66e802c0b0 tx=0x7f66e80323b0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.203+0000 7f66fd800640 1 -- 172.21.15.42:0/1032988764 shutdown_connections 2024-04-20T10:35:11.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.203+0000 7f66fd800640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f66dc0a3f40 0x7f66dc0a6400 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.203+0000 7f66fd800640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f819a8d0 0x7f66f819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.203+0000 7f66fd800640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f81094e0 0x7f66f81a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.203+0000 7f66fd800640 1 --2- 172.21.15.42:0/1032988764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66f8104410 0x7f66f81a0720 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.203+0000 7f66fd800640 1 -- 172.21.15.42:0/1032988764 >> 172.21.15.42:0/1032988764 conn(0x7f66f80ffcf0 msgr2=0x7f66f81010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:11.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.204+0000 7f66fd800640 1 -- 172.21.15.42:0/1032988764 shutdown_connections 2024-04-20T10:35:11.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.204+0000 7f66fd800640 1 -- 172.21.15.42:0/1032988764 wait complete. 2024-04-20T10:35:11.222 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:35:11.223 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:35:11.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.417+0000 7f9f26a00640 1 -- 172.21.15.42:0/1519760266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f20112d90 msgr2=0x7f9f20115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.417+0000 7f9f26a00640 1 --2- 172.21.15.42:0/1519760266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f20112d90 0x7f9f20115180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009be0 tx=0x7f9f1802bf80 comp rx=0 tx=0).stop 2024-04-20T10:35:11.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.417+0000 7f9f26a00640 1 -- 172.21.15.42:0/1519760266 shutdown_connections 2024-04-20T10:35:11.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.417+0000 7f9f26a00640 1 --2- 172.21.15.42:0/1519760266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f20112d90 0x7f9f20115180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.417+0000 7f9f26a00640 1 --2- 172.21.15.42:0/1519760266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20074c20 0x7f9f20112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.417+0000 7f9f26a00640 1 --2- 172.21.15.42:0/1519760266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f20074300 0x7f9f200746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.417+0000 7f9f26a00640 1 -- 172.21.15.42:0/1519760266 >> 172.21.15.42:0/1519760266 conn(0x7f9f200fd890 msgr2=0x7f9f200ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:11.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.417+0000 7f9f26a00640 1 -- 172.21.15.42:0/1519760266 shutdown_connections 2024-04-20T10:35:11.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.417+0000 7f9f26a00640 1 -- 172.21.15.42:0/1519760266 wait complete. 2024-04-20T10:35:11.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.418+0000 7f9f26a00640 1 Processor -- start 2024-04-20T10:35:11.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.419+0000 7f9f26a00640 1 -- start start 2024-04-20T10:35:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.419+0000 7f9f26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f20074300 0x7f9f201a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.419+0000 7f9f26a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20074c20 0x7f9f201a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.419+0000 7f9f26a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f20112d90 0x7f9f2019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.419+0000 7f9f26a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201187a0 con 0x7f9f20074300 2024-04-20T10:35:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.419+0000 7f9f26a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20118620 con 0x7f9f20074c20 2024-04-20T10:35:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.419+0000 7f9f26a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20118920 con 0x7f9f20112d90 2024-04-20T10:35:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.419+0000 7f9f25600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f20074300 0x7f9f201a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.419+0000 7f9f25600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f20074300 0x7f9f201a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37012/0 (socket says 172.21.15.42:37012) 2024-04-20T10:35:11.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.419+0000 7f9f25600640 1 -- 172.21.15.42:0/2444869026 learned_addr learned my addr 172.21.15.42:0/2444869026 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:11.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.420+0000 7f9f24c00640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20074c20 0x7f9f201a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.420+0000 7f9f26000640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f20112d90 0x7f9f2019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.420+0000 7f9f25600640 1 -- 172.21.15.42:0/2444869026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f20112d90 msgr2=0x7f9f2019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.420+0000 7f9f25600640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f20112d90 0x7f9f2019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.420+0000 7f9f25600640 1 -- 172.21.15.42:0/2444869026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20074c20 msgr2=0x7f9f201a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.420+0000 7f9f25600640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20074c20 0x7f9f201a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.420+0000 7f9f25600640 1 -- 172.21.15.42:0/2444869026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2019f220 con 0x7f9f20074300 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.420+0000 7f9f26000640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f20112d90 0x7f9f2019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.420+0000 7f9f24c00640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20074c20 0x7f9f201a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.420+0000 7f9f25600640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f20074300 0x7f9f201a48f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400e9c0 tx=0x7f9f1400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.421+0000 7f9f1e000640 1 -- 172.21.15.42:0/2444869026 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400cde0 con 0x7f9f20074300 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.421+0000 7f9f26a00640 1 -- 172.21.15.42:0/2444869026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2019f510 con 0x7f9f20074300 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.421+0000 7f9f1e000640 1 -- 172.21.15.42:0/2444869026 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f140035f0 con 0x7f9f20074300 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.421+0000 7f9f1e000640 1 -- 172.21.15.42:0/2444869026 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14010650 con 0x7f9f20074300 2024-04-20T10:35:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.421+0000 7f9f26a00640 1 -- 172.21.15.42:0/2444869026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f201ab7b0 con 0x7f9f20074300 2024-04-20T10:35:11.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.422+0000 7f9f26a00640 1 -- 172.21.15.42:0/2444869026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f20068ca0 con 0x7f9f20074300 2024-04-20T10:35:11.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.424+0000 7f9f1e000640 1 -- 172.21.15.42:0/2444869026 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7f9f140037a0 con 0x7f9f20074300 2024-04-20T10:35:11.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.426+0000 7f9f1e000640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9f080a3ec0 0x7f9f080a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.426+0000 7f9f1e000640 1 -- 172.21.15.42:0/2444869026 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9f140d0e60 con 0x7f9f20074300 2024-04-20T10:35:11.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.426+0000 7f9f24c00640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9f080a3ec0 0x7f9f080a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:11.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.427+0000 7f9f24c00640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9f080a3ec0 0x7f9f080a6380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9f200fff90 tx=0x7f9f0c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:11.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.431+0000 7f9f1e000640 1 -- 172.21.15.42:0/2444869026 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9f1409d250 con 0x7f9f20074300 2024-04-20T10:35:11.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:11 smithi042 ceph-mon[17078]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T10:35:11.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:11 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:35:11.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:11 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:35:11.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:11 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:35:11.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.801+0000 7f9f26a00640 1 -- 172.21.15.42:0/2444869026 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f2019ffd0 con 0x7f9f20074300 2024-04-20T10:35:11.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.802+0000 7f9f1e000640 1 -- 172.21.15.42:0/2444869026 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f9f1409cb50 con 0x7f9f20074300 2024-04-20T10:35:11.801 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:35:11.802 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:35:11.802 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:35:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:11 smithi046 ceph-mon[21574]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T10:35:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:11 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:35:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:11 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:35:11.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:11 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:35:11.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.804+0000 7f9f26a00640 1 -- 172.21.15.42:0/2444869026 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9f080a3ec0 msgr2=0x7f9f080a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.804+0000 7f9f26a00640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9f080a3ec0 0x7f9f080a6380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9f200fff90 tx=0x7f9f0c005f90 comp rx=0 tx=0).stop 2024-04-20T10:35:11.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.804+0000 7f9f26a00640 1 -- 172.21.15.42:0/2444869026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f20074300 msgr2=0x7f9f201a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.804+0000 7f9f26a00640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f20074300 0x7f9f201a48f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400e9c0 tx=0x7f9f1400ee80 comp rx=0 tx=0).stop 2024-04-20T10:35:11.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.804+0000 7f9f26a00640 1 -- 172.21.15.42:0/2444869026 shutdown_connections 2024-04-20T10:35:11.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.804+0000 7f9f26a00640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9f080a3ec0 0x7f9f080a6380 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.804+0000 7f9f26a00640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f20112d90 0x7f9f2019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.804+0000 7f9f26a00640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20074c20 0x7f9f201a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.804+0000 7f9f26a00640 1 --2- 172.21.15.42:0/2444869026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f20074300 0x7f9f201a48f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.804+0000 7f9f26a00640 1 -- 172.21.15.42:0/2444869026 >> 172.21.15.42:0/2444869026 conn(0x7f9f200fd890 msgr2=0x7f9f200ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:11.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.804+0000 7f9f26a00640 1 -- 172.21.15.42:0/2444869026 shutdown_connections 2024-04-20T10:35:11.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.805+0000 7f9f26a00640 1 -- 172.21.15.42:0/2444869026 wait complete. 2024-04-20T10:35:11.825 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:35:11.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:11 smithi162 ceph-mon[21854]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-20T10:35:11.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:11 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:35:11.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:11 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:35:11.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:11 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:35:11.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.997+0000 7f8f4e000640 1 -- 172.21.15.42:0/1411240931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48102b50 msgr2=0x7f8f48106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:11.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.997+0000 7f8f4e000640 1 --2- 172.21.15.42:0/1411240931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48102b50 0x7f8f48106b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009900 tx=0x7f8f3402bc40 comp rx=0 tx=0).stop 2024-04-20T10:35:11.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.997+0000 7f8f4e000640 1 -- 172.21.15.42:0/1411240931 shutdown_connections 2024-04-20T10:35:11.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.997+0000 7f8f4e000640 1 --2- 172.21.15.42:0/1411240931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f48107270 0x7f8f4810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.997+0000 7f8f4e000640 1 --2- 172.21.15.42:0/1411240931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48102b50 0x7f8f48106b90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.997+0000 7f8f4e000640 1 --2- 172.21.15.42:0/1411240931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f481021a0 0x7f8f48102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:11.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.997+0000 7f8f4e000640 1 -- 172.21.15.42:0/1411240931 >> 172.21.15.42:0/1411240931 conn(0x7f8f480793b0 msgr2=0x7f8f480797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:11.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.997+0000 7f8f4e000640 1 -- 172.21.15.42:0/1411240931 shutdown_connections 2024-04-20T10:35:11.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.997+0000 7f8f4e000640 1 -- 172.21.15.42:0/1411240931 wait complete. 2024-04-20T10:35:11.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.999+0000 7f8f4e000640 1 Processor -- start 2024-04-20T10:35:11.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.999+0000 7f8f4e000640 1 -- start start 2024-04-20T10:35:11.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.999+0000 7f8f4e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f481021a0 0x7f8f481a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.999+0000 7f8f4e000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f48107270 0x7f8f481a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:11.999+0000 7f8f4e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f4819a8a0 0x7f8f4819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:11.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f4e000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48112330 con 0x7f8f4819a8a0 2024-04-20T10:35:12.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f4e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481121b0 con 0x7f8f481021a0 2024-04-20T10:35:12.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f4e000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481124b0 con 0x7f8f48107270 2024-04-20T10:35:12.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f4819a8a0 0x7f8f4819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f46a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f48107270 0x7f8f481a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f46a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f48107270 0x7f8f481a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:35270/0 (socket says 172.21.15.42:35270) 2024-04-20T10:35:12.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f46a00640 1 -- 172.21.15.42:0/1781212545 learned_addr learned my addr 172.21.15.42:0/1781212545 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:12.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f47400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f481021a0 0x7f8f481a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f4819a8a0 0x7f8f4819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37034/0 (socket says 172.21.15.42:37034) 2024-04-20T10:35:12.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f46a00640 1 -- 172.21.15.42:0/1781212545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f481021a0 msgr2=0x7f8f481a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:12.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f46a00640 1 --2- 172.21.15.42:0/1781212545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f481021a0 0x7f8f481a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f46a00640 1 -- 172.21.15.42:0/1781212545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f4819a8a0 msgr2=0x7f8f4819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:12.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f46a00640 1 --2- 172.21.15.42:0/1781212545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f4819a8a0 0x7f8f4819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.000+0000 7f8f46a00640 1 -- 172.21.15.42:0/1781212545 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4819b3c0 con 0x7f8f48107270 2024-04-20T10:35:12.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.001+0000 7f8f46a00640 1 --2- 172.21.15.42:0/1781212545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f48107270 0x7f8f481a0c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f3402c250 tx=0x7f8f340042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:12.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.001+0000 7f8f3fe00640 1 -- 172.21.15.42:0/1781212545 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34041070 con 0x7f8f48107270 2024-04-20T10:35:12.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.001+0000 7f8f4e000640 1 -- 172.21.15.42:0/1781212545 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4819b650 con 0x7f8f48107270 2024-04-20T10:35:12.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.001+0000 7f8f3fe00640 1 -- 172.21.15.42:0/1781212545 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3402cca0 con 0x7f8f48107270 2024-04-20T10:35:12.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.001+0000 7f8f3fe00640 1 -- 172.21.15.42:0/1781212545 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34003cf0 con 0x7f8f48107270 2024-04-20T10:35:12.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.001+0000 7f8f4e000640 1 -- 172.21.15.42:0/1781212545 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f481a78a0 con 0x7f8f48107270 2024-04-20T10:35:12.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.005+0000 7f8f50c01740 1 Processor -- start 2024-04-20T10:35:12.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.006+0000 7f8f50c01740 1 -- start start 2024-04-20T10:35:12.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.006+0000 7f8f50c01740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50b9ffb0 0x559c50ba0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.006+0000 7f8f50c01740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50ba08d0 0x559c50ba8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.006+0000 7f8f50c01740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba94b0 0x559c50bab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.006+0000 7f8f50c01740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559c50b05050 con 0x559c50b9ffb0 2024-04-20T10:35:12.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.006+0000 7f8f50c01740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559c50aab490 con 0x559c50ba94b0 2024-04-20T10:35:12.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.006+0000 7f8f50c01740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559c50b01010 con 0x559c50ba08d0 2024-04-20T10:35:12.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.006+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50b9ffb0 0x559c50ba0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.007+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50b9ffb0 0x559c50ba0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37038/0 (socket says 172.21.15.42:37038) 2024-04-20T10:35:12.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.007+0000 7f8f47e00640 1 -- 172.21.15.42:0/3369953419 learned_addr learned my addr 172.21.15.42:0/3369953419 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:12.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.007+0000 7f8f47400640 1 --2- 172.21.15.42:0/3369953419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50ba08d0 0x559c50ba8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.007+0000 7f8f47400640 1 -- 172.21.15.42:0/3369953419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba94b0 msgr2=0x559c50bab8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:35:12.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.007+0000 7f8f47400640 1 --2- 172.21.15.42:0/3369953419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba94b0 0x559c50bab8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.007+0000 7f8f47400640 1 -- 172.21.15.42:0/3369953419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50b9ffb0 msgr2=0x559c50ba0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:12.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.007+0000 7f8f47400640 1 --2- 172.21.15.42:0/3369953419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50b9ffb0 0x559c50ba0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.007+0000 7f8f47400640 1 -- 172.21.15.42:0/3369953419 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c50ab3070 con 0x559c50ba08d0 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.007+0000 7f8f3fe00640 1 -- 172.21.15.42:0/1781212545 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7f8f34044070 con 0x7f8f48107270 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.007+0000 7f8f47400640 1 --2- 172.21.15.42:0/3369953419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50ba08d0 0x559c50ba8f70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009bf0 tx=0x7f8f38011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8523228d76d8b983 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.008+0000 7f8f33e00640 1 -- 172.21.15.42:0/3369953419 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38004050 con 0x559c50ba08d0 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.008+0000 7f8f33e00640 1 -- 172.21.15.42:0/3369953419 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f38005500 con 0x559c50ba08d0 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.008+0000 7f8f33e00640 1 -- 172.21.15.42:0/3369953419 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38004db0 con 0x559c50ba08d0 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.009+0000 7f8f50c01740 1 -- 172.21.15.42:0/3369953419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50ba08d0 msgr2=0x559c50ba8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.009+0000 7f8f50c01740 1 --2- 172.21.15.42:0/3369953419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50ba08d0 0x559c50ba8f70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009bf0 tx=0x7f8f38011810 comp rx=0 tx=0).stop 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.009+0000 7f8f50c01740 1 -- 172.21.15.42:0/3369953419 shutdown_connections 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.009+0000 7f8f50c01740 1 --2- 172.21.15.42:0/3369953419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba94b0 0x559c50bab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.009+0000 7f8f50c01740 1 --2- 172.21.15.42:0/3369953419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50ba08d0 0x559c50ba8f70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.009+0000 7f8f50c01740 1 --2- 172.21.15.42:0/3369953419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50b9ffb0 0x559c50ba0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.009+0000 7f8f50c01740 1 -- 172.21.15.42:0/3369953419 >> 172.21.15.42:0/3369953419 conn(0x559c50b9a6e0 msgr2=0x559c50b9aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:12.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.009+0000 7f8f50c01740 1 -- 172.21.15.42:0/3369953419 shutdown_connections 2024-04-20T10:35:12.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.009+0000 7f8f50c01740 1 -- 172.21.15.42:0/3369953419 wait complete. 2024-04-20T10:35:12.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.009+0000 7f8f3fe00640 1 --2- 172.21.15.42:0/1781212545 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8f2c0a3d50 0x7f8f2c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.009+0000 7f8f3fe00640 1 -- 172.21.15.42:0/1781212545 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8f340f2630 con 0x7f8f48107270 2024-04-20T10:35:12.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.009+0000 7f8f50c01740 1 -- start start 2024-04-20T10:35:12.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.010+0000 7f8f47400640 1 --2- 172.21.15.42:0/1781212545 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8f2c0a3d50 0x7f8f2c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.010+0000 7f8f50c01740 1 Processor -- start 2024-04-20T10:35:12.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.010+0000 7f8f50c01740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50ba6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.010+0000 7f8f50c01740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba08d0 0x559c50ba6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.010+0000 7f8f50c01740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba94b0 0x559c50ba6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.010+0000 7f8f50c01740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559c50bac2a0 con 0x559c50ba94b0 2024-04-20T10:35:12.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.010+0000 7f8f50c01740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559c50abe860 con 0x559c50ba08d0 2024-04-20T10:35:12.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.010+0000 7f8f50c01740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559c50bac420 con 0x559c50b9ffb0 2024-04-20T10:35:12.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.010+0000 7f8f50c01740 10 client.0 _ll_register_callbacks cb 0x559c50a8cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:12.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.011+0000 7f8f46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba08d0 0x559c50ba6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.011+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba94b0 0x559c50ba6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.011+0000 7f8f46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba08d0 0x559c50ba6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52426/0 (socket says 172.21.15.42:52426) 2024-04-20T10:35:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.011+0000 7f8f46a00640 1 -- 172.21.15.42:0/1838382710 learned_addr learned my addr 172.21.15.42:0/1838382710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.011+0000 7f8f47400640 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50ba6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.011+0000 7f8f47400640 1 --2- 172.21.15.42:0/1781212545 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8f2c0a3d50 0x7f8f2c0a6210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f38011d10 tx=0x7f8f380048b0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.011+0000 7f8f46a00640 1 -- 172.21.15.42:0/1838382710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 msgr2=0x559c50ba6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.011+0000 7f8f46a00640 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50ba6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.011+0000 7f8f46a00640 1 -- 172.21.15.42:0/1838382710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba94b0 msgr2=0x559c50ba6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.011+0000 7f8f46a00640 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba94b0 0x559c50ba6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.011+0000 7f8f46a00640 1 -- 172.21.15.42:0/1838382710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c50ac2950 con 0x559c50ba08d0 2024-04-20T10:35:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.011+0000 7f8f47e00640 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba94b0 0x559c50ba6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.012+0000 7f8f46a00640 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba08d0 0x559c50ba6a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8f340be470 tx=0x7f8f340045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c723b77d573b5de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:12.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.012+0000 7f8f30c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.012+0000 7f8f30c00640 1 -- 172.21.15.42:0/1838382710 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3404b070 con 0x559c50ba08d0 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.012+0000 7f8f30c00640 1 -- 172.21.15.42:0/1838382710 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8f340be940 con 0x559c50ba08d0 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.012+0000 7f8f50c01740 1 -- 172.21.15.42:0/1838382710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f38005500 con 0x559c50ba08d0 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.012+0000 7f8f30c00640 1 -- 172.21.15.42:0/1838382710 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8f340be020 con 0x559c50ba08d0 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.012+0000 7f8f50c01740 10 client.25472 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.012+0000 7f8f50c01740 1 -- 172.21.15.42:0/1838382710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559c50a90820 con 0x559c50ba08d0 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.013+0000 7f8f30c00640 1 -- 172.21.15.42:0/1838382710 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f38005500 con 0x559c50ba08d0 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.013+0000 7f8f30c00640 1 -- 172.21.15.42:0/1838382710 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f8f340c6c10 con 0x559c50ba08d0 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.013+0000 7f8f50c01740 10 client.25472 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.013+0000 7f8f50c01740 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.013+0000 7f8f50c01740 20 client.25472 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.013+0000 7f8f50c01740 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559c50bb9460 0x559c50bbb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.013+0000 7f8f50c01740 4 client.25472 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:35:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.013+0000 7f8f50c01740 1 -- 172.21.15.42:0/1838382710 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c50b12a70 con 0x559c50bb9460 2024-04-20T10:35:12.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.013+0000 7f8f47400640 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559c50bb9460 0x559c50bbb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.014+0000 7f8f47400640 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559c50bb9460 0x559c50bbb880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b76f4c07fa16dba9 server_cookie=f5b51cf3b42f08b4 in_seq=0 out_seq=0 2024-04-20T10:35:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.014+0000 7f8f30c00640 10 client.25472 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f30c00640 1 -- 172.21.15.42:0/1838382710 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c50b12a70 con 0x559c50bb9460 2024-04-20T10:35:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f30c00640 10 client.25472 handle_command_reply: tid=0 2024-04-20T10:35:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f50c01740 1 client.25472 shutdown 2024-04-20T10:35:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f50c01740 1 -- 172.21.15.42:0/1838382710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba08d0 msgr2=0x559c50ba6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f50c01740 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba08d0 0x559c50ba6a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8f340be470 tx=0x7f8f340045a0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f50c01740 1 -- 172.21.15.42:0/1838382710 shutdown_connections 2024-04-20T10:35:12.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f50c01740 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559c50bb9460 0x559c50bbb880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f50c01740 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba94b0 0x559c50ba6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f50c01740 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba08d0 0x559c50ba6a50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f50c01740 1 --2- 172.21.15.42:0/1838382710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50ba6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f50c01740 1 -- 172.21.15.42:0/1838382710 >> 172.21.15.42:0/1838382710 conn(0x559c50b9a6e0 msgr2=0x559c50b9c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f30c00640 0 client.25472 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.015+0000 7f8f50c01740 1 -- 172.21.15.42:0/1838382710 shutdown_connections 2024-04-20T10:35:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.016+0000 7f8f50c01740 1 -- 172.21.15.42:0/1838382710 wait complete. 2024-04-20T10:35:12.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.016+0000 7f8f50c01740 20 client.25472 trim_cache size 0 max 16384 2024-04-20T10:35:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.043+0000 7f8f50c01740 1 Processor -- start 2024-04-20T10:35:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.043+0000 7f8f50c01740 1 -- start start 2024-04-20T10:35:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.044+0000 7f8f50c01740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50ba8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.044+0000 7f8f50c01740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba08d0 0x559c50bb5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.044+0000 7f8f50c01740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba94b0 0x559c50bc1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.044+0000 7f8f50c01740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481adf20 con 0x559c50ba94b0 2024-04-20T10:35:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.044+0000 7f8f50c01740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481b07a0 con 0x559c50ba08d0 2024-04-20T10:35:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.044+0000 7f8f50c01740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481ae590 con 0x559c50b9ffb0 2024-04-20T10:35:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.044+0000 7f8f47400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50ba8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.044+0000 7f8f47400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50ba8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:35304/0 (socket says 172.21.15.42:35304) 2024-04-20T10:35:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.044+0000 7f8f47400640 1 -- 172.21.15.42:0/2328858292 learned_addr learned my addr 172.21.15.42:0/2328858292 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.044+0000 7f8f47e00640 1 --2- 172.21.15.42:0/2328858292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba94b0 0x559c50bc1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.044+0000 7f8f46a00640 1 --2- 172.21.15.42:0/2328858292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba08d0 0x559c50bb5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.045+0000 7f8f47400640 1 -- 172.21.15.42:0/2328858292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba08d0 msgr2=0x559c50bb5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.045+0000 7f8f47400640 1 --2- 172.21.15.42:0/2328858292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba08d0 0x559c50bb5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.045+0000 7f8f47400640 1 -- 172.21.15.42:0/2328858292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba94b0 msgr2=0x559c50bc1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.045+0000 7f8f47400640 1 --2- 172.21.15.42:0/2328858292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba94b0 0x559c50bc1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.045+0000 7f8f47400640 1 -- 172.21.15.42:0/2328858292 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c508173f0 con 0x559c50b9ffb0 2024-04-20T10:35:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.045+0000 7f8f47400640 1 --2- 172.21.15.42:0/2328858292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50ba8c40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f38004920 tx=0x7f8f38023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5627ccbfc548ce79 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.045+0000 7f8f33400640 1 -- 172.21.15.42:0/2328858292 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3803f070 con 0x559c50b9ffb0 2024-04-20T10:35:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.045+0000 7f8f33400640 1 -- 172.21.15.42:0/2328858292 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3802a040 con 0x559c50b9ffb0 2024-04-20T10:35:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.045+0000 7f8f33400640 1 -- 172.21.15.42:0/2328858292 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f380326b0 con 0x559c50b9ffb0 2024-04-20T10:35:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.046+0000 7f8f50c01740 1 -- 172.21.15.42:0/2328858292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 msgr2=0x559c50ba8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.046+0000 7f8f50c01740 1 --2- 172.21.15.42:0/2328858292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50ba8c40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f38004920 tx=0x7f8f38023000 comp rx=0 tx=0).stop 2024-04-20T10:35:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.046+0000 7f8f50c01740 1 -- 172.21.15.42:0/2328858292 shutdown_connections 2024-04-20T10:35:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.046+0000 7f8f50c01740 1 --2- 172.21.15.42:0/2328858292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba94b0 0x559c50bc1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.046+0000 7f8f50c01740 1 --2- 172.21.15.42:0/2328858292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba08d0 0x559c50bb5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.046+0000 7f8f50c01740 1 --2- 172.21.15.42:0/2328858292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50ba8c40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.046+0000 7f8f50c01740 1 -- 172.21.15.42:0/2328858292 >> 172.21.15.42:0/2328858292 conn(0x559c50b9a6e0 msgr2=0x559c50ba18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.047+0000 7f8f50c01740 1 -- 172.21.15.42:0/2328858292 shutdown_connections 2024-04-20T10:35:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.047+0000 7f8f50c01740 1 -- 172.21.15.42:0/2328858292 wait complete. 2024-04-20T10:35:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.047+0000 7f8f50c01740 1 -- start start 2024-04-20T10:35:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.047+0000 7f8f50c01740 1 Processor -- start 2024-04-20T10:35:12.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.048+0000 7f8f50c01740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50bb1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.048+0000 7f8f50c01740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba08d0 0x559c50bb1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.048+0000 7f8f50c01740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba94b0 0x559c50ba29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.048+0000 7f8f50c01740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481aec60 con 0x559c50ba08d0 2024-04-20T10:35:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.048+0000 7f8f50c01740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481b00d0 con 0x559c50ba94b0 2024-04-20T10:35:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.048+0000 7f8f50c01740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481af330 con 0x559c50b9ffb0 2024-04-20T10:35:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.048+0000 7f8f50c01740 10 client.0 _ll_register_callbacks cb 0x7f8f34005600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.048+0000 7f8f47400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50bb1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.048+0000 7f8f47400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50bb1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:35314/0 (socket says 172.21.15.42:35314) 2024-04-20T10:35:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.048+0000 7f8f47400640 1 -- 172.21.15.42:0/202983568 learned_addr learned my addr 172.21.15.42:0/202983568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.049+0000 7f8f47e00640 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba94b0 0x559c50ba29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.049+0000 7f8f46a00640 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba08d0 0x559c50bb1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.049+0000 7f8f47400640 1 -- 172.21.15.42:0/202983568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba94b0 msgr2=0x559c50ba29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.049+0000 7f8f47400640 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba94b0 0x559c50ba29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.049+0000 7f8f47400640 1 -- 172.21.15.42:0/202983568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba08d0 msgr2=0x559c50bb1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.049+0000 7f8f47400640 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba08d0 0x559c50bb1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.049+0000 7f8f47400640 1 -- 172.21.15.42:0/202983568 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c508e17d0 con 0x559c50b9ffb0 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.049+0000 7f8f47e00640 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba94b0 0x559c50ba29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.049+0000 7f8f47400640 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50bb1670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8f38004880 tx=0x7f8f38004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b6e1f97481bb82 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.049+0000 7f8f30c00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.050+0000 7f8f30c00640 1 -- 172.21.15.42:0/202983568 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38048060 con 0x559c50b9ffb0 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.050+0000 7f8f50c01740 1 -- 172.21.15.42:0/202983568 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f3802a040 con 0x559c50b9ffb0 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.050+0000 7f8f30c00640 1 -- 172.21.15.42:0/202983568 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f8f380040d0 con 0x559c50b9ffb0 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.050+0000 7f8f30c00640 1 -- 172.21.15.42:0/202983568 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8f3803f040 con 0x559c50b9ffb0 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.050+0000 7f8f50c01740 10 client.25501 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.050+0000 7f8f50c01740 1 -- 172.21.15.42:0/202983568 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559c50c1f2a0 con 0x559c50b9ffb0 2024-04-20T10:35:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.050+0000 7f8f30c00640 1 -- 172.21.15.42:0/202983568 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3802a040 con 0x559c50b9ffb0 2024-04-20T10:35:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.050+0000 7f8f30c00640 1 -- 172.21.15.42:0/202983568 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f8f38039030 con 0x559c50b9ffb0 2024-04-20T10:35:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.051+0000 7f8f50c01740 10 client.25501 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.051+0000 7f8f50c01740 10 client.25501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.051+0000 7f8f50c01740 20 client.25501 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.051+0000 7f8f50c01740 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559c50c1f720 0x559c50c21b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.051+0000 7f8f50c01740 4 client.25501 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:35:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.051+0000 7f8f50c01740 1 -- 172.21.15.42:0/202983568 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c50b0c8e0 con 0x559c50c1f720 2024-04-20T10:35:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.051+0000 7f8f46a00640 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559c50c1f720 0x559c50c21b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:12.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f46a00640 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559c50c1f720 0x559c50c21b10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f870c8098a58a7 server_cookie=36001bf3aa56ab39 in_seq=0 out_seq=0 2024-04-20T10:35:12.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f30c00640 10 client.25501 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f30c00640 1 -- 172.21.15.42:0/202983568 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c50b0c8e0 con 0x559c50c1f720 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f30c00640 10 client.25501 handle_command_reply: tid=0 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f50c01740 1 client.25501 shutdown 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f50c01740 1 -- 172.21.15.42:0/202983568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 msgr2=0x559c50bb1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f50c01740 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50bb1670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8f38004880 tx=0x7f8f38004950 comp rx=0 tx=0).stop 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f50c01740 1 -- 172.21.15.42:0/202983568 shutdown_connections 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f50c01740 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559c50c1f720 0x559c50c21b10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f50c01740 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c50ba94b0 0x559c50ba29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f50c01740 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559c50ba08d0 0x559c50bb1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f50c01740 1 --2- 172.21.15.42:0/202983568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559c50b9ffb0 0x559c50bb1670 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f50c01740 1 -- 172.21.15.42:0/202983568 >> 172.21.15.42:0/202983568 conn(0x559c50b9a6e0 msgr2=0x559c50ba64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f30c00640 0 client.25501 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f50c01740 1 -- 172.21.15.42:0/202983568 shutdown_connections 2024-04-20T10:35:12.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.052+0000 7f8f50c01740 1 -- 172.21.15.42:0/202983568 wait complete. 2024-04-20T10:35:12.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.053+0000 7f8f50c01740 20 client.25501 trim_cache size 0 max 16384 2024-04-20T10:35:12.056 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:35:12.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.053+0000 7f8f4e000640 1 -- 172.21.15.42:0/1781212545 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8f2c0a3d50 msgr2=0x7f8f2c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:12.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.053+0000 7f8f4e000640 1 --2- 172.21.15.42:0/1781212545 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8f2c0a3d50 0x7f8f2c0a6210 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8f38011d10 tx=0x7f8f380048b0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.053+0000 7f8f4e000640 1 -- 172.21.15.42:0/1781212545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f48107270 msgr2=0x7f8f481a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:12.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.053+0000 7f8f4e000640 1 --2- 172.21.15.42:0/1781212545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f48107270 0x7f8f481a0c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f3402c250 tx=0x7f8f340042f0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.054+0000 7f8f4e000640 1 -- 172.21.15.42:0/1781212545 shutdown_connections 2024-04-20T10:35:12.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.054+0000 7f8f4e000640 1 --2- 172.21.15.42:0/1781212545 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8f2c0a3d50 0x7f8f2c0a6210 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.054+0000 7f8f4e000640 1 --2- 172.21.15.42:0/1781212545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f4819a8a0 0x7f8f4819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.054+0000 7f8f4e000640 1 --2- 172.21.15.42:0/1781212545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f48107270 0x7f8f481a0c30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.054+0000 7f8f4e000640 1 --2- 172.21.15.42:0/1781212545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f481021a0 0x7f8f481a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:12.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.054+0000 7f8f4e000640 1 -- 172.21.15.42:0/1781212545 >> 172.21.15.42:0/1781212545 conn(0x7f8f480793b0 msgr2=0x7f8f480ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:12.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.054+0000 7f8f4e000640 1 -- 172.21.15.42:0/1781212545 shutdown_connections 2024-04-20T10:35:12.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:12.054+0000 7f8f4e000640 1 -- 172.21.15.42:0/1781212545 wait complete. 2024-04-20T10:35:12.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:12 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2444869026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:35:12.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:12 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2444869026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:35:12.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:12 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2444869026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:35:13.075 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:35:13.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:35:13.075 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:35:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.261+0000 7fd514e00640 1 -- 172.21.15.42:0/3462877724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510102ba0 msgr2=0x7fd510106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.261+0000 7fd514e00640 1 --2- 172.21.15.42:0/3462877724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510102ba0 0x7fd510106be0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009900 tx=0x7fd4fc02bc30 comp rx=0 tx=0).stop 2024-04-20T10:35:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.261+0000 7fd514e00640 1 -- 172.21.15.42:0/3462877724 shutdown_connections 2024-04-20T10:35:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.261+0000 7fd514e00640 1 --2- 172.21.15.42:0/3462877724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5101072c0 0x7fd51010ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.261+0000 7fd514e00640 1 --2- 172.21.15.42:0/3462877724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510102ba0 0x7fd510106be0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.261+0000 7fd514e00640 1 --2- 172.21.15.42:0/3462877724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5101021f0 0x7fd5101025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.261+0000 7fd514e00640 1 -- 172.21.15.42:0/3462877724 >> 172.21.15.42:0/3462877724 conn(0x7fd5100fda70 msgr2=0x7fd5100ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.261+0000 7fd514e00640 1 -- 172.21.15.42:0/3462877724 shutdown_connections 2024-04-20T10:35:13.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.261+0000 7fd514e00640 1 -- 172.21.15.42:0/3462877724 wait complete. 2024-04-20T10:35:13.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.262+0000 7fd514e00640 1 Processor -- start 2024-04-20T10:35:13.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.263+0000 7fd514e00640 1 -- start start 2024-04-20T10:35:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.263+0000 7fd514e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5101021f0 0x7fd5101a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.263+0000 7fd514e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5101072c0 0x7fd5101a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.263+0000 7fd514e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51019a8e0 0x7fd51019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.263+0000 7fd514e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd5101123a0 con 0x7fd5101072c0 2024-04-20T10:35:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.263+0000 7fd514e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd510112220 con 0x7fd51019a8e0 2024-04-20T10:35:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.263+0000 7fd514e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd510112520 con 0x7fd5101021f0 2024-04-20T10:35:13.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.263+0000 7fd50de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5101021f0 0x7fd5101a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.263+0000 7fd50d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5101072c0 0x7fd5101a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.263+0000 7fd50de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5101021f0 0x7fd5101a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:35338/0 (socket says 172.21.15.42:35338) 2024-04-20T10:35:13.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.263+0000 7fd50de00640 1 -- 172.21.15.42:0/535759169 learned_addr learned my addr 172.21.15.42:0/535759169 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:13.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.264+0000 7fd50e800640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51019a8e0 0x7fd51019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.264+0000 7fd50de00640 1 -- 172.21.15.42:0/535759169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51019a8e0 msgr2=0x7fd51019ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.264+0000 7fd50de00640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51019a8e0 0x7fd51019ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.264+0000 7fd50de00640 1 -- 172.21.15.42:0/535759169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5101072c0 msgr2=0x7fd5101a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.264+0000 7fd50de00640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5101072c0 0x7fd5101a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.264+0000 7fd50de00640 1 -- 172.21.15.42:0/535759169 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51019b400 con 0x7fd5101021f0 2024-04-20T10:35:13.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.264+0000 7fd50de00640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5101021f0 0x7fd5101a0730 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd50000e9f0 tx=0x7fd50000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:13.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.264+0000 7fd506a00640 1 -- 172.21.15.42:0/535759169 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50000cd80 con 0x7fd5101021f0 2024-04-20T10:35:13.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.265+0000 7fd506a00640 1 -- 172.21.15.42:0/535759169 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd500010910 con 0x7fd5101021f0 2024-04-20T10:35:13.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.265+0000 7fd514e00640 1 -- 172.21.15.42:0/535759169 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51019b6f0 con 0x7fd5101021f0 2024-04-20T10:35:13.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.265+0000 7fd506a00640 1 -- 172.21.15.42:0/535759169 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500018990 con 0x7fd5101021f0 2024-04-20T10:35:13.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.265+0000 7fd514e00640 1 -- 172.21.15.42:0/535759169 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5101a7990 con 0x7fd5101021f0 2024-04-20T10:35:13.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.268+0000 7fd517bef740 1 Processor -- start 2024-04-20T10:35:13.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.268+0000 7fd517bef740 1 -- start start 2024-04-20T10:35:13.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.268+0000 7fd517bef740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25b2f10 0x557bb25b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.268+0000 7fd517bef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.268+0000 7fd517bef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25bc410 0x557bb25be800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.268+0000 7fd517bef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557bb2517d20 con 0x557bb25b3830 2024-04-20T10:35:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.268+0000 7fd517bef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557bb24be3e0 con 0x557bb25bc410 2024-04-20T10:35:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.268+0000 7fd517bef740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x557bb2513ce0 con 0x557bb25b2f10 2024-04-20T10:35:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.269+0000 7fd50e800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25b2f10 0x557bb25b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.269+0000 7fd50e800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25b2f10 0x557bb25b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:35342/0 (socket says 172.21.15.42:35342) 2024-04-20T10:35:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.269+0000 7fd50e800640 1 -- 172.21.15.42:0/693752563 learned_addr learned my addr 172.21.15.42:0/693752563 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.269+0000 7fd50d400640 1 --2- 172.21.15.42:0/693752563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25bc410 0x557bb25be800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.269+0000 7fd50e800640 1 -- 172.21.15.42:0/693752563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25bc410 msgr2=0x557bb25be800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.269+0000 7fd50e800640 1 --2- 172.21.15.42:0/693752563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25bc410 0x557bb25be800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.269+0000 7fd50e800640 1 -- 172.21.15.42:0/693752563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 msgr2=0x557bb25bbed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:35:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.269+0000 7fd50e800640 1 --2- 172.21.15.42:0/693752563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25bbed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.269+0000 7fd50e800640 1 -- 172.21.15.42:0/693752563 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb24dd980 con 0x557bb25b2f10 2024-04-20T10:35:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.269+0000 7fd506a00640 1 -- 172.21.15.42:0/535759169 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7fd500010ac0 con 0x7fd5101021f0 2024-04-20T10:35:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.269+0000 7fd50e800640 1 --2- 172.21.15.42:0/693752563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25b2f10 0x557bb25b32f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd508009f40 tx=0x7fd508010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c4006f94e14c3c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.270+0000 7fd4fb400640 1 -- 172.21.15.42:0/693752563 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508003b70 con 0x557bb25b2f10 2024-04-20T10:35:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.270+0000 7fd4fb400640 1 -- 172.21.15.42:0/693752563 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd508003d10 con 0x557bb25b2f10 2024-04-20T10:35:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.270+0000 7fd4fb400640 1 -- 172.21.15.42:0/693752563 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508011af0 con 0x557bb25b2f10 2024-04-20T10:35:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.271+0000 7fd517bef740 1 -- 172.21.15.42:0/693752563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25b2f10 msgr2=0x557bb25b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.271+0000 7fd517bef740 1 --2- 172.21.15.42:0/693752563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25b2f10 0x557bb25b32f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd508009f40 tx=0x7fd508010860 comp rx=0 tx=0).stop 2024-04-20T10:35:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.271+0000 7fd517bef740 1 -- 172.21.15.42:0/693752563 shutdown_connections 2024-04-20T10:35:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.271+0000 7fd517bef740 1 --2- 172.21.15.42:0/693752563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25bc410 0x557bb25be800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.271+0000 7fd517bef740 1 --2- 172.21.15.42:0/693752563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25bbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.271+0000 7fd517bef740 1 --2- 172.21.15.42:0/693752563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25b2f10 0x557bb25b32f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.271+0000 7fd517bef740 1 -- 172.21.15.42:0/693752563 >> 172.21.15.42:0/693752563 conn(0x557bb25ad640 msgr2=0x557bb25ada20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.271+0000 7fd506a00640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd4f40a3ec0 0x7fd4f40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.271+0000 7fd506a00640 1 -- 172.21.15.42:0/535759169 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd5000d0de0 con 0x7fd5101021f0 2024-04-20T10:35:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.271+0000 7fd517bef740 1 -- 172.21.15.42:0/693752563 shutdown_connections 2024-04-20T10:35:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.271+0000 7fd517bef740 1 -- 172.21.15.42:0/693752563 wait complete. 2024-04-20T10:35:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.272+0000 7fd50e800640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd4f40a3ec0 0x7fd4f40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.272+0000 7fd517bef740 1 -- start start 2024-04-20T10:35:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.272+0000 7fd517bef740 1 Processor -- start 2024-04-20T10:35:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd517bef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 0x557bb25b9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd517bef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd517bef740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25bc410 0x557bb25b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd517bef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557bb25bf200 con 0x557bb25b3830 2024-04-20T10:35:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd517bef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557bb24d17a0 con 0x557bb25b2f10 2024-04-20T10:35:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd517bef740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x557bb25bf380 con 0x557bb25bc410 2024-04-20T10:35:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd50e800640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd4f40a3ec0 0x7fd4f40a6380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd508005fd0 tx=0x7fd508009fd0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd517bef740 10 client.0 _ll_register_callbacks cb 0x557bb24d3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd50de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 0x557bb25b9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd50de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 0x557bb25b9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:52478/0 (socket says 172.21.15.42:52478) 2024-04-20T10:35:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd50de00640 1 -- 172.21.15.42:0/763673443 learned_addr learned my addr 172.21.15.42:0/763673443 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd50d400640 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.273+0000 7fd50e800640 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25bc410 0x557bb25b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.274+0000 7fd50de00640 1 -- 172.21.15.42:0/763673443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25bc410 msgr2=0x557bb25b9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.274+0000 7fd50de00640 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25bc410 0x557bb25b9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.274+0000 7fd50de00640 1 -- 172.21.15.42:0/763673443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 msgr2=0x557bb25b9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.274+0000 7fd50de00640 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.274+0000 7fd50de00640 1 -- 172.21.15.42:0/763673443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bb22f27e0 con 0x557bb25b2f10 2024-04-20T10:35:13.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.274+0000 7fd50de00640 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 0x557bb25b9510 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd5000ce4c0 tx=0x7fd50009d230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6502f135b449c9e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:13.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.274+0000 7fd4efe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:35:13.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.274+0000 7fd4efe00640 1 -- 172.21.15.42:0/763673443 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50009de60 con 0x557bb25b2f10 2024-04-20T10:35:13.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.275+0000 7fd517bef740 1 -- 172.21.15.42:0/763673443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd508003d10 con 0x557bb25b2f10 2024-04-20T10:35:13.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.275+0000 7fd4efe00640 1 -- 172.21.15.42:0/763673443 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd5000ce890 con 0x557bb25b2f10 2024-04-20T10:35:13.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.275+0000 7fd4efe00640 1 -- 172.21.15.42:0/763673443 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd5000a57a0 con 0x557bb25b2f10 2024-04-20T10:35:13.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.275+0000 7fd517bef740 10 client.25490 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.275+0000 7fd517bef740 1 -- 172.21.15.42:0/763673443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557bb2352f70 con 0x557bb25b2f10 2024-04-20T10:35:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.275+0000 7fd4efe00640 1 -- 172.21.15.42:0/763673443 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd508003d10 con 0x557bb25b2f10 2024-04-20T10:35:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.275+0000 7fd4efe00640 1 -- 172.21.15.42:0/763673443 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fd50009b9f0 con 0x557bb25b2f10 2024-04-20T10:35:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.276+0000 7fd517bef740 10 client.25490 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.276+0000 7fd517bef740 10 client.25490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.276+0000 7fd517bef740 20 client.25490 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.276+0000 7fd517bef740 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x557bb25d4fe0 0x557bb25d73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.276+0000 7fd517bef740 4 client.25490 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:35:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.276+0000 7fd517bef740 1 -- 172.21.15.42:0/763673443 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bb2525740 con 0x557bb25d4fe0 2024-04-20T10:35:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.276+0000 7fd50d400640 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x557bb25d4fe0 0x557bb25d73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.277+0000 7fd50d400640 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x557bb25d4fe0 0x557bb25d73d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c4a3517e6a4417 server_cookie=5dd0e0b6015f7d70 in_seq=0 out_seq=0 2024-04-20T10:35:13.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.277+0000 7fd4efe00640 10 client.25490 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:13.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.277+0000 7fd4efe00640 1 -- 172.21.15.42:0/763673443 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bb2525740 con 0x557bb25d4fe0 2024-04-20T10:35:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.277+0000 7fd4efe00640 10 client.25490 handle_command_reply: tid=0 2024-04-20T10:35:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.277+0000 7fd517bef740 1 client.25490 shutdown 2024-04-20T10:35:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.278+0000 7fd517bef740 1 -- 172.21.15.42:0/763673443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 msgr2=0x557bb25b9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.278+0000 7fd517bef740 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 0x557bb25b9510 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd5000ce4c0 tx=0x7fd50009d230 comp rx=0 tx=0).stop 2024-04-20T10:35:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.278+0000 7fd517bef740 1 -- 172.21.15.42:0/763673443 shutdown_connections 2024-04-20T10:35:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.278+0000 7fd517bef740 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x557bb25d4fe0 0x557bb25d73d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.278+0000 7fd517bef740 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25bc410 0x557bb25b9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.278+0000 7fd517bef740 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.278+0000 7fd517bef740 1 --2- 172.21.15.42:0/763673443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 0x557bb25b9510 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.278+0000 7fd4efe00640 0 client.25490 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:13.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.278+0000 7fd517bef740 1 -- 172.21.15.42:0/763673443 >> 172.21.15.42:0/763673443 conn(0x557bb25ad640 msgr2=0x557bb25af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:13.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.278+0000 7fd517bef740 1 -- 172.21.15.42:0/763673443 shutdown_connections 2024-04-20T10:35:13.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.278+0000 7fd517bef740 1 -- 172.21.15.42:0/763673443 wait complete. 2024-04-20T10:35:13.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.278+0000 7fd517bef740 20 client.25490 trim_cache size 0 max 16384 2024-04-20T10:35:13.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.303+0000 7fd517bef740 1 Processor -- start 2024-04-20T10:35:13.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.303+0000 7fd517bef740 1 -- start start 2024-04-20T10:35:13.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.303+0000 7fd517bef740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25b2f10 0x557bb25c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.303+0000 7fd517bef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.303+0000 7fd517bef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25bc410 0x557bb25dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.303+0000 7fd517bef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd5101a9c40 con 0x557bb25b3830 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.303+0000 7fd517bef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5101ac460 con 0x557bb25bc410 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.303+0000 7fd517bef740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd5101aa290 con 0x557bb25b2f10 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.304+0000 7fd50d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.304+0000 7fd50d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37122/0 (socket says 172.21.15.42:37122) 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.304+0000 7fd50d400640 1 -- 172.21.15.42:0/845576249 learned_addr learned my addr 172.21.15.42:0/845576249 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.304+0000 7fd50e800640 1 --2- 172.21.15.42:0/845576249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25bc410 0x557bb25dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.304+0000 7fd50de00640 1 --2- 172.21.15.42:0/845576249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25b2f10 0x557bb25c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.304+0000 7fd50d400640 1 -- 172.21.15.42:0/845576249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25b2f10 msgr2=0x557bb25c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.304+0000 7fd50d400640 1 --2- 172.21.15.42:0/845576249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25b2f10 0x557bb25c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.304+0000 7fd50d400640 1 -- 172.21.15.42:0/845576249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25bc410 msgr2=0x557bb25dcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.304+0000 7fd50d400640 1 --2- 172.21.15.42:0/845576249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25bc410 0x557bb25dcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.304+0000 7fd50d400640 1 -- 172.21.15.42:0/845576249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb21ea170 con 0x557bb25b3830 2024-04-20T10:35:13.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.304+0000 7fd50e800640 1 --2- 172.21.15.42:0/845576249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25bc410 0x557bb25dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:35:13.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.305+0000 7fd50d400640 1 --2- 172.21.15.42:0/845576249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b9dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd51006b640 tx=0x7fd4fc02df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a1e81e4496c850 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:13.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.305+0000 7fd4faa00640 1 -- 172.21.15.42:0/845576249 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc004040 con 0x557bb25b3830 2024-04-20T10:35:13.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.305+0000 7fd4faa00640 1 -- 172.21.15.42:0/845576249 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4fc050030 con 0x557bb25b3830 2024-04-20T10:35:13.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.305+0000 7fd4faa00640 1 -- 172.21.15.42:0/845576249 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc04c790 con 0x557bb25b3830 2024-04-20T10:35:13.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.306+0000 7fd517bef740 1 -- 172.21.15.42:0/845576249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 msgr2=0x557bb25b9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.306+0000 7fd517bef740 1 --2- 172.21.15.42:0/845576249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b9dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd51006b640 tx=0x7fd4fc02df90 comp rx=0 tx=0).stop 2024-04-20T10:35:13.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.306+0000 7fd517bef740 1 -- 172.21.15.42:0/845576249 shutdown_connections 2024-04-20T10:35:13.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.306+0000 7fd517bef740 1 --2- 172.21.15.42:0/845576249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25bc410 0x557bb25dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.306+0000 7fd517bef740 1 --2- 172.21.15.42:0/845576249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b9dc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.306+0000 7fd517bef740 1 --2- 172.21.15.42:0/845576249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25b2f10 0x557bb25c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.306+0000 7fd517bef740 1 -- 172.21.15.42:0/845576249 >> 172.21.15.42:0/845576249 conn(0x557bb25ad640 msgr2=0x557bb25af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:13.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.306+0000 7fd517bef740 1 -- 172.21.15.42:0/845576249 shutdown_connections 2024-04-20T10:35:13.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.306+0000 7fd517bef740 1 -- 172.21.15.42:0/845576249 wait complete. 2024-04-20T10:35:13.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.307+0000 7fd517bef740 1 -- start start 2024-04-20T10:35:13.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.307+0000 7fd517bef740 1 Processor -- start 2024-04-20T10:35:13.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.307+0000 7fd517bef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 0x557bb25b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.307+0000 7fd517bef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.307+0000 7fd517bef740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25bc410 0x557bb263a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.307+0000 7fd517bef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd5101aa920 con 0x557bb25b3830 2024-04-20T10:35:13.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.307+0000 7fd517bef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5101abd90 con 0x557bb25b2f10 2024-04-20T10:35:13.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.307+0000 7fd517bef740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd5101aaff0 con 0x557bb25bc410 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.308+0000 7fd517bef740 10 client.0 _ll_register_callbacks cb 0x7fd50000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.308+0000 7fd50e800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25bc410 0x557bb263a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.308+0000 7fd50d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.308+0000 7fd50e800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25bc410 0x557bb263a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:35384/0 (socket says 172.21.15.42:35384) 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.308+0000 7fd50e800640 1 -- 172.21.15.42:0/1875616828 learned_addr learned my addr 172.21.15.42:0/1875616828 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.308+0000 7fd50de00640 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 0x557bb25b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.308+0000 7fd50e800640 1 -- 172.21.15.42:0/1875616828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 msgr2=0x557bb25b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.308+0000 7fd50e800640 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 0x557bb25b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.308+0000 7fd50e800640 1 -- 172.21.15.42:0/1875616828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 msgr2=0x557bb25b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.308+0000 7fd50e800640 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.308+0000 7fd50e800640 1 -- 172.21.15.42:0/1875616828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bb263aa30 con 0x557bb25bc410 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.308+0000 7fd50de00640 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 0x557bb25b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.309+0000 7fd50e800640 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25bc410 0x557bb263a4a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd50801a000 tx=0x7fd5080116d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc16cb0fc88354df server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:13.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.309+0000 7fd4efe00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:35:13.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.309+0000 7fd4efe00640 1 -- 172.21.15.42:0/1875616828 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508037060 con 0x557bb25bc410 2024-04-20T10:35:13.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.309+0000 7fd517bef740 1 -- 172.21.15.42:0/1875616828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4fc050030 con 0x557bb25bc410 2024-04-20T10:35:13.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.309+0000 7fd4efe00640 1 -- 172.21.15.42:0/1875616828 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd508003b30 con 0x557bb25bc410 2024-04-20T10:35:13.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.309+0000 7fd4efe00640 1 -- 172.21.15.42:0/1875616828 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd508003d30 con 0x557bb25bc410 2024-04-20T10:35:13.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.309+0000 7fd517bef740 10 client.25519 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:13.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.309+0000 7fd517bef740 1 -- 172.21.15.42:0/1875616828 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557bb263bd70 con 0x557bb25bc410 2024-04-20T10:35:13.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.310+0000 7fd4efe00640 1 -- 172.21.15.42:0/1875616828 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4fc050030 con 0x557bb25bc410 2024-04-20T10:35:13.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.310+0000 7fd4efe00640 1 -- 172.21.15.42:0/1875616828 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fd508026530 con 0x557bb25bc410 2024-04-20T10:35:13.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.310+0000 7fd517bef740 10 client.25519 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:13.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.310+0000 7fd517bef740 10 client.25519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:13.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.310+0000 7fd517bef740 20 client.25519 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:13.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.310+0000 7fd517bef740 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x557bb263c300 0x557bb263e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.310+0000 7fd517bef740 4 client.25519 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:35:13.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.310+0000 7fd517bef740 1 -- 172.21.15.42:0/1875616828 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557bb251f5b0 con 0x557bb263c300 2024-04-20T10:35:13.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.310+0000 7fd50de00640 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x557bb263c300 0x557bb263e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.311+0000 7fd50de00640 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x557bb263c300 0x557bb263e720 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a52856e36221fd37 server_cookie=77563d33a041d4b in_seq=0 out_seq=0 2024-04-20T10:35:13.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.311+0000 7fd4efe00640 10 client.25519 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:13.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.312+0000 7fd4efe00640 1 -- 172.21.15.42:0/1875616828 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557bb251f5b0 con 0x557bb263c300 2024-04-20T10:35:13.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.312+0000 7fd4efe00640 10 client.25519 handle_command_reply: tid=0 2024-04-20T10:35:13.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.312+0000 7fd517bef740 1 client.25519 shutdown 2024-04-20T10:35:13.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.313+0000 7fd517bef740 1 -- 172.21.15.42:0/1875616828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25bc410 msgr2=0x557bb263a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.313+0000 7fd517bef740 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25bc410 0x557bb263a4a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd50801a000 tx=0x7fd5080116d0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.313+0000 7fd517bef740 1 -- 172.21.15.42:0/1875616828 shutdown_connections 2024-04-20T10:35:13.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.313+0000 7fd517bef740 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x557bb263c300 0x557bb263e720 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.313+0000 7fd517bef740 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557bb25bc410 0x557bb263a4a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.313+0000 7fd517bef740 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557bb25b3830 0x557bb25b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.313+0000 7fd517bef740 1 --2- 172.21.15.42:0/1875616828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bb25b2f10 0x557bb25b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.313+0000 7fd517bef740 1 -- 172.21.15.42:0/1875616828 >> 172.21.15.42:0/1875616828 conn(0x557bb25ad640 msgr2=0x557bb25abdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:13.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.313+0000 7fd4efe00640 0 client.25519 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:13.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.313+0000 7fd517bef740 1 -- 172.21.15.42:0/1875616828 shutdown_connections 2024-04-20T10:35:13.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.313+0000 7fd517bef740 1 -- 172.21.15.42:0/1875616828 wait complete. 2024-04-20T10:35:13.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.313+0000 7fd517bef740 20 client.25519 trim_cache size 0 max 16384 2024-04-20T10:35:13.315 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:35:13.315 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:35:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "64e6ee31-3956-493b-9186-b4b388330c9d", 2024-04-20T10:35:13.316 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:35:13.316 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:35:13.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.314+0000 7fd514e00640 1 -- 172.21.15.42:0/535759169 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd4f40a3ec0 msgr2=0x7fd4f40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.314+0000 7fd514e00640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd4f40a3ec0 0x7fd4f40a6380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd508005fd0 tx=0x7fd508009fd0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.314+0000 7fd514e00640 1 -- 172.21.15.42:0/535759169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5101021f0 msgr2=0x7fd5101a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.314+0000 7fd514e00640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5101021f0 0x7fd5101a0730 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd50000e9f0 tx=0x7fd50000eeb0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.314+0000 7fd514e00640 1 -- 172.21.15.42:0/535759169 shutdown_connections 2024-04-20T10:35:13.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.314+0000 7fd514e00640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd4f40a3ec0 0x7fd4f40a6380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.314+0000 7fd514e00640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd51019a8e0 0x7fd51019ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.314+0000 7fd514e00640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5101072c0 0x7fd5101a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.314+0000 7fd514e00640 1 --2- 172.21.15.42:0/535759169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd5101021f0 0x7fd5101a0730 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.314+0000 7fd514e00640 1 -- 172.21.15.42:0/535759169 >> 172.21.15.42:0/535759169 conn(0x7fd5100fda70 msgr2=0x7fd5100ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:13.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.314+0000 7fd514e00640 1 -- 172.21.15.42:0/535759169 shutdown_connections 2024-04-20T10:35:13.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.315+0000 7fd514e00640 1 -- 172.21.15.42:0/535759169 wait complete. 2024-04-20T10:35:13.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:35:13.339 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:35:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.541+0000 7f9e15000640 1 -- 172.21.15.42:0/1609259571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e10104dc0 msgr2=0x7f9e10108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.541+0000 7f9e15000640 1 --2- 172.21.15.42:0/1609259571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e10104dc0 0x7f9e10108e00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009860 tx=0x7f9dfc02bb70 comp rx=0 tx=0).stop 2024-04-20T10:35:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.541+0000 7f9e15000640 1 -- 172.21.15.42:0/1609259571 shutdown_connections 2024-04-20T10:35:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.541+0000 7f9e15000640 1 --2- 172.21.15.42:0/1609259571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e101094e0 0x7f9e101110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.541+0000 7f9e15000640 1 --2- 172.21.15.42:0/1609259571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e10104dc0 0x7f9e10108e00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.541+0000 7f9e15000640 1 --2- 172.21.15.42:0/1609259571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e10104410 0x7f9e101047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.541+0000 7f9e15000640 1 -- 172.21.15.42:0/1609259571 >> 172.21.15.42:0/1609259571 conn(0x7f9e100ffcf0 msgr2=0x7f9e10102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.541+0000 7f9e15000640 1 -- 172.21.15.42:0/1609259571 shutdown_connections 2024-04-20T10:35:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.541+0000 7f9e15000640 1 -- 172.21.15.42:0/1609259571 wait complete. 2024-04-20T10:35:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.542+0000 7f9e15000640 1 Processor -- start 2024-04-20T10:35:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.542+0000 7f9e15000640 1 -- start start 2024-04-20T10:35:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.543+0000 7f9e15000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e10104410 0x7f9e101a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.543+0000 7f9e15000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e101094e0 0x7f9e101a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.543+0000 7f9e15000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1019a9d0 0x7f9e1019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.543+0000 7f9e15000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10114570 con 0x7f9e101094e0 2024-04-20T10:35:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.543+0000 7f9e15000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101143f0 con 0x7f9e1019a9d0 2024-04-20T10:35:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.543+0000 7f9e15000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101146f0 con 0x7f9e10104410 2024-04-20T10:35:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.543+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e101094e0 0x7f9e101a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.543+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e101094e0 0x7f9e101a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37146/0 (socket says 172.21.15.42:37146) 2024-04-20T10:35:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.543+0000 7f9e0d400640 1 -- 172.21.15.42:0/2806366254 learned_addr learned my addr 172.21.15.42:0/2806366254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.543+0000 7f9e0de00640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e10104410 0x7f9e101a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.543+0000 7f9e0e800640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1019a9d0 0x7f9e1019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.543+0000 7f9e0d400640 1 -- 172.21.15.42:0/2806366254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e10104410 msgr2=0x7f9e101a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.544+0000 7f9e0d400640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e10104410 0x7f9e101a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.544+0000 7f9e0d400640 1 -- 172.21.15.42:0/2806366254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1019a9d0 msgr2=0x7f9e1019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.544+0000 7f9e0d400640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1019a9d0 0x7f9e1019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.544+0000 7f9e0d400640 1 -- 172.21.15.42:0/2806366254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1019b4f0 con 0x7f9e101094e0 2024-04-20T10:35:13.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.544+0000 7f9e0d400640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e101094e0 0x7f9e101a0d10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc037040 tx=0x7f9dfc0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:13.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.544+0000 7f9e06a00640 1 -- 172.21.15.42:0/2806366254 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc041020 con 0x7f9e101094e0 2024-04-20T10:35:13.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.544+0000 7f9e06a00640 1 -- 172.21.15.42:0/2806366254 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dfc02c9f0 con 0x7f9e101094e0 2024-04-20T10:35:13.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.544+0000 7f9e15000640 1 -- 172.21.15.42:0/2806366254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1019b780 con 0x7f9e101094e0 2024-04-20T10:35:13.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.545+0000 7f9e15000640 1 -- 172.21.15.42:0/2806366254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e101a7920 con 0x7f9e101094e0 2024-04-20T10:35:13.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.545+0000 7f9e06a00640 1 -- 172.21.15.42:0/2806366254 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc039040 con 0x7f9e101094e0 2024-04-20T10:35:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.548+0000 7f9e06a00640 1 -- 172.21.15.42:0/2806366254 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 138879+0+0 (secure 0 0 0) 0x7f9dfc044020 con 0x7f9e101094e0 2024-04-20T10:35:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.549+0000 7f9e17d90740 1 Processor -- start 2024-04-20T10:35:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.549+0000 7f9e17d90740 1 -- start start 2024-04-20T10:35:13.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.549+0000 7f9e17d90740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e377751a0 0x558e37775580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.549+0000 7f9e17d90740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e3777e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.549+0000 7f9e17d90740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e3777e6a0 0x558e37780a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.549+0000 7f9e17d90740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558e376db150 con 0x558e377751a0 2024-04-20T10:35:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e17d90740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558e37681370 con 0x558e37775ac0 2024-04-20T10:35:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e17d90740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x558e376d7110 con 0x558e3777e6a0 2024-04-20T10:35:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e0e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e377751a0 0x558e37775580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e0e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e377751a0 0x558e37775580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37156/0 (socket says 172.21.15.42:37156) 2024-04-20T10:35:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e0e800640 1 -- 172.21.15.42:0/1813831754 learned_addr learned my addr 172.21.15.42:0/1813831754 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e0de00640 1 --2- 172.21.15.42:0/1813831754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e3777e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e0d400640 1 --2- 172.21.15.42:0/1813831754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e3777e6a0 0x558e37780a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e06a00640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9df40a3e40 0x7f9df40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e0de00640 1 -- 172.21.15.42:0/1813831754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e3777e6a0 msgr2=0x558e37780a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e0de00640 1 --2- 172.21.15.42:0/1813831754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e3777e6a0 0x558e37780a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e0de00640 1 -- 172.21.15.42:0/1813831754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e377751a0 msgr2=0x558e37775580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e0de00640 1 --2- 172.21.15.42:0/1813831754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e377751a0 0x558e37775580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e0de00640 1 -- 172.21.15.42:0/1813831754 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e374b77d0 con 0x558e37775ac0 2024-04-20T10:35:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e0e800640 1 --2- 172.21.15.42:0/1813831754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e377751a0 0x558e37775580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.550+0000 7f9e06a00640 1 -- 172.21.15.42:0/2806366254 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9dfc0f21e0 con 0x7f9e101094e0 2024-04-20T10:35:13.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.551+0000 7f9e0e800640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9df40a3e40 0x7f9df40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.551+0000 7f9e0de00640 1 --2- 172.21.15.42:0/1813831754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e3777e160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009530 tx=0x7f9e00011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c0c551aab1f324b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:13.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.551+0000 7f9dfb400640 1 -- 172.21.15.42:0/1813831754 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00004220 con 0x558e37775ac0 2024-04-20T10:35:13.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.551+0000 7f9dfb400640 1 -- 172.21.15.42:0/1813831754 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e000043c0 con 0x558e37775ac0 2024-04-20T10:35:13.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.551+0000 7f9dfb400640 1 -- 172.21.15.42:0/1813831754 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000197f0 con 0x558e37775ac0 2024-04-20T10:35:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.552+0000 7f9e17d90740 1 -- 172.21.15.42:0/1813831754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 msgr2=0x558e3777e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.552+0000 7f9e0e800640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9df40a3e40 0x7f9df40a6300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9e080107d0 tx=0x7f9e08004a40 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.552+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1813831754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e3777e160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009530 tx=0x7f9e00011810 comp rx=0 tx=0).stop 2024-04-20T10:35:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.552+0000 7f9e17d90740 1 -- 172.21.15.42:0/1813831754 shutdown_connections 2024-04-20T10:35:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.552+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1813831754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e3777e6a0 0x558e37780a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.552+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1813831754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e3777e160 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.552+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1813831754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e377751a0 0x558e37775580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.552+0000 7f9e17d90740 1 -- 172.21.15.42:0/1813831754 >> 172.21.15.42:0/1813831754 conn(0x558e3776f8d0 msgr2=0x558e3776fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:13.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.553+0000 7f9e17d90740 1 -- 172.21.15.42:0/1813831754 shutdown_connections 2024-04-20T10:35:13.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.553+0000 7f9e17d90740 1 -- 172.21.15.42:0/1813831754 wait complete. 2024-04-20T10:35:13.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.553+0000 7f9e17d90740 1 -- start start 2024-04-20T10:35:13.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.553+0000 7f9e17d90740 1 Processor -- start 2024-04-20T10:35:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.554+0000 7f9e17d90740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 0x558e3777b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.554+0000 7f9e17d90740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e3777bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.554+0000 7f9e17d90740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e3777e6a0 0x558e3777c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.554+0000 7f9e17d90740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558e37781490 con 0x558e3777e6a0 2024-04-20T10:35:13.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.554+0000 7f9e17d90740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558e37694740 con 0x558e37775ac0 2024-04-20T10:35:13.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.554+0000 7f9e17d90740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x558e37781610 con 0x558e377751a0 2024-04-20T10:35:13.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.554+0000 7f9e17d90740 10 client.0 _ll_register_callbacks cb 0x558e3768fb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:13.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.554+0000 7f9e0de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 0x558e3777b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.555+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e3777bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.555+0000 7f9e0e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e3777e6a0 0x558e3777c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.555+0000 7f9e0de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 0x558e3777b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:35444/0 (socket says 172.21.15.42:35444) 2024-04-20T10:35:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.555+0000 7f9e0de00640 1 -- 172.21.15.42:0/1239299298 learned_addr learned my addr 172.21.15.42:0/1239299298 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.555+0000 7f9e0de00640 1 -- 172.21.15.42:0/1239299298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 msgr2=0x558e3777bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.555+0000 7f9e0de00640 1 --2- 172.21.15.42:0/1239299298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e3777bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.555+0000 7f9e0de00640 1 -- 172.21.15.42:0/1239299298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e3777e6a0 msgr2=0x558e3777c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.555+0000 7f9e0de00640 1 --2- 172.21.15.42:0/1239299298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e3777e6a0 0x558e3777c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.555+0000 7f9e0de00640 1 -- 172.21.15.42:0/1239299298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e373ed3f0 con 0x558e377751a0 2024-04-20T10:35:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.555+0000 7f9e0de00640 1 --2- 172.21.15.42:0/1239299298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 0x558e3777b730 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e00003f10 tx=0x7f9e00003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdf7f53dcc2f7491 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.555+0000 7f9defe00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:35:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.556+0000 7f9defe00640 1 -- 172.21.15.42:0/1239299298 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00028070 con 0x558e377751a0 2024-04-20T10:35:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.556+0000 7f9e17d90740 1 -- 172.21.15.42:0/1239299298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e000043c0 con 0x558e377751a0 2024-04-20T10:35:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.556+0000 7f9defe00640 1 -- 172.21.15.42:0/1239299298 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9e00005040 con 0x558e377751a0 2024-04-20T10:35:13.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.556+0000 7f9defe00640 1 -- 172.21.15.42:0/1239299298 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9e00034050 con 0x558e377751a0 2024-04-20T10:35:13.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.556+0000 7f9e17d90740 10 client.25531 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:13.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.556+0000 7f9e17d90740 1 -- 172.21.15.42:0/1239299298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558e373ad0f0 con 0x558e377751a0 2024-04-20T10:35:13.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.557+0000 7f9defe00640 1 -- 172.21.15.42:0/1239299298 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e000043c0 con 0x558e377751a0 2024-04-20T10:35:13.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.557+0000 7f9defe00640 1 -- 172.21.15.42:0/1239299298 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f9e00004860 con 0x558e377751a0 2024-04-20T10:35:13.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.557+0000 7f9e17d90740 10 client.25531 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:13.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.557+0000 7f9e17d90740 10 client.25531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:13.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.557+0000 7f9e17d90740 20 client.25531 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:13.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.557+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1239299298 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558e3778e9d0 0x558e37790dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.557+0000 7f9e17d90740 4 client.25531 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:35:13.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.557+0000 7f9e17d90740 1 -- 172.21.15.42:0/1239299298 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e376e8b70 con 0x558e3778e9d0 2024-04-20T10:35:13.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.558+0000 7f9e0d400640 1 --2- 172.21.15.42:0/1239299298 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558e3778e9d0 0x558e37790dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.558+0000 7f9e0d400640 1 --2- 172.21.15.42:0/1239299298 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558e3778e9d0 0x558e37790dc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c589c0289ad36d server_cookie=2366f70c6c2f3514 in_seq=0 out_seq=0 2024-04-20T10:35:13.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.558+0000 7f9defe00640 10 client.25531 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:13.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9defe00640 1 -- 172.21.15.42:0/1239299298 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e376e8b70 con 0x558e3778e9d0 2024-04-20T10:35:13.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9defe00640 10 client.25531 handle_command_reply: tid=0 2024-04-20T10:35:13.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9e17d90740 1 client.25531 shutdown 2024-04-20T10:35:13.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9e17d90740 1 -- 172.21.15.42:0/1239299298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 msgr2=0x558e3777b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1239299298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 0x558e3777b730 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e00003f10 tx=0x7f9e00003820 comp rx=0 tx=0).stop 2024-04-20T10:35:13.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9e17d90740 1 -- 172.21.15.42:0/1239299298 shutdown_connections 2024-04-20T10:35:13.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1239299298 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558e3778e9d0 0x558e37790dc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1239299298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e3777e6a0 0x558e3777c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1239299298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e3777bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1239299298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 0x558e3777b730 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9e17d90740 1 -- 172.21.15.42:0/1239299298 >> 172.21.15.42:0/1239299298 conn(0x558e3776f8d0 msgr2=0x558e37771620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9defe00640 0 client.25531 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9e17d90740 1 -- 172.21.15.42:0/1239299298 shutdown_connections 2024-04-20T10:35:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.559+0000 7f9e17d90740 1 -- 172.21.15.42:0/1239299298 wait complete. 2024-04-20T10:35:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.560+0000 7f9e17d90740 20 client.25531 trim_cache size 0 max 16384 2024-04-20T10:35:13.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.588+0000 7f9e17d90740 1 Processor -- start 2024-04-20T10:35:13.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.589+0000 7f9e17d90740 1 -- start start 2024-04-20T10:35:13.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.589+0000 7f9e17d90740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e3776f8d0 0x558e3777de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.589+0000 7f9e17d90740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 0x558e3778a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.589+0000 7f9e17d90740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e37771590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.589+0000 7f9e17d90740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101adf70 con 0x558e3776f8d0 2024-04-20T10:35:13.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.589+0000 7f9e17d90740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101b07f0 con 0x558e37775ac0 2024-04-20T10:35:13.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.589+0000 7f9e17d90740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101ae5e0 con 0x558e377751a0 2024-04-20T10:35:13.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.589+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 0x558e3778a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.589+0000 7f9e0de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e3776f8d0 0x558e3777de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.589+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 0x558e3778a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:35450/0 (socket says 172.21.15.42:35450) 2024-04-20T10:35:13.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.589+0000 7f9e0d400640 1 -- 172.21.15.42:0/1414041685 learned_addr learned my addr 172.21.15.42:0/1414041685 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:13.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.590+0000 7f9e0e800640 1 --2- 172.21.15.42:0/1414041685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e37771590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.590+0000 7f9e0d400640 1 -- 172.21.15.42:0/1414041685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 msgr2=0x558e37771590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.590+0000 7f9e0d400640 1 --2- 172.21.15.42:0/1414041685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e37771590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.590+0000 7f9e0d400640 1 -- 172.21.15.42:0/1414041685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e3776f8d0 msgr2=0x558e3777de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.590+0000 7f9e0d400640 1 --2- 172.21.15.42:0/1414041685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e3776f8d0 0x558e3777de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.590+0000 7f9e0d400640 1 -- 172.21.15.42:0/1414041685 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e37780a90 con 0x558e377751a0 2024-04-20T10:35:13.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.590+0000 7f9e0e800640 1 --2- 172.21.15.42:0/1414041685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e37771590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:35:13.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.590+0000 7f9e0d400640 1 --2- 172.21.15.42:0/1414041685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 0x558e3778a350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc02cbc0 tx=0x7f9dfc0f46f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d93529a87db2b37f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:13.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.590+0000 7f9dfaa00640 1 -- 172.21.15.42:0/1414041685 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc09ce60 con 0x558e377751a0 2024-04-20T10:35:13.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.590+0000 7f9dfaa00640 1 -- 172.21.15.42:0/1414041685 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dfc032640 con 0x558e377751a0 2024-04-20T10:35:13.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.590+0000 7f9dfaa00640 1 -- 172.21.15.42:0/1414041685 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0c6cb0 con 0x558e377751a0 2024-04-20T10:35:13.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.591+0000 7f9e17d90740 1 -- 172.21.15.42:0/1414041685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 msgr2=0x558e3778a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.591+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1414041685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 0x558e3778a350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc02cbc0 tx=0x7f9dfc0f46f0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.591+0000 7f9e17d90740 1 -- 172.21.15.42:0/1414041685 shutdown_connections 2024-04-20T10:35:13.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.591+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1414041685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e37771590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.591+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1414041685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e377751a0 0x558e3778a350 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.591+0000 7f9e17d90740 1 --2- 172.21.15.42:0/1414041685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e3776f8d0 0x558e3777de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.591+0000 7f9e17d90740 1 -- 172.21.15.42:0/1414041685 >> 172.21.15.42:0/1414041685 conn(0x558e37777100 msgr2=0x558e3776e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:13.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.592+0000 7f9e17d90740 1 -- 172.21.15.42:0/1414041685 shutdown_connections 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.592+0000 7f9e17d90740 1 -- 172.21.15.42:0/1414041685 wait complete. 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.592+0000 7f9e17d90740 1 -- start start 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.593+0000 7f9e17d90740 1 Processor -- start 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.593+0000 7f9e17d90740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e3776f8d0 0x558e37786ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.593+0000 7f9e17d90740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e377751a0 0x558e37781e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.593+0000 7f9e17d90740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e37782360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.593+0000 7f9e17d90740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101aecb0 con 0x558e377751a0 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.593+0000 7f9e17d90740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101b0120 con 0x558e37775ac0 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.593+0000 7f9e17d90740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101af380 con 0x558e3776f8d0 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.593+0000 7f9e17d90740 10 client.0 _ll_register_callbacks cb 0x7f9e00011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9e0de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e3776f8d0 0x558e37786ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9e0de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e3776f8d0 0x558e37786ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:35456/0 (socket says 172.21.15.42:35456) 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9e0de00640 1 -- 172.21.15.42:0/239638227 learned_addr learned my addr 172.21.15.42:0/239638227 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9e0d400640 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e377751a0 0x558e37781e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9e0e800640 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e37782360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9e0de00640 1 -- 172.21.15.42:0/239638227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 msgr2=0x558e37782360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9e0de00640 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e37782360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9e0de00640 1 -- 172.21.15.42:0/239638227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e377751a0 msgr2=0x558e37781e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9e0de00640 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e377751a0 0x558e37781e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9e0de00640 1 -- 172.21.15.42:0/239638227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e377829d0 con 0x558e3776f8d0 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9e0e800640 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e37782360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9e0de00640 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e3776f8d0 0x558e37786ab0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9e00003f40 tx=0x7f9e00005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e684306fc702cd51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.594+0000 7f9defe00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.595+0000 7f9defe00640 1 -- 172.21.15.42:0/239638227 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0003d060 con 0x558e3776f8d0 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.595+0000 7f9defe00640 1 -- 172.21.15.42:0/239638227 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f9e00004280 con 0x558e3776f8d0 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.595+0000 7f9e17d90740 1 -- 172.21.15.42:0/239638227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558e376e29e0 con 0x558e3776f8d0 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.595+0000 7f9defe00640 1 -- 172.21.15.42:0/239638227 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9e00028070 con 0x558e3776f8d0 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.595+0000 7f9e17d90740 10 client.25537 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:13.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.595+0000 7f9e17d90740 1 -- 172.21.15.42:0/239638227 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558e377fd450 con 0x558e3776f8d0 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.595+0000 7f9defe00640 1 -- 172.21.15.42:0/239638227 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558e376e29e0 con 0x558e3776f8d0 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.596+0000 7f9defe00640 1 -- 172.21.15.42:0/239638227 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f9e00019b10 con 0x558e3776f8d0 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.596+0000 7f9e17d90740 10 client.25537 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.596+0000 7f9e17d90740 10 client.25537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.596+0000 7f9e17d90740 20 client.25537 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.596+0000 7f9e17d90740 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558e3777e6a0 0x558e377ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.596+0000 7f9e17d90740 4 client.25537 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.596+0000 7f9e17d90740 1 -- 172.21.15.42:0/239638227 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e3765e450 con 0x558e3777e6a0 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.596+0000 7f9e0d400640 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558e3777e6a0 0x558e377ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.597+0000 7f9e0d400640 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558e3777e6a0 0x558e377ffb50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa69167708f6200 server_cookie=54f2c520c910ccd in_seq=0 out_seq=0 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.597+0000 7f9defe00640 10 client.25537 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.597+0000 7f9defe00640 1 -- 172.21.15.42:0/239638227 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558e3765e450 con 0x558e3777e6a0 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.597+0000 7f9defe00640 10 client.25537 handle_command_reply: tid=0 2024-04-20T10:35:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.597+0000 7f9e17d90740 1 client.25537 shutdown 2024-04-20T10:35:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.598+0000 7f9e17d90740 1 -- 172.21.15.42:0/239638227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e3776f8d0 msgr2=0x558e37786ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.598+0000 7f9e17d90740 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e3776f8d0 0x558e37786ab0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9e00003f40 tx=0x7f9e00005e60 comp rx=0 tx=0).stop 2024-04-20T10:35:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.598+0000 7f9e17d90740 1 -- 172.21.15.42:0/239638227 shutdown_connections 2024-04-20T10:35:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.598+0000 7f9e17d90740 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558e3777e6a0 0x558e377ffb50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.598+0000 7f9e17d90740 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e37775ac0 0x558e37782360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.598+0000 7f9e17d90740 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558e377751a0 0x558e37781e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.598+0000 7f9e17d90740 1 --2- 172.21.15.42:0/239638227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558e3776f8d0 0x558e37786ab0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.598+0000 7f9e17d90740 1 -- 172.21.15.42:0/239638227 >> 172.21.15.42:0/239638227 conn(0x558e37777100 msgr2=0x558e37770b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.598+0000 7f9defe00640 0 client.25537 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.598+0000 7f9e17d90740 1 -- 172.21.15.42:0/239638227 shutdown_connections 2024-04-20T10:35:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.598+0000 7f9e17d90740 1 -- 172.21.15.42:0/239638227 wait complete. 2024-04-20T10:35:13.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.598+0000 7f9e17d90740 20 client.25537 trim_cache size 0 max 16384 2024-04-20T10:35:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:35:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T10:35:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:35:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "64e6ee31-3956-493b-9186-b4b388330c9d": { 2024-04-20T10:35:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:35:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "64e6ee31-3956-493b-9186-b4b388330c9d", 2024-04-20T10:35:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:35:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:35:13.611 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:35:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:35:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.599+0000 7f9e15000640 1 -- 172.21.15.42:0/2806366254 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9df40a3e40 msgr2=0x7f9df40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.599+0000 7f9e15000640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9df40a3e40 0x7f9df40a6300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9e080107d0 tx=0x7f9e08004a40 comp rx=0 tx=0).stop 2024-04-20T10:35:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.599+0000 7f9e15000640 1 -- 172.21.15.42:0/2806366254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e101094e0 msgr2=0x7f9e101a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.599+0000 7f9e15000640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e101094e0 0x7f9e101a0d10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc037040 tx=0x7f9dfc0045b0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.599+0000 7f9e15000640 1 -- 172.21.15.42:0/2806366254 shutdown_connections 2024-04-20T10:35:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.599+0000 7f9e15000640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9df40a3e40 0x7f9df40a6300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.599+0000 7f9e15000640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1019a9d0 0x7f9e1019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.599+0000 7f9e15000640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e101094e0 0x7f9e101a0d10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.599+0000 7f9e15000640 1 --2- 172.21.15.42:0/2806366254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e10104410 0x7f9e101a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.599+0000 7f9e15000640 1 -- 172.21.15.42:0/2806366254 >> 172.21.15.42:0/2806366254 conn(0x7f9e100ffcf0 msgr2=0x7f9e10101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.600+0000 7f9e15000640 1 -- 172.21.15.42:0/2806366254 shutdown_connections 2024-04-20T10:35:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:13.600+0000 7f9e15000640 1 -- 172.21.15.42:0/2806366254 wait complete. 2024-04-20T10:35:13.623 INFO:tasks.cephfs.filesystem:scrub status for tag:64e6ee31-3956-493b-9186-b4b388330c9d - {'path': '/', 'tag': '64e6ee31-3956-493b-9186-b4b388330c9d', 'options': 'recursive,force'} 2024-04-20T10:35:13.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:13 smithi046 ceph-mon[21574]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T10:35:13.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:13 smithi042 ceph-mon[17078]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T10:35:13.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:13 smithi162 ceph-mon[21854]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-20T10:35:14.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:14 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:35:14.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:14 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:35:14.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:14 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:35:14.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:14 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:35:14.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:14 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:35:14.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:14 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:35:14.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:14 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:35:14.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:14 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:35:14.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:14 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:35:14.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:14 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:35:14.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:14 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:35:14.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:14 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:35:15.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:15 smithi046 ceph-mon[21574]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-20T10:35:15.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:15 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:35:15.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:15 smithi042 ceph-mon[17078]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-20T10:35:15.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:15 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:35:15.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:15 smithi162 ceph-mon[21854]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-20T10:35:15.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:15 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:35:17.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:17 smithi046 ceph-mon[21574]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-20T10:35:17.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:17 smithi042 ceph-mon[17078]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-20T10:35:17.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:17 smithi162 ceph-mon[21854]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-20T10:35:19.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:19 smithi046 ceph-mon[21574]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-20T10:35:19.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:19 smithi042 ceph-mon[17078]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-20T10:35:19.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:19 smithi162 ceph-mon[21854]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-20T10:35:21.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:21 smithi046 ceph-mon[21574]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-20T10:35:21.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:21 smithi042 ceph-mon[17078]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-20T10:35:21.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:21 smithi162 ceph-mon[21854]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-04-20T10:35:23.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:23 smithi046 ceph-mon[21574]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-20T10:35:23.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:23 smithi042 ceph-mon[17078]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-20T10:35:23.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:23 smithi162 ceph-mon[21854]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-20T10:35:25.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:25 smithi046 ceph-mon[21574]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T10:35:25.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:25 smithi042 ceph-mon[17078]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T10:35:25.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:25 smithi162 ceph-mon[21854]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-20T10:35:27.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:27 smithi046 ceph-mon[21574]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T10:35:27.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:27 smithi042 ceph-mon[17078]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T10:35:27.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:27 smithi162 ceph-mon[21854]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-20T10:35:29.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:29 smithi046 ceph-mon[21574]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T10:35:29.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:29 smithi042 ceph-mon[17078]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T10:35:29.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:29 smithi162 ceph-mon[21854]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T10:35:31.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:31 smithi046 ceph-mon[21574]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T10:35:31.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:31 smithi042 ceph-mon[17078]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T10:35:31.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:31 smithi162 ceph-mon[21854]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-20T10:35:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:33 smithi046 ceph-mon[21574]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:33 smithi046 ceph-mon[21574]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-20T10:35:33.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:33 smithi042 ceph-mon[17078]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:33.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:33 smithi042 ceph-mon[17078]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-20T10:35:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:33 smithi162 ceph-mon[21854]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:33 smithi162 ceph-mon[21854]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-20T10:35:35.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:35 smithi046 ceph-mon[21574]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:35.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:35 smithi042 ceph-mon[17078]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:35 smithi162 ceph-mon[21854]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:37.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:37 smithi042 ceph-mon[17078]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:37 smithi162 ceph-mon[21854]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:37 smithi046 ceph-mon[21574]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:39.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:39 smithi042 ceph-mon[17078]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:39 smithi162 ceph-mon[21854]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:39 smithi046 ceph-mon[21574]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:40.912 INFO:tasks.workunit.client.0.smithi042.stderr:Updating files: 58% (7162/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 95% (11729/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-20T10:35:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:41 smithi042 ceph-mon[17078]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:41 smithi162 ceph-mon[21854]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:41.950 INFO:tasks.workunit.client.0.smithi042.stderr:Note: switching to 'a8c84f5997950fd7d008204839253291d52e6eda'. 2024-04-20T10:35:41.950 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-20T10:35:41.950 INFO:tasks.workunit.client.0.smithi042.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-20T10:35:41.950 INFO:tasks.workunit.client.0.smithi042.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-20T10:35:41.950 INFO:tasks.workunit.client.0.smithi042.stderr:state without impacting any branches by switching back to a branch. 2024-04-20T10:35:41.950 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-20T10:35:41.950 INFO:tasks.workunit.client.0.smithi042.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-20T10:35:41.951 INFO:tasks.workunit.client.0.smithi042.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-20T10:35:41.951 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-20T10:35:41.951 INFO:tasks.workunit.client.0.smithi042.stderr: git switch -c 2024-04-20T10:35:41.951 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-20T10:35:41.951 INFO:tasks.workunit.client.0.smithi042.stderr:Or undo this operation with: 2024-04-20T10:35:41.951 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-20T10:35:41.951 INFO:tasks.workunit.client.0.smithi042.stderr: git switch - 2024-04-20T10:35:41.951 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-20T10:35:41.951 INFO:tasks.workunit.client.0.smithi042.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-20T10:35:41.951 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-20T10:35:41.951 INFO:tasks.workunit.client.0.smithi042.stderr:HEAD is now at a8c84f59979 Merge PR #51659 into wip-rishabh-testing-20240416.193735 2024-04-20T10:35:41.965 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-20T10:35:42.025 INFO:tasks.workunit.client.0.smithi042.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-20T10:35:42.032 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T10:35:42.033 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-20T10:35:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:41 smithi046 ceph-mon[21574]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:42.810 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-20T10:35:42.950 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-20T10:35:43.039 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T10:35:43.045 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T10:35:43.045 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-20T10:35:43.135 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T10:35:43.146 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:35:43.146 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-20T10:35:43.201 INFO:tasks.workunit:Running workunits matching kernel_untar_build.sh on client.0... 2024-04-20T10:35:43.202 INFO:tasks.workunit:Running workunit kernel_untar_build.sh... 2024-04-20T10:35:43.202 DEBUG:teuthology.orchestra.run.smithi042:workunit test kernel_untar_build.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh 2024-04-20T10:35:43.276 INFO:tasks.workunit.client.0.smithi042.stderr:+ wget -O linux.tar.xz http://download.ceph.com/qa/linux-6.5.11.tar.xz 2024-04-20T10:35:43.345 INFO:tasks.workunit.client.0.smithi042.stderr:--2024-04-20 10:35:43-- http://download.ceph.com/qa/linux-6.5.11.tar.xz 2024-04-20T10:35:43.346 INFO:tasks.workunit.client.0.smithi042.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-04-20T10:35:43.368 INFO:tasks.workunit.client.0.smithi042.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-04-20T10:35:43.391 INFO:tasks.workunit.client.0.smithi042.stderr:HTTP request sent, awaiting response... 200 OK 2024-04-20T10:35:43.391 INFO:tasks.workunit.client.0.smithi042.stderr:Length: 139010916 (133M) [application/octet-stream] 2024-04-20T10:35:43.391 INFO:tasks.workunit.client.0.smithi042.stderr:Saving to: ‘linux.tar.xz’ 2024-04-20T10:35:43.398 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-20T10:35:43.433 INFO:tasks.workunit.client.0.smithi042.stderr: 0K .......... .......... .......... .......... .......... 0% 1.18M 1m52s 2024-04-20T10:35:43.453 INFO:tasks.workunit.client.0.smithi042.stderr: 50K .......... .......... .......... .......... .......... 0% 2.40M 84s 2024-04-20T10:35:43.454 INFO:tasks.workunit.client.0.smithi042.stderr: 100K .......... .......... .......... .......... .......... 0% 36.1M 57s 2024-04-20T10:35:43.476 INFO:tasks.workunit.client.0.smithi042.stderr: 150K .......... .......... .......... .......... .......... 0% 2.45M 56s 2024-04-20T10:35:43.476 INFO:tasks.workunit.client.0.smithi042.stderr: 200K .......... .......... .......... .......... .......... 0% 67.9M 45s 2024-04-20T10:35:43.476 INFO:tasks.workunit.client.0.smithi042.stderr: 250K .......... .......... .......... .......... .......... 0% 73.3M 38s 2024-04-20T10:35:43.477 INFO:tasks.workunit.client.0.smithi042.stderr: 300K .......... .......... .......... .......... .......... 0% 77.1M 33s 2024-04-20T10:35:43.497 INFO:tasks.workunit.client.0.smithi042.stderr: 350K .......... .......... .......... .......... .......... 0% 2.69M 35s 2024-04-20T10:35:43.497 INFO:tasks.workunit.client.0.smithi042.stderr: 400K .......... .......... .......... .......... .......... 0% 47.5M 31s 2024-04-20T10:35:43.497 INFO:tasks.workunit.client.0.smithi042.stderr: 450K .......... .......... .......... .......... .......... 0% 63.1M 28s 2024-04-20T10:35:43.497 INFO:tasks.workunit.client.0.smithi042.stderr: 500K .......... .......... .......... .......... .......... 0% 70.0M 26s 2024-04-20T10:35:43.518 INFO:tasks.workunit.client.0.smithi042.stderr: 550K .......... .......... .......... .......... .......... 0% 69.8M 24s 2024-04-20T10:35:43.518 INFO:tasks.workunit.client.0.smithi042.stderr: 600K .......... .......... .......... .......... .......... 0% 2.67M 26s 2024-04-20T10:35:43.519 INFO:tasks.workunit.client.0.smithi042.stderr: 650K .......... .......... .......... .......... .......... 0% 73.6M 24s 2024-04-20T10:35:43.519 INFO:tasks.workunit.client.0.smithi042.stderr: 700K .......... .......... .......... .......... .......... 0% 75.1M 23s 2024-04-20T10:35:43.519 INFO:tasks.workunit.client.0.smithi042.stderr: 750K .......... .......... .......... .......... .......... 0% 73.5M 21s 2024-04-20T10:35:43.541 INFO:tasks.workunit.client.0.smithi042.stderr: 800K .......... .......... .......... .......... .......... 0% 2.68M 23s 2024-04-20T10:35:43.541 INFO:tasks.workunit.client.0.smithi042.stderr: 850K .......... .......... .......... .......... .......... 0% 57.1M 22s 2024-04-20T10:35:43.541 INFO:tasks.workunit.client.0.smithi042.stderr: 900K .......... .......... .......... .......... .......... 0% 74.4M 21s 2024-04-20T10:35:43.541 INFO:tasks.workunit.client.0.smithi042.stderr: 950K .......... .......... .......... .......... .......... 0% 69.4M 20s 2024-04-20T10:35:43.541 INFO:tasks.workunit.client.0.smithi042.stderr: 1000K .......... .......... .......... .......... .......... 0% 72.8M 19s 2024-04-20T10:35:43.541 INFO:tasks.workunit.client.0.smithi042.stderr: 1050K .......... .......... .......... .......... .......... 0% 64.2M 18s 2024-04-20T10:35:43.541 INFO:tasks.workunit.client.0.smithi042.stderr: 1100K .......... .......... .......... .......... .......... 0% 73.9M 17s 2024-04-20T10:35:43.542 INFO:tasks.workunit.client.0.smithi042.stderr: 1150K .......... .......... .......... .......... .......... 0% 73.6M 17s 2024-04-20T10:35:43.542 INFO:tasks.workunit.client.0.smithi042.stderr: 1200K .......... .......... .......... .......... .......... 0% 74.9M 16s 2024-04-20T10:35:43.543 INFO:tasks.workunit.client.0.smithi042.stderr: 1250K .......... .......... .......... .......... .......... 0% 34.9M 16s 2024-04-20T10:35:43.545 INFO:tasks.workunit.client.0.smithi042.stderr: 1300K .......... .......... .......... .......... .......... 0% 25.3M 15s 2024-04-20T10:35:43.547 INFO:tasks.workunit.client.0.smithi042.stderr: 1350K .......... .......... .......... .......... .......... 1% 24.5M 15s 2024-04-20T10:35:43.549 INFO:tasks.workunit.client.0.smithi042.stderr: 1400K .......... .......... .......... .......... .......... 1% 35.9M 15s 2024-04-20T10:35:43.560 INFO:tasks.workunit.client.0.smithi042.stderr: 1450K .......... .......... .......... .......... .......... 1% 6.18M 15s 2024-04-20T10:35:43.560 INFO:tasks.workunit.client.0.smithi042.stderr: 1500K .......... .......... .......... .......... .......... 1% 51.3M 14s 2024-04-20T10:35:43.560 INFO:tasks.workunit.client.0.smithi042.stderr: 1550K .......... .......... .......... .......... .......... 1% 62.5M 14s 2024-04-20T10:35:43.560 INFO:tasks.workunit.client.0.smithi042.stderr: 1600K .......... .......... .......... .......... .......... 1% 74.7M 14s 2024-04-20T10:35:43.561 INFO:tasks.workunit.client.0.smithi042.stderr: 1650K .......... .......... .......... .......... .......... 1% 74.6M 13s 2024-04-20T10:35:43.561 INFO:tasks.workunit.client.0.smithi042.stderr: 1700K .......... .......... .......... .......... .......... 1% 63.3M 13s 2024-04-20T10:35:43.561 INFO:tasks.workunit.client.0.smithi042.stderr: 1750K .......... .......... .......... .......... .......... 1% 75.1M 13s 2024-04-20T10:35:43.562 INFO:tasks.workunit.client.0.smithi042.stderr: 1800K .......... .......... .......... .......... .......... 1% 41.0M 12s 2024-04-20T10:35:43.564 INFO:tasks.workunit.client.0.smithi042.stderr: 1850K .......... .......... .......... .......... .......... 1% 25.6M 12s 2024-04-20T10:35:43.565 INFO:tasks.workunit.client.0.smithi042.stderr: 1900K .......... .......... .......... .......... .......... 1% 26.8M 12s 2024-04-20T10:35:43.566 INFO:tasks.workunit.client.0.smithi042.stderr: 1950K .......... .......... .......... .......... .......... 1% 43.2M 12s 2024-04-20T10:35:43.571 INFO:tasks.workunit.client.0.smithi042.stderr: 2000K .......... .......... .......... .......... .......... 1% 24.8M 12s 2024-04-20T10:35:43.571 INFO:tasks.workunit.client.0.smithi042.stderr: 2050K .......... .......... .......... .......... .......... 1% 20.5M 11s 2024-04-20T10:35:43.572 INFO:tasks.workunit.client.0.smithi042.stderr: 2100K .......... .......... .......... .......... .......... 1% 27.7M 11s 2024-04-20T10:35:43.574 INFO:tasks.workunit.client.0.smithi042.stderr: 2150K .......... .......... .......... .......... .......... 1% 39.7M 11s 2024-04-20T10:35:43.575 INFO:tasks.workunit.client.0.smithi042.stderr: 2200K .......... .......... .......... .......... .......... 1% 40.3M 11s 2024-04-20T10:35:43.577 INFO:tasks.workunit.client.0.smithi042.stderr: 2250K .......... .......... .......... .......... .......... 1% 26.5M 11s 2024-04-20T10:35:43.579 INFO:tasks.workunit.client.0.smithi042.stderr: 2300K .......... .......... .......... .......... .......... 1% 28.4M 11s 2024-04-20T10:35:43.580 INFO:tasks.workunit.client.0.smithi042.stderr: 2350K .......... .......... .......... .......... .......... 1% 28.8M 11s 2024-04-20T10:35:43.582 INFO:tasks.workunit.client.0.smithi042.stderr: 2400K .......... .......... .......... .......... .......... 1% 27.4M 10s 2024-04-20T10:35:43.582 INFO:tasks.workunit.client.0.smithi042.stderr: 2450K .......... .......... .......... .......... .......... 1% 70.3M 10s 2024-04-20T10:35:43.585 INFO:tasks.workunit.client.0.smithi042.stderr: 2500K .......... .......... .......... .......... .......... 1% 24.3M 10s 2024-04-20T10:35:43.587 INFO:tasks.workunit.client.0.smithi042.stderr: 2550K .......... .......... .......... .......... .......... 1% 19.5M 10s 2024-04-20T10:35:43.589 INFO:tasks.workunit.client.0.smithi042.stderr: 2600K .......... .......... .......... .......... .......... 1% 25.5M 10s 2024-04-20T10:35:43.590 INFO:tasks.workunit.client.0.smithi042.stderr: 2650K .......... .......... .......... .......... .......... 1% 31.7M 10s 2024-04-20T10:35:43.591 INFO:tasks.workunit.client.0.smithi042.stderr: 2700K .......... .......... .......... .......... .......... 2% 66.3M 10s 2024-04-20T10:35:43.594 INFO:tasks.workunit.client.0.smithi042.stderr: 2750K .......... .......... .......... .......... .......... 2% 19.7M 10s 2024-04-20T10:35:43.596 INFO:tasks.workunit.client.0.smithi042.stderr: 2800K .......... .......... .......... .......... .......... 2% 24.6M 10s 2024-04-20T10:35:43.598 INFO:tasks.workunit.client.0.smithi042.stderr: 2850K .......... .......... .......... .......... .......... 2% 24.5M 9s 2024-04-20T10:35:43.601 INFO:tasks.workunit.client.0.smithi042.stderr: 2900K .......... .......... .......... .......... .......... 2% 23.5M 9s 2024-04-20T10:35:43.601 INFO:tasks.workunit.client.0.smithi042.stderr: 2950K .......... .......... .......... .......... .......... 2% 67.8M 9s 2024-04-20T10:35:43.602 INFO:tasks.workunit.client.0.smithi042.stderr: 3000K .......... .......... .......... .......... .......... 2% 34.9M 9s 2024-04-20T10:35:43.604 INFO:tasks.workunit.client.0.smithi042.stderr: 3050K .......... .......... .......... .......... .......... 2% 27.3M 9s 2024-04-20T10:35:43.605 INFO:tasks.workunit.client.0.smithi042.stderr: 3100K .......... .......... .......... .......... .......... 2% 31.4M 9s 2024-04-20T10:35:43.609 INFO:tasks.workunit.client.0.smithi042.stderr: 3150K .......... .......... .......... .......... .......... 2% 17.1M 9s 2024-04-20T10:35:43.609 INFO:tasks.workunit.client.0.smithi042.stderr: 3200K .......... .......... .......... .......... .......... 2% 75.1M 9s 2024-04-20T10:35:43.610 INFO:tasks.workunit.client.0.smithi042.stderr: 3250K .......... .......... .......... .......... .......... 2% 26.8M 9s 2024-04-20T10:35:43.613 INFO:tasks.workunit.client.0.smithi042.stderr: 3300K .......... .......... .......... .......... .......... 2% 24.6M 9s 2024-04-20T10:35:43.615 INFO:tasks.workunit.client.0.smithi042.stderr: 3350K .......... .......... .......... .......... .......... 2% 27.7M 9s 2024-04-20T10:35:43.617 INFO:tasks.workunit.client.0.smithi042.stderr: 3400K .......... .......... .......... .......... .......... 2% 25.5M 9s 2024-04-20T10:35:43.619 INFO:tasks.workunit.client.0.smithi042.stderr: 3450K .......... .......... .......... .......... .......... 2% 72.1M 9s 2024-04-20T10:35:43.619 INFO:tasks.workunit.client.0.smithi042.stderr: 3500K .......... .......... .......... .......... .......... 2% 25.4M 8s 2024-04-20T10:35:43.621 INFO:tasks.workunit.client.0.smithi042.stderr: 3550K .......... .......... .......... .......... .......... 2% 26.9M 8s 2024-04-20T10:35:43.622 INFO:tasks.workunit.client.0.smithi042.stderr: 3600K .......... .......... .......... .......... .......... 2% 26.4M 8s 2024-04-20T10:35:43.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:35:43.624 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:35:43.626 INFO:tasks.workunit.client.0.smithi042.stderr: 3650K .......... .......... .......... .......... .......... 2% 24.0M 8s 2024-04-20T10:35:43.626 INFO:tasks.workunit.client.0.smithi042.stderr: 3700K .......... .......... .......... .......... .......... 2% 65.7M 8s 2024-04-20T10:35:43.635 INFO:tasks.workunit.client.0.smithi042.stderr: 3750K .......... .......... .......... .......... .......... 2% 26.9M 8s 2024-04-20T10:35:43.636 INFO:tasks.workunit.client.0.smithi042.stderr: 3800K .......... .......... .......... .......... .......... 2% 23.1M 8s 2024-04-20T10:35:43.636 INFO:tasks.workunit.client.0.smithi042.stderr: 3850K .......... .......... .......... .......... .......... 2% 23.2M 8s 2024-04-20T10:35:43.636 INFO:tasks.workunit.client.0.smithi042.stderr: 3900K .......... .......... .......... .......... .......... 2% 14.1M 8s 2024-04-20T10:35:43.636 INFO:tasks.workunit.client.0.smithi042.stderr: 3950K .......... .......... .......... .......... .......... 2% 27.1M 8s 2024-04-20T10:35:43.638 INFO:tasks.workunit.client.0.smithi042.stderr: 4000K .......... .......... .......... .......... .......... 2% 29.3M 8s 2024-04-20T10:35:43.640 INFO:tasks.workunit.client.0.smithi042.stderr: 4050K .......... .......... .......... .......... .......... 3% 32.2M 8s 2024-04-20T10:35:43.641 INFO:tasks.workunit.client.0.smithi042.stderr: 4100K .......... .......... .......... .......... .......... 3% 29.0M 8s 2024-04-20T10:35:43.643 INFO:tasks.workunit.client.0.smithi042.stderr: 4150K .......... .......... .......... .......... .......... 3% 27.1M 8s 2024-04-20T10:35:43.646 INFO:tasks.workunit.client.0.smithi042.stderr: 4200K .......... .......... .......... .......... .......... 3% 23.9M 8s 2024-04-20T10:35:43.646 INFO:tasks.workunit.client.0.smithi042.stderr: 4250K .......... .......... .......... .......... .......... 3% 53.2M 8s 2024-04-20T10:35:43.648 INFO:tasks.workunit.client.0.smithi042.stderr: 4300K .......... .......... .......... .......... .......... 3% 25.9M 8s 2024-04-20T10:35:43.650 INFO:tasks.workunit.client.0.smithi042.stderr: 4350K .......... .......... .......... .......... .......... 3% 25.6M 8s 2024-04-20T10:35:43.652 INFO:tasks.workunit.client.0.smithi042.stderr: 4400K .......... .......... .......... .......... .......... 3% 31.1M 8s 2024-04-20T10:35:43.654 INFO:tasks.workunit.client.0.smithi042.stderr: 4450K .......... .......... .......... .......... .......... 3% 25.1M 8s 2024-04-20T10:35:43.654 INFO:tasks.workunit.client.0.smithi042.stderr: 4500K .......... .......... .......... .......... .......... 3% 46.3M 8s 2024-04-20T10:35:43.656 INFO:tasks.workunit.client.0.smithi042.stderr: 4550K .......... .......... .......... .......... .......... 3% 25.8M 8s 2024-04-20T10:35:43.659 INFO:tasks.workunit.client.0.smithi042.stderr: 4600K .......... .......... .......... .......... .......... 3% 26.2M 8s 2024-04-20T10:35:43.661 INFO:tasks.workunit.client.0.smithi042.stderr: 4650K .......... .......... .......... .......... .......... 3% 29.2M 8s 2024-04-20T10:35:43.662 INFO:tasks.workunit.client.0.smithi042.stderr: 4700K .......... .......... .......... .......... .......... 3% 24.5M 7s 2024-04-20T10:35:43.663 INFO:tasks.workunit.client.0.smithi042.stderr: 4750K .......... .......... .......... .......... .......... 3% 32.3M 7s 2024-04-20T10:35:43.666 INFO:tasks.workunit.client.0.smithi042.stderr: 4800K .......... .......... .......... .......... .......... 3% 24.1M 7s 2024-04-20T10:35:43.668 INFO:tasks.workunit.client.0.smithi042.stderr: 4850K .......... .......... .......... .......... .......... 3% 26.8M 7s 2024-04-20T10:35:43.669 INFO:tasks.workunit.client.0.smithi042.stderr: 4900K .......... .......... .......... .......... .......... 3% 26.8M 7s 2024-04-20T10:35:43.671 INFO:tasks.workunit.client.0.smithi042.stderr: 4950K .......... .......... .......... .......... .......... 3% 25.4M 7s 2024-04-20T10:35:43.672 INFO:tasks.workunit.client.0.smithi042.stderr: 5000K .......... .......... .......... .......... .......... 3% 38.3M 7s 2024-04-20T10:35:43.674 INFO:tasks.workunit.client.0.smithi042.stderr: 5050K .......... .......... .......... .......... .......... 3% 27.1M 7s 2024-04-20T10:35:43.675 INFO:tasks.workunit.client.0.smithi042.stderr: 5100K .......... .......... .......... .......... .......... 3% 37.1M 7s 2024-04-20T10:35:43.679 INFO:tasks.workunit.client.0.smithi042.stderr: 5150K .......... .......... .......... .......... .......... 3% 25.4M 7s 2024-04-20T10:35:43.679 INFO:tasks.workunit.client.0.smithi042.stderr: 5200K .......... .......... .......... .......... .......... 3% 39.7M 7s 2024-04-20T10:35:43.681 INFO:tasks.workunit.client.0.smithi042.stderr: 5250K .......... .......... .......... .......... .......... 3% 26.0M 7s 2024-04-20T10:35:43.682 INFO:tasks.workunit.client.0.smithi042.stderr: 5300K .......... .......... .......... .......... .......... 3% 30.6M 7s 2024-04-20T10:35:43.684 INFO:tasks.workunit.client.0.smithi042.stderr: 5350K .......... .......... .......... .......... .......... 3% 29.0M 7s 2024-04-20T10:35:43.686 INFO:tasks.workunit.client.0.smithi042.stderr: 5400K .......... .......... .......... .......... .......... 4% 22.4M 7s 2024-04-20T10:35:43.689 INFO:tasks.workunit.client.0.smithi042.stderr: 5450K .......... .......... .......... .......... .......... 4% 24.1M 7s 2024-04-20T10:35:43.689 INFO:tasks.workunit.client.0.smithi042.stderr: 5500K .......... .......... .......... .......... .......... 4% 44.9M 7s 2024-04-20T10:35:43.691 INFO:tasks.workunit.client.0.smithi042.stderr: 5550K .......... .......... .......... .......... .......... 4% 24.9M 7s 2024-04-20T10:35:43.692 INFO:tasks.workunit.client.0.smithi042.stderr: 5600K .......... .......... .......... .......... .......... 4% 33.9M 7s 2024-04-20T10:35:43.695 INFO:tasks.workunit.client.0.smithi042.stderr: 5650K .......... .......... .......... .......... .......... 4% 25.4M 7s 2024-04-20T10:35:43.697 INFO:tasks.workunit.client.0.smithi042.stderr: 5700K .......... .......... .......... .......... .......... 4% 25.0M 7s 2024-04-20T10:35:43.697 INFO:tasks.workunit.client.0.smithi042.stderr: 5750K .......... .......... .......... .......... .......... 4% 47.7M 7s 2024-04-20T10:35:43.699 INFO:tasks.workunit.client.0.smithi042.stderr: 5800K .......... .......... .......... .......... .......... 4% 23.8M 7s 2024-04-20T10:35:43.702 INFO:tasks.workunit.client.0.smithi042.stderr: 5850K .......... .......... .......... .......... .......... 4% 33.9M 7s 2024-04-20T10:35:43.704 INFO:tasks.workunit.client.0.smithi042.stderr: 5900K .......... .......... .......... .......... .......... 4% 22.7M 7s 2024-04-20T10:35:43.705 INFO:tasks.workunit.client.0.smithi042.stderr: 5950K .......... .......... .......... .......... .......... 4% 23.7M 7s 2024-04-20T10:35:43.706 INFO:tasks.workunit.client.0.smithi042.stderr: 6000K .......... .......... .......... .......... .......... 4% 40.5M 7s 2024-04-20T10:35:43.709 INFO:tasks.workunit.client.0.smithi042.stderr: 6050K .......... .......... .......... .......... .......... 4% 24.9M 7s 2024-04-20T10:35:43.710 INFO:tasks.workunit.client.0.smithi042.stderr: 6100K .......... .......... .......... .......... .......... 4% 32.0M 7s 2024-04-20T10:35:43.711 INFO:tasks.workunit.client.0.smithi042.stderr: 6150K .......... .......... .......... .......... .......... 4% 27.1M 7s 2024-04-20T10:35:43.714 INFO:tasks.workunit.client.0.smithi042.stderr: 6200K .......... .......... .......... .......... .......... 4% 23.5M 7s 2024-04-20T10:35:43.715 INFO:tasks.workunit.client.0.smithi042.stderr: 6250K .......... .......... .......... .......... .......... 4% 47.1M 7s 2024-04-20T10:35:43.717 INFO:tasks.workunit.client.0.smithi042.stderr: 6300K .......... .......... .......... .......... .......... 4% 22.4M 7s 2024-04-20T10:35:43.719 INFO:tasks.workunit.client.0.smithi042.stderr: 6350K .......... .......... .......... .......... .......... 4% 29.4M 7s 2024-04-20T10:35:43.720 INFO:tasks.workunit.client.0.smithi042.stderr: 6400K .......... .......... .......... .......... .......... 4% 30.9M 7s 2024-04-20T10:35:43.721 INFO:tasks.workunit.client.0.smithi042.stderr: 6450K .......... .......... .......... .......... .......... 4% 35.9M 7s 2024-04-20T10:35:43.723 INFO:tasks.workunit.client.0.smithi042.stderr: 6500K .......... .......... .......... .......... .......... 4% 29.3M 7s 2024-04-20T10:35:43.726 INFO:tasks.workunit.client.0.smithi042.stderr: 6550K .......... .......... .......... .......... .......... 4% 21.9M 7s 2024-04-20T10:35:43.727 INFO:tasks.workunit.client.0.smithi042.stderr: 6600K .......... .......... .......... .......... .......... 4% 28.3M 7s 2024-04-20T10:35:43.729 INFO:tasks.workunit.client.0.smithi042.stderr: 6650K .......... .......... .......... .......... .......... 4% 33.6M 7s 2024-04-20T10:35:43.731 INFO:tasks.workunit.client.0.smithi042.stderr: 6700K .......... .......... .......... .......... .......... 4% 25.6M 6s 2024-04-20T10:35:43.732 INFO:tasks.workunit.client.0.smithi042.stderr: 6750K .......... .......... .......... .......... .......... 5% 36.1M 6s 2024-04-20T10:35:43.734 INFO:tasks.workunit.client.0.smithi042.stderr: 6800K .......... .......... .......... .......... .......... 5% 21.6M 6s 2024-04-20T10:35:43.736 INFO:tasks.workunit.client.0.smithi042.stderr: 6850K .......... .......... .......... .......... .......... 5% 25.1M 6s 2024-04-20T10:35:43.737 INFO:tasks.workunit.client.0.smithi042.stderr: 6900K .......... .......... .......... .......... .......... 5% 37.2M 6s 2024-04-20T10:35:43.741 INFO:tasks.workunit.client.0.smithi042.stderr: 6950K .......... .......... .......... .......... .......... 5% 21.5M 6s 2024-04-20T10:35:43.741 INFO:tasks.workunit.client.0.smithi042.stderr: 7000K .......... .......... .......... .......... .......... 5% 39.1M 6s 2024-04-20T10:35:43.743 INFO:tasks.workunit.client.0.smithi042.stderr: 7050K .......... .......... .......... .......... .......... 5% 30.8M 6s 2024-04-20T10:35:43.745 INFO:tasks.workunit.client.0.smithi042.stderr: 7100K .......... .......... .......... .......... .......... 5% 24.7M 6s 2024-04-20T10:35:43.746 INFO:tasks.workunit.client.0.smithi042.stderr: 7150K .......... .......... .......... .......... .......... 5% 25.5M 6s 2024-04-20T10:35:43.749 INFO:tasks.workunit.client.0.smithi042.stderr: 7200K .......... .......... .......... .......... .......... 5% 26.7M 6s 2024-04-20T10:35:43.749 INFO:tasks.workunit.client.0.smithi042.stderr: 7250K .......... .......... .......... .......... .......... 5% 43.9M 6s 2024-04-20T10:35:43.752 INFO:tasks.workunit.client.0.smithi042.stderr: 7300K .......... .......... .......... .......... .......... 5% 26.5M 6s 2024-04-20T10:35:43.754 INFO:tasks.workunit.client.0.smithi042.stderr: 7350K .......... .......... .......... .......... .......... 5% 26.5M 6s 2024-04-20T10:35:43.756 INFO:tasks.workunit.client.0.smithi042.stderr: 7400K .......... .......... .......... .......... .......... 5% 26.7M 6s 2024-04-20T10:35:43.757 INFO:tasks.workunit.client.0.smithi042.stderr: 7450K .......... .......... .......... .......... .......... 5% 25.0M 6s 2024-04-20T10:35:43.758 INFO:tasks.workunit.client.0.smithi042.stderr: 7500K .......... .......... .......... .......... .......... 5% 50.5M 6s 2024-04-20T10:35:43.760 INFO:tasks.workunit.client.0.smithi042.stderr: 7550K .......... .......... .......... .......... .......... 5% 23.5M 6s 2024-04-20T10:35:43.762 INFO:tasks.workunit.client.0.smithi042.stderr: 7600K .......... .......... .......... .......... .......... 5% 30.7M 6s 2024-04-20T10:35:43.764 INFO:tasks.workunit.client.0.smithi042.stderr: 7650K .......... .......... .......... .......... .......... 5% 22.4M 6s 2024-04-20T10:35:43.765 INFO:tasks.workunit.client.0.smithi042.stderr: 7700K .......... .......... .......... .......... .......... 5% 27.5M 6s 2024-04-20T10:35:43.767 INFO:tasks.workunit.client.0.smithi042.stderr: 7750K .......... .......... .......... .......... .......... 5% 36.4M 6s 2024-04-20T10:35:43.768 INFO:tasks.workunit.client.0.smithi042.stderr: 7800K .......... .......... .......... .......... .......... 5% 28.9M 6s 2024-04-20T10:35:43.770 INFO:tasks.workunit.client.0.smithi042.stderr: 7850K .......... .......... .......... .......... .......... 5% 27.1M 6s 2024-04-20T10:35:43.772 INFO:tasks.workunit.client.0.smithi042.stderr: 7900K .......... .......... .......... .......... .......... 5% 30.8M 6s 2024-04-20T10:35:43.774 INFO:tasks.workunit.client.0.smithi042.stderr: 7950K .......... .......... .......... .......... .......... 5% 26.3M 6s 2024-04-20T10:35:43.775 INFO:tasks.workunit.client.0.smithi042.stderr: 8000K .......... .......... .......... .......... .......... 5% 36.3M 6s 2024-04-20T10:35:43.777 INFO:tasks.workunit.client.0.smithi042.stderr: 8050K .......... .......... .......... .......... .......... 5% 21.9M 6s 2024-04-20T10:35:43.779 INFO:tasks.workunit.client.0.smithi042.stderr: 8100K .......... .......... .......... .......... .......... 6% 32.3M 6s 2024-04-20T10:35:43.780 INFO:tasks.workunit.client.0.smithi042.stderr: 8150K .......... .......... .......... .......... .......... 6% 28.8M 6s 2024-04-20T10:35:43.782 INFO:tasks.workunit.client.0.smithi042.stderr: 8200K .......... .......... .......... .......... .......... 6% 26.8M 6s 2024-04-20T10:35:43.783 INFO:tasks.workunit.client.0.smithi042.stderr: 8250K .......... .......... .......... .......... .......... 6% 38.0M 6s 2024-04-20T10:35:43.785 INFO:tasks.workunit.client.0.smithi042.stderr: 8300K .......... .......... .......... .......... .......... 6% 27.8M 6s 2024-04-20T10:35:43.787 INFO:tasks.workunit.client.0.smithi042.stderr: 8350K .......... .......... .......... .......... .......... 6% 27.0M 6s 2024-04-20T10:35:43.789 INFO:tasks.workunit.client.0.smithi042.stderr: 8400K .......... .......... .......... .......... .......... 6% 20.9M 6s 2024-04-20T10:35:43.792 INFO:tasks.workunit.client.0.smithi042.stderr: 8450K .......... .......... .......... .......... .......... 6% 17.5M 6s 2024-04-20T10:35:43.794 INFO:tasks.workunit.client.0.smithi042.stderr: 8500K .......... .......... .......... .......... .......... 6% 36.9M 6s 2024-04-20T10:35:43.796 INFO:tasks.workunit.client.0.smithi042.stderr: 8550K .......... .......... .......... .......... .......... 6% 23.0M 6s 2024-04-20T10:35:43.798 INFO:tasks.workunit.client.0.smithi042.stderr: 8600K .......... .......... .......... .......... .......... 6% 27.1M 6s 2024-04-20T10:35:43.799 INFO:tasks.workunit.client.0.smithi042.stderr: 8650K .......... .......... .......... .......... .......... 6% 38.6M 6s 2024-04-20T10:35:43.801 INFO:tasks.workunit.client.0.smithi042.stderr: 8700K .......... .......... .......... .......... .......... 6% 25.1M 6s 2024-04-20T10:35:43.803 INFO:tasks.workunit.client.0.smithi042.stderr: 8750K .......... .......... .......... .......... .......... 6% 34.6M 6s 2024-04-20T10:35:43.804 INFO:tasks.workunit.client.0.smithi042.stderr: 8800K .......... .......... .......... .......... .......... 6% 25.2M 6s 2024-04-20T10:35:43.806 INFO:tasks.workunit.client.0.smithi042.stderr: 8850K .......... .......... .......... .......... .......... 6% 23.9M 6s 2024-04-20T10:35:43.807 INFO:tasks.workunit.client.0.smithi042.stderr: 8900K .......... .......... .......... .......... .......... 6% 35.7M 6s 2024-04-20T10:35:43.810 INFO:tasks.workunit.client.0.smithi042.stderr: 8950K .......... .......... .......... .......... .......... 6% 22.8M 6s 2024-04-20T10:35:43.811 INFO:tasks.workunit.client.0.smithi042.stderr: 9000K .......... .......... .......... .......... .......... 6% 64.3M 6s 2024-04-20T10:35:43.812 INFO:tasks.workunit.client.0.smithi042.stderr: 9050K .......... .......... .......... .......... .......... 6% 24.2M 6s 2024-04-20T10:35:43.814 INFO:tasks.workunit.client.0.smithi042.stderr: 9100K .......... .......... .......... .......... .......... 6% 24.6M 6s 2024-04-20T10:35:43.817 INFO:tasks.workunit.client.0.smithi042.stderr: 9150K .......... .......... .......... .......... .......... 6% 23.0M 6s 2024-04-20T10:35:43.819 INFO:tasks.workunit.client.0.smithi042.stderr: 9200K .......... .......... .......... .......... .......... 6% 33.8M 6s 2024-04-20T10:35:43.820 INFO:tasks.workunit.client.0.smithi042.stderr: 9250K .......... .......... .......... .......... .......... 6% 27.8M 6s 2024-04-20T10:35:43.821 INFO:tasks.workunit.client.0.smithi042.stderr: 9300K .......... .......... .......... .......... .......... 6% 32.6M 6s 2024-04-20T10:35:43.824 INFO:tasks.workunit.client.0.smithi042.stderr: 9350K .......... .......... .......... .......... .......... 6% 20.9M 6s 2024-04-20T10:35:43.826 INFO:tasks.workunit.client.0.smithi042.stderr: 9400K .......... .......... .......... .......... .......... 6% 23.8M 6s 2024-04-20T10:35:43.828 INFO:tasks.workunit.client.0.smithi042.stderr: 9450K .......... .......... .......... .......... .......... 6% 34.2M 6s 2024-04-20T10:35:43.829 INFO:tasks.workunit.client.0.smithi042.stderr: 9500K .......... .......... .......... .......... .......... 7% 28.5M 6s 2024-04-20T10:35:43.830 INFO:tasks.workunit.client.0.smithi042.stderr: 9550K .......... .......... .......... .......... .......... 7% 34.9M 6s 2024-04-20T10:35:43.832 INFO:tasks.workunit.client.0.smithi042.stderr: 9600K .......... .......... .......... .......... .......... 7% 27.2M 6s 2024-04-20T10:35:43.833 INFO:tasks.workunit.client.0.smithi042.stderr: 9650K .......... .......... .......... .......... .......... 7% 31.1M 6s 2024-04-20T10:35:43.836 INFO:tasks.workunit.client.0.smithi042.stderr: 9700K .......... .......... .......... .......... .......... 7% 25.5M 6s 2024-04-20T10:35:43.838 INFO:tasks.workunit.client.0.smithi042.stderr: 9750K .......... .......... .......... .......... .......... 7% 25.3M 6s 2024-04-20T10:35:43.838 INFO:tasks.workunit.client.0.smithi042.stderr: 9800K .......... .......... .......... .......... .......... 7% 40.2M 6s 2024-04-20T10:35:43.840 INFO:tasks.workunit.client.0.smithi042.stderr: 9850K .......... .......... .......... .......... .......... 7% 26.3M 6s 2024-04-20T10:35:43.842 INFO:tasks.workunit.client.0.smithi042.stderr: 9900K .......... .......... .......... .......... .......... 7% 29.4M 6s 2024-04-20T10:35:43.844 INFO:tasks.workunit.client.0.smithi042.stderr: 9950K .......... .......... .......... .......... .......... 7% 23.7M 6s 2024-04-20T10:35:43.847 INFO:tasks.workunit.client.0.smithi042.stderr: 10000K .......... .......... .......... .......... .......... 7% 25.3M 6s 2024-04-20T10:35:43.847 INFO:tasks.workunit.client.0.smithi042.stderr: 10050K .......... .......... .......... .......... .......... 7% 47.7M 6s 2024-04-20T10:35:43.849 INFO:tasks.workunit.client.0.smithi042.stderr: 10100K .......... .......... .......... .......... .......... 7% 27.2M 6s 2024-04-20T10:35:43.851 INFO:tasks.workunit.client.0.smithi042.stderr: 10150K .......... .......... .......... .......... .......... 7% 25.5M 6s 2024-04-20T10:35:43.852 INFO:tasks.workunit.client.0.smithi042.stderr: 10200K .......... .......... .......... .......... .......... 7% 32.8M 6s 2024-04-20T10:35:43.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:43 smithi042 ceph-mon[17078]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:43.854 INFO:tasks.workunit.client.0.smithi042.stderr: 10250K .......... .......... .......... .......... .......... 7% 40.0M 6s 2024-04-20T10:35:43.855 INFO:tasks.workunit.client.0.smithi042.stderr: 10300K .......... .......... .......... .......... .......... 7% 27.9M 6s 2024-04-20T10:35:43.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.856+0000 7ff8b3e00640 1 -- 172.21.15.42:0/2889326664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b410d3d0 msgr2=0x7ff8b410d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:43.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.856+0000 7ff8b3e00640 1 --2- 172.21.15.42:0/2889326664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b410d3d0 0x7ff8b410d7b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009860 tx=0x7ff8a402bc10 comp rx=0 tx=0).stop 2024-04-20T10:35:43.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.856+0000 7ff8b3e00640 1 -- 172.21.15.42:0/2889326664 shutdown_connections 2024-04-20T10:35:43.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.856+0000 7ff8b3e00640 1 --2- 172.21.15.42:0/2889326664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b4108ea0 0x7ff8b41152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.856+0000 7ff8b3e00640 1 --2- 172.21.15.42:0/2889326664 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8b4108500 0x7ff8b4108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.856+0000 7ff8b3e00640 1 --2- 172.21.15.42:0/2889326664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b410d3d0 0x7ff8b410d7b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.856+0000 7ff8b3e00640 1 -- 172.21.15.42:0/2889326664 >> 172.21.15.42:0/2889326664 conn(0x7ff8b4077d80 msgr2=0x7ff8b4106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:43.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.856+0000 7ff8b3e00640 1 -- 172.21.15.42:0/2889326664 shutdown_connections 2024-04-20T10:35:43.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.856+0000 7ff8b3e00640 1 -- 172.21.15.42:0/2889326664 wait complete. 2024-04-20T10:35:43.858 INFO:tasks.workunit.client.0.smithi042.stderr: 10350K .......... .......... .......... .......... .......... 7% 23.7M 6s 2024-04-20T10:35:43.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.857+0000 7ff8b3e00640 1 Processor -- start 2024-04-20T10:35:43.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.857+0000 7ff8b3e00640 1 -- start start 2024-04-20T10:35:43.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b4108500 0x7ff8b419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8b4108ea0 0x7ff8b419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b410d3d0 0x7ff8b41a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b3e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41187f0 con 0x7ff8b4108500 2024-04-20T10:35:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b4118670 con 0x7ff8b410d3d0 2024-04-20T10:35:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b3e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b4118970 con 0x7ff8b4108ea0 2024-04-20T10:35:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b4108500 0x7ff8b419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b4108500 0x7ff8b419ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54426/0 (socket says 172.21.15.42:54426) 2024-04-20T10:35:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b2a00640 1 -- 172.21.15.42:0/3460894409 learned_addr learned my addr 172.21.15.42:0/3460894409 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:43.861 INFO:tasks.workunit.client.0.smithi042.stderr: 10400K .......... .......... .......... .......... .......... 7% 35.6M 6s 2024-04-20T10:35:43.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b3400640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b410d3d0 0x7ff8b41a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b2000640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8b4108ea0 0x7ff8b419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b2a00640 1 -- 172.21.15.42:0/3460894409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8b4108ea0 msgr2=0x7ff8b419f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:43.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b2a00640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8b4108ea0 0x7ff8b419f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b2a00640 1 -- 172.21.15.42:0/3460894409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b410d3d0 msgr2=0x7ff8b41a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:43.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b2a00640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b410d3d0 0x7ff8b41a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.858+0000 7ff8b2a00640 1 -- 172.21.15.42:0/3460894409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b41a3e20 con 0x7ff8b4108500 2024-04-20T10:35:43.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.859+0000 7ff8b3400640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b410d3d0 0x7ff8b41a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:35:43.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.859+0000 7ff8b2a00640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b4108500 0x7ff8b419ee30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4037040 tx=0x7ff8a40344c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:43.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.859+0000 7ff8ab400640 1 -- 172.21.15.42:0/3460894409 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4041020 con 0x7ff8b4108500 2024-04-20T10:35:43.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.859+0000 7ff8ab400640 1 -- 172.21.15.42:0/3460894409 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a402ca90 con 0x7ff8b4108500 2024-04-20T10:35:43.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.859+0000 7ff8b3e00640 1 -- 172.21.15.42:0/3460894409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b41a40b0 con 0x7ff8b4108500 2024-04-20T10:35:43.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.859+0000 7ff8ab400640 1 -- 172.21.15.42:0/3460894409 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4039040 con 0x7ff8b4108500 2024-04-20T10:35:43.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.859+0000 7ff8b3e00640 1 -- 172.21.15.42:0/3460894409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b41ab9a0 con 0x7ff8b4108500 2024-04-20T10:35:43.867 INFO:tasks.workunit.client.0.smithi042.stderr: 10450K .......... .......... .......... .......... .......... 7% 30.8M 6s 2024-04-20T10:35:43.867 INFO:tasks.workunit.client.0.smithi042.stderr: 10500K .......... .......... .......... .......... .......... 7% 20.4M 6s 2024-04-20T10:35:43.867 INFO:tasks.workunit.client.0.smithi042.stderr: 10550K .......... .......... .......... .......... .......... 7% 48.8M 6s 2024-04-20T10:35:43.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.864+0000 7ff8bc672740 1 Processor -- start 2024-04-20T10:35:43.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.864+0000 7ff8bc672740 1 -- start start 2024-04-20T10:35:43.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.864+0000 7ff8ab400640 1 -- 172.21.15.42:0/3460894409 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff8a4044020 con 0x7ff8b4108500 2024-04-20T10:35:43.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.864+0000 7ff8bc672740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 0x5559cee61760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.864+0000 7ff8bc672740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61ca0 0x5559cee6a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.864+0000 7ff8bc672740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee6a880 0x5559cee6cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.864+0000 7ff8bc672740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5559cedc7150 con 0x5559cee6a880 2024-04-20T10:35:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.864+0000 7ff8bc672740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5559ced6d370 con 0x5559cee61380 2024-04-20T10:35:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.864+0000 7ff8bc672740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5559cedc3110 con 0x5559cee61ca0 2024-04-20T10:35:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.865+0000 7ff8b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 0x5559cee61760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.865+0000 7ff8b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 0x5559cee61760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:48682/0 (socket says 172.21.15.42:48682) 2024-04-20T10:35:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.865+0000 7ff8b3400640 1 -- 172.21.15.42:0/4163556089 learned_addr learned my addr 172.21.15.42:0/4163556089 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.865+0000 7ff8b2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee6a880 0x5559cee6cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.865+0000 7ff8b2a00640 1 --2- 172.21.15.42:0/4163556089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61ca0 0x5559cee6a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.865+0000 7ff8b3400640 1 -- 172.21.15.42:0/4163556089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61ca0 msgr2=0x5559cee6a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.865+0000 7ff8b3400640 1 --2- 172.21.15.42:0/4163556089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61ca0 0x5559cee6a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.865+0000 7ff8b3400640 1 -- 172.21.15.42:0/4163556089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee6a880 msgr2=0x5559cee6cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:43.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.865+0000 7ff8b3400640 1 --2- 172.21.15.42:0/4163556089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee6a880 0x5559cee6cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.865+0000 7ff8b3400640 1 -- 172.21.15.42:0/4163556089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ceba37d0 con 0x5559cee61380 2024-04-20T10:35:43.871 INFO:tasks.workunit.client.0.smithi042.stderr: 10600K .......... .......... .......... .......... .......... 7% 23.3M 6s 2024-04-20T10:35:43.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.866+0000 7ff8b3400640 1 --2- 172.21.15.42:0/4163556089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 0x5559cee61760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009830 tx=0x7ff8ac00f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=164aad4ba844547d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:43.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.866+0000 7ff8ab400640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff8980a3d20 0x7ff8980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.866+0000 7ff89fe00640 1 -- 172.21.15.42:0/4163556089 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac01d070 con 0x5559cee61380 2024-04-20T10:35:43.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.866+0000 7ff89fe00640 1 -- 172.21.15.42:0/4163556089 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8ac00dad0 con 0x5559cee61380 2024-04-20T10:35:43.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.866+0000 7ff89fe00640 1 -- 172.21.15.42:0/4163556089 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac018c00 con 0x5559cee61380 2024-04-20T10:35:43.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.866+0000 7ff8ab400640 1 -- 172.21.15.42:0/3460894409 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff8a40f1fb0 con 0x7ff8b4108500 2024-04-20T10:35:43.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.866+0000 7ff8b2a00640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff8980a3d20 0x7ff8980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.867+0000 7ff8bc672740 1 -- 172.21.15.42:0/4163556089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 msgr2=0x5559cee61760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:43.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.867+0000 7ff8bc672740 1 --2- 172.21.15.42:0/4163556089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 0x5559cee61760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009830 tx=0x7ff8ac00f7d0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.874 INFO:tasks.workunit.client.0.smithi042.stderr: 10650K .......... .......... .......... .......... .......... 7% 35.1M 6s 2024-04-20T10:35:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.867+0000 7ff8b2a00640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff8980a3d20 0x7ff8980a61e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5559cee60e50 tx=0x7ff8a40f5a60 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.867+0000 7ff8bc672740 1 -- 172.21.15.42:0/4163556089 shutdown_connections 2024-04-20T10:35:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.867+0000 7ff8bc672740 1 --2- 172.21.15.42:0/4163556089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee6a880 0x5559cee6cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.867+0000 7ff8bc672740 1 --2- 172.21.15.42:0/4163556089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61ca0 0x5559cee6a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.867+0000 7ff8bc672740 1 --2- 172.21.15.42:0/4163556089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 0x5559cee61760 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.867+0000 7ff8bc672740 1 -- 172.21.15.42:0/4163556089 >> 172.21.15.42:0/4163556089 conn(0x5559cee5bab0 msgr2=0x5559cee5be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.867+0000 7ff8bc672740 1 -- 172.21.15.42:0/4163556089 shutdown_connections 2024-04-20T10:35:43.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.867+0000 7ff8bc672740 1 -- 172.21.15.42:0/4163556089 wait complete. 2024-04-20T10:35:43.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.867+0000 7ff8bc672740 1 -- start start 2024-04-20T10:35:43.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.868+0000 7ff8bc672740 1 Processor -- start 2024-04-20T10:35:43.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.868+0000 7ff8bc672740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 0x5559cee67920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.868+0000 7ff8bc672740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee67e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.868+0000 7ff8bc672740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee6a880 0x5559cee683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.868+0000 7ff8bc672740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5559cee6d670 con 0x5559cee61ca0 2024-04-20T10:35:43.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.868+0000 7ff8bc672740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5559ced80740 con 0x5559cee61380 2024-04-20T10:35:43.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.868+0000 7ff8bc672740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5559cee6d7f0 con 0x5559cee6a880 2024-04-20T10:35:43.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.868+0000 7ff8bc672740 10 client.0 _ll_register_callbacks cb 0x5559ced7bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:43.878 INFO:tasks.workunit.client.0.smithi042.stderr: 10700K .......... .......... .......... .......... .......... 7% 26.3M 6s 2024-04-20T10:35:43.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff8b2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee67e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff8b2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee67e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54456/0 (socket says 172.21.15.42:54456) 2024-04-20T10:35:43.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff8b2000640 1 -- 172.21.15.42:0/2326833220 learned_addr learned my addr 172.21.15.42:0/2326833220 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:43.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff8b3400640 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 0x5559cee67920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff8b3400640 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee6a880 0x5559cee683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff8b2000640 1 -- 172.21.15.42:0/2326833220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee6a880 msgr2=0x5559cee683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff8b2000640 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee6a880 0x5559cee683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff8b2000640 1 -- 172.21.15.42:0/2326833220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 msgr2=0x5559cee67920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff8b2000640 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 0x5559cee67920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff8b2000640 1 -- 172.21.15.42:0/2326833220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559cead93f0 con 0x5559cee61ca0 2024-04-20T10:35:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff8b3400640 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 0x5559cee67920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff8b2000640 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee67e60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0012700 tx=0x7ff8a0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb77a2b968ddca33 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:43.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.869+0000 7ff89cc00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:35:43.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.870+0000 7ff89cc00640 1 -- 172.21.15.42:0/2326833220 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0004320 con 0x5559cee61ca0 2024-04-20T10:35:43.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.870+0000 7ff89cc00640 1 -- 172.21.15.42:0/2326833220 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff8a00119a0 con 0x5559cee61ca0 2024-04-20T10:35:43.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.870+0000 7ff8bc672740 1 -- 172.21.15.42:0/2326833220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ac00dad0 con 0x5559cee61ca0 2024-04-20T10:35:43.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.870+0000 7ff89cc00640 1 -- 172.21.15.42:0/2326833220 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff8a00044c0 con 0x5559cee61ca0 2024-04-20T10:35:43.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.870+0000 7ff8bc672740 10 client.15414 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:43.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.870+0000 7ff8bc672740 1 -- 172.21.15.42:0/2326833220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5559cea990f0 con 0x5559cee61ca0 2024-04-20T10:35:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.870+0000 7ff89cc00640 1 -- 172.21.15.42:0/2326833220 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8ac00dad0 con 0x5559cee61ca0 2024-04-20T10:35:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.871+0000 7ff89cc00640 1 -- 172.21.15.42:0/2326833220 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7ff8a000f440 con 0x5559cee61ca0 2024-04-20T10:35:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.871+0000 7ff8bc672740 10 client.15414 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.871+0000 7ff8bc672740 10 client.15414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.871+0000 7ff8bc672740 20 client.15414 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.871+0000 7ff8bc672740 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5559cee7ad50 0x5559cee7d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.871+0000 7ff8bc672740 4 client.15414 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:35:43.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.871+0000 7ff8bc672740 1 -- 172.21.15.42:0/2326833220 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559cedd4b70 con 0x5559cee7ad50 2024-04-20T10:35:43.884 INFO:tasks.workunit.client.0.smithi042.stderr: 10750K .......... .......... .......... .......... .......... 7% 20.8M 6s 2024-04-20T10:35:43.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.871+0000 7ff8b3400640 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5559cee7ad50 0x5559cee7d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.872+0000 7ff8b3400640 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5559cee7ad50 0x5559cee7d160 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24dbd72b26a7dc89 server_cookie=94e07f846a280e0e in_seq=0 out_seq=0 2024-04-20T10:35:43.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.872+0000 7ff89cc00640 10 client.15414 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:43.885 INFO:tasks.workunit.client.0.smithi042.stderr: 10800K .......... .......... .......... .......... .......... 7% 46.9M 6s 2024-04-20T10:35:43.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.872+0000 7ff89cc00640 1 -- 172.21.15.42:0/2326833220 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559cedd4b70 con 0x5559cee7ad50 2024-04-20T10:35:43.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.872+0000 7ff89cc00640 10 client.15414 handle_command_reply: tid=0 2024-04-20T10:35:43.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff8bc672740 1 client.15414 shutdown 2024-04-20T10:35:43.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff8bc672740 1 -- 172.21.15.42:0/2326833220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 msgr2=0x5559cee67e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:43.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff8bc672740 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee67e60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0012700 tx=0x7ff8a0012bc0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff8bc672740 1 -- 172.21.15.42:0/2326833220 shutdown_connections 2024-04-20T10:35:43.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff8bc672740 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5559cee7ad50 0x5559cee7d160 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff8bc672740 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee6a880 0x5559cee683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff89cc00640 0 client.15414 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:43.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff8bc672740 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee67e60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff8bc672740 1 --2- 172.21.15.42:0/2326833220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee61380 0x5559cee67920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff8bc672740 1 -- 172.21.15.42:0/2326833220 >> 172.21.15.42:0/2326833220 conn(0x5559cee5bab0 msgr2=0x5559cee5d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:43.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff8bc672740 1 -- 172.21.15.42:0/2326833220 shutdown_connections 2024-04-20T10:35:43.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff8bc672740 1 -- 172.21.15.42:0/2326833220 wait complete. 2024-04-20T10:35:43.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.873+0000 7ff8bc672740 20 client.15414 trim_cache size 0 max 16384 2024-04-20T10:35:43.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:43 smithi162 ceph-mon[21854]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:43.905 INFO:tasks.workunit.client.0.smithi042.stderr: 10850K .......... .......... .......... .......... .......... 8% 23.0M 6s 2024-04-20T10:35:43.905 INFO:tasks.workunit.client.0.smithi042.stderr: 10900K .......... .......... .......... .......... .......... 8% 31.7M 6s 2024-04-20T10:35:43.905 INFO:tasks.workunit.client.0.smithi042.stderr: 10950K .......... .......... .......... .......... .......... 8% 31.6M 6s 2024-04-20T10:35:43.905 INFO:tasks.workunit.client.0.smithi042.stderr: 11000K .......... .......... .......... .......... .......... 8% 20.6M 6s 2024-04-20T10:35:43.905 INFO:tasks.workunit.client.0.smithi042.stderr: 11050K .......... .......... .......... .......... .......... 8% 43.2M 6s 2024-04-20T10:35:43.905 INFO:tasks.workunit.client.0.smithi042.stderr: 11100K .......... .......... .......... .......... .......... 8% 26.5M 6s 2024-04-20T10:35:43.905 INFO:tasks.workunit.client.0.smithi042.stderr: 11150K .......... .......... .......... .......... .......... 8% 33.6M 5s 2024-04-20T10:35:43.906 INFO:tasks.workunit.client.0.smithi042.stderr: 11200K .......... .......... .......... .......... .......... 8% 24.6M 5s 2024-04-20T10:35:43.906 INFO:tasks.workunit.client.0.smithi042.stderr: 11250K .......... .......... .......... .......... .......... 8% 24.8M 5s 2024-04-20T10:35:43.906 INFO:tasks.workunit.client.0.smithi042.stderr: 11300K .......... .......... .......... .......... .......... 8% 51.7M 5s 2024-04-20T10:35:43.906 INFO:tasks.workunit.client.0.smithi042.stderr: 11350K .......... .......... .......... .......... .......... 8% 22.5M 5s 2024-04-20T10:35:43.906 INFO:tasks.workunit.client.0.smithi042.stderr: 11400K .......... .......... .......... .......... .......... 8% 39.3M 5s 2024-04-20T10:35:43.906 INFO:tasks.workunit.client.0.smithi042.stderr: 11450K .......... .......... .......... .......... .......... 8% 24.4M 5s 2024-04-20T10:35:43.906 INFO:tasks.workunit.client.0.smithi042.stderr: 11500K .......... .......... .......... .......... .......... 8% 23.5M 5s 2024-04-20T10:35:43.906 INFO:tasks.workunit.client.0.smithi042.stderr: 11550K .......... .......... .......... .......... .......... 8% 48.1M 5s 2024-04-20T10:35:43.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.897+0000 7ff8bc672740 1 Processor -- start 2024-04-20T10:35:43.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.897+0000 7ff8bc672740 1 -- start start 2024-04-20T10:35:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.898+0000 7ff8bc672740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61380 0x5559cee6a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.898+0000 7ff8bc672740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.898+0000 7ff8bc672740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee6a880 0x5559cee82a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.898+0000 7ff8bc672740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41b2160 con 0x5559cee61ca0 2024-04-20T10:35:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.898+0000 7ff8bc672740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41b49e0 con 0x5559cee6a880 2024-04-20T10:35:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.898+0000 7ff8bc672740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41b27d0 con 0x5559cee61380 2024-04-20T10:35:43.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.898+0000 7ff8b2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.898+0000 7ff8b2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee765c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54460/0 (socket says 172.21.15.42:54460) 2024-04-20T10:35:43.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.898+0000 7ff8b2000640 1 -- 172.21.15.42:0/1294866890 learned_addr learned my addr 172.21.15.42:0/1294866890 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:43.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.898+0000 7ff8b3400640 1 --2- 172.21.15.42:0/1294866890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61380 0x5559cee6a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.898+0000 7ff8b3400640 1 --2- 172.21.15.42:0/1294866890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee6a880 0x5559cee82a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.899+0000 7ff8b2000640 1 -- 172.21.15.42:0/1294866890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61380 msgr2=0x5559cee6a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:43.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.899+0000 7ff8b2000640 1 --2- 172.21.15.42:0/1294866890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61380 0x5559cee6a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.899+0000 7ff8b2000640 1 -- 172.21.15.42:0/1294866890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee6a880 msgr2=0x5559cee82a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:43.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.899+0000 7ff8b2000640 1 --2- 172.21.15.42:0/1294866890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee6a880 0x5559cee82a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.899+0000 7ff8b2000640 1 -- 172.21.15.42:0/1294866890 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559cee634d0 con 0x5559cee61ca0 2024-04-20T10:35:43.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.899+0000 7ff8b3400640 1 --2- 172.21.15.42:0/1294866890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61380 0x5559cee6a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:43.911 INFO:tasks.workunit.client.0.smithi042.stderr: 11600K .......... .......... .......... .......... .......... 8% 30.7M 5s 2024-04-20T10:35:43.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.899+0000 7ff8b2000640 1 --2- 172.21.15.42:0/1294866890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee765c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000f8e0 tx=0x7ff8a0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ee627a274eea2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:43.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.899+0000 7ff89f400640 1 -- 172.21.15.42:0/1294866890 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0032070 con 0x5559cee61ca0 2024-04-20T10:35:43.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.900+0000 7ff89f400640 1 -- 172.21.15.42:0/1294866890 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a0011e80 con 0x5559cee61ca0 2024-04-20T10:35:43.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.900+0000 7ff89f400640 1 -- 172.21.15.42:0/1294866890 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0029040 con 0x5559cee61ca0 2024-04-20T10:35:43.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.900+0000 7ff8bc672740 1 -- 172.21.15.42:0/1294866890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 msgr2=0x5559cee765c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:43.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.900+0000 7ff8bc672740 1 --2- 172.21.15.42:0/1294866890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee765c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000f8e0 tx=0x7ff8a0011810 comp rx=0 tx=0).stop 2024-04-20T10:35:43.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.900+0000 7ff8bc672740 1 -- 172.21.15.42:0/1294866890 shutdown_connections 2024-04-20T10:35:43.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.900+0000 7ff8bc672740 1 --2- 172.21.15.42:0/1294866890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee6a880 0x5559cee82a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.900+0000 7ff8bc672740 1 --2- 172.21.15.42:0/1294866890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee765c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.900+0000 7ff8bc672740 1 --2- 172.21.15.42:0/1294866890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61380 0x5559cee6a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.900+0000 7ff8bc672740 1 -- 172.21.15.42:0/1294866890 >> 172.21.15.42:0/1294866890 conn(0x5559cee5bab0 msgr2=0x5559cee661f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:43.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.900+0000 7ff8bc672740 1 -- 172.21.15.42:0/1294866890 shutdown_connections 2024-04-20T10:35:43.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.900+0000 7ff8bc672740 1 -- 172.21.15.42:0/1294866890 wait complete. 2024-04-20T10:35:43.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.901+0000 7ff8bc672740 1 -- start start 2024-04-20T10:35:43.914 INFO:tasks.workunit.client.0.smithi042.stderr: 11650K .......... .......... .......... .......... .......... 8% 23.6M 5s 2024-04-20T10:35:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.901+0000 7ff8bc672740 1 Processor -- start 2024-04-20T10:35:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8bc672740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61380 0x5559cee72b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8bc672740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee6e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8bc672740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee6a880 0x5559cee6e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8bc672740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41b2ea0 con 0x5559cee61ca0 2024-04-20T10:35:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8bc672740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41b4310 con 0x5559cee6a880 2024-04-20T10:35:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8bc672740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41b3570 con 0x5559cee61380 2024-04-20T10:35:43.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8bc672740 10 client.0 _ll_register_callbacks cb 0x7ff8a00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:43.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8b2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee6e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8b2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee6e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54472/0 (socket says 172.21.15.42:54472) 2024-04-20T10:35:43.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8b2000640 1 -- 172.21.15.42:0/85160909 learned_addr learned my addr 172.21.15.42:0/85160909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:43.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8b3400640 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61380 0x5559cee72b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8b3400640 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee6a880 0x5559cee6e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8b2000640 1 -- 172.21.15.42:0/85160909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61380 msgr2=0x5559cee72b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:43.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8b2000640 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61380 0x5559cee72b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8b2000640 1 -- 172.21.15.42:0/85160909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee6a880 msgr2=0x5559cee6e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:43.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8b2000640 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee6a880 0x5559cee6e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.902+0000 7ff8b2000640 1 -- 172.21.15.42:0/85160909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559cee6eb30 con 0x5559cee61ca0 2024-04-20T10:35:43.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.903+0000 7ff8b2000640 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee6e0b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0012c00 tx=0x7ff8a0009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874bf558fa83cd0d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:43.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.903+0000 7ff89cc00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:35:43.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.903+0000 7ff89cc00640 1 -- 172.21.15.42:0/85160909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0032070 con 0x5559cee61ca0 2024-04-20T10:35:43.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.903+0000 7ff89cc00640 1 -- 172.21.15.42:0/85160909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff8a00119a0 con 0x5559cee61ca0 2024-04-20T10:35:43.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.903+0000 7ff8bc672740 1 -- 172.21.15.42:0/85160909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5559cedce9e0 con 0x5559cee61ca0 2024-04-20T10:35:43.921 INFO:tasks.workunit.client.0.smithi042.stderr: 11700K .......... .......... .......... .......... .......... 8% 21.0M 5s 2024-04-20T10:35:43.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.903+0000 7ff89cc00640 1 -- 172.21.15.42:0/85160909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff8a000f650 con 0x5559cee61ca0 2024-04-20T10:35:43.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.903+0000 7ff8bc672740 10 client.15420 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:43.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.903+0000 7ff8bc672740 1 -- 172.21.15.42:0/85160909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5559ceee0fe0 con 0x5559cee61ca0 2024-04-20T10:35:43.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.904+0000 7ff89cc00640 1 -- 172.21.15.42:0/85160909 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5559cedce9e0 con 0x5559cee61ca0 2024-04-20T10:35:43.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.904+0000 7ff89cc00640 1 -- 172.21.15.42:0/85160909 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7ff8a0004b90 con 0x5559cee61ca0 2024-04-20T10:35:43.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.904+0000 7ff8bc672740 10 client.15420 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:43.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.904+0000 7ff8bc672740 10 client.15420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:43.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.904+0000 7ff8bc672740 20 client.15420 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:43.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.904+0000 7ff8bc672740 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5559ceee15f0 0x5559ceee3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:43.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.904+0000 7ff8bc672740 4 client.15420 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:35:43.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.904+0000 7ff8bc672740 1 -- 172.21.15.42:0/85160909 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5559ced4a450 con 0x5559ceee15f0 2024-04-20T10:35:43.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.904+0000 7ff8b3400640 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5559ceee15f0 0x5559ceee3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:43.926 INFO:tasks.workunit.client.0.smithi042.stderr: 11750K .......... .......... .......... .......... .......... 8% 33.2M 5s 2024-04-20T10:35:43.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff8b3400640 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5559ceee15f0 0x5559ceee3a10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d8200560df8a3d server_cookie=ed84122abd7cc58d in_seq=0 out_seq=0 2024-04-20T10:35:43.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff89cc00640 10 client.15420 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:43.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff89cc00640 1 -- 172.21.15.42:0/85160909 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5559ced4a450 con 0x5559ceee15f0 2024-04-20T10:35:43.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff89cc00640 10 client.15420 handle_command_reply: tid=0 2024-04-20T10:35:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff8bc672740 1 client.15420 shutdown 2024-04-20T10:35:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff8bc672740 1 -- 172.21.15.42:0/85160909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 msgr2=0x5559cee6e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff8bc672740 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee6e0b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0012c00 tx=0x7ff8a0009e20 comp rx=0 tx=0).stop 2024-04-20T10:35:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff8bc672740 1 -- 172.21.15.42:0/85160909 shutdown_connections 2024-04-20T10:35:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff8bc672740 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5559ceee15f0 0x5559ceee3a10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff8bc672740 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559cee6a880 0x5559cee6e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff8bc672740 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5559cee61ca0 0x5559cee6e0b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff8bc672740 1 --2- 172.21.15.42:0/85160909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5559cee61380 0x5559cee72b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff89cc00640 0 client.15420 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.905+0000 7ff8bc672740 1 -- 172.21.15.42:0/85160909 >> 172.21.15.42:0/85160909 conn(0x5559cee5bab0 msgr2=0x5559cee678c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:43.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.906+0000 7ff8bc672740 1 -- 172.21.15.42:0/85160909 shutdown_connections 2024-04-20T10:35:43.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.906+0000 7ff8bc672740 1 -- 172.21.15.42:0/85160909 wait complete. 2024-04-20T10:35:43.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.906+0000 7ff8bc672740 20 client.15420 trim_cache size 0 max 16384 2024-04-20T10:35:43.929 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:35:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:35:43.929 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:35:43.929 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:35:43.930 INFO:tasks.workunit.client.0.smithi042.stderr: 11800K .......... .......... .......... .......... .......... 8% 33.6M 5s 2024-04-20T10:35:43.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.906+0000 7ff8b3e00640 1 -- 172.21.15.42:0/3460894409 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff8980a3d20 msgr2=0x7ff8980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:43.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.906+0000 7ff8b3e00640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff8980a3d20 0x7ff8980a61e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5559cee60e50 tx=0x7ff8a40f5a60 comp rx=0 tx=0).stop 2024-04-20T10:35:43.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.906+0000 7ff8b3e00640 1 -- 172.21.15.42:0/3460894409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b4108500 msgr2=0x7ff8b419ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:43.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.906+0000 7ff8b3e00640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b4108500 0x7ff8b419ee30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4037040 tx=0x7ff8a40344c0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.906+0000 7ff8b3e00640 1 -- 172.21.15.42:0/3460894409 shutdown_connections 2024-04-20T10:35:43.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.906+0000 7ff8b3e00640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff8980a3d20 0x7ff8980a61e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.907+0000 7ff8b3e00640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b410d3d0 0x7ff8b41a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.907+0000 7ff8b3e00640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8b4108ea0 0x7ff8b419f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.907+0000 7ff8b3e00640 1 --2- 172.21.15.42:0/3460894409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b4108500 0x7ff8b419ee30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:43.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.907+0000 7ff8b3e00640 1 -- 172.21.15.42:0/3460894409 >> 172.21.15.42:0/3460894409 conn(0x7ff8b4077d80 msgr2=0x7ff8b4078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:43.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.907+0000 7ff8b3e00640 1 -- 172.21.15.42:0/3460894409 shutdown_connections 2024-04-20T10:35:43.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:43.907+0000 7ff8b3e00640 1 -- 172.21.15.42:0/3460894409 wait complete. 2024-04-20T10:35:43.941 INFO:tasks.workunit.client.0.smithi042.stderr: 11850K .......... .......... .......... .......... .......... 8% 26.2M 5s 2024-04-20T10:35:43.941 INFO:tasks.workunit.client.0.smithi042.stderr: 11900K .......... .......... .......... .......... .......... 8% 28.9M 5s 2024-04-20T10:35:43.941 INFO:tasks.workunit.client.0.smithi042.stderr: 11950K .......... .......... .......... .......... .......... 8% 23.7M 5s 2024-04-20T10:35:43.941 INFO:tasks.workunit.client.0.smithi042.stderr: 12000K .......... .......... .......... .......... .......... 8% 27.3M 5s 2024-04-20T10:35:43.941 INFO:tasks.workunit.client.0.smithi042.stderr: 12050K .......... .......... .......... .......... .......... 8% 38.0M 5s 2024-04-20T10:35:43.941 INFO:tasks.workunit.client.0.smithi042.stderr: 12100K .......... .......... .......... .......... .......... 8% 25.0M 5s 2024-04-20T10:35:43.941 INFO:tasks.workunit.client.0.smithi042.stderr: 12150K .......... .......... .......... .......... .......... 8% 26.5M 5s 2024-04-20T10:35:43.941 INFO:tasks.workunit.client.0.smithi042.stderr: 12200K .......... .......... .......... .......... .......... 9% 28.7M 5s 2024-04-20T10:35:43.941 INFO:tasks.workunit.client.0.smithi042.stderr: 12250K .......... .......... .......... .......... .......... 9% 27.5M 5s 2024-04-20T10:35:43.941 INFO:tasks.workunit.client.0.smithi042.stderr: 12300K .......... .......... .......... .......... .......... 9% 45.8M 5s 2024-04-20T10:35:43.942 INFO:tasks.workunit.client.0.smithi042.stderr: 12350K .......... .......... .......... .......... .......... 9% 26.9M 5s 2024-04-20T10:35:43.943 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:35:43.943 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 12400K .......... .......... .......... .......... .......... 9% 27.8M 5s 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 12450K .......... .......... .......... .......... .......... 9% 22.8M 5s 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 12500K .......... .......... .......... .......... .......... 9% 25.9M 5s 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 12550K .......... .......... .......... .......... .......... 9% 48.8M 5s 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 12600K .......... .......... .......... .......... .......... 9% 26.8M 5s 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 12650K .......... .......... .......... .......... .......... 9% 27.6M 5s 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 12700K .......... .......... .......... .......... .......... 9% 26.9M 5s 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 12750K .......... .......... .......... .......... .......... 9% 25.7M 5s 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 12800K .......... .......... .......... .......... .......... 9% 44.1M 5s 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 12850K .......... .......... .......... .......... .......... 9% 24.8M 5s 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 12900K .......... .......... .......... .......... .......... 9% 25.6M 5s 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 12950K .......... .......... .......... .......... .......... 9% 35.5M 5s 2024-04-20T10:35:43.956 INFO:tasks.workunit.client.0.smithi042.stderr: 13000K .......... .......... .......... .......... .......... 9% 22.2M 5s 2024-04-20T10:35:43.957 INFO:tasks.workunit.client.0.smithi042.stderr: 13050K .......... .......... .......... .......... .......... 9% 44.1M 5s 2024-04-20T10:35:43.957 INFO:tasks.workunit.client.0.smithi042.stderr: 13100K .......... .......... .......... .......... .......... 9% 24.5M 5s 2024-04-20T10:35:43.960 INFO:tasks.workunit.client.0.smithi042.stderr: 13150K .......... .......... .......... .......... .......... 9% 33.5M 5s 2024-04-20T10:35:43.960 INFO:tasks.workunit.client.0.smithi042.stderr: 13200K .......... .......... .......... .......... .......... 9% 22.4M 5s 2024-04-20T10:35:43.960 INFO:tasks.workunit.client.0.smithi042.stderr: 13250K .......... .......... .......... .......... .......... 9% 14.8M 5s 2024-04-20T10:35:43.960 INFO:tasks.workunit.client.0.smithi042.stderr: 13300K .......... .......... .......... .......... .......... 9% 35.9M 5s 2024-04-20T10:35:43.961 INFO:tasks.workunit.client.0.smithi042.stderr: 13350K .......... .......... .......... .......... .......... 9% 26.7M 5s 2024-04-20T10:35:43.964 INFO:tasks.workunit.client.0.smithi042.stderr: 13400K .......... .......... .......... .......... .......... 9% 23.9M 5s 2024-04-20T10:35:43.965 INFO:tasks.workunit.client.0.smithi042.stderr: 13450K .......... .......... .......... .......... .......... 9% 34.8M 5s 2024-04-20T10:35:43.968 INFO:tasks.workunit.client.0.smithi042.stderr: 13500K .......... .......... .......... .......... .......... 9% 24.5M 5s 2024-04-20T10:35:43.968 INFO:tasks.workunit.client.0.smithi042.stderr: 13550K .......... .......... .......... .......... .......... 10% 61.1M 5s 2024-04-20T10:35:43.969 INFO:tasks.workunit.client.0.smithi042.stderr: 13600K .......... .......... .......... .......... .......... 10% 29.2M 5s 2024-04-20T10:35:43.971 INFO:tasks.workunit.client.0.smithi042.stderr: 13650K .......... .......... .......... .......... .......... 10% 26.4M 5s 2024-04-20T10:35:43.973 INFO:tasks.workunit.client.0.smithi042.stderr: 13700K .......... .......... .......... .......... .......... 10% 24.7M 5s 2024-04-20T10:35:43.975 INFO:tasks.workunit.client.0.smithi042.stderr: 13750K .......... .......... .......... .......... .......... 10% 24.3M 5s 2024-04-20T10:35:43.977 INFO:tasks.workunit.client.0.smithi042.stderr: 13800K .......... .......... .......... .......... .......... 10% 40.8M 5s 2024-04-20T10:35:43.979 INFO:tasks.workunit.client.0.smithi042.stderr: 13850K .......... .......... .......... .......... .......... 10% 24.8M 5s 2024-04-20T10:35:43.980 INFO:tasks.workunit.client.0.smithi042.stderr: 13900K .......... .......... .......... .......... .......... 10% 25.8M 5s 2024-04-20T10:35:43.982 INFO:tasks.workunit.client.0.smithi042.stderr: 13950K .......... .......... .......... .......... .......... 10% 34.0M 5s 2024-04-20T10:35:43.984 INFO:tasks.workunit.client.0.smithi042.stderr: 14000K .......... .......... .......... .......... .......... 10% 25.3M 5s 2024-04-20T10:35:43.986 INFO:tasks.workunit.client.0.smithi042.stderr: 14050K .......... .......... .......... .......... .......... 10% 47.8M 5s 2024-04-20T10:35:43.987 INFO:tasks.workunit.client.0.smithi042.stderr: 14100K .......... .......... .......... .......... .......... 10% 24.2M 5s 2024-04-20T10:35:43.989 INFO:tasks.workunit.client.0.smithi042.stderr: 14150K .......... .......... .......... .......... .......... 10% 21.8M 5s 2024-04-20T10:35:43.992 INFO:tasks.workunit.client.0.smithi042.stderr: 14200K .......... .......... .......... .......... .......... 10% 21.4M 5s 2024-04-20T10:35:43.994 INFO:tasks.workunit.client.0.smithi042.stderr: 14250K .......... .......... .......... .......... .......... 10% 27.7M 5s 2024-04-20T10:35:43.995 INFO:tasks.workunit.client.0.smithi042.stderr: 14300K .......... .......... .......... .......... .......... 10% 65.4M 5s 2024-04-20T10:35:43.996 INFO:tasks.workunit.client.0.smithi042.stderr: 14350K .......... .......... .......... .......... .......... 10% 22.8M 5s 2024-04-20T10:35:43.998 INFO:tasks.workunit.client.0.smithi042.stderr: 14400K .......... .......... .......... .......... .......... 10% 23.9M 5s 2024-04-20T10:35:44.000 INFO:tasks.workunit.client.0.smithi042.stderr: 14450K .......... .......... .......... .......... .......... 10% 27.3M 5s 2024-04-20T10:35:44.002 INFO:tasks.workunit.client.0.smithi042.stderr: 14500K .......... .......... .......... .......... .......... 10% 31.1M 5s 2024-04-20T10:35:44.004 INFO:tasks.workunit.client.0.smithi042.stderr: 14550K .......... .......... .......... .......... .......... 10% 24.8M 5s 2024-04-20T10:35:44.004 INFO:tasks.workunit.client.0.smithi042.stderr: 14600K .......... .......... .......... .......... .......... 10% 51.4M 5s 2024-04-20T10:35:44.006 INFO:tasks.workunit.client.0.smithi042.stderr: 14650K .......... .......... .......... .......... .......... 10% 24.5M 5s 2024-04-20T10:35:44.008 INFO:tasks.workunit.client.0.smithi042.stderr: 14700K .......... .......... .......... .......... .......... 10% 27.8M 5s 2024-04-20T10:35:44.010 INFO:tasks.workunit.client.0.smithi042.stderr: 14750K .......... .......... .......... .......... .......... 10% 28.2M 5s 2024-04-20T10:35:44.012 INFO:tasks.workunit.client.0.smithi042.stderr: 14800K .......... .......... .......... .......... .......... 10% 25.2M 5s 2024-04-20T10:35:44.013 INFO:tasks.workunit.client.0.smithi042.stderr: 14850K .......... .......... .......... .......... .......... 10% 40.0M 5s 2024-04-20T10:35:44.015 INFO:tasks.workunit.client.0.smithi042.stderr: 14900K .......... .......... .......... .......... .......... 11% 23.7M 5s 2024-04-20T10:35:44.017 INFO:tasks.workunit.client.0.smithi042.stderr: 14950K .......... .......... .......... .......... .......... 11% 37.6M 5s 2024-04-20T10:35:44.018 INFO:tasks.workunit.client.0.smithi042.stderr: 15000K .......... .......... .......... .......... .......... 11% 24.5M 5s 2024-04-20T10:35:44.020 INFO:tasks.workunit.client.0.smithi042.stderr: 15050K .......... .......... .......... .......... .......... 11% 24.4M 5s 2024-04-20T10:35:44.021 INFO:tasks.workunit.client.0.smithi042.stderr: 15100K .......... .......... .......... .......... .......... 11% 59.9M 5s 2024-04-20T10:35:44.023 INFO:tasks.workunit.client.0.smithi042.stderr: 15150K .......... .......... .......... .......... .......... 11% 20.8M 5s 2024-04-20T10:35:44.025 INFO:tasks.workunit.client.0.smithi042.stderr: 15200K .......... .......... .......... .......... .......... 11% 35.7M 5s 2024-04-20T10:35:44.027 INFO:tasks.workunit.client.0.smithi042.stderr: 15250K .......... .......... .......... .......... .......... 11% 24.3M 5s 2024-04-20T10:35:44.029 INFO:tasks.workunit.client.0.smithi042.stderr: 15300K .......... .......... .......... .......... .......... 11% 23.3M 5s 2024-04-20T10:35:44.030 INFO:tasks.workunit.client.0.smithi042.stderr: 15350K .......... .......... .......... .......... .......... 11% 45.6M 5s 2024-04-20T10:35:44.032 INFO:tasks.workunit.client.0.smithi042.stderr: 15400K .......... .......... .......... .......... .......... 11% 24.3M 5s 2024-04-20T10:35:44.033 INFO:tasks.workunit.client.0.smithi042.stderr: 15450K .......... .......... .......... .......... .......... 11% 31.9M 5s 2024-04-20T10:35:44.036 INFO:tasks.workunit.client.0.smithi042.stderr: 15500K .......... .......... .......... .......... .......... 11% 24.7M 5s 2024-04-20T10:35:44.037 INFO:tasks.workunit.client.0.smithi042.stderr: 15550K .......... .......... .......... .......... .......... 11% 36.6M 5s 2024-04-20T10:35:44.038 INFO:tasks.workunit.client.0.smithi042.stderr: 15600K .......... .......... .......... .......... .......... 11% 29.5M 5s 2024-04-20T10:35:44.040 INFO:tasks.workunit.client.0.smithi042.stderr: 15650K .......... .......... .......... .......... .......... 11% 24.0M 5s 2024-04-20T10:35:44.042 INFO:tasks.workunit.client.0.smithi042.stderr: 15700K .......... .......... .......... .......... .......... 11% 36.9M 5s 2024-04-20T10:35:44.043 INFO:tasks.workunit.client.0.smithi042.stderr: 15750K .......... .......... .......... .......... .......... 11% 28.4M 5s 2024-04-20T10:35:44.046 INFO:tasks.workunit.client.0.smithi042.stderr: 15800K .......... .......... .......... .......... .......... 11% 20.2M 5s 2024-04-20T10:35:44.047 INFO:tasks.workunit.client.0.smithi042.stderr: 15850K .......... .......... .......... .......... .......... 11% 48.7M 5s 2024-04-20T10:35:44.049 INFO:tasks.workunit.client.0.smithi042.stderr: 15900K .......... .......... .......... .......... .......... 11% 23.4M 5s 2024-04-20T10:35:44.050 INFO:tasks.workunit.client.0.smithi042.stderr: 15950K .......... .......... .......... .......... .......... 11% 32.0M 5s 2024-04-20T10:35:44.052 INFO:tasks.workunit.client.0.smithi042.stderr: 16000K .......... .......... .......... .......... .......... 11% 26.6M 5s 2024-04-20T10:35:44.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:43 smithi046 ceph-mon[21574]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-20T10:35:44.054 INFO:tasks.workunit.client.0.smithi042.stderr: 16050K .......... .......... .......... .......... .......... 11% 24.4M 5s 2024-04-20T10:35:44.055 INFO:tasks.workunit.client.0.smithi042.stderr: 16100K .......... .......... .......... .......... .......... 11% 45.5M 5s 2024-04-20T10:35:44.057 INFO:tasks.workunit.client.0.smithi042.stderr: 16150K .......... .......... .......... .......... .......... 11% 26.8M 5s 2024-04-20T10:35:44.059 INFO:tasks.workunit.client.0.smithi042.stderr: 16200K .......... .......... .......... .......... .......... 11% 30.0M 5s 2024-04-20T10:35:44.061 INFO:tasks.workunit.client.0.smithi042.stderr: 16250K .......... .......... .......... .......... .......... 12% 23.3M 5s 2024-04-20T10:35:44.063 INFO:tasks.workunit.client.0.smithi042.stderr: 16300K .......... .......... .......... .......... .......... 12% 27.0M 5s 2024-04-20T10:35:44.064 INFO:tasks.workunit.client.0.smithi042.stderr: 16350K .......... .......... .......... .......... .......... 12% 42.1M 5s 2024-04-20T10:35:44.066 INFO:tasks.workunit.client.0.smithi042.stderr: 16400K .......... .......... .......... .......... .......... 12% 26.7M 5s 2024-04-20T10:35:44.068 INFO:tasks.workunit.client.0.smithi042.stderr: 16450K .......... .......... .......... .......... .......... 12% 28.5M 5s 2024-04-20T10:35:44.069 INFO:tasks.workunit.client.0.smithi042.stderr: 16500K .......... .......... .......... .......... .......... 12% 25.8M 5s 2024-04-20T10:35:44.072 INFO:tasks.workunit.client.0.smithi042.stderr: 16550K .......... .......... .......... .......... .......... 12% 21.4M 5s 2024-04-20T10:35:44.073 INFO:tasks.workunit.client.0.smithi042.stderr: 16600K .......... .......... .......... .......... .......... 12% 42.6M 5s 2024-04-20T10:35:44.075 INFO:tasks.workunit.client.0.smithi042.stderr: 16650K .......... .......... .......... .......... .......... 12% 24.6M 5s 2024-04-20T10:35:44.076 INFO:tasks.workunit.client.0.smithi042.stderr: 16700K .......... .......... .......... .......... .......... 12% 29.4M 5s 2024-04-20T10:35:44.080 INFO:tasks.workunit.client.0.smithi042.stderr: 16750K .......... .......... .......... .......... .......... 12% 21.1M 5s 2024-04-20T10:35:44.080 INFO:tasks.workunit.client.0.smithi042.stderr: 16800K .......... .......... .......... .......... .......... 12% 52.8M 5s 2024-04-20T10:35:44.082 INFO:tasks.workunit.client.0.smithi042.stderr: 16850K .......... .......... .......... .......... .......... 12% 23.3M 5s 2024-04-20T10:35:44.084 INFO:tasks.workunit.client.0.smithi042.stderr: 16900K .......... .......... .......... .......... .......... 12% 22.5M 5s 2024-04-20T10:35:44.087 INFO:tasks.workunit.client.0.smithi042.stderr: 16950K .......... .......... .......... .......... .......... 12% 26.2M 5s 2024-04-20T10:35:44.087 INFO:tasks.workunit.client.0.smithi042.stderr: 17000K .......... .......... .......... .......... .......... 12% 50.2M 5s 2024-04-20T10:35:44.089 INFO:tasks.workunit.client.0.smithi042.stderr: 17050K .......... .......... .......... .......... .......... 12% 22.5M 5s 2024-04-20T10:35:44.091 INFO:tasks.workunit.client.0.smithi042.stderr: 17100K .......... .......... .......... .......... .......... 12% 29.7M 5s 2024-04-20T10:35:44.093 INFO:tasks.workunit.client.0.smithi042.stderr: 17150K .......... .......... .......... .......... .......... 12% 26.8M 5s 2024-04-20T10:35:44.095 INFO:tasks.workunit.client.0.smithi042.stderr: 17200K .......... .......... .......... .......... .......... 12% 31.5M 5s 2024-04-20T10:35:44.096 INFO:tasks.workunit.client.0.smithi042.stderr: 17250K .......... .......... .......... .......... .......... 12% 27.5M 5s 2024-04-20T10:35:44.099 INFO:tasks.workunit.client.0.smithi042.stderr: 17300K .......... .......... .......... .......... .......... 12% 23.6M 5s 2024-04-20T10:35:44.099 INFO:tasks.workunit.client.0.smithi042.stderr: 17350K .......... .......... .......... .......... .......... 12% 48.1M 5s 2024-04-20T10:35:44.101 INFO:tasks.workunit.client.0.smithi042.stderr: 17400K .......... .......... .......... .......... .......... 12% 23.3M 5s 2024-04-20T10:35:44.103 INFO:tasks.workunit.client.0.smithi042.stderr: 17450K .......... .......... .......... .......... .......... 12% 31.4M 5s 2024-04-20T10:35:44.105 INFO:tasks.workunit.client.0.smithi042.stderr: 17500K .......... .......... .......... .......... .......... 12% 25.7M 5s 2024-04-20T10:35:44.106 INFO:tasks.workunit.client.0.smithi042.stderr: 17550K .......... .......... .......... .......... .......... 12% 27.3M 5s 2024-04-20T10:35:44.107 INFO:tasks.workunit.client.0.smithi042.stderr: 17600K .......... .......... .......... .......... .......... 13% 38.5M 5s 2024-04-20T10:35:44.109 INFO:tasks.workunit.client.0.smithi042.stderr: 17650K .......... .......... .......... .......... .......... 13% 27.2M 5s 2024-04-20T10:35:44.111 INFO:tasks.workunit.client.0.smithi042.stderr: 17700K .......... .......... .......... .......... .......... 13% 25.5M 5s 2024-04-20T10:35:44.114 INFO:tasks.workunit.client.0.smithi042.stderr: 17750K .......... .......... .......... .......... .......... 13% 22.6M 5s 2024-04-20T10:35:44.116 INFO:tasks.workunit.client.0.smithi042.stderr: 17800K .......... .......... .......... .......... .......... 13% 30.4M 5s 2024-04-20T10:35:44.116 INFO:tasks.workunit.client.0.smithi042.stderr: 17850K .......... .......... .......... .......... .......... 13% 51.3M 5s 2024-04-20T10:35:44.118 INFO:tasks.workunit.client.0.smithi042.stderr: 17900K .......... .......... .......... .......... .......... 13% 24.2M 5s 2024-04-20T10:35:44.120 INFO:tasks.workunit.client.0.smithi042.stderr: 17950K .......... .......... .......... .......... .......... 13% 30.6M 5s 2024-04-20T10:35:44.123 INFO:tasks.workunit.client.0.smithi042.stderr: 18000K .......... .......... .......... .......... .......... 13% 26.0M 5s 2024-04-20T10:35:44.125 INFO:tasks.workunit.client.0.smithi042.stderr: 18050K .......... .......... .......... .......... .......... 13% 24.5M 5s 2024-04-20T10:35:44.126 INFO:tasks.workunit.client.0.smithi042.stderr: 18100K .......... .......... .......... .......... .......... 13% 47.5M 5s 2024-04-20T10:35:44.127 INFO:tasks.workunit.client.0.smithi042.stderr: 18150K .......... .......... .......... .......... .......... 13% 21.9M 5s 2024-04-20T10:35:44.135 INFO:tasks.workunit.client.0.smithi042.stderr: 18200K .......... .......... .......... .......... .......... 13% 9.92M 5s 2024-04-20T10:35:44.135 INFO:tasks.workunit.client.0.smithi042.stderr: 18250K .......... .......... .......... .......... .......... 13% 60.9M 5s 2024-04-20T10:35:44.135 INFO:tasks.workunit.client.0.smithi042.stderr: 18300K .......... .......... .......... .......... .......... 13% 40.1M 5s 2024-04-20T10:35:44.135 INFO:tasks.workunit.client.0.smithi042.stderr: 18350K .......... .......... .......... .......... .......... 13% 37.1M 5s 2024-04-20T10:35:44.137 INFO:tasks.workunit.client.0.smithi042.stderr: 18400K .......... .......... .......... .......... .......... 13% 25.9M 5s 2024-04-20T10:35:44.139 INFO:tasks.workunit.client.0.smithi042.stderr: 18450K .......... .......... .......... .......... .......... 13% 29.9M 5s 2024-04-20T10:35:44.141 INFO:tasks.workunit.client.0.smithi042.stderr: 18500K .......... .......... .......... .......... .......... 13% 25.8M 5s 2024-04-20T10:35:44.143 INFO:tasks.workunit.client.0.smithi042.stderr: 18550K .......... .......... .......... .......... .......... 13% 24.4M 5s 2024-04-20T10:35:44.143 INFO:tasks.workunit.client.0.smithi042.stderr: 18600K .......... .......... .......... .......... .......... 13% 46.1M 5s 2024-04-20T10:35:44.145 INFO:tasks.workunit.client.0.smithi042.stderr: 18650K .......... .......... .......... .......... .......... 13% 23.3M 5s 2024-04-20T10:35:44.147 INFO:tasks.workunit.client.0.smithi042.stderr: 18700K .......... .......... .......... .......... .......... 13% 28.4M 5s 2024-04-20T10:35:44.150 INFO:tasks.workunit.client.0.smithi042.stderr: 18750K .......... .......... .......... .......... .......... 13% 33.5M 5s 2024-04-20T10:35:44.152 INFO:tasks.workunit.client.0.smithi042.stderr: 18800K .......... .......... .......... .......... .......... 13% 23.8M 5s 2024-04-20T10:35:44.152 INFO:tasks.workunit.client.0.smithi042.stderr: 18850K .......... .......... .......... .......... .......... 13% 48.8M 5s 2024-04-20T10:35:44.154 INFO:tasks.workunit.client.0.smithi042.stderr: 18900K .......... .......... .......... .......... .......... 13% 25.4M 5s 2024-04-20T10:35:44.156 INFO:tasks.workunit.client.0.smithi042.stderr: 18950K .......... .......... .......... .......... .......... 13% 24.1M 5s 2024-04-20T10:35:44.158 INFO:tasks.workunit.client.0.smithi042.stderr: 19000K .......... .......... .......... .......... .......... 14% 31.7M 5s 2024-04-20T10:35:44.160 INFO:tasks.workunit.client.0.smithi042.stderr: 19050K .......... .......... .......... .......... .......... 14% 24.8M 5s 2024-04-20T10:35:44.160 INFO:tasks.workunit.client.0.smithi042.stderr: 19100K .......... .......... .......... .......... .......... 14% 49.0M 5s 2024-04-20T10:35:44.162 INFO:tasks.workunit.client.0.smithi042.stderr: 19150K .......... .......... .......... .......... .......... 14% 23.7M 5s 2024-04-20T10:35:44.164 INFO:tasks.workunit.client.0.smithi042.stderr: 19200K .......... .......... .......... .......... .......... 14% 25.3M 5s 2024-04-20T10:35:44.166 INFO:tasks.workunit.client.0.smithi042.stderr: 19250K .......... .......... .......... .......... .......... 14% 31.5M 5s 2024-04-20T10:35:44.168 INFO:tasks.workunit.client.0.smithi042.stderr: 19300K .......... .......... .......... .......... .......... 14% 19.6M 5s 2024-04-20T10:35:44.173 INFO:tasks.workunit.client.0.smithi042.stderr: 19350K .......... .......... .......... .......... .......... 14% 41.1M 5s 2024-04-20T10:35:44.174 INFO:tasks.workunit.client.0.smithi042.stderr: 19400K .......... .......... .......... .......... .......... 14% 25.9M 5s 2024-04-20T10:35:44.174 INFO:tasks.workunit.client.0.smithi042.stderr: 19450K .......... .......... .......... .......... .......... 14% 25.2M 5s 2024-04-20T10:35:44.178 INFO:tasks.workunit.client.0.smithi042.stderr: 19500K .......... .......... .......... .......... .......... 14% 33.4M 5s 2024-04-20T10:35:44.178 INFO:tasks.workunit.client.0.smithi042.stderr: 19550K .......... .......... .......... .......... .......... 14% 24.1M 5s 2024-04-20T10:35:44.180 INFO:tasks.workunit.client.0.smithi042.stderr: 19600K .......... .......... .......... .......... .......... 14% 25.8M 5s 2024-04-20T10:35:44.180 INFO:tasks.workunit.client.0.smithi042.stderr: 19650K .......... .......... .......... .......... .......... 14% 41.7M 5s 2024-04-20T10:35:44.184 INFO:tasks.workunit.client.0.smithi042.stderr: 19700K .......... .......... .......... .......... .......... 14% 25.6M 5s 2024-04-20T10:35:44.184 INFO:tasks.workunit.client.0.smithi042.stderr: 19750K .......... .......... .......... .......... .......... 14% 29.7M 5s 2024-04-20T10:35:44.186 INFO:tasks.workunit.client.0.smithi042.stderr: 19800K .......... .......... .......... .......... .......... 14% 27.5M 5s 2024-04-20T10:35:44.187 INFO:tasks.workunit.client.0.smithi042.stderr: 19850K .......... .......... .......... .......... .......... 14% 25.2M 5s 2024-04-20T10:35:44.188 INFO:tasks.workunit.client.0.smithi042.stderr: 19900K .......... .......... .......... .......... .......... 14% 49.0M 5s 2024-04-20T10:35:44.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.188+0000 7f0b55a00640 1 -- 172.21.15.42:0/943374771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50107080 msgr2=0x7f0b5010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.188+0000 7f0b55a00640 1 --2- 172.21.15.42:0/943374771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50107080 0x7f0b5010ec50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009ca0 tx=0x7f0b4802c0f0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.188+0000 7f0b55a00640 1 -- 172.21.15.42:0/943374771 shutdown_connections 2024-04-20T10:35:44.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.188+0000 7f0b55a00640 1 --2- 172.21.15.42:0/943374771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50107080 0x7f0b5010ec50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.188+0000 7f0b55a00640 1 --2- 172.21.15.42:0/943374771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b50102960 0x7f0b501069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.188+0000 7f0b55a00640 1 --2- 172.21.15.42:0/943374771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b50101fb0 0x7f0b50102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.188+0000 7f0b55a00640 1 -- 172.21.15.42:0/943374771 >> 172.21.15.42:0/943374771 conn(0x7f0b500fd890 msgr2=0x7f0b500ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:44.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.188+0000 7f0b55a00640 1 -- 172.21.15.42:0/943374771 shutdown_connections 2024-04-20T10:35:44.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.188+0000 7f0b55a00640 1 -- 172.21.15.42:0/943374771 wait complete. 2024-04-20T10:35:44.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.189+0000 7f0b55a00640 1 Processor -- start 2024-04-20T10:35:44.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.189+0000 7f0b55a00640 1 -- start start 2024-04-20T10:35:44.191 INFO:tasks.workunit.client.0.smithi042.stderr: 19950K .......... .......... .......... .......... .......... 14% 24.5M 5s 2024-04-20T10:35:44.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.190+0000 7f0b55a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b50101fb0 0x7f0b501a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.190+0000 7f0b55a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50107080 0x7f0b501a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.190+0000 7f0b55a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b5019a620 0x7f0b5019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.190+0000 7f0b55a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0b50112140 con 0x7f0b50107080 2024-04-20T10:35:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.190+0000 7f0b55a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b50111fc0 con 0x7f0b5019a620 2024-04-20T10:35:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.190+0000 7f0b55a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0b501122c0 con 0x7f0b50101fb0 2024-04-20T10:35:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.190+0000 7f0b4f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50107080 0x7f0b501a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.190+0000 7f0b4f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50107080 0x7f0b501a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54502/0 (socket says 172.21.15.42:54502) 2024-04-20T10:35:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.190+0000 7f0b4f400640 1 -- 172.21.15.42:0/506951028 learned_addr learned my addr 172.21.15.42:0/506951028 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.190+0000 7f0b55000640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b5019a620 0x7f0b5019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.190+0000 7f0b4fe00640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b50101fb0 0x7f0b501a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.191+0000 7f0b4f400640 1 -- 172.21.15.42:0/506951028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b50101fb0 msgr2=0x7f0b501a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.191+0000 7f0b4f400640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b50101fb0 0x7f0b501a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.191+0000 7f0b4f400640 1 -- 172.21.15.42:0/506951028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b5019a620 msgr2=0x7f0b5019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.191+0000 7f0b4f400640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b5019a620 0x7f0b5019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.191+0000 7f0b4f400640 1 -- 172.21.15.42:0/506951028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5019b140 con 0x7f0b50107080 2024-04-20T10:35:44.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.191+0000 7f0b55000640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b5019a620 0x7f0b5019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:44.195 INFO:tasks.workunit.client.0.smithi042.stderr: 20000K .......... .......... .......... .......... .......... 14% 33.3M 5s 2024-04-20T10:35:44.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.191+0000 7f0b4f400640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50107080 0x7f0b501a09b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00e9f0 tx=0x7f0b3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.191+0000 7f0b4cc00640 1 -- 172.21.15.42:0/506951028 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00ccc0 con 0x7f0b50107080 2024-04-20T10:35:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.191+0000 7f0b55a00640 1 -- 172.21.15.42:0/506951028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5019b430 con 0x7f0b50107080 2024-04-20T10:35:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.191+0000 7f0b4cc00640 1 -- 172.21.15.42:0/506951028 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b3c005500 con 0x7f0b50107080 2024-04-20T10:35:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.191+0000 7f0b4cc00640 1 -- 172.21.15.42:0/506951028 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c010690 con 0x7f0b50107080 2024-04-20T10:35:44.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.192+0000 7f0b55a00640 1 -- 172.21.15.42:0/506951028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b501a7630 con 0x7f0b50107080 2024-04-20T10:35:44.197 INFO:tasks.workunit.client.0.smithi042.stderr: 20050K .......... .......... .......... .......... .......... 14% 26.0M 5s 2024-04-20T10:35:44.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.193+0000 7f0b55a00640 1 -- 172.21.15.42:0/506951028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b500690f0 con 0x7f0b50107080 2024-04-20T10:35:44.198 INFO:tasks.workunit.client.0.smithi042.stderr: 20100K .......... .......... .......... .......... .......... 14% 23.4M 5s 2024-04-20T10:35:44.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.196+0000 7f0b4cc00640 1 -- 172.21.15.42:0/506951028 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0b3c0046c0 con 0x7f0b50107080 2024-04-20T10:35:44.200 INFO:tasks.workunit.client.0.smithi042.stderr: 20150K .......... .......... .......... .......... .......... 14% 50.5M 5s 2024-04-20T10:35:44.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.197+0000 7f0b4cc00640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0b340a3d20 0x7f0b340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.198+0000 7f0b4cc00640 1 -- 172.21.15.42:0/506951028 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0b3c0d0b20 con 0x7f0b50107080 2024-04-20T10:35:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.198+0000 7f0b4fe00640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0b340a3d20 0x7f0b340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.202 INFO:tasks.workunit.client.0.smithi042.stderr: 20200K .......... .......... .......... .......... .......... 14% 24.1M 5s 2024-04-20T10:35:44.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.198+0000 7f0b4fe00640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0b340a3d20 0x7f0b340a61e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009bd0 tx=0x7f0b40005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:44.205 INFO:tasks.workunit.client.0.smithi042.stderr: 20250K .......... .......... .......... .......... .......... 14% 29.6M 5s 2024-04-20T10:35:44.206 INFO:tasks.workunit.client.0.smithi042.stderr: 20300K .......... .......... .......... .......... .......... 14% 24.0M 5s 2024-04-20T10:35:44.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.203+0000 7f0b4cc00640 1 -- 172.21.15.42:0/506951028 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0b3c09d160 con 0x7f0b50107080 2024-04-20T10:35:44.223 INFO:tasks.workunit.client.0.smithi042.stderr: 20350K .......... .......... .......... .......... .......... 15% 25.9M 5s 2024-04-20T10:35:44.223 INFO:tasks.workunit.client.0.smithi042.stderr: 20400K .......... .......... .......... .......... .......... 15% 42.6M 5s 2024-04-20T10:35:44.224 INFO:tasks.workunit.client.0.smithi042.stderr: 20450K .......... .......... .......... .......... .......... 15% 25.0M 5s 2024-04-20T10:35:44.224 INFO:tasks.workunit.client.0.smithi042.stderr: 20500K .......... .......... .......... .......... .......... 15% 29.5M 5s 2024-04-20T10:35:44.224 INFO:tasks.workunit.client.0.smithi042.stderr: 20550K .......... .......... .......... .......... .......... 15% 27.5M 5s 2024-04-20T10:35:44.224 INFO:tasks.workunit.client.0.smithi042.stderr: 20600K .......... .......... .......... .......... .......... 15% 22.4M 5s 2024-04-20T10:35:44.224 INFO:tasks.workunit.client.0.smithi042.stderr: 20650K .......... .......... .......... .......... .......... 15% 42.4M 5s 2024-04-20T10:35:44.224 INFO:tasks.workunit.client.0.smithi042.stderr: 20700K .......... .......... .......... .......... .......... 15% 26.6M 5s 2024-04-20T10:35:44.224 INFO:tasks.workunit.client.0.smithi042.stderr: 20750K .......... .......... .......... .......... .......... 15% 29.5M 5s 2024-04-20T10:35:44.224 INFO:tasks.workunit.client.0.smithi042.stderr: 20800K .......... .......... .......... .......... .......... 15% 27.3M 5s 2024-04-20T10:35:44.224 INFO:tasks.workunit.client.0.smithi042.stderr: 20850K .......... .......... .......... .......... .......... 15% 24.7M 5s 2024-04-20T10:35:44.224 INFO:tasks.workunit.client.0.smithi042.stderr: 20900K .......... .......... .......... .......... .......... 15% 41.9M 5s 2024-04-20T10:35:44.228 INFO:tasks.workunit.client.0.smithi042.stderr: 20950K .......... .......... .......... .......... .......... 15% 26.7M 5s 2024-04-20T10:35:44.229 INFO:tasks.workunit.client.0.smithi042.stderr: 21000K .......... .......... .......... .......... .......... 15% 32.1M 5s 2024-04-20T10:35:44.229 INFO:tasks.workunit.client.0.smithi042.stderr: 21050K .......... .......... .......... .......... .......... 15% 25.3M 5s 2024-04-20T10:35:44.230 INFO:tasks.workunit.client.0.smithi042.stderr: 21100K .......... .......... .......... .......... .......... 15% 26.0M 5s 2024-04-20T10:35:44.231 INFO:tasks.workunit.client.0.smithi042.stderr: 21150K .......... .......... .......... .......... .......... 15% 42.5M 5s 2024-04-20T10:35:44.233 INFO:tasks.workunit.client.0.smithi042.stderr: 21200K .......... .......... .......... .......... .......... 15% 23.4M 5s 2024-04-20T10:35:44.235 INFO:tasks.workunit.client.0.smithi042.stderr: 21250K .......... .......... .......... .......... .......... 15% 32.5M 5s 2024-04-20T10:35:44.237 INFO:tasks.workunit.client.0.smithi042.stderr: 21300K .......... .......... .......... .......... .......... 15% 24.8M 5s 2024-04-20T10:35:44.240 INFO:tasks.workunit.client.0.smithi042.stderr: 21350K .......... .......... .......... .......... .......... 15% 27.2M 5s 2024-04-20T10:35:44.240 INFO:tasks.workunit.client.0.smithi042.stderr: 21400K .......... .......... .......... .......... .......... 15% 45.5M 5s 2024-04-20T10:35:44.244 INFO:tasks.workunit.client.0.smithi042.stderr: 21450K .......... .......... .......... .......... .......... 15% 22.3M 5s 2024-04-20T10:35:44.244 INFO:tasks.workunit.client.0.smithi042.stderr: 21500K .......... .......... .......... .......... .......... 15% 33.1M 5s 2024-04-20T10:35:44.245 INFO:tasks.workunit.client.0.smithi042.stderr: 21550K .......... .......... .......... .......... .......... 15% 25.7M 5s 2024-04-20T10:35:44.248 INFO:tasks.workunit.client.0.smithi042.stderr: 21600K .......... .......... .......... .......... .......... 15% 26.1M 5s 2024-04-20T10:35:44.248 INFO:tasks.workunit.client.0.smithi042.stderr: 21650K .......... .......... .......... .......... .......... 15% 41.7M 5s 2024-04-20T10:35:44.250 INFO:tasks.workunit.client.0.smithi042.stderr: 21700K .......... .......... .......... .......... .......... 16% 21.1M 5s 2024-04-20T10:35:44.252 INFO:tasks.workunit.client.0.smithi042.stderr: 21750K .......... .......... .......... .......... .......... 16% 30.3M 4s 2024-04-20T10:35:44.254 INFO:tasks.workunit.client.0.smithi042.stderr: 21800K .......... .......... .......... .......... .......... 16% 24.2M 4s 2024-04-20T10:35:44.257 INFO:tasks.workunit.client.0.smithi042.stderr: 21850K .......... .......... .......... .......... .......... 16% 26.6M 4s 2024-04-20T10:35:44.257 INFO:tasks.workunit.client.0.smithi042.stderr: 21900K .......... .......... .......... .......... .......... 16% 41.0M 4s 2024-04-20T10:35:44.261 INFO:tasks.workunit.client.0.smithi042.stderr: 21950K .......... .......... .......... .......... .......... 16% 27.3M 4s 2024-04-20T10:35:44.261 INFO:tasks.workunit.client.0.smithi042.stderr: 22000K .......... .......... .......... .......... .......... 16% 30.7M 4s 2024-04-20T10:35:44.264 INFO:tasks.workunit.client.0.smithi042.stderr: 22050K .......... .......... .......... .......... .......... 16% 21.9M 4s 2024-04-20T10:35:44.264 INFO:tasks.workunit.client.0.smithi042.stderr: 22100K .......... .......... .......... .......... .......... 16% 43.0M 4s 2024-04-20T10:35:44.265 INFO:tasks.workunit.client.0.smithi042.stderr: 22150K .......... .......... .......... .......... .......... 16% 25.7M 4s 2024-04-20T10:35:44.267 INFO:tasks.workunit.client.0.smithi042.stderr: 22200K .......... .......... .......... .......... .......... 16% 27.5M 4s 2024-04-20T10:35:44.269 INFO:tasks.workunit.client.0.smithi042.stderr: 22250K .......... .......... .......... .......... .......... 16% 26.8M 4s 2024-04-20T10:35:44.270 INFO:tasks.workunit.client.0.smithi042.stderr: 22300K .......... .......... .......... .......... .......... 16% 30.9M 4s 2024-04-20T10:35:44.272 INFO:tasks.workunit.client.0.smithi042.stderr: 22350K .......... .......... .......... .......... .......... 16% 32.0M 4s 2024-04-20T10:35:44.273 INFO:tasks.workunit.client.0.smithi042.stderr: 22400K .......... .......... .......... .......... .......... 16% 30.0M 4s 2024-04-20T10:35:44.276 INFO:tasks.workunit.client.0.smithi042.stderr: 22450K .......... .......... .......... .......... .......... 16% 24.6M 4s 2024-04-20T10:35:44.278 INFO:tasks.workunit.client.0.smithi042.stderr: 22500K .......... .......... .......... .......... .......... 16% 24.7M 4s 2024-04-20T10:35:44.279 INFO:tasks.workunit.client.0.smithi042.stderr: 22550K .......... .......... .......... .......... .......... 16% 33.9M 4s 2024-04-20T10:35:44.280 INFO:tasks.workunit.client.0.smithi042.stderr: 22600K .......... .......... .......... .......... .......... 16% 36.3M 4s 2024-04-20T10:35:44.282 INFO:tasks.workunit.client.0.smithi042.stderr: 22650K .......... .......... .......... .......... .......... 16% 25.6M 4s 2024-04-20T10:35:44.284 INFO:tasks.workunit.client.0.smithi042.stderr: 22700K .......... .......... .......... .......... .......... 16% 28.5M 4s 2024-04-20T10:35:44.286 INFO:tasks.workunit.client.0.smithi042.stderr: 22750K .......... .......... .......... .......... .......... 16% 30.0M 4s 2024-04-20T10:35:44.288 INFO:tasks.workunit.client.0.smithi042.stderr: 22800K .......... .......... .......... .......... .......... 16% 26.8M 4s 2024-04-20T10:35:44.290 INFO:tasks.workunit.client.0.smithi042.stderr: 22850K .......... .......... .......... .......... .......... 16% 25.8M 4s 2024-04-20T10:35:44.290 INFO:tasks.workunit.client.0.smithi042.stderr: 22900K .......... .......... .......... .......... .......... 16% 42.8M 4s 2024-04-20T10:35:44.292 INFO:tasks.workunit.client.0.smithi042.stderr: 22950K .......... .......... .......... .......... .......... 16% 25.0M 4s 2024-04-20T10:35:44.294 INFO:tasks.workunit.client.0.smithi042.stderr: 23000K .......... .......... .......... .......... .......... 16% 30.6M 4s 2024-04-20T10:35:44.297 INFO:tasks.workunit.client.0.smithi042.stderr: 23050K .......... .......... .......... .......... .......... 17% 26.7M 4s 2024-04-20T10:35:44.298 INFO:tasks.workunit.client.0.smithi042.stderr: 23100K .......... .......... .......... .......... .......... 17% 24.9M 4s 2024-04-20T10:35:44.299 INFO:tasks.workunit.client.0.smithi042.stderr: 23150K .......... .......... .......... .......... .......... 17% 44.8M 4s 2024-04-20T10:35:44.301 INFO:tasks.workunit.client.0.smithi042.stderr: 23200K .......... .......... .......... .......... .......... 17% 24.7M 4s 2024-04-20T10:35:44.303 INFO:tasks.workunit.client.0.smithi042.stderr: 23250K .......... .......... .......... .......... .......... 17% 28.3M 4s 2024-04-20T10:35:44.307 INFO:tasks.workunit.client.0.smithi042.stderr: 23300K .......... .......... .......... .......... .......... 17% 27.2M 4s 2024-04-20T10:35:44.307 INFO:tasks.workunit.client.0.smithi042.stderr: 23350K .......... .......... .......... .......... .......... 17% 25.2M 4s 2024-04-20T10:35:44.307 INFO:tasks.workunit.client.0.smithi042.stderr: 23400K .......... .......... .......... .......... .......... 17% 49.0M 4s 2024-04-20T10:35:44.310 INFO:tasks.workunit.client.0.smithi042.stderr: 23450K .......... .......... .......... .......... .......... 17% 18.6M 4s 2024-04-20T10:35:44.311 INFO:tasks.workunit.client.0.smithi042.stderr: 23500K .......... .......... .......... .......... .......... 17% 31.0M 4s 2024-04-20T10:35:44.315 INFO:tasks.workunit.client.0.smithi042.stderr: 23550K .......... .......... .......... .......... .......... 17% 19.5M 4s 2024-04-20T10:35:44.316 INFO:tasks.workunit.client.0.smithi042.stderr: 23600K .......... .......... .......... .......... .......... 17% 20.9M 4s 2024-04-20T10:35:44.317 INFO:tasks.workunit.client.0.smithi042.stderr: 23650K .......... .......... .......... .......... .......... 17% 36.9M 4s 2024-04-20T10:35:44.319 INFO:tasks.workunit.client.0.smithi042.stderr: 23700K .......... .......... .......... .......... .......... 17% 26.8M 4s 2024-04-20T10:35:44.322 INFO:tasks.workunit.client.0.smithi042.stderr: 23750K .......... .......... .......... .......... .......... 17% 24.6M 4s 2024-04-20T10:35:44.324 INFO:tasks.workunit.client.0.smithi042.stderr: 23800K .......... .......... .......... .......... .......... 17% 34.2M 4s 2024-04-20T10:35:44.326 INFO:tasks.workunit.client.0.smithi042.stderr: 23850K .......... .......... .......... .......... .......... 17% 24.1M 4s 2024-04-20T10:35:44.326 INFO:tasks.workunit.client.0.smithi042.stderr: 23900K .......... .......... .......... .......... .......... 17% 39.7M 4s 2024-04-20T10:35:44.328 INFO:tasks.workunit.client.0.smithi042.stderr: 23950K .......... .......... .......... .......... .......... 17% 26.0M 4s 2024-04-20T10:35:44.329 INFO:tasks.workunit.client.0.smithi042.stderr: 24000K .......... .......... .......... .......... .......... 17% 34.7M 4s 2024-04-20T10:35:44.332 INFO:tasks.workunit.client.0.smithi042.stderr: 24050K .......... .......... .......... .......... .......... 17% 24.4M 4s 2024-04-20T10:35:44.334 INFO:tasks.workunit.client.0.smithi042.stderr: 24100K .......... .......... .......... .......... .......... 17% 24.5M 4s 2024-04-20T10:35:44.334 INFO:tasks.workunit.client.0.smithi042.stderr: 24150K .......... .......... .......... .......... .......... 17% 55.3M 4s 2024-04-20T10:35:44.336 INFO:tasks.workunit.client.0.smithi042.stderr: 24200K .......... .......... .......... .......... .......... 17% 23.2M 4s 2024-04-20T10:35:44.338 INFO:tasks.workunit.client.0.smithi042.stderr: 24250K .......... .......... .......... .......... .......... 17% 28.3M 4s 2024-04-20T10:35:44.341 INFO:tasks.workunit.client.0.smithi042.stderr: 24300K .......... .......... .......... .......... .......... 17% 28.3M 4s 2024-04-20T10:35:44.343 INFO:tasks.workunit.client.0.smithi042.stderr: 24350K .......... .......... .......... .......... .......... 17% 24.0M 4s 2024-04-20T10:35:44.343 INFO:tasks.workunit.client.0.smithi042.stderr: 24400K .......... .......... .......... .......... .......... 18% 46.0M 4s 2024-04-20T10:35:44.345 INFO:tasks.workunit.client.0.smithi042.stderr: 24450K .......... .......... .......... .......... .......... 18% 24.2M 4s 2024-04-20T10:35:44.347 INFO:tasks.workunit.client.0.smithi042.stderr: 24500K .......... .......... .......... .......... .......... 18% 26.1M 4s 2024-04-20T10:35:44.349 INFO:tasks.workunit.client.0.smithi042.stderr: 24550K .......... .......... .......... .......... .......... 18% 26.4M 4s 2024-04-20T10:35:44.351 INFO:tasks.workunit.client.0.smithi042.stderr: 24600K .......... .......... .......... .......... .......... 18% 20.8M 4s 2024-04-20T10:35:44.444 INFO:tasks.workunit.client.0.smithi042.stderr: 24650K .......... .......... .......... .......... .......... 18% 595K 5s 2024-04-20T10:35:44.444 INFO:tasks.workunit.client.0.smithi042.stderr: 24700K .......... .......... .......... .......... .......... 18% 60.3M 5s 2024-04-20T10:35:44.444 INFO:tasks.workunit.client.0.smithi042.stderr: 24750K .......... .......... .......... .......... .......... 18% 72.3M 5s 2024-04-20T10:35:44.444 INFO:tasks.workunit.client.0.smithi042.stderr: 24800K .......... .......... .......... .......... .......... 18% 53.5M 5s 2024-04-20T10:35:44.444 INFO:tasks.workunit.client.0.smithi042.stderr: 24850K .......... .......... .......... .......... .......... 18% 61.2M 5s 2024-04-20T10:35:44.444 INFO:tasks.workunit.client.0.smithi042.stderr: 24900K .......... .......... .......... .......... .......... 18% 59.7M 5s 2024-04-20T10:35:44.444 INFO:tasks.workunit.client.0.smithi042.stderr: 24950K .......... .......... .......... .......... .......... 18% 66.4M 5s 2024-04-20T10:35:44.444 INFO:tasks.workunit.client.0.smithi042.stderr: 25000K .......... .......... .......... .......... .......... 18% 63.6M 5s 2024-04-20T10:35:44.445 INFO:tasks.workunit.client.0.smithi042.stderr: 25050K .......... .......... .......... .......... .......... 18% 57.8M 5s 2024-04-20T10:35:44.445 INFO:tasks.workunit.client.0.smithi042.stderr: 25100K .......... .......... .......... .......... .......... 18% 72.3M 5s 2024-04-20T10:35:44.445 INFO:tasks.workunit.client.0.smithi042.stderr: 25150K .......... .......... .......... .......... .......... 18% 60.2M 5s 2024-04-20T10:35:44.463 INFO:tasks.workunit.client.0.smithi042.stderr: 25200K .......... .......... .......... .......... .......... 18% 8.93M 5s 2024-04-20T10:35:44.463 INFO:tasks.workunit.client.0.smithi042.stderr: 25250K .......... .......... .......... .......... .......... 18% 60.1M 5s 2024-04-20T10:35:44.463 INFO:tasks.workunit.client.0.smithi042.stderr: 25300K .......... .......... .......... .......... .......... 18% 59.6M 5s 2024-04-20T10:35:44.463 INFO:tasks.workunit.client.0.smithi042.stderr: 25350K .......... .......... .......... .......... .......... 18% 52.7M 5s 2024-04-20T10:35:44.463 INFO:tasks.workunit.client.0.smithi042.stderr: 25400K .......... .......... .......... .......... .......... 18% 64.9M 5s 2024-04-20T10:35:44.463 INFO:tasks.workunit.client.0.smithi042.stderr: 25450K .......... .......... .......... .......... .......... 18% 72.8M 5s 2024-04-20T10:35:44.463 INFO:tasks.workunit.client.0.smithi042.stderr: 25500K .......... .......... .......... .......... .......... 18% 52.4M 5s 2024-04-20T10:35:44.463 INFO:tasks.workunit.client.0.smithi042.stderr: 25550K .......... .......... .......... .......... .......... 18% 75.9M 5s 2024-04-20T10:35:44.463 INFO:tasks.workunit.client.0.smithi042.stderr: 25600K .......... .......... .......... .......... .......... 18% 62.8M 5s 2024-04-20T10:35:44.463 INFO:tasks.workunit.client.0.smithi042.stderr: 25650K .......... .......... .......... .......... .......... 18% 63.2M 5s 2024-04-20T10:35:44.463 INFO:tasks.workunit.client.0.smithi042.stderr: 25700K .......... .......... .......... .......... .......... 18% 56.8M 5s 2024-04-20T10:35:44.464 INFO:tasks.workunit.client.0.smithi042.stderr: 25750K .......... .......... .......... .......... .......... 19% 72.0M 5s 2024-04-20T10:35:44.464 INFO:tasks.workunit.client.0.smithi042.stderr: 25800K .......... .......... .......... .......... .......... 19% 61.3M 5s 2024-04-20T10:35:44.464 INFO:tasks.workunit.client.0.smithi042.stderr: 25850K .......... .......... .......... .......... .......... 19% 57.8M 5s 2024-04-20T10:35:44.464 INFO:tasks.workunit.client.0.smithi042.stderr: 25900K .......... .......... .......... .......... .......... 19% 37.5M 5s 2024-04-20T10:35:44.464 INFO:tasks.workunit.client.0.smithi042.stderr: 25950K .......... .......... .......... .......... .......... 19% 66.3M 5s 2024-04-20T10:35:44.464 INFO:tasks.workunit.client.0.smithi042.stderr: 26000K .......... .......... .......... .......... .......... 19% 68.0M 5s 2024-04-20T10:35:44.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.461+0000 7f0b55a00640 1 -- 172.21.15.42:0/506951028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b5019bb50 con 0x7f0b50107080 2024-04-20T10:35:44.467 INFO:tasks.workunit.client.0.smithi042.stderr: 26050K .......... .......... .......... .......... .......... 19% 51.7M 5s 2024-04-20T10:35:44.467 INFO:tasks.workunit.client.0.smithi042.stderr: 26100K .......... .......... .......... .......... .......... 19% 98.1M 4s 2024-04-20T10:35:44.467 INFO:tasks.workunit.client.0.smithi042.stderr: 26150K .......... .......... .......... .......... .......... 19% 119M 4s 2024-04-20T10:35:44.467 INFO:tasks.workunit.client.0.smithi042.stderr: 26200K .......... .......... .......... .......... .......... 19% 110M 4s 2024-04-20T10:35:44.467 INFO:tasks.workunit.client.0.smithi042.stderr: 26250K .......... .......... .......... .......... .......... 19% 116M 4s 2024-04-20T10:35:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.464+0000 7f0b4cc00640 1 -- 172.21.15.42:0/506951028 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f0b5019bb50 con 0x7f0b50107080 2024-04-20T10:35:44.468 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:35:44.469 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:35:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:35:44.472 INFO:tasks.workunit.client.0.smithi042.stderr: 26300K .......... .......... .......... .......... .......... 19% 75.6M 4s 2024-04-20T10:35:44.472 INFO:tasks.workunit.client.0.smithi042.stderr: 26350K .......... .......... .......... .......... .......... 19% 63.7M 4s 2024-04-20T10:35:44.472 INFO:tasks.workunit.client.0.smithi042.stderr: 26400K .......... .......... .......... .......... .......... 19% 58.8M 4s 2024-04-20T10:35:44.472 INFO:tasks.workunit.client.0.smithi042.stderr: 26450K .......... .......... .......... .......... .......... 19% 65.0M 4s 2024-04-20T10:35:44.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.467+0000 7f0b55a00640 1 -- 172.21.15.42:0/506951028 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0b340a3d20 msgr2=0x7f0b340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.467+0000 7f0b55a00640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0b340a3d20 0x7f0b340a61e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009bd0 tx=0x7f0b40005f90 comp rx=0 tx=0).stop 2024-04-20T10:35:44.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.467+0000 7f0b55a00640 1 -- 172.21.15.42:0/506951028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50107080 msgr2=0x7f0b501a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.467+0000 7f0b55a00640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50107080 0x7f0b501a09b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00e9f0 tx=0x7f0b3c00eeb0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.467+0000 7f0b55a00640 1 -- 172.21.15.42:0/506951028 shutdown_connections 2024-04-20T10:35:44.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.467+0000 7f0b55a00640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0b340a3d20 0x7f0b340a61e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.467+0000 7f0b55a00640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b5019a620 0x7f0b5019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.467+0000 7f0b55a00640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b50107080 0x7f0b501a09b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.467+0000 7f0b55a00640 1 --2- 172.21.15.42:0/506951028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b50101fb0 0x7f0b501a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.467+0000 7f0b55a00640 1 -- 172.21.15.42:0/506951028 >> 172.21.15.42:0/506951028 conn(0x7f0b500fd890 msgr2=0x7f0b500ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:44.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.467+0000 7f0b55a00640 1 -- 172.21.15.42:0/506951028 shutdown_connections 2024-04-20T10:35:44.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.468+0000 7f0b55a00640 1 -- 172.21.15.42:0/506951028 wait complete. 2024-04-20T10:35:44.501 INFO:tasks.workunit.client.0.smithi042.stderr: 26500K .......... .......... .......... .......... .......... 19% 41.4M 4s 2024-04-20T10:35:44.501 INFO:tasks.workunit.client.0.smithi042.stderr: 26550K .......... .......... .......... .......... .......... 19% 102M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 26600K .......... .......... .......... .......... .......... 19% 74.2M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 26650K .......... .......... .......... .......... .......... 19% 97.6M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 26700K .......... .......... .......... .......... .......... 19% 82.6M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 26750K .......... .......... .......... .......... .......... 19% 73.3M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 26800K .......... .......... .......... .......... .......... 19% 92.5M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 26850K .......... .......... .......... .......... .......... 19% 71.8M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 26900K .......... .......... .......... .......... .......... 19% 79.3M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 26950K .......... .......... .......... .......... .......... 19% 81.1M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 27000K .......... .......... .......... .......... .......... 19% 90.3M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 27050K .......... .......... .......... .......... .......... 19% 99.7M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 27100K .......... .......... .......... .......... .......... 19% 53.1M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 27150K .......... .......... .......... .......... .......... 20% 93.8M 4s 2024-04-20T10:35:44.502 INFO:tasks.workunit.client.0.smithi042.stderr: 27200K .......... .......... .......... .......... .......... 20% 76.7M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27250K .......... .......... .......... .......... .......... 20% 77.0M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27300K .......... .......... .......... .......... .......... 20% 72.2M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27350K .......... .......... .......... .......... .......... 20% 94.7M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27400K .......... .......... .......... .......... .......... 20% 96.6M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27450K .......... .......... .......... .......... .......... 20% 64.1M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27500K .......... .......... .......... .......... .......... 20% 54.9M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27550K .......... .......... .......... .......... .......... 20% 86.6M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27600K .......... .......... .......... .......... .......... 20% 72.2M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27650K .......... .......... .......... .......... .......... 20% 66.2M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27700K .......... .......... .......... .......... .......... 20% 54.2M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27750K .......... .......... .......... .......... .......... 20% 96.8M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27800K .......... .......... .......... .......... .......... 20% 151M 4s 2024-04-20T10:35:44.503 INFO:tasks.workunit.client.0.smithi042.stderr: 27850K .......... .......... .......... .......... .......... 20% 147M 4s 2024-04-20T10:35:44.504 INFO:tasks.workunit.client.0.smithi042.stderr: 27900K .......... .......... .......... .......... .......... 20% 72.4M 4s 2024-04-20T10:35:44.504 INFO:tasks.workunit.client.0.smithi042.stderr: 27950K .......... .......... .......... .......... .......... 20% 140M 4s 2024-04-20T10:35:44.504 INFO:tasks.workunit.client.0.smithi042.stderr: 28000K .......... .......... .......... .......... .......... 20% 139M 4s 2024-04-20T10:35:44.504 INFO:tasks.workunit.client.0.smithi042.stderr: 28050K .......... .......... .......... .......... .......... 20% 151M 4s 2024-04-20T10:35:44.504 INFO:tasks.workunit.client.0.smithi042.stderr: 28100K .......... .......... .......... .......... .......... 20% 105M 4s 2024-04-20T10:35:44.504 INFO:tasks.workunit.client.0.smithi042.stderr: 28150K .......... .......... .......... .......... .......... 20% 138M 4s 2024-04-20T10:35:44.505 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:35:44.520 INFO:tasks.workunit.client.0.smithi042.stderr: 28200K .......... .......... .......... .......... .......... 20% 126M 4s 2024-04-20T10:35:44.520 INFO:tasks.workunit.client.0.smithi042.stderr: 28250K .......... .......... .......... .......... .......... 20% 136M 4s 2024-04-20T10:35:44.520 INFO:tasks.workunit.client.0.smithi042.stderr: 28300K .......... .......... .......... .......... .......... 20% 108M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28350K .......... .......... .......... .......... .......... 20% 147M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28400K .......... .......... .......... .......... .......... 20% 137M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28450K .......... .......... .......... .......... .......... 20% 145M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28500K .......... .......... .......... .......... .......... 21% 98.4M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28550K .......... .......... .......... .......... .......... 21% 151M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28600K .......... .......... .......... .......... .......... 21% 28.8M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28650K .......... .......... .......... .......... .......... 21% 64.1M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28700K .......... .......... .......... .......... .......... 21% 38.7M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28750K .......... .......... .......... .......... .......... 21% 24.3M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28800K .......... .......... .......... .......... .......... 21% 26.5M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28850K .......... .......... .......... .......... .......... 21% 32.2M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28900K .......... .......... .......... .......... .......... 21% 28.0M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 28950K .......... .......... .......... .......... .......... 21% 39.2M 4s 2024-04-20T10:35:44.521 INFO:tasks.workunit.client.0.smithi042.stderr: 29000K .......... .......... .......... .......... .......... 21% 26.3M 4s 2024-04-20T10:35:44.522 INFO:tasks.workunit.client.0.smithi042.stderr: 29050K .......... .......... .......... .......... .......... 21% 24.8M 4s 2024-04-20T10:35:44.522 INFO:tasks.workunit.client.0.smithi042.stderr: 29100K .......... .......... .......... .......... .......... 21% 33.3M 4s 2024-04-20T10:35:44.522 INFO:tasks.workunit.client.0.smithi042.stderr: 29150K .......... .......... .......... .......... .......... 21% 26.4M 4s 2024-04-20T10:35:44.522 INFO:tasks.workunit.client.0.smithi042.stderr: 29200K .......... .......... .......... .......... .......... 21% 46.2M 4s 2024-04-20T10:35:44.522 INFO:tasks.workunit.client.0.smithi042.stderr: 29250K .......... .......... .......... .......... .......... 21% 22.1M 4s 2024-04-20T10:35:44.522 INFO:tasks.workunit.client.0.smithi042.stderr: 29300K .......... .......... .......... .......... .......... 21% 27.3M 4s 2024-04-20T10:35:44.522 INFO:tasks.workunit.client.0.smithi042.stderr: 29350K .......... .......... .......... .......... .......... 21% 32.5M 4s 2024-04-20T10:35:44.525 INFO:tasks.workunit.client.0.smithi042.stderr: 29400K .......... .......... .......... .......... .......... 21% 26.4M 4s 2024-04-20T10:35:44.526 INFO:tasks.workunit.client.0.smithi042.stderr: 29450K .......... .......... .......... .......... .......... 21% 34.9M 4s 2024-04-20T10:35:44.526 INFO:tasks.workunit.client.0.smithi042.stderr: 29500K .......... .......... .......... .......... .......... 21% 26.5M 4s 2024-04-20T10:35:44.526 INFO:tasks.workunit.client.0.smithi042.stderr: 29550K .......... .......... .......... .......... .......... 21% 33.7M 4s 2024-04-20T10:35:44.526 INFO:tasks.workunit.client.0.smithi042.stderr: 29600K .......... .......... .......... .......... .......... 21% 26.7M 4s 2024-04-20T10:35:44.527 INFO:tasks.workunit.client.0.smithi042.stderr: 29650K .......... .......... .......... .......... .......... 21% 26.1M 4s 2024-04-20T10:35:44.527 INFO:tasks.workunit.client.0.smithi042.stderr: 29700K .......... .......... .......... .......... .......... 21% 36.2M 4s 2024-04-20T10:35:44.530 INFO:tasks.workunit.client.0.smithi042.stderr: 29750K .......... .......... .......... .......... .......... 21% 25.8M 4s 2024-04-20T10:35:44.531 INFO:tasks.workunit.client.0.smithi042.stderr: 29800K .......... .......... .......... .......... .......... 21% 41.2M 4s 2024-04-20T10:35:44.532 INFO:tasks.workunit.client.0.smithi042.stderr: 29850K .......... .......... .......... .......... .......... 22% 30.4M 4s 2024-04-20T10:35:44.537 INFO:tasks.workunit.client.0.smithi042.stderr: 29900K .......... .......... .......... .......... .......... 22% 14.1M 4s 2024-04-20T10:35:44.537 INFO:tasks.workunit.client.0.smithi042.stderr: 29950K .......... .......... .......... .......... .......... 22% 54.7M 4s 2024-04-20T10:35:44.538 INFO:tasks.workunit.client.0.smithi042.stderr: 30000K .......... .......... .......... .......... .......... 22% 44.4M 4s 2024-04-20T10:35:44.540 INFO:tasks.workunit.client.0.smithi042.stderr: 30050K .......... .......... .......... .......... .......... 22% 26.3M 4s 2024-04-20T10:35:44.541 INFO:tasks.workunit.client.0.smithi042.stderr: 30100K .......... .......... .......... .......... .......... 22% 32.4M 4s 2024-04-20T10:35:44.542 INFO:tasks.workunit.client.0.smithi042.stderr: 30150K .......... .......... .......... .......... .......... 22% 32.4M 4s 2024-04-20T10:35:44.544 INFO:tasks.workunit.client.0.smithi042.stderr: 30200K .......... .......... .......... .......... .......... 22% 27.9M 4s 2024-04-20T10:35:44.546 INFO:tasks.workunit.client.0.smithi042.stderr: 30250K .......... .......... .......... .......... .......... 22% 25.2M 4s 2024-04-20T10:35:44.549 INFO:tasks.workunit.client.0.smithi042.stderr: 30300K .......... .......... .......... .......... .......... 22% 21.7M 4s 2024-04-20T10:35:44.551 INFO:tasks.workunit.client.0.smithi042.stderr: 30350K .......... .......... .......... .......... .......... 22% 33.2M 4s 2024-04-20T10:35:44.553 INFO:tasks.workunit.client.0.smithi042.stderr: 30400K .......... .......... .......... .......... .......... 22% 24.2M 4s 2024-04-20T10:35:44.553 INFO:tasks.workunit.client.0.smithi042.stderr: 30450K .......... .......... .......... .......... .......... 22% 93.9M 4s 2024-04-20T10:35:44.556 INFO:tasks.workunit.client.0.smithi042.stderr: 30500K .......... .......... .......... .......... .......... 22% 19.0M 4s 2024-04-20T10:35:44.558 INFO:tasks.workunit.client.0.smithi042.stderr: 30550K .......... .......... .......... .......... .......... 22% 31.6M 4s 2024-04-20T10:35:44.559 INFO:tasks.workunit.client.0.smithi042.stderr: 30600K .......... .......... .......... .......... .......... 22% 23.1M 4s 2024-04-20T10:35:44.564 INFO:tasks.workunit.client.0.smithi042.stderr: 30650K .......... .......... .......... .......... .......... 22% 19.7M 4s 2024-04-20T10:35:44.564 INFO:tasks.workunit.client.0.smithi042.stderr: 30700K .......... .......... .......... .......... .......... 22% 137M 4s 2024-04-20T10:35:44.564 INFO:tasks.workunit.client.0.smithi042.stderr: 30750K .......... .......... .......... .......... .......... 22% 29.2M 4s 2024-04-20T10:35:44.566 INFO:tasks.workunit.client.0.smithi042.stderr: 30800K .......... .......... .......... .......... .......... 22% 24.8M 4s 2024-04-20T10:35:44.568 INFO:tasks.workunit.client.0.smithi042.stderr: 30850K .......... .......... .......... .......... .......... 22% 25.8M 4s 2024-04-20T10:35:44.580 INFO:tasks.workunit.client.0.smithi042.stderr: 30900K .......... .......... .......... .......... .......... 22% 21.1M 4s 2024-04-20T10:35:44.580 INFO:tasks.workunit.client.0.smithi042.stderr: 30950K .......... .......... .......... .......... .......... 22% 36.9M 4s 2024-04-20T10:35:44.580 INFO:tasks.workunit.client.0.smithi042.stderr: 31000K .......... .......... .......... .......... .......... 22% 31.9M 4s 2024-04-20T10:35:44.580 INFO:tasks.workunit.client.0.smithi042.stderr: 31050K .......... .......... .......... .......... .......... 22% 22.1M 4s 2024-04-20T10:35:44.580 INFO:tasks.workunit.client.0.smithi042.stderr: 31100K .......... .......... .......... .......... .......... 22% 32.5M 4s 2024-04-20T10:35:44.580 INFO:tasks.workunit.client.0.smithi042.stderr: 31150K .......... .......... .......... .......... .......... 22% 24.7M 4s 2024-04-20T10:35:44.580 INFO:tasks.workunit.client.0.smithi042.stderr: 31200K .......... .......... .......... .......... .......... 23% 37.1M 4s 2024-04-20T10:35:44.581 INFO:tasks.workunit.client.0.smithi042.stderr: 31250K .......... .......... .......... .......... .......... 23% 34.2M 4s 2024-04-20T10:35:44.583 INFO:tasks.workunit.client.0.smithi042.stderr: 31300K .......... .......... .......... .......... .......... 23% 24.7M 4s 2024-04-20T10:35:44.585 INFO:tasks.workunit.client.0.smithi042.stderr: 31350K .......... .......... .......... .......... .......... 23% 31.2M 4s 2024-04-20T10:35:44.587 INFO:tasks.workunit.client.0.smithi042.stderr: 31400K .......... .......... .......... .......... .......... 23% 25.2M 4s 2024-04-20T10:35:44.589 INFO:tasks.workunit.client.0.smithi042.stderr: 31450K .......... .......... .......... .......... .......... 23% 38.3M 4s 2024-04-20T10:35:44.589 INFO:tasks.workunit.client.0.smithi042.stderr: 31500K .......... .......... .......... .......... .......... 23% 29.9M 4s 2024-04-20T10:35:44.591 INFO:tasks.workunit.client.0.smithi042.stderr: 31550K .......... .......... .......... .......... .......... 23% 25.8M 4s 2024-04-20T10:35:44.593 INFO:tasks.workunit.client.0.smithi042.stderr: 31600K .......... .......... .......... .......... .......... 23% 30.9M 4s 2024-04-20T10:35:44.595 INFO:tasks.workunit.client.0.smithi042.stderr: 31650K .......... .......... .......... .......... .......... 23% 24.1M 4s 2024-04-20T10:35:44.597 INFO:tasks.workunit.client.0.smithi042.stderr: 31700K .......... .......... .......... .......... .......... 23% 52.7M 4s 2024-04-20T10:35:44.598 INFO:tasks.workunit.client.0.smithi042.stderr: 31750K .......... .......... .......... .......... .......... 23% 18.7M 4s 2024-04-20T10:35:44.600 INFO:tasks.workunit.client.0.smithi042.stderr: 31800K .......... .......... .......... .......... .......... 23% 26.6M 4s 2024-04-20T10:35:44.601 INFO:tasks.workunit.client.0.smithi042.stderr: 31850K .......... .......... .......... .......... .......... 23% 35.5M 4s 2024-04-20T10:35:44.603 INFO:tasks.workunit.client.0.smithi042.stderr: 31900K .......... .......... .......... .......... .......... 23% 27.5M 4s 2024-04-20T10:35:44.605 INFO:tasks.workunit.client.0.smithi042.stderr: 31950K .......... .......... .......... .......... .......... 23% 22.2M 4s 2024-04-20T10:35:44.606 INFO:tasks.workunit.client.0.smithi042.stderr: 32000K .......... .......... .......... .......... .......... 23% 48.5M 4s 2024-04-20T10:35:44.608 INFO:tasks.workunit.client.0.smithi042.stderr: 32050K .......... .......... .......... .......... .......... 23% 24.7M 4s 2024-04-20T10:35:44.610 INFO:tasks.workunit.client.0.smithi042.stderr: 32100K .......... .......... .......... .......... .......... 23% 37.1M 4s 2024-04-20T10:35:44.613 INFO:tasks.workunit.client.0.smithi042.stderr: 32150K .......... .......... .......... .......... .......... 23% 20.2M 4s 2024-04-20T10:35:44.616 INFO:tasks.workunit.client.0.smithi042.stderr: 32200K .......... .......... .......... .......... .......... 23% 25.4M 4s 2024-04-20T10:35:44.616 INFO:tasks.workunit.client.0.smithi042.stderr: 32250K .......... .......... .......... .......... .......... 23% 47.5M 4s 2024-04-20T10:35:44.618 INFO:tasks.workunit.client.0.smithi042.stderr: 32300K .......... .......... .......... .......... .......... 23% 24.8M 4s 2024-04-20T10:35:44.619 INFO:tasks.workunit.client.0.smithi042.stderr: 32350K .......... .......... .......... .......... .......... 23% 34.6M 4s 2024-04-20T10:35:44.621 INFO:tasks.workunit.client.0.smithi042.stderr: 32400K .......... .......... .......... .......... .......... 23% 27.2M 4s 2024-04-20T10:35:44.624 INFO:tasks.workunit.client.0.smithi042.stderr: 32450K .......... .......... .......... .......... .......... 23% 26.0M 4s 2024-04-20T10:35:44.624 INFO:tasks.workunit.client.0.smithi042.stderr: 32500K .......... .......... .......... .......... .......... 23% 36.9M 4s 2024-04-20T10:35:44.625 INFO:tasks.workunit.client.0.smithi042.stderr: 32550K .......... .......... .......... .......... .......... 24% 26.4M 4s 2024-04-20T10:35:44.628 INFO:tasks.workunit.client.0.smithi042.stderr: 32600K .......... .......... .......... .......... .......... 24% 31.2M 4s 2024-04-20T10:35:44.630 INFO:tasks.workunit.client.0.smithi042.stderr: 32650K .......... .......... .......... .......... .......... 24% 26.2M 4s 2024-04-20T10:35:44.633 INFO:tasks.workunit.client.0.smithi042.stderr: 32700K .......... .......... .......... .......... .......... 24% 19.3M 4s 2024-04-20T10:35:44.633 INFO:tasks.workunit.client.0.smithi042.stderr: 32750K .......... .......... .......... .......... .......... 24% 43.0M 4s 2024-04-20T10:35:44.636 INFO:tasks.workunit.client.0.smithi042.stderr: 32800K .......... .......... .......... .......... .......... 24% 15.8M 4s 2024-04-20T10:35:44.638 INFO:tasks.workunit.client.0.smithi042.stderr: 32850K .......... .......... .......... .......... .......... 24% 24.9M 4s 2024-04-20T10:35:44.640 INFO:tasks.workunit.client.0.smithi042.stderr: 32900K .......... .......... .......... .......... .......... 24% 25.7M 4s 2024-04-20T10:35:44.643 INFO:tasks.workunit.client.0.smithi042.stderr: 32950K .......... .......... .......... .......... .......... 24% 18.4M 4s 2024-04-20T10:35:44.643 INFO:tasks.workunit.client.0.smithi042.stderr: 33000K .......... .......... .......... .......... .......... 24% 38.6M 4s 2024-04-20T10:35:44.646 INFO:tasks.workunit.client.0.smithi042.stderr: 33050K .......... .......... .......... .......... .......... 24% 27.5M 4s 2024-04-20T10:35:44.647 INFO:tasks.workunit.client.0.smithi042.stderr: 33100K .......... .......... .......... .......... .......... 24% 33.8M 4s 2024-04-20T10:35:44.649 INFO:tasks.workunit.client.0.smithi042.stderr: 33150K .......... .......... .......... .......... .......... 24% 24.9M 4s 2024-04-20T10:35:44.652 INFO:tasks.workunit.client.0.smithi042.stderr: 33200K .......... .......... .......... .......... .......... 24% 24.9M 4s 2024-04-20T10:35:44.652 INFO:tasks.workunit.client.0.smithi042.stderr: 33250K .......... .......... .......... .......... .......... 24% 52.4M 4s 2024-04-20T10:35:44.653 INFO:tasks.workunit.client.0.smithi042.stderr: 33300K .......... .......... .......... .......... .......... 24% 27.1M 4s 2024-04-20T10:35:44.656 INFO:tasks.workunit.client.0.smithi042.stderr: 33350K .......... .......... .......... .......... .......... 24% 26.3M 4s 2024-04-20T10:35:44.658 INFO:tasks.workunit.client.0.smithi042.stderr: 33400K .......... .......... .......... .......... .......... 24% 23.6M 4s 2024-04-20T10:35:44.660 INFO:tasks.workunit.client.0.smithi042.stderr: 33450K .......... .......... .......... .......... .......... 24% 25.1M 4s 2024-04-20T10:35:44.661 INFO:tasks.workunit.client.0.smithi042.stderr: 33500K .......... .......... .......... .......... .......... 24% 43.9M 4s 2024-04-20T10:35:44.662 INFO:tasks.workunit.client.0.smithi042.stderr: 33550K .......... .......... .......... .......... .......... 24% 27.7M 4s 2024-04-20T10:35:44.664 INFO:tasks.workunit.client.0.smithi042.stderr: 33600K .......... .......... .......... .......... .......... 24% 32.0M 4s 2024-04-20T10:35:44.666 INFO:tasks.workunit.client.0.smithi042.stderr: 33650K .......... .......... .......... .......... .......... 24% 21.2M 4s 2024-04-20T10:35:44.668 INFO:tasks.workunit.client.0.smithi042.stderr: 33700K .......... .......... .......... .......... .......... 24% 31.2M 4s 2024-04-20T10:35:44.669 INFO:tasks.workunit.client.0.smithi042.stderr: 33750K .......... .......... .......... .......... .......... 24% 40.7M 4s 2024-04-20T10:35:44.671 INFO:tasks.workunit.client.0.smithi042.stderr: 33800K .......... .......... .......... .......... .......... 24% 23.3M 4s 2024-04-20T10:35:44.672 INFO:tasks.workunit.client.0.smithi042.stderr: 33850K .......... .......... .......... .......... .......... 24% 35.4M 4s 2024-04-20T10:35:44.674 INFO:tasks.workunit.client.0.smithi042.stderr: 33900K .......... .......... .......... .......... .......... 25% 23.3M 4s 2024-04-20T10:35:44.676 INFO:tasks.workunit.client.0.smithi042.stderr: 33950K .......... .......... .......... .......... .......... 25% 28.1M 4s 2024-04-20T10:35:44.677 INFO:tasks.workunit.client.0.smithi042.stderr: 34000K .......... .......... .......... .......... .......... 25% 38.7M 4s 2024-04-20T10:35:44.679 INFO:tasks.workunit.client.0.smithi042.stderr: 34050K .......... .......... .......... .......... .......... 25% 24.7M 4s 2024-04-20T10:35:44.680 INFO:tasks.workunit.client.0.smithi042.stderr: 34100K .......... .......... .......... .......... .......... 25% 33.8M 4s 2024-04-20T10:35:44.682 INFO:tasks.workunit.client.0.smithi042.stderr: 34150K .......... .......... .......... .......... .......... 25% 24.7M 4s 2024-04-20T10:35:44.683 INFO:tasks.workunit.client.0.smithi042.stderr: 34200K .......... .......... .......... .......... .......... 25% 43.3M 4s 2024-04-20T10:35:44.685 INFO:tasks.workunit.client.0.smithi042.stderr: 34250K .......... .......... .......... .......... .......... 25% 29.0M 4s 2024-04-20T10:35:44.688 INFO:tasks.workunit.client.0.smithi042.stderr: 34300K .......... .......... .......... .......... .......... 25% 22.0M 4s 2024-04-20T10:35:44.689 INFO:tasks.workunit.client.0.smithi042.stderr: 34350K .......... .......... .......... .......... .......... 25% 36.8M 4s 2024-04-20T10:35:44.691 INFO:tasks.workunit.client.0.smithi042.stderr: 34400K .......... .......... .......... .......... .......... 25% 24.1M 4s 2024-04-20T10:35:44.693 INFO:tasks.workunit.client.0.smithi042.stderr: 34450K .......... .......... .......... .......... .......... 25% 27.7M 4s 2024-04-20T10:35:44.694 INFO:tasks.workunit.client.0.smithi042.stderr: 34500K .......... .......... .......... .......... .......... 25% 34.1M 4s 2024-04-20T10:35:44.696 INFO:tasks.workunit.client.0.smithi042.stderr: 34550K .......... .......... .......... .......... .......... 25% 23.9M 4s 2024-04-20T10:35:44.697 INFO:tasks.workunit.client.0.smithi042.stderr: 34600K .......... .......... .......... .......... .......... 25% 36.7M 4s 2024-04-20T10:35:44.699 INFO:tasks.workunit.client.0.smithi042.stderr: 34650K .......... .......... .......... .......... .......... 25% 25.2M 4s 2024-04-20T10:35:44.701 INFO:tasks.workunit.client.0.smithi042.stderr: 34700K .......... .......... .......... .......... .......... 25% 27.2M 4s 2024-04-20T10:35:44.702 INFO:tasks.workunit.client.0.smithi042.stderr: 34750K .......... .......... .......... .......... .......... 25% 42.3M 4s 2024-04-20T10:35:44.704 INFO:tasks.workunit.client.0.smithi042.stderr: 34800K .......... .......... .......... .......... .......... 25% 26.7M 4s 2024-04-20T10:35:44.706 INFO:tasks.workunit.client.0.smithi042.stderr: 34850K .......... .......... .......... .......... .......... 25% 29.7M 4s 2024-04-20T10:35:44.708 INFO:tasks.workunit.client.0.smithi042.stderr: 34900K .......... .......... .......... .......... .......... 25% 26.2M 4s 2024-04-20T10:35:44.710 INFO:tasks.workunit.client.0.smithi042.stderr: 34950K .......... .......... .......... .......... .......... 25% 26.3M 4s 2024-04-20T10:35:44.710 INFO:tasks.workunit.client.0.smithi042.stderr: 35000K .......... .......... .......... .......... .......... 25% 36.7M 4s 2024-04-20T10:35:44.713 INFO:tasks.workunit.client.0.smithi042.stderr: 35050K .......... .......... .......... .......... .......... 25% 26.1M 4s 2024-04-20T10:35:44.715 INFO:tasks.workunit.client.0.smithi042.stderr: 35100K .......... .......... .......... .......... .......... 25% 25.6M 4s 2024-04-20T10:35:44.716 INFO:tasks.workunit.client.0.smithi042.stderr: 35150K .......... .......... .......... .......... .......... 25% 33.7M 4s 2024-04-20T10:35:44.718 INFO:tasks.workunit.client.0.smithi042.stderr: 35200K .......... .......... .......... .......... .......... 25% 27.0M 4s 2024-04-20T10:35:44.719 INFO:tasks.workunit.client.0.smithi042.stderr: 35250K .......... .......... .......... .......... .......... 26% 36.0M 4s 2024-04-20T10:35:44.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.736+0000 7fb16e200640 1 -- 172.21.15.42:0/1566591459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb168102a20 msgr2=0x7fb168106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.736+0000 7fb16e200640 1 --2- 172.21.15.42:0/1566591459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb168102a20 0x7fb168106a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb150009860 tx=0x7fb15002bb70 comp rx=0 tx=0).stop 2024-04-20T10:35:44.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.737+0000 7fb16e200640 1 -- 172.21.15.42:0/1566591459 shutdown_connections 2024-04-20T10:35:44.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.737+0000 7fb16e200640 1 --2- 172.21.15.42:0/1566591459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168107140 0x7fb16810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.737+0000 7fb16e200640 1 --2- 172.21.15.42:0/1566591459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb168102a20 0x7fb168106a60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.737+0000 7fb16e200640 1 --2- 172.21.15.42:0/1566591459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb168102070 0x7fb168102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.737+0000 7fb16e200640 1 -- 172.21.15.42:0/1566591459 >> 172.21.15.42:0/1566591459 conn(0x7fb1680fd930 msgr2=0x7fb1680ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:44.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.737+0000 7fb16e200640 1 -- 172.21.15.42:0/1566591459 shutdown_connections 2024-04-20T10:35:44.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.737+0000 7fb16e200640 1 -- 172.21.15.42:0/1566591459 wait complete. 2024-04-20T10:35:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.738+0000 7fb16e200640 1 Processor -- start 2024-04-20T10:35:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.738+0000 7fb16e200640 1 -- start start 2024-04-20T10:35:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.738+0000 7fb16e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168102070 0x7fb168198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.738+0000 7fb16e200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb168107140 0x7fb168198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.738+0000 7fb16e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb16819cfa0 0x7fb16819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.738+0000 7fb16e200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681121d0 con 0x7fb16819cfa0 2024-04-20T10:35:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.738+0000 7fb16e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb168112050 con 0x7fb168102070 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.738+0000 7fb16e200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb168112350 con 0x7fb168107140 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.738+0000 7fb16d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb16819cfa0 0x7fb16819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.738+0000 7fb16ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168102070 0x7fb168198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.739+0000 7fb16ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168102070 0x7fb168198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:48738/0 (socket says 172.21.15.42:48738) 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.739+0000 7fb16ce00640 1 -- 172.21.15.42:0/357830592 learned_addr learned my addr 172.21.15.42:0/357830592 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.739+0000 7fb15fe00640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb168107140 0x7fb168198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.739+0000 7fb16ce00640 1 -- 172.21.15.42:0/357830592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb168107140 msgr2=0x7fb168198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.739+0000 7fb16ce00640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb168107140 0x7fb168198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.739+0000 7fb16ce00640 1 -- 172.21.15.42:0/357830592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb16819cfa0 msgr2=0x7fb16819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.739+0000 7fb16ce00640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb16819cfa0 0x7fb16819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.739+0000 7fb16ce00640 1 -- 172.21.15.42:0/357830592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16819da90 con 0x7fb168102070 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.739+0000 7fb16d800640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb16819cfa0 0x7fb16819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.739+0000 7fb16ce00640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168102070 0x7fb168198730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb15800e9c0 tx=0x7fb15800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.740+0000 7fb166000640 1 -- 172.21.15.42:0/357830592 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15800cde0 con 0x7fb168102070 2024-04-20T10:35:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.740+0000 7fb16e200640 1 -- 172.21.15.42:0/357830592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16819dd80 con 0x7fb168102070 2024-04-20T10:35:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.740+0000 7fb166000640 1 -- 172.21.15.42:0/357830592 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1580035f0 con 0x7fb168102070 2024-04-20T10:35:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.740+0000 7fb166000640 1 -- 172.21.15.42:0/357830592 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158010650 con 0x7fb168102070 2024-04-20T10:35:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.740+0000 7fb16e200640 1 -- 172.21.15.42:0/357830592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1681a5590 con 0x7fb168102070 2024-04-20T10:35:44.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.743+0000 7fb1725c3740 1 Processor -- start 2024-04-20T10:35:44.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.743+0000 7fb166000640 1 -- 172.21.15.42:0/357830592 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb1580037a0 con 0x7fb168102070 2024-04-20T10:35:44.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.743+0000 7fb1725c3740 1 -- start start 2024-04-20T10:35:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.743+0000 7fb1725c3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1a2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.743+0000 7fb1725c3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1a2780 0x55869c1af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.743+0000 7fb1725c3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1afa80 0x55869c1b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.743+0000 7fb1725c3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55869c107050 con 0x55869c1a2780 2024-04-20T10:35:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.743+0000 7fb1725c3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55869c0ad490 con 0x55869c1afa80 2024-04-20T10:35:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.743+0000 7fb1725c3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55869c103010 con 0x55869c1a1e60 2024-04-20T10:35:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.744+0000 7fb16d800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.744+0000 7fb16ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1a2780 0x55869c1af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.744+0000 7fb16d800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1a2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:51232/0 (socket says 172.21.15.42:51232) 2024-04-20T10:35:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.744+0000 7fb16d800640 1 -- 172.21.15.42:0/3466242914 learned_addr learned my addr 172.21.15.42:0/3466242914 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.744+0000 7fb15fe00640 1 --2- 172.21.15.42:0/3466242914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1afa80 0x55869c1b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.744+0000 7fb16d800640 1 -- 172.21.15.42:0/3466242914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1afa80 msgr2=0x55869c1b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.744+0000 7fb16d800640 1 --2- 172.21.15.42:0/3466242914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1afa80 0x55869c1b1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.744+0000 7fb16d800640 1 -- 172.21.15.42:0/3466242914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1a2780 msgr2=0x55869c1af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.744+0000 7fb16d800640 1 --2- 172.21.15.42:0/3466242914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1a2780 0x55869c1af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.744+0000 7fb16d800640 1 -- 172.21.15.42:0/3466242914 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869c0b5070 con 0x55869c1a1e60 2024-04-20T10:35:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.745+0000 7fb16d800640 1 --2- 172.21.15.42:0/3466242914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1a2240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb160011790 tx=0x7fb160011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4059124d2e1df9dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:44.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.745+0000 7fb15e000640 1 -- 172.21.15.42:0/3466242914 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160003c30 con 0x55869c1a1e60 2024-04-20T10:35:44.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.745+0000 7fb15e000640 1 -- 172.21.15.42:0/3466242914 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb16000fe80 con 0x55869c1a1e60 2024-04-20T10:35:44.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.745+0000 7fb15e000640 1 -- 172.21.15.42:0/3466242914 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1600193e0 con 0x55869c1a1e60 2024-04-20T10:35:44.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.745+0000 7fb166000640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb14c0a3da0 0x7fb14c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.745+0000 7fb166000640 1 -- 172.21.15.42:0/357830592 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb1580d0b00 con 0x7fb168102070 2024-04-20T10:35:44.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.746+0000 7fb16ce00640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb14c0a3da0 0x7fb14c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.746+0000 7fb1725c3740 1 -- 172.21.15.42:0/3466242914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 msgr2=0x55869c1a2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.746+0000 7fb1725c3740 1 --2- 172.21.15.42:0/3466242914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1a2240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb160011790 tx=0x7fb160011c50 comp rx=0 tx=0).stop 2024-04-20T10:35:44.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.746+0000 7fb1725c3740 1 -- 172.21.15.42:0/3466242914 shutdown_connections 2024-04-20T10:35:44.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.746+0000 7fb1725c3740 1 --2- 172.21.15.42:0/3466242914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1afa80 0x55869c1b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.746+0000 7fb1725c3740 1 --2- 172.21.15.42:0/3466242914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1a2780 0x55869c1af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.746+0000 7fb1725c3740 1 --2- 172.21.15.42:0/3466242914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1a2240 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.746+0000 7fb1725c3740 1 -- 172.21.15.42:0/3466242914 >> 172.21.15.42:0/3466242914 conn(0x55869c19c8a0 msgr2=0x55869c19cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.746+0000 7fb1725c3740 1 -- 172.21.15.42:0/3466242914 shutdown_connections 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.746+0000 7fb1725c3740 1 -- 172.21.15.42:0/3466242914 wait complete. 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.746+0000 7fb16ce00640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb14c0a3da0 0x7fb14c0a6260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55869c1a3080 tx=0x7fb1580d45b0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.747+0000 7fb1725c3740 1 -- start start 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.747+0000 7fb1725c3740 1 Processor -- start 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.747+0000 7fb1725c3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1aca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.747+0000 7fb1725c3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a2780 0x55869c1acf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.747+0000 7fb1725c3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1afa80 0x55869c1ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.747+0000 7fb1725c3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55869c1b2870 con 0x55869c1afa80 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.747+0000 7fb1725c3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55869c0c0860 con 0x55869c1a2780 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.747+0000 7fb1725c3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55869c1b29f0 con 0x55869c1a1e60 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.747+0000 7fb1725c3740 10 client.0 _ll_register_callbacks cb 0x55869c08efe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb15fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a2780 0x55869c1acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb15fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a2780 0x55869c1acf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:48754/0 (socket says 172.21.15.42:48754) 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb15fe00640 1 -- 172.21.15.42:0/559948508 learned_addr learned my addr 172.21.15.42:0/559948508 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb16d800640 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb16d800640 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1afa80 0x55869c1ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb15fe00640 1 -- 172.21.15.42:0/559948508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 msgr2=0x55869c1aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb15fe00640 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1aca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb15fe00640 1 -- 172.21.15.42:0/559948508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1afa80 msgr2=0x55869c1ad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb15fe00640 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1afa80 0x55869c1ad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb15fe00640 1 -- 172.21.15.42:0/559948508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869c0c4950 con 0x55869c1a2780 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb16d800640 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1afa80 0x55869c1ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb15fe00640 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a2780 0x55869c1acf60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb16806b4c0 tx=0x7fb150009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41440c102fd8f62b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.748+0000 7fb156a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.749+0000 7fb156a00640 1 -- 172.21.15.42:0/559948508 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150043030 con 0x55869c1a2780 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.749+0000 7fb156a00640 1 -- 172.21.15.42:0/559948508 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb150004250 con 0x55869c1a2780 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.749+0000 7fb1725c3740 1 -- 172.21.15.42:0/559948508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55869c114a70 con 0x55869c1a2780 2024-04-20T10:35:44.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.749+0000 7fb156a00640 1 -- 172.21.15.42:0/559948508 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb150039040 con 0x55869c1a2780 2024-04-20T10:35:44.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.749+0000 7fb1725c3740 10 client.25550 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:44.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.749+0000 7fb1725c3740 1 -- 172.21.15.42:0/559948508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55869c092820 con 0x55869c1a2780 2024-04-20T10:35:44.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.749+0000 7fb156a00640 1 -- 172.21.15.42:0/559948508 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55869c114a70 con 0x55869c1a2780 2024-04-20T10:35:44.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.750+0000 7fb156a00640 1 -- 172.21.15.42:0/559948508 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fb15002ca70 con 0x55869c1a2780 2024-04-20T10:35:44.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.750+0000 7fb1725c3740 10 client.25550 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:44.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.750+0000 7fb1725c3740 10 client.25550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:44.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.750+0000 7fb1725c3740 20 client.25550 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:44.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.750+0000 7fb1725c3740 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55869c1bf9e0 0x55869c1c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.750+0000 7fb1725c3740 4 client.25550 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:35:44.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.750+0000 7fb1725c3740 1 -- 172.21.15.42:0/559948508 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55869c10e8e0 con 0x55869c1bf9e0 2024-04-20T10:35:44.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.750+0000 7fb16d800640 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55869c1bf9e0 0x55869c1c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.751+0000 7fb16d800640 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55869c1bf9e0 0x55869c1c1e20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d05eee094f431179 server_cookie=44f38be10efbdf5d in_seq=0 out_seq=0 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.751+0000 7fb156a00640 10 client.25550 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb156a00640 1 -- 172.21.15.42:0/559948508 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55869c10e8e0 con 0x55869c1bf9e0 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb156a00640 10 client.25550 handle_command_reply: tid=0 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb1725c3740 1 client.25550 shutdown 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb1725c3740 1 -- 172.21.15.42:0/559948508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a2780 msgr2=0x55869c1acf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb1725c3740 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a2780 0x55869c1acf60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb16806b4c0 tx=0x7fb150009490 comp rx=0 tx=0).stop 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb1725c3740 1 -- 172.21.15.42:0/559948508 shutdown_connections 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb1725c3740 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55869c1bf9e0 0x55869c1c1e20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb1725c3740 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1afa80 0x55869c1ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb1725c3740 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a2780 0x55869c1acf60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb156a00640 0 client.25550 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb1725c3740 1 --2- 172.21.15.42:0/559948508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb1725c3740 1 -- 172.21.15.42:0/559948508 >> 172.21.15.42:0/559948508 conn(0x55869c19c8a0 msgr2=0x55869c19e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb1725c3740 1 -- 172.21.15.42:0/559948508 shutdown_connections 2024-04-20T10:35:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb1725c3740 1 -- 172.21.15.42:0/559948508 wait complete. 2024-04-20T10:35:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.752+0000 7fb1725c3740 20 client.25550 trim_cache size 0 max 16384 2024-04-20T10:35:44.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.767+0000 7fb1725c3740 1 Processor -- start 2024-04-20T10:35:44.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.767+0000 7fb1725c3740 1 -- start start 2024-04-20T10:35:44.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.768+0000 7fb1725c3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a1e60 0x55869c1af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.768+0000 7fb1725c3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a2780 0x55869c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.768+0000 7fb1725c3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1afa80 0x55869c1ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.768+0000 7fb1725c3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681a9970 con 0x55869c1afa80 2024-04-20T10:35:44.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.768+0000 7fb1725c3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681ac1f0 con 0x55869c1a1e60 2024-04-20T10:35:44.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.768+0000 7fb1725c3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681a9fe0 con 0x55869c1a2780 2024-04-20T10:35:44.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.768+0000 7fb16d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a1e60 0x55869c1af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.768+0000 7fb16d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a1e60 0x55869c1af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:48762/0 (socket says 172.21.15.42:48762) 2024-04-20T10:35:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.768+0000 7fb16d800640 1 -- 172.21.15.42:0/4293937027 learned_addr learned my addr 172.21.15.42:0/4293937027 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.768+0000 7fb15fe00640 1 --2- 172.21.15.42:0/4293937027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a2780 0x55869c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.768+0000 7fb16d800640 1 --2- 172.21.15.42:0/4293937027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1afa80 0x55869c1ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.769+0000 7fb16d800640 1 -- 172.21.15.42:0/4293937027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a2780 msgr2=0x55869c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.769+0000 7fb16d800640 1 --2- 172.21.15.42:0/4293937027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a2780 0x55869c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.769+0000 7fb16d800640 1 -- 172.21.15.42:0/4293937027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1afa80 msgr2=0x55869c1ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.769+0000 7fb16d800640 1 --2- 172.21.15.42:0/4293937027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1afa80 0x55869c1ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.769+0000 7fb16d800640 1 -- 172.21.15.42:0/4293937027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869c1a25b0 con 0x55869c1a1e60 2024-04-20T10:35:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.769+0000 7fb15fe00640 1 --2- 172.21.15.42:0/4293937027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a2780 0x55869c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.769+0000 7fb16d800640 1 --2- 172.21.15.42:0/4293937027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a1e60 0x55869c1af150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb160014040 tx=0x7fb16001c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15e147049aaf1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.769+0000 7fb15d600640 1 -- 172.21.15.42:0/4293937027 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1600042c0 con 0x55869c1a1e60 2024-04-20T10:35:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.769+0000 7fb15d600640 1 -- 172.21.15.42:0/4293937027 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb160004460 con 0x55869c1a1e60 2024-04-20T10:35:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.769+0000 7fb15d600640 1 -- 172.21.15.42:0/4293937027 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160004b50 con 0x55869c1a1e60 2024-04-20T10:35:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.770+0000 7fb1725c3740 1 -- 172.21.15.42:0/4293937027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a1e60 msgr2=0x55869c1af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.770+0000 7fb1725c3740 1 --2- 172.21.15.42:0/4293937027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a1e60 0x55869c1af150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb160014040 tx=0x7fb16001c040 comp rx=0 tx=0).stop 2024-04-20T10:35:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.770+0000 7fb1725c3740 1 -- 172.21.15.42:0/4293937027 shutdown_connections 2024-04-20T10:35:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.770+0000 7fb1725c3740 1 --2- 172.21.15.42:0/4293937027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1afa80 0x55869c1ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.770+0000 7fb1725c3740 1 --2- 172.21.15.42:0/4293937027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a2780 0x55869c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.770+0000 7fb1725c3740 1 --2- 172.21.15.42:0/4293937027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1a1e60 0x55869c1af150 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.770+0000 7fb1725c3740 1 -- 172.21.15.42:0/4293937027 >> 172.21.15.42:0/4293937027 conn(0x55869c19c8a0 msgr2=0x55869c19b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.770+0000 7fb1725c3740 1 -- 172.21.15.42:0/4293937027 shutdown_connections 2024-04-20T10:35:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.770+0000 7fb1725c3740 1 -- 172.21.15.42:0/4293937027 wait complete. 2024-04-20T10:35:44.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.771+0000 7fb1725c3740 1 -- start start 2024-04-20T10:35:44.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.771+0000 7fb1725c3740 1 Processor -- start 2024-04-20T10:35:44.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.771+0000 7fb1725c3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1a9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.771+0000 7fb1725c3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1a2780 0x55869c1a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.771+0000 7fb1725c3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1afa80 0x55869c1aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.771+0000 7fb1725c3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681aa6b0 con 0x55869c1a2780 2024-04-20T10:35:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.771+0000 7fb1725c3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681abb20 con 0x55869c1afa80 2024-04-20T10:35:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.771+0000 7fb1725c3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb1681aad80 con 0x55869c1a1e60 2024-04-20T10:35:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.771+0000 7fb1725c3740 10 client.0 _ll_register_callbacks cb 0x7fb150034e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.772+0000 7fb15fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1a2780 0x55869c1a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.772+0000 7fb16d800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.772+0000 7fb16d800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1a9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:51256/0 (socket says 172.21.15.42:51256) 2024-04-20T10:35:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.772+0000 7fb16d800640 1 -- 172.21.15.42:0/3708007433 learned_addr learned my addr 172.21.15.42:0/3708007433 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.772+0000 7fb16d800640 1 --2- 172.21.15.42:0/3708007433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1afa80 0x55869c1aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.772+0000 7fb15fe00640 1 -- 172.21.15.42:0/3708007433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 msgr2=0x55869c1a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.772+0000 7fb15fe00640 1 --2- 172.21.15.42:0/3708007433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.772+0000 7fb15fe00640 1 -- 172.21.15.42:0/3708007433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1afa80 msgr2=0x55869c1aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.772+0000 7fb15fe00640 1 --2- 172.21.15.42:0/3708007433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1afa80 0x55869c1aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.772+0000 7fb15fe00640 1 -- 172.21.15.42:0/3708007433 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869be193f0 con 0x55869c1a2780 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.772+0000 7fb15fe00640 1 --2- 172.21.15.42:0/3708007433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1a2780 0x55869c1a9e50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb150032850 tx=0x7fb150009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0722e140e98e3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.772+0000 7fb156a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.773+0000 7fb156a00640 1 -- 172.21.15.42:0/3708007433 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150043070 con 0x55869c1a2780 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.773+0000 7fb156a00640 1 -- 172.21.15.42:0/3708007433 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb150004460 con 0x55869c1a2780 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.773+0000 7fb1725c3740 1 -- 172.21.15.42:0/3708007433 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb160004460 con 0x55869c1a2780 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.773+0000 7fb156a00640 1 -- 172.21.15.42:0/3708007433 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb150000e60 con 0x55869c1a2780 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.773+0000 7fb1725c3740 10 client.15444 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.773+0000 7fb1725c3740 1 -- 172.21.15.42:0/3708007433 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55869bee37d0 con 0x55869c1a2780 2024-04-20T10:35:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.773+0000 7fb156a00640 1 -- 172.21.15.42:0/3708007433 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb160004460 con 0x55869c1a2780 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.774+0000 7fb156a00640 1 -- 172.21.15.42:0/3708007433 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fb15002c8d0 con 0x55869c1a2780 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.774+0000 7fb1725c3740 10 client.15444 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.774+0000 7fb1725c3740 10 client.15444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.774+0000 7fb1725c3740 20 client.15444 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.774+0000 7fb1725c3740 1 --2- 172.21.15.42:0/3708007433 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55869c226860 0x55869c228c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.774+0000 7fb1725c3740 4 client.15444 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.774+0000 7fb1725c3740 1 -- 172.21.15.42:0/3708007433 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55869c08a5a0 con 0x55869c226860 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.774+0000 7fb16d800640 1 --2- 172.21.15.42:0/3708007433 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55869c226860 0x55869c228c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb16d800640 1 --2- 172.21.15.42:0/3708007433 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55869c226860 0x55869c228c80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f00ced020113db2 server_cookie=d3db78a60c313bc9 in_seq=0 out_seq=0 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb156a00640 10 client.15444 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb156a00640 1 -- 172.21.15.42:0/3708007433 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55869c08a5a0 con 0x55869c226860 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb156a00640 10 client.15444 handle_command_reply: tid=0 2024-04-20T10:35:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb1725c3740 1 client.15444 shutdown 2024-04-20T10:35:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb1725c3740 1 -- 172.21.15.42:0/3708007433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1a2780 msgr2=0x55869c1a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb1725c3740 1 --2- 172.21.15.42:0/3708007433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1a2780 0x55869c1a9e50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb150032850 tx=0x7fb150009720 comp rx=0 tx=0).stop 2024-04-20T10:35:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb1725c3740 1 -- 172.21.15.42:0/3708007433 shutdown_connections 2024-04-20T10:35:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb1725c3740 1 --2- 172.21.15.42:0/3708007433 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55869c226860 0x55869c228c80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb1725c3740 1 --2- 172.21.15.42:0/3708007433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55869c1afa80 0x55869c1aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb1725c3740 1 --2- 172.21.15.42:0/3708007433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55869c1a2780 0x55869c1a9e50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb1725c3740 1 --2- 172.21.15.42:0/3708007433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55869c1a1e60 0x55869c1a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb1725c3740 1 -- 172.21.15.42:0/3708007433 >> 172.21.15.42:0/3708007433 conn(0x55869c19c8a0 msgr2=0x55869c19e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb156a00640 0 client.15444 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb1725c3740 1 -- 172.21.15.42:0/3708007433 shutdown_connections 2024-04-20T10:35:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb1725c3740 1 -- 172.21.15.42:0/3708007433 wait complete. 2024-04-20T10:35:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.775+0000 7fb1725c3740 20 client.15444 trim_cache size 0 max 16384 2024-04-20T10:35:44.781 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:35:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.776+0000 7fb16e200640 1 -- 172.21.15.42:0/357830592 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb14c0a3da0 msgr2=0x7fb14c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.776+0000 7fb16e200640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb14c0a3da0 0x7fb14c0a6260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55869c1a3080 tx=0x7fb1580d45b0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.776+0000 7fb16e200640 1 -- 172.21.15.42:0/357830592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168102070 msgr2=0x7fb168198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.776+0000 7fb16e200640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168102070 0x7fb168198730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb15800e9c0 tx=0x7fb15800ee80 comp rx=0 tx=0).stop 2024-04-20T10:35:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.776+0000 7fb16e200640 1 -- 172.21.15.42:0/357830592 shutdown_connections 2024-04-20T10:35:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.776+0000 7fb16e200640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb14c0a3da0 0x7fb14c0a6260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.776+0000 7fb16e200640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb16819cfa0 0x7fb16819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.776+0000 7fb16e200640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb168107140 0x7fb168198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.776+0000 7fb16e200640 1 --2- 172.21.15.42:0/357830592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb168102070 0x7fb168198730 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.776+0000 7fb16e200640 1 -- 172.21.15.42:0/357830592 >> 172.21.15.42:0/357830592 conn(0x7fb1680fd930 msgr2=0x7fb1680fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.776+0000 7fb16e200640 1 -- 172.21.15.42:0/357830592 shutdown_connections 2024-04-20T10:35:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:44.776+0000 7fb16e200640 1 -- 172.21.15.42:0/357830592 wait complete. 2024-04-20T10:35:44.791 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:44 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/506951028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:35:44.886 INFO:tasks.workunit.client.0.smithi042.stderr: 35300K .......... .......... .......... .......... .......... 26% 309K 4s 2024-04-20T10:35:44.886 INFO:tasks.workunit.client.0.smithi042.stderr: 35350K .......... .......... .......... .......... .......... 26% 42.6M 4s 2024-04-20T10:35:44.886 INFO:tasks.workunit.client.0.smithi042.stderr: 35400K .......... .......... .......... .......... .......... 26% 39.7M 4s 2024-04-20T10:35:44.886 INFO:tasks.workunit.client.0.smithi042.stderr: 35450K .......... .......... .......... .......... .......... 26% 56.7M 4s 2024-04-20T10:35:44.886 INFO:tasks.workunit.client.0.smithi042.stderr: 35500K .......... .......... .......... .......... .......... 26% 67.8M 4s 2024-04-20T10:35:44.892 INFO:tasks.workunit.client.0.smithi042.stderr: 35550K .......... .......... .......... .......... .......... 26% 23.8M 4s 2024-04-20T10:35:44.892 INFO:tasks.workunit.client.0.smithi042.stderr: 35600K .......... .......... .......... .......... .......... 26% 61.8M 4s 2024-04-20T10:35:44.892 INFO:tasks.workunit.client.0.smithi042.stderr: 35650K .......... .......... .......... .......... .......... 26% 59.4M 4s 2024-04-20T10:35:44.892 INFO:tasks.workunit.client.0.smithi042.stderr: 35700K .......... .......... .......... .......... .......... 26% 60.8M 4s 2024-04-20T10:35:44.893 INFO:tasks.workunit.client.0.smithi042.stderr: 35750K .......... .......... .......... .......... .......... 26% 58.8M 4s 2024-04-20T10:35:44.893 INFO:tasks.workunit.client.0.smithi042.stderr: 35800K .......... .......... .......... .......... .......... 26% 63.9M 4s 2024-04-20T10:35:44.894 INFO:tasks.workunit.client.0.smithi042.stderr: 35850K .......... .......... .......... .......... .......... 26% 22.8M 4s 2024-04-20T10:35:44.895 INFO:tasks.workunit.client.0.smithi042.stderr: 35900K .......... .......... .......... .......... .......... 26% 48.0M 4s 2024-04-20T10:35:44.896 INFO:tasks.workunit.client.0.smithi042.stderr: 35950K .......... .......... .......... .......... .......... 26% 23.8M 4s 2024-04-20T10:35:44.897 INFO:tasks.workunit.client.0.smithi042.stderr: 36000K .......... .......... .......... .......... .......... 26% 35.8M 4s 2024-04-20T10:35:44.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:44 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/506951028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:35:44.905 INFO:tasks.workunit.client.0.smithi042.stderr: 36050K .......... .......... .......... .......... .......... 26% 8.55M 4s 2024-04-20T10:35:44.905 INFO:tasks.workunit.client.0.smithi042.stderr: 36100K .......... .......... .......... .......... .......... 26% 59.9M 4s 2024-04-20T10:35:44.905 INFO:tasks.workunit.client.0.smithi042.stderr: 36150K .......... .......... .......... .......... .......... 26% 59.2M 4s 2024-04-20T10:35:44.910 INFO:tasks.workunit.client.0.smithi042.stderr: 36200K .......... .......... .......... .......... .......... 26% 18.9M 4s 2024-04-20T10:35:44.910 INFO:tasks.workunit.client.0.smithi042.stderr: 36250K .......... .......... .......... .......... .......... 26% 61.8M 4s 2024-04-20T10:35:44.910 INFO:tasks.workunit.client.0.smithi042.stderr: 36300K .......... .......... .......... .......... .......... 26% 44.7M 4s 2024-04-20T10:35:44.912 INFO:tasks.workunit.client.0.smithi042.stderr: 36350K .......... .......... .......... .......... .......... 26% 29.4M 4s 2024-04-20T10:35:44.912 INFO:tasks.workunit.client.0.smithi042.stderr: 36400K .......... .......... .......... .......... .......... 26% 45.9M 4s 2024-04-20T10:35:44.914 INFO:tasks.workunit.client.0.smithi042.stderr: 36450K .......... .......... .......... .......... .......... 26% 21.5M 4s 2024-04-20T10:35:44.917 INFO:tasks.workunit.client.0.smithi042.stderr: 36500K .......... .......... .......... .......... .......... 26% 24.7M 4s 2024-04-20T10:35:45.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:44 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/506951028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:35:45.346 INFO:tasks.workunit.client.0.smithi042.stderr: 36550K .......... .......... .......... .......... .......... 26% 118K 5s 2024-04-20T10:35:45.346 INFO:tasks.workunit.client.0.smithi042.stderr: 36600K .......... .......... .......... .......... .......... 26% 37.0M 5s 2024-04-20T10:35:45.346 INFO:tasks.workunit.client.0.smithi042.stderr: 36650K .......... .......... .......... .......... .......... 27% 43.1M 5s 2024-04-20T10:35:45.358 INFO:tasks.workunit.client.0.smithi042.stderr: 36700K .......... .......... .......... .......... .......... 27% 71.4M 5s 2024-04-20T10:35:45.358 INFO:tasks.workunit.client.0.smithi042.stderr: 36750K .......... .......... .......... .......... .......... 27% 70.8M 5s 2024-04-20T10:35:45.358 INFO:tasks.workunit.client.0.smithi042.stderr: 36800K .......... .......... .......... .......... .......... 27% 49.2M 5s 2024-04-20T10:35:45.359 INFO:tasks.workunit.client.0.smithi042.stderr: 36850K .......... .......... .......... .......... .......... 27% 64.0M 5s 2024-04-20T10:35:45.359 INFO:tasks.workunit.client.0.smithi042.stderr: 36900K .......... .......... .......... .......... .......... 27% 64.8M 5s 2024-04-20T10:35:45.359 INFO:tasks.workunit.client.0.smithi042.stderr: 36950K .......... .......... .......... .......... .......... 27% 61.6M 5s 2024-04-20T10:35:45.359 INFO:tasks.workunit.client.0.smithi042.stderr: 37000K .......... .......... .......... .......... .......... 27% 56.1M 5s 2024-04-20T10:35:45.359 INFO:tasks.workunit.client.0.smithi042.stderr: 37050K .......... .......... .......... .......... .......... 27% 70.0M 5s 2024-04-20T10:35:45.359 INFO:tasks.workunit.client.0.smithi042.stderr: 37100K .......... .......... .......... .......... .......... 27% 56.5M 5s 2024-04-20T10:35:45.359 INFO:tasks.workunit.client.0.smithi042.stderr: 37150K .......... .......... .......... .......... .......... 27% 24.1M 5s 2024-04-20T10:35:45.359 INFO:tasks.workunit.client.0.smithi042.stderr: 37200K .......... .......... .......... .......... .......... 27% 23.9M 5s 2024-04-20T10:35:45.359 INFO:tasks.workunit.client.0.smithi042.stderr: 37250K .......... .......... .......... .......... .......... 27% 28.0M 5s 2024-04-20T10:35:45.359 INFO:tasks.workunit.client.0.smithi042.stderr: 37300K .......... .......... .......... .......... .......... 27% 59.2M 5s 2024-04-20T10:35:45.361 INFO:tasks.workunit.client.0.smithi042.stderr: 37350K .......... .......... .......... .......... .......... 27% 25.2M 5s 2024-04-20T10:35:45.362 INFO:tasks.workunit.client.0.smithi042.stderr: 37400K .......... .......... .......... .......... .......... 27% 24.7M 5s 2024-04-20T10:35:45.365 INFO:tasks.workunit.client.0.smithi042.stderr: 37450K .......... .......... .......... .......... .......... 27% 20.5M 5s 2024-04-20T10:35:45.367 INFO:tasks.workunit.client.0.smithi042.stderr: 37500K .......... .......... .......... .......... .......... 27% 64.0M 5s 2024-04-20T10:35:45.367 INFO:tasks.workunit.client.0.smithi042.stderr: 37550K .......... .......... .......... .......... .......... 27% 25.9M 5s 2024-04-20T10:35:45.369 INFO:tasks.workunit.client.0.smithi042.stderr: 37600K .......... .......... .......... .......... .......... 27% 24.3M 5s 2024-04-20T10:35:45.371 INFO:tasks.workunit.client.0.smithi042.stderr: 37650K .......... .......... .......... .......... .......... 27% 25.6M 5s 2024-04-20T10:35:45.446 INFO:tasks.workunit.client.0.smithi042.stderr: 37700K .......... .......... .......... .......... .......... 27% 685K 5s 2024-04-20T10:35:45.447 INFO:tasks.workunit.client.0.smithi042.stderr: 37750K .......... .......... .......... .......... .......... 27% 50.1M 5s 2024-04-20T10:35:45.447 INFO:tasks.workunit.client.0.smithi042.stderr: 37800K .......... .......... .......... .......... .......... 27% 68.3M 5s 2024-04-20T10:35:45.460 INFO:tasks.workunit.client.0.smithi042.stderr: 37850K .......... .......... .......... .......... .......... 27% 7.06M 5s 2024-04-20T10:35:45.460 INFO:tasks.workunit.client.0.smithi042.stderr: 37900K .......... .......... .......... .......... .......... 27% 57.8M 5s 2024-04-20T10:35:45.460 INFO:tasks.workunit.client.0.smithi042.stderr: 37950K .......... .......... .......... .......... .......... 27% 55.0M 5s 2024-04-20T10:35:45.461 INFO:tasks.workunit.client.0.smithi042.stderr: 38000K .......... .......... .......... .......... .......... 28% 67.4M 5s 2024-04-20T10:35:45.461 INFO:tasks.workunit.client.0.smithi042.stderr: 38050K .......... .......... .......... .......... .......... 28% 69.4M 5s 2024-04-20T10:35:45.461 INFO:tasks.workunit.client.0.smithi042.stderr: 38100K .......... .......... .......... .......... .......... 28% 59.9M 5s 2024-04-20T10:35:45.461 INFO:tasks.workunit.client.0.smithi042.stderr: 38150K .......... .......... .......... .......... .......... 28% 55.1M 5s 2024-04-20T10:35:45.461 INFO:tasks.workunit.client.0.smithi042.stderr: 38200K .......... .......... .......... .......... .......... 28% 62.7M 5s 2024-04-20T10:35:45.461 INFO:tasks.workunit.client.0.smithi042.stderr: 38250K .......... .......... .......... .......... .......... 28% 60.7M 5s 2024-04-20T10:35:45.461 INFO:tasks.workunit.client.0.smithi042.stderr: 38300K .......... .......... .......... .......... .......... 28% 67.2M 5s 2024-04-20T10:35:45.464 INFO:tasks.workunit.client.0.smithi042.stderr: 38350K .......... .......... .......... .......... .......... 28% 26.6M 5s 2024-04-20T10:35:45.464 INFO:tasks.workunit.client.0.smithi042.stderr: 38400K .......... .......... .......... .......... .......... 28% 31.7M 5s 2024-04-20T10:35:45.464 INFO:tasks.workunit.client.0.smithi042.stderr: 38450K .......... .......... .......... .......... .......... 28% 62.7M 5s 2024-04-20T10:35:45.466 INFO:tasks.workunit.client.0.smithi042.stderr: 38500K .......... .......... .......... .......... .......... 28% 23.0M 5s 2024-04-20T10:35:45.467 INFO:tasks.workunit.client.0.smithi042.stderr: 38550K .......... .......... .......... .......... .......... 28% 25.7M 5s 2024-04-20T10:35:45.470 INFO:tasks.workunit.client.0.smithi042.stderr: 38600K .......... .......... .......... .......... .......... 28% 25.8M 5s 2024-04-20T10:35:45.473 INFO:tasks.workunit.client.0.smithi042.stderr: 38650K .......... .......... .......... .......... .......... 28% 28.7M 5s 2024-04-20T10:35:45.473 INFO:tasks.workunit.client.0.smithi042.stderr: 38700K .......... .......... .......... .......... .......... 28% 57.7M 5s 2024-04-20T10:35:45.475 INFO:tasks.workunit.client.0.smithi042.stderr: 38750K .......... .......... .......... .......... .......... 28% 21.2M 5s 2024-04-20T10:35:45.477 INFO:tasks.workunit.client.0.smithi042.stderr: 38800K .......... .......... .......... .......... .......... 28% 28.0M 5s 2024-04-20T10:35:45.479 INFO:tasks.workunit.client.0.smithi042.stderr: 38850K .......... .......... .......... .......... .......... 28% 26.6M 5s 2024-04-20T10:35:45.481 INFO:tasks.workunit.client.0.smithi042.stderr: 38900K .......... .......... .......... .......... .......... 28% 60.0M 5s 2024-04-20T10:35:45.481 INFO:tasks.workunit.client.0.smithi042.stderr: 38950K .......... .......... .......... .......... .......... 28% 26.2M 5s 2024-04-20T10:35:45.483 INFO:tasks.workunit.client.0.smithi042.stderr: 39000K .......... .......... .......... .......... .......... 28% 23.5M 5s 2024-04-20T10:35:45.485 INFO:tasks.workunit.client.0.smithi042.stderr: 39050K .......... .......... .......... .......... .......... 28% 27.1M 5s 2024-04-20T10:35:45.485 INFO:tasks.workunit.client.0.smithi042.stderr: 39100K .......... .......... .......... .......... .......... 28% 57.0M 5s 2024-04-20T10:35:45.487 INFO:tasks.workunit.client.0.smithi042.stderr: 39150K .......... .......... .......... .......... .......... 28% 24.1M 5s 2024-04-20T10:35:45.490 INFO:tasks.workunit.client.0.smithi042.stderr: 39200K .......... .......... .......... .......... .......... 28% 26.3M 5s 2024-04-20T10:35:45.492 INFO:tasks.workunit.client.0.smithi042.stderr: 39250K .......... .......... .......... .......... .......... 28% 23.9M 5s 2024-04-20T10:35:45.494 INFO:tasks.workunit.client.0.smithi042.stderr: 39300K .......... .......... .......... .......... .......... 28% 26.4M 5s 2024-04-20T10:35:45.494 INFO:tasks.workunit.client.0.smithi042.stderr: 39350K .......... .......... .......... .......... .......... 29% 67.6M 5s 2024-04-20T10:35:45.496 INFO:tasks.workunit.client.0.smithi042.stderr: 39400K .......... .......... .......... .......... .......... 29% 21.9M 5s 2024-04-20T10:35:45.498 INFO:tasks.workunit.client.0.smithi042.stderr: 39450K .......... .......... .......... .......... .......... 29% 28.4M 5s 2024-04-20T10:35:45.501 INFO:tasks.workunit.client.0.smithi042.stderr: 39500K .......... .......... .......... .......... .......... 29% 22.5M 5s 2024-04-20T10:35:45.501 INFO:tasks.workunit.client.0.smithi042.stderr: 39550K .......... .......... .......... .......... .......... 29% 44.0M 5s 2024-04-20T10:35:45.503 INFO:tasks.workunit.client.0.smithi042.stderr: 39600K .......... .......... .......... .......... .......... 29% 28.0M 5s 2024-04-20T10:35:45.505 INFO:tasks.workunit.client.0.smithi042.stderr: 39650K .......... .......... .......... .......... .......... 29% 21.8M 5s 2024-04-20T10:35:45.508 INFO:tasks.workunit.client.0.smithi042.stderr: 39700K .......... .......... .......... .......... .......... 29% 27.1M 5s 2024-04-20T10:35:45.511 INFO:tasks.workunit.client.0.smithi042.stderr: 39750K .......... .......... .......... .......... .......... 29% 52.3M 5s 2024-04-20T10:35:45.511 INFO:tasks.workunit.client.0.smithi042.stderr: 39800K .......... .......... .......... .......... .......... 29% 26.7M 5s 2024-04-20T10:35:45.511 INFO:tasks.workunit.client.0.smithi042.stderr: 39850K .......... .......... .......... .......... .......... 29% 53.4M 5s 2024-04-20T10:35:45.514 INFO:tasks.workunit.client.0.smithi042.stderr: 39900K .......... .......... .......... .......... .......... 29% 16.9M 5s 2024-04-20T10:35:45.516 INFO:tasks.workunit.client.0.smithi042.stderr: 39950K .......... .......... .......... .......... .......... 29% 37.9M 5s 2024-04-20T10:35:45.516 INFO:tasks.workunit.client.0.smithi042.stderr: 40000K .......... .......... .......... .......... .......... 29% 48.8M 5s 2024-04-20T10:35:45.520 INFO:tasks.workunit.client.0.smithi042.stderr: 40050K .......... .......... .......... .......... .......... 29% 22.9M 5s 2024-04-20T10:35:45.520 INFO:tasks.workunit.client.0.smithi042.stderr: 40100K .......... .......... .......... .......... .......... 29% 37.9M 5s 2024-04-20T10:35:45.521 INFO:tasks.workunit.client.0.smithi042.stderr: 40150K .......... .......... .......... .......... .......... 29% 27.0M 5s 2024-04-20T10:35:45.522 INFO:tasks.workunit.client.0.smithi042.stderr: 40200K .......... .......... .......... .......... .......... 29% 35.9M 5s 2024-04-20T10:35:45.525 INFO:tasks.workunit.client.0.smithi042.stderr: 40250K .......... .......... .......... .......... .......... 29% 19.3M 5s 2024-04-20T10:35:45.527 INFO:tasks.workunit.client.0.smithi042.stderr: 40300K .......... .......... .......... .......... .......... 29% 32.7M 5s 2024-04-20T10:35:45.529 INFO:tasks.workunit.client.0.smithi042.stderr: 40350K .......... .......... .......... .......... .......... 29% 24.3M 5s 2024-04-20T10:35:45.531 INFO:tasks.workunit.client.0.smithi042.stderr: 40400K .......... .......... .......... .......... .......... 29% 25.7M 5s 2024-04-20T10:35:45.532 INFO:tasks.workunit.client.0.smithi042.stderr: 40450K .......... .......... .......... .......... .......... 29% 45.0M 5s 2024-04-20T10:35:45.535 INFO:tasks.workunit.client.0.smithi042.stderr: 40500K .......... .......... .......... .......... .......... 29% 15.5M 5s 2024-04-20T10:35:45.535 INFO:tasks.workunit.client.0.smithi042.stderr: 40550K .......... .......... .......... .......... .......... 29% 60.0M 5s 2024-04-20T10:35:45.537 INFO:tasks.workunit.client.0.smithi042.stderr: 40600K .......... .......... .......... .......... .......... 29% 31.6M 5s 2024-04-20T10:35:45.541 INFO:tasks.workunit.client.0.smithi042.stderr: 40650K .......... .......... .......... .......... .......... 29% 18.6M 5s 2024-04-20T10:35:45.541 INFO:tasks.workunit.client.0.smithi042.stderr: 40700K .......... .......... .......... .......... .......... 30% 52.7M 5s 2024-04-20T10:35:45.544 INFO:tasks.workunit.client.0.smithi042.stderr: 40750K .......... .......... .......... .......... .......... 30% 18.0M 5s 2024-04-20T10:35:45.544 INFO:tasks.workunit.client.0.smithi042.stderr: 40800K .......... .......... .......... .......... .......... 30% 62.4M 5s 2024-04-20T10:35:45.546 INFO:tasks.workunit.client.0.smithi042.stderr: 40850K .......... .......... .......... .......... .......... 30% 26.6M 5s 2024-04-20T10:35:45.548 INFO:tasks.workunit.client.0.smithi042.stderr: 40900K .......... .......... .......... .......... .......... 30% 27.4M 5s 2024-04-20T10:35:45.550 INFO:tasks.workunit.client.0.smithi042.stderr: 40950K .......... .......... .......... .......... .......... 30% 20.5M 5s 2024-04-20T10:35:45.554 INFO:tasks.workunit.client.0.smithi042.stderr: 41000K .......... .......... .......... .......... .......... 30% 15.8M 5s 2024-04-20T10:35:45.555 INFO:tasks.workunit.client.0.smithi042.stderr: 41050K .......... .......... .......... .......... .......... 30% 58.9M 5s 2024-04-20T10:35:45.556 INFO:tasks.workunit.client.0.smithi042.stderr: 41100K .......... .......... .......... .......... .......... 30% 24.0M 5s 2024-04-20T10:35:45.558 INFO:tasks.workunit.client.0.smithi042.stderr: 41150K .......... .......... .......... .......... .......... 30% 29.0M 5s 2024-04-20T10:35:45.560 INFO:tasks.workunit.client.0.smithi042.stderr: 41200K .......... .......... .......... .......... .......... 30% 26.5M 5s 2024-04-20T10:35:45.562 INFO:tasks.workunit.client.0.smithi042.stderr: 41250K .......... .......... .......... .......... .......... 30% 24.8M 5s 2024-04-20T10:35:45.562 INFO:tasks.workunit.client.0.smithi042.stderr: 41300K .......... .......... .......... .......... .......... 30% 58.5M 5s 2024-04-20T10:35:45.564 INFO:tasks.workunit.client.0.smithi042.stderr: 41350K .......... .......... .......... .......... .......... 30% 25.7M 5s 2024-04-20T10:35:45.566 INFO:tasks.workunit.client.0.smithi042.stderr: 41400K .......... .......... .......... .......... .......... 30% 26.2M 5s 2024-04-20T10:35:45.569 INFO:tasks.workunit.client.0.smithi042.stderr: 41450K .......... .......... .......... .......... .......... 30% 24.4M 5s 2024-04-20T10:35:45.571 INFO:tasks.workunit.client.0.smithi042.stderr: 41500K .......... .......... .......... .......... .......... 30% 20.8M 5s 2024-04-20T10:35:45.572 INFO:tasks.workunit.client.0.smithi042.stderr: 41550K .......... .......... .......... .......... .......... 30% 58.4M 5s 2024-04-20T10:35:45.574 INFO:tasks.workunit.client.0.smithi042.stderr: 41600K .......... .......... .......... .......... .......... 30% 23.9M 5s 2024-04-20T10:35:45.576 INFO:tasks.workunit.client.0.smithi042.stderr: 41650K .......... .......... .......... .......... .......... 30% 27.8M 5s 2024-04-20T10:35:45.577 INFO:tasks.workunit.client.0.smithi042.stderr: 41700K .......... .......... .......... .......... .......... 30% 25.6M 5s 2024-04-20T10:35:45.580 INFO:tasks.workunit.client.0.smithi042.stderr: 41750K .......... .......... .......... .......... .......... 30% 26.1M 5s 2024-04-20T10:35:45.580 INFO:tasks.workunit.client.0.smithi042.stderr: 41800K .......... .......... .......... .......... .......... 30% 43.1M 5s 2024-04-20T10:35:45.582 INFO:tasks.workunit.client.0.smithi042.stderr: 41850K .......... .......... .......... .......... .......... 30% 26.8M 5s 2024-04-20T10:35:45.583 INFO:tasks.workunit.client.0.smithi042.stderr: 41900K .......... .......... .......... .......... .......... 30% 28.8M 5s 2024-04-20T10:35:45.585 INFO:tasks.workunit.client.0.smithi042.stderr: 41950K .......... .......... .......... .......... .......... 30% 24.2M 5s 2024-04-20T10:35:45.588 INFO:tasks.workunit.client.0.smithi042.stderr: 42000K .......... .......... .......... .......... .......... 30% 27.6M 5s 2024-04-20T10:35:45.588 INFO:tasks.workunit.client.0.smithi042.stderr: 42050K .......... .......... .......... .......... .......... 31% 42.9M 5s 2024-04-20T10:35:45.590 INFO:tasks.workunit.client.0.smithi042.stderr: 42100K .......... .......... .......... .......... .......... 31% 25.2M 5s 2024-04-20T10:35:45.592 INFO:tasks.workunit.client.0.smithi042.stderr: 42150K .......... .......... .......... .......... .......... 31% 28.1M 5s 2024-04-20T10:35:45.595 INFO:tasks.workunit.client.0.smithi042.stderr: 42200K .......... .......... .......... .......... .......... 31% 25.4M 5s 2024-04-20T10:35:45.596 INFO:tasks.workunit.client.0.smithi042.stderr: 42250K .......... .......... .......... .......... .......... 31% 27.3M 5s 2024-04-20T10:35:45.597 INFO:tasks.workunit.client.0.smithi042.stderr: 42300K .......... .......... .......... .......... .......... 31% 47.0M 5s 2024-04-20T10:35:45.599 INFO:tasks.workunit.client.0.smithi042.stderr: 42350K .......... .......... .......... .......... .......... 31% 26.5M 5s 2024-04-20T10:35:45.601 INFO:tasks.workunit.client.0.smithi042.stderr: 42400K .......... .......... .......... .......... .......... 31% 26.1M 5s 2024-04-20T10:35:45.602 INFO:tasks.workunit.client.0.smithi042.stderr: 42450K .......... .......... .......... .......... .......... 31% 27.3M 5s 2024-04-20T10:35:45.605 INFO:tasks.workunit.client.0.smithi042.stderr: 42500K .......... .......... .......... .......... .......... 31% 26.3M 5s 2024-04-20T10:35:45.605 INFO:tasks.workunit.client.0.smithi042.stderr: 42550K .......... .......... .......... .......... .......... 31% 44.4M 5s 2024-04-20T10:35:45.608 INFO:tasks.workunit.client.0.smithi042.stderr: 42600K .......... .......... .......... .......... .......... 31% 24.2M 5s 2024-04-20T10:35:45.608 INFO:tasks.workunit.client.0.smithi042.stderr: 42650K .......... .......... .......... .......... .......... 31% 34.2M 5s 2024-04-20T10:35:45.621 INFO:tasks.workunit.client.0.smithi042.stderr: 42700K .......... .......... .......... .......... .......... 31% 23.3M 5s 2024-04-20T10:35:45.621 INFO:tasks.workunit.client.0.smithi042.stderr: 42750K .......... .......... .......... .......... .......... 31% 26.3M 5s 2024-04-20T10:35:45.621 INFO:tasks.workunit.client.0.smithi042.stderr: 42800K .......... .......... .......... .......... .......... 31% 31.3M 5s 2024-04-20T10:35:45.621 INFO:tasks.workunit.client.0.smithi042.stderr: 42850K .......... .......... .......... .......... .......... 31% 29.5M 5s 2024-04-20T10:35:45.622 INFO:tasks.workunit.client.0.smithi042.stderr: 42900K .......... .......... .......... .......... .......... 31% 27.5M 5s 2024-04-20T10:35:45.622 INFO:tasks.workunit.client.0.smithi042.stderr: 42950K .......... .......... .......... .......... .......... 31% 26.2M 5s 2024-04-20T10:35:45.622 INFO:tasks.workunit.client.0.smithi042.stderr: 43000K .......... .......... .......... .......... .......... 31% 38.5M 5s 2024-04-20T10:35:45.623 INFO:tasks.workunit.client.0.smithi042.stderr: 43050K .......... .......... .......... .......... .......... 31% 24.8M 5s 2024-04-20T10:35:45.625 INFO:tasks.workunit.client.0.smithi042.stderr: 43100K .......... .......... .......... .......... .......... 31% 27.4M 5s 2024-04-20T10:35:45.626 INFO:tasks.workunit.client.0.smithi042.stderr: 43150K .......... .......... .......... .......... .......... 31% 30.1M 5s 2024-04-20T10:35:45.629 INFO:tasks.workunit.client.0.smithi042.stderr: 43200K .......... .......... .......... .......... .......... 31% 16.2M 5s 2024-04-20T10:35:45.631 INFO:tasks.workunit.client.0.smithi042.stderr: 43250K .......... .......... .......... .......... .......... 31% 37.0M 5s 2024-04-20T10:35:45.632 INFO:tasks.workunit.client.0.smithi042.stderr: 43300K .......... .......... .......... .......... .......... 31% 37.9M 5s 2024-04-20T10:35:45.634 INFO:tasks.workunit.client.0.smithi042.stderr: 43350K .......... .......... .......... .......... .......... 31% 37.3M 5s 2024-04-20T10:35:45.635 INFO:tasks.workunit.client.0.smithi042.stderr: 43400K .......... .......... .......... .......... .......... 32% 28.2M 5s 2024-04-20T10:35:45.637 INFO:tasks.workunit.client.0.smithi042.stderr: 43450K .......... .......... .......... .......... .......... 32% 26.2M 5s 2024-04-20T10:35:45.638 INFO:tasks.workunit.client.0.smithi042.stderr: 43500K .......... .......... .......... .......... .......... 32% 24.7M 5s 2024-04-20T10:35:45.640 INFO:tasks.workunit.client.0.smithi042.stderr: 43550K .......... .......... .......... .......... .......... 32% 35.2M 5s 2024-04-20T10:35:45.642 INFO:tasks.workunit.client.0.smithi042.stderr: 43600K .......... .......... .......... .......... .......... 32% 26.7M 5s 2024-04-20T10:35:45.643 INFO:tasks.workunit.client.0.smithi042.stderr: 43650K .......... .......... .......... .......... .......... 32% 26.5M 5s 2024-04-20T10:35:45.645 INFO:tasks.workunit.client.0.smithi042.stderr: 43700K .......... .......... .......... .......... .......... 32% 31.1M 5s 2024-04-20T10:35:45.647 INFO:tasks.workunit.client.0.smithi042.stderr: 43750K .......... .......... .......... .......... .......... 32% 25.5M 5s 2024-04-20T10:35:45.648 INFO:tasks.workunit.client.0.smithi042.stderr: 43800K .......... .......... .......... .......... .......... 32% 36.2M 5s 2024-04-20T10:35:45.650 INFO:tasks.workunit.client.0.smithi042.stderr: 43850K .......... .......... .......... .......... .......... 32% 29.3M 5s 2024-04-20T10:35:45.652 INFO:tasks.workunit.client.0.smithi042.stderr: 43900K .......... .......... .......... .......... .......... 32% 24.7M 5s 2024-04-20T10:35:45.653 INFO:tasks.workunit.client.0.smithi042.stderr: 43950K .......... .......... .......... .......... .......... 32% 35.0M 5s 2024-04-20T10:35:45.655 INFO:tasks.workunit.client.0.smithi042.stderr: 44000K .......... .......... .......... .......... .......... 32% 26.0M 5s 2024-04-20T10:35:45.657 INFO:tasks.workunit.client.0.smithi042.stderr: 44050K .......... .......... .......... .......... .......... 32% 34.5M 5s 2024-04-20T10:35:45.658 INFO:tasks.workunit.client.0.smithi042.stderr: 44100K .......... .......... .......... .......... .......... 32% 27.8M 5s 2024-04-20T10:35:45.660 INFO:tasks.workunit.client.0.smithi042.stderr: 44150K .......... .......... .......... .......... .......... 32% 26.6M 5s 2024-04-20T10:35:45.662 INFO:tasks.workunit.client.0.smithi042.stderr: 44200K .......... .......... .......... .......... .......... 32% 29.2M 5s 2024-04-20T10:35:45.684 INFO:tasks.workunit.client.0.smithi042.stderr: 44250K .......... .......... .......... .......... .......... 32% 25.7M 5s 2024-04-20T10:35:45.684 INFO:tasks.workunit.client.0.smithi042.stderr: 44300K .......... .......... .......... .......... .......... 32% 38.0M 5s 2024-04-20T10:35:45.684 INFO:tasks.workunit.client.0.smithi042.stderr: 44350K .......... .......... .......... .......... .......... 32% 26.5M 5s 2024-04-20T10:35:45.684 INFO:tasks.workunit.client.0.smithi042.stderr: 44400K .......... .......... .......... .......... .......... 32% 24.8M 5s 2024-04-20T10:35:45.684 INFO:tasks.workunit.client.0.smithi042.stderr: 44450K .......... .......... .......... .......... .......... 32% 33.9M 5s 2024-04-20T10:35:45.685 INFO:tasks.workunit.client.0.smithi042.stderr: 44500K .......... .......... .......... .......... .......... 32% 26.3M 5s 2024-04-20T10:35:45.685 INFO:tasks.workunit.client.0.smithi042.stderr: 44550K .......... .......... .......... .......... .......... 32% 44.7M 5s 2024-04-20T10:35:45.685 INFO:tasks.workunit.client.0.smithi042.stderr: 44600K .......... .......... .......... .......... .......... 32% 26.2M 5s 2024-04-20T10:35:45.685 INFO:tasks.workunit.client.0.smithi042.stderr: 44650K .......... .......... .......... .......... .......... 32% 24.5M 5s 2024-04-20T10:35:45.685 INFO:tasks.workunit.client.0.smithi042.stderr: 44700K .......... .......... .......... .......... .......... 32% 38.6M 5s 2024-04-20T10:35:45.685 INFO:tasks.workunit.client.0.smithi042.stderr: 44750K .......... .......... .......... .......... .......... 33% 22.1M 5s 2024-04-20T10:35:45.685 INFO:tasks.workunit.client.0.smithi042.stderr: 44800K .......... .......... .......... .......... .......... 33% 43.0M 5s 2024-04-20T10:35:45.685 INFO:tasks.workunit.client.0.smithi042.stderr: 44850K .......... .......... .......... .......... .......... 33% 24.6M 5s 2024-04-20T10:35:45.686 INFO:tasks.workunit.client.0.smithi042.stderr: 44900K .......... .......... .......... .......... .......... 33% 25.6M 5s 2024-04-20T10:35:45.687 INFO:tasks.workunit.client.0.smithi042.stderr: 44950K .......... .......... .......... .......... .......... 33% 31.5M 5s 2024-04-20T10:35:45.689 INFO:tasks.workunit.client.0.smithi042.stderr: 45000K .......... .......... .......... .......... .......... 33% 24.7M 5s 2024-04-20T10:35:45.692 INFO:tasks.workunit.client.0.smithi042.stderr: 45050K .......... .......... .......... .......... .......... 33% 34.4M 5s 2024-04-20T10:35:45.693 INFO:tasks.workunit.client.0.smithi042.stderr: 45100K .......... .......... .......... .......... .......... 33% 24.4M 5s 2024-04-20T10:35:45.695 INFO:tasks.workunit.client.0.smithi042.stderr: 45150K .......... .......... .......... .......... .......... 33% 25.1M 5s 2024-04-20T10:35:45.696 INFO:tasks.workunit.client.0.smithi042.stderr: 45200K .......... .......... .......... .......... .......... 33% 32.1M 5s 2024-04-20T10:35:45.698 INFO:tasks.workunit.client.0.smithi042.stderr: 45250K .......... .......... .......... .......... .......... 33% 26.0M 5s 2024-04-20T10:35:45.700 INFO:tasks.workunit.client.0.smithi042.stderr: 45300K .......... .......... .......... .......... .......... 33% 23.6M 5s 2024-04-20T10:35:45.702 INFO:tasks.workunit.client.0.smithi042.stderr: 45350K .......... .......... .......... .......... .......... 33% 30.3M 5s 2024-04-20T10:35:45.704 INFO:tasks.workunit.client.0.smithi042.stderr: 45400K .......... .......... .......... .......... .......... 33% 24.9M 5s 2024-04-20T10:35:45.705 INFO:tasks.workunit.client.0.smithi042.stderr: 45450K .......... .......... .......... .......... .......... 33% 31.6M 5s 2024-04-20T10:35:45.707 INFO:tasks.workunit.client.0.smithi042.stderr: 45500K .......... .......... .......... .......... .......... 33% 24.7M 5s 2024-04-20T10:35:45.710 INFO:tasks.workunit.client.0.smithi042.stderr: 45550K .......... .......... .......... .......... .......... 33% 25.3M 5s 2024-04-20T10:35:45.710 INFO:tasks.workunit.client.0.smithi042.stderr: 45600K .......... .......... .......... .......... .......... 33% 53.8M 5s 2024-04-20T10:35:45.713 INFO:tasks.workunit.client.0.smithi042.stderr: 45650K .......... .......... .......... .......... .......... 33% 21.2M 5s 2024-04-20T10:35:45.714 INFO:tasks.workunit.client.0.smithi042.stderr: 45700K .......... .......... .......... .......... .......... 33% 32.3M 5s 2024-04-20T10:35:45.716 INFO:tasks.workunit.client.0.smithi042.stderr: 45750K .......... .......... .......... .......... .......... 33% 20.7M 5s 2024-04-20T10:35:45.720 INFO:tasks.workunit.client.0.smithi042.stderr: 45800K .......... .......... .......... .......... .......... 33% 16.9M 5s 2024-04-20T10:35:45.721 INFO:tasks.workunit.client.0.smithi042.stderr: 45850K .......... .......... .......... .......... .......... 33% 29.7M 5s 2024-04-20T10:35:45.723 INFO:tasks.workunit.client.0.smithi042.stderr: 45900K .......... .......... .......... .......... .......... 33% 26.3M 5s 2024-04-20T10:35:45.724 INFO:tasks.workunit.client.0.smithi042.stderr: 45950K .......... .......... .......... .......... .......... 33% 30.1M 5s 2024-04-20T10:35:45.726 INFO:tasks.workunit.client.0.smithi042.stderr: 46000K .......... .......... .......... .......... .......... 33% 27.1M 5s 2024-04-20T10:35:45.729 INFO:tasks.workunit.client.0.smithi042.stderr: 46050K .......... .......... .......... .......... .......... 33% 24.7M 5s 2024-04-20T10:35:45.729 INFO:tasks.workunit.client.0.smithi042.stderr: 46100K .......... .......... .......... .......... .......... 33% 49.8M 5s 2024-04-20T10:35:45.731 INFO:tasks.workunit.client.0.smithi042.stderr: 46150K .......... .......... .......... .......... .......... 34% 24.4M 5s 2024-04-20T10:35:45.733 INFO:tasks.workunit.client.0.smithi042.stderr: 46200K .......... .......... .......... .......... .......... 34% 31.4M 5s 2024-04-20T10:35:45.734 INFO:tasks.workunit.client.0.smithi042.stderr: 46250K .......... .......... .......... .......... .......... 34% 31.5M 5s 2024-04-20T10:35:45.737 INFO:tasks.workunit.client.0.smithi042.stderr: 46300K .......... .......... .......... .......... .......... 34% 22.2M 5s 2024-04-20T10:35:45.738 INFO:tasks.workunit.client.0.smithi042.stderr: 46350K .......... .......... .......... .......... .......... 34% 25.6M 5s 2024-04-20T10:35:45.740 INFO:tasks.workunit.client.0.smithi042.stderr: 46400K .......... .......... .......... .......... .......... 34% 27.6M 5s 2024-04-20T10:35:45.742 INFO:tasks.workunit.client.0.smithi042.stderr: 46450K .......... .......... .......... .......... .......... 34% 32.1M 5s 2024-04-20T10:35:45.743 INFO:tasks.workunit.client.0.smithi042.stderr: 46500K .......... .......... .......... .......... .......... 34% 26.3M 5s 2024-04-20T10:35:45.746 INFO:tasks.workunit.client.0.smithi042.stderr: 46550K .......... .......... .......... .......... .......... 34% 23.2M 5s 2024-04-20T10:35:45.747 INFO:tasks.workunit.client.0.smithi042.stderr: 46600K .......... .......... .......... .......... .......... 34% 40.3M 4s 2024-04-20T10:35:45.749 INFO:tasks.workunit.client.0.smithi042.stderr: 46650K .......... .......... .......... .......... .......... 34% 26.3M 4s 2024-04-20T10:35:45.750 INFO:tasks.workunit.client.0.smithi042.stderr: 46700K .......... .......... .......... .......... .......... 34% 31.9M 4s 2024-04-20T10:35:45.752 INFO:tasks.workunit.client.0.smithi042.stderr: 46750K .......... .......... .......... .......... .......... 34% 25.2M 4s 2024-04-20T10:35:45.755 INFO:tasks.workunit.client.0.smithi042.stderr: 46800K .......... .......... .......... .......... .......... 34% 26.2M 4s 2024-04-20T10:35:45.755 INFO:tasks.workunit.client.0.smithi042.stderr: 46850K .......... .......... .......... .......... .......... 34% 41.8M 4s 2024-04-20T10:35:45.759 INFO:tasks.workunit.client.0.smithi042.stderr: 46900K .......... .......... .......... .......... .......... 34% 26.5M 4s 2024-04-20T10:35:45.759 INFO:tasks.workunit.client.0.smithi042.stderr: 46950K .......... .......... .......... .......... .......... 34% 29.3M 4s 2024-04-20T10:35:45.761 INFO:tasks.workunit.client.0.smithi042.stderr: 47000K .......... .......... .......... .......... .......... 34% 25.4M 4s 2024-04-20T10:35:45.763 INFO:tasks.workunit.client.0.smithi042.stderr: 47050K .......... .......... .......... .......... .......... 34% 26.3M 4s 2024-04-20T10:35:45.764 INFO:tasks.workunit.client.0.smithi042.stderr: 47100K .......... .......... .......... .......... .......... 34% 39.4M 4s 2024-04-20T10:35:45.766 INFO:tasks.workunit.client.0.smithi042.stderr: 47150K .......... .......... .......... .......... .......... 34% 24.8M 4s 2024-04-20T10:35:45.767 INFO:tasks.workunit.client.0.smithi042.stderr: 47200K .......... .......... .......... .......... .......... 34% 35.3M 4s 2024-04-20T10:35:45.769 INFO:tasks.workunit.client.0.smithi042.stderr: 47250K .......... .......... .......... .......... .......... 34% 25.2M 4s 2024-04-20T10:35:45.772 INFO:tasks.workunit.client.0.smithi042.stderr: 47300K .......... .......... .......... .......... .......... 34% 24.7M 4s 2024-04-20T10:35:45.772 INFO:tasks.workunit.client.0.smithi042.stderr: 47350K .......... .......... .......... .......... .......... 34% 46.5M 4s 2024-04-20T10:35:45.774 INFO:tasks.workunit.client.0.smithi042.stderr: 47400K .......... .......... .......... .......... .......... 34% 22.3M 4s 2024-04-20T10:35:45.775 INFO:tasks.workunit.client.0.smithi042.stderr: 47450K .......... .......... .......... .......... .......... 34% 36.3M 4s 2024-04-20T10:35:45.778 INFO:tasks.workunit.client.0.smithi042.stderr: 47500K .......... .......... .......... .......... .......... 35% 22.8M 4s 2024-04-20T10:35:45.780 INFO:tasks.workunit.client.0.smithi042.stderr: 47550K .......... .......... .......... .......... .......... 35% 25.8M 4s 2024-04-20T10:35:45.781 INFO:tasks.workunit.client.0.smithi042.stderr: 47600K .......... .......... .......... .......... .......... 35% 44.6M 4s 2024-04-20T10:35:45.783 INFO:tasks.workunit.client.0.smithi042.stderr: 47650K .......... .......... .......... .......... .......... 35% 23.2M 4s 2024-04-20T10:35:45.784 INFO:tasks.workunit.client.0.smithi042.stderr: 47700K .......... .......... .......... .......... .......... 35% 36.6M 4s 2024-04-20T10:35:45.786 INFO:tasks.workunit.client.0.smithi042.stderr: 47750K .......... .......... .......... .......... .......... 35% 24.3M 4s 2024-04-20T10:35:45.788 INFO:tasks.workunit.client.0.smithi042.stderr: 47800K .......... .......... .......... .......... .......... 35% 27.6M 4s 2024-04-20T10:35:45.789 INFO:tasks.workunit.client.0.smithi042.stderr: 47850K .......... .......... .......... .......... .......... 35% 34.7M 4s 2024-04-20T10:35:45.791 INFO:tasks.workunit.client.0.smithi042.stderr: 47900K .......... .......... .......... .......... .......... 35% 25.0M 4s 2024-04-20T10:35:45.793 INFO:tasks.workunit.client.0.smithi042.stderr: 47950K .......... .......... .......... .......... .......... 35% 37.4M 4s 2024-04-20T10:35:45.794 INFO:tasks.workunit.client.0.smithi042.stderr: 48000K .......... .......... .......... .......... .......... 35% 24.7M 4s 2024-04-20T10:35:45.795 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:35:45.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:35:45.796 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:35:45.797 INFO:tasks.workunit.client.0.smithi042.stderr: 48050K .......... .......... .......... .......... .......... 35% 27.5M 4s 2024-04-20T10:35:45.798 INFO:tasks.workunit.client.0.smithi042.stderr: 48100K .......... .......... .......... .......... .......... 35% 35.6M 4s 2024-04-20T10:35:45.804 INFO:tasks.workunit.client.0.smithi042.stderr: 48150K .......... .......... .......... .......... .......... 35% 25.3M 4s 2024-04-20T10:35:45.804 INFO:tasks.workunit.client.0.smithi042.stderr: 48200K .......... .......... .......... .......... .......... 35% 35.2M 4s 2024-04-20T10:35:45.804 INFO:tasks.workunit.client.0.smithi042.stderr: 48250K .......... .......... .......... .......... .......... 35% 25.4M 4s 2024-04-20T10:35:45.806 INFO:tasks.workunit.client.0.smithi042.stderr: 48300K .......... .......... .......... .......... .......... 35% 26.8M 4s 2024-04-20T10:35:45.806 INFO:tasks.workunit.client.0.smithi042.stderr: 48350K .......... .......... .......... .......... .......... 35% 50.1M 4s 2024-04-20T10:35:45.808 INFO:tasks.workunit.client.0.smithi042.stderr: 48400K .......... .......... .......... .......... .......... 35% 21.7M 4s 2024-04-20T10:35:45.810 INFO:tasks.workunit.client.0.smithi042.stderr: 48450K .......... .......... .......... .......... .......... 35% 25.3M 4s 2024-04-20T10:35:45.811 INFO:tasks.workunit.client.0.smithi042.stderr: 48500K .......... .......... .......... .......... .......... 35% 35.6M 4s 2024-04-20T10:35:45.813 INFO:tasks.workunit.client.0.smithi042.stderr: 48550K .......... .......... .......... .......... .......... 35% 25.2M 4s 2024-04-20T10:35:45.814 INFO:tasks.workunit.client.0.smithi042.stderr: 48600K .......... .......... .......... .......... .......... 35% 39.1M 4s 2024-04-20T10:35:45.816 INFO:tasks.workunit.client.0.smithi042.stderr: 48650K .......... .......... .......... .......... .......... 35% 26.3M 4s 2024-04-20T10:35:45.818 INFO:tasks.workunit.client.0.smithi042.stderr: 48700K .......... .......... .......... .......... .......... 35% 22.5M 4s 2024-04-20T10:35:45.820 INFO:tasks.workunit.client.0.smithi042.stderr: 48750K .......... .......... .......... .......... .......... 35% 35.1M 4s 2024-04-20T10:35:45.822 INFO:tasks.workunit.client.0.smithi042.stderr: 48800K .......... .......... .......... .......... .......... 35% 24.7M 4s 2024-04-20T10:35:45.823 INFO:tasks.workunit.client.0.smithi042.stderr: 48850K .......... .......... .......... .......... .......... 36% 35.4M 4s 2024-04-20T10:35:45.825 INFO:tasks.workunit.client.0.smithi042.stderr: 48900K .......... .......... .......... .......... .......... 36% 27.7M 4s 2024-04-20T10:35:45.827 INFO:tasks.workunit.client.0.smithi042.stderr: 48950K .......... .......... .......... .......... .......... 36% 24.9M 4s 2024-04-20T10:35:45.828 INFO:tasks.workunit.client.0.smithi042.stderr: 49000K .......... .......... .......... .......... .......... 36% 33.5M 4s 2024-04-20T10:35:45.830 INFO:tasks.workunit.client.0.smithi042.stderr: 49050K .......... .......... .......... .......... .......... 36% 26.0M 4s 2024-04-20T10:35:45.831 INFO:tasks.workunit.client.0.smithi042.stderr: 49100K .......... .......... .......... .......... .......... 36% 42.6M 4s 2024-04-20T10:35:45.833 INFO:tasks.workunit.client.0.smithi042.stderr: 49150K .......... .......... .......... .......... .......... 36% 25.3M 4s 2024-04-20T10:35:45.836 INFO:tasks.workunit.client.0.smithi042.stderr: 49200K .......... .......... .......... .......... .......... 36% 24.3M 4s 2024-04-20T10:35:45.837 INFO:tasks.workunit.client.0.smithi042.stderr: 49250K .......... .......... .......... .......... .......... 36% 37.7M 4s 2024-04-20T10:35:45.839 INFO:tasks.workunit.client.0.smithi042.stderr: 49300K .......... .......... .......... .......... .......... 36% 23.7M 4s 2024-04-20T10:35:45.840 INFO:tasks.workunit.client.0.smithi042.stderr: 49350K .......... .......... .......... .......... .......... 36% 43.0M 4s 2024-04-20T10:35:45.842 INFO:tasks.workunit.client.0.smithi042.stderr: 49400K .......... .......... .......... .......... .......... 36% 23.2M 4s 2024-04-20T10:35:45.844 INFO:tasks.workunit.client.0.smithi042.stderr: 49450K .......... .......... .......... .......... .......... 36% 24.6M 4s 2024-04-20T10:35:45.846 INFO:tasks.workunit.client.0.smithi042.stderr: 49500K .......... .......... .......... .......... .......... 36% 35.2M 4s 2024-04-20T10:35:45.847 INFO:tasks.workunit.client.0.smithi042.stderr: 49550K .......... .......... .......... .......... .......... 36% 23.7M 4s 2024-04-20T10:35:45.848 INFO:tasks.workunit.client.0.smithi042.stderr: 49600K .......... .......... .......... .......... .......... 36% 48.1M 4s 2024-04-20T10:35:45.850 INFO:tasks.workunit.client.0.smithi042.stderr: 49650K .......... .......... .......... .......... .......... 36% 25.7M 4s 2024-04-20T10:35:45.852 INFO:tasks.workunit.client.0.smithi042.stderr: 49700K .......... .......... .......... .......... .......... 36% 23.2M 4s 2024-04-20T10:35:45.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:45 smithi042 ceph-mon[17078]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-20T10:35:45.854 INFO:tasks.workunit.client.0.smithi042.stderr: 49750K .......... .......... .......... .......... .......... 36% 39.1M 4s 2024-04-20T10:35:45.856 INFO:tasks.workunit.client.0.smithi042.stderr: 49800K .......... .......... .......... .......... .......... 36% 19.5M 4s 2024-04-20T10:35:45.857 INFO:tasks.workunit.client.0.smithi042.stderr: 49850K .......... .......... .......... .......... .......... 36% 39.0M 4s 2024-04-20T10:35:45.859 INFO:tasks.workunit.client.0.smithi042.stderr: 49900K .......... .......... .......... .......... .......... 36% 27.3M 4s 2024-04-20T10:35:45.861 INFO:tasks.workunit.client.0.smithi042.stderr: 49950K .......... .......... .......... .......... .......... 36% 24.6M 4s 2024-04-20T10:35:45.862 INFO:tasks.workunit.client.0.smithi042.stderr: 50000K .......... .......... .......... .......... .......... 36% 36.7M 4s 2024-04-20T10:35:45.864 INFO:tasks.workunit.client.0.smithi042.stderr: 50050K .......... .......... .......... .......... .......... 36% 24.4M 4s 2024-04-20T10:35:45.867 INFO:tasks.workunit.client.0.smithi042.stderr: 50100K .......... .......... .......... .......... .......... 36% 37.4M 4s 2024-04-20T10:35:45.868 INFO:tasks.workunit.client.0.smithi042.stderr: 50150K .......... .......... .......... .......... .......... 36% 24.1M 4s 2024-04-20T10:35:45.870 INFO:tasks.workunit.client.0.smithi042.stderr: 50200K .......... .......... .......... .......... .......... 37% 24.5M 4s 2024-04-20T10:35:45.871 INFO:tasks.workunit.client.0.smithi042.stderr: 50250K .......... .......... .......... .......... .......... 37% 35.5M 4s 2024-04-20T10:35:45.873 INFO:tasks.workunit.client.0.smithi042.stderr: 50300K .......... .......... .......... .......... .......... 37% 24.0M 4s 2024-04-20T10:35:45.874 INFO:tasks.workunit.client.0.smithi042.stderr: 50350K .......... .......... .......... .......... .......... 37% 47.0M 4s 2024-04-20T10:35:45.876 INFO:tasks.workunit.client.0.smithi042.stderr: 50400K .......... .......... .......... .......... .......... 37% 22.7M 4s 2024-04-20T10:35:45.879 INFO:tasks.workunit.client.0.smithi042.stderr: 50450K .......... .......... .......... .......... .......... 37% 21.4M 4s 2024-04-20T10:35:45.880 INFO:tasks.workunit.client.0.smithi042.stderr: 50500K .......... .......... .......... .......... .......... 37% 43.1M 4s 2024-04-20T10:35:45.882 INFO:tasks.workunit.client.0.smithi042.stderr: 50550K .......... .......... .......... .......... .......... 37% 20.8M 4s 2024-04-20T10:35:45.884 INFO:tasks.workunit.client.0.smithi042.stderr: 50600K .......... .......... .......... .......... .......... 37% 24.0M 4s 2024-04-20T10:35:45.887 INFO:tasks.workunit.client.0.smithi042.stderr: 50650K .......... .......... .......... .......... .......... 37% 28.5M 4s 2024-04-20T10:35:45.889 INFO:tasks.workunit.client.0.smithi042.stderr: 50700K .......... .......... .......... .......... .......... 37% 20.8M 4s 2024-04-20T10:35:45.891 INFO:tasks.workunit.client.0.smithi042.stderr: 50750K .......... .......... .......... .......... .......... 37% 23.9M 4s 2024-04-20T10:35:45.893 INFO:tasks.workunit.client.0.smithi042.stderr: 50800K .......... .......... .......... .......... .......... 37% 23.5M 4s 2024-04-20T10:35:45.894 INFO:tasks.workunit.client.0.smithi042.stderr: 50850K .......... .......... .......... .......... .......... 37% 29.2M 4s 2024-04-20T10:35:45.895 INFO:tasks.workunit.client.0.smithi042.stderr: 50900K .......... .......... .......... .......... .......... 37% 40.8M 4s 2024-04-20T10:35:45.897 INFO:tasks.workunit.client.0.smithi042.stderr: 50950K .......... .......... .......... .......... .......... 37% 36.1M 4s 2024-04-20T10:35:45.900 INFO:tasks.workunit.client.0.smithi042.stderr: 51000K .......... .......... .......... .......... .......... 37% 22.7M 4s 2024-04-20T10:35:45.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:45 smithi162 ceph-mon[21854]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-20T10:35:45.906 INFO:tasks.workunit.client.0.smithi042.stderr: 51050K .......... .......... .......... .......... .......... 37% 25.4M 4s 2024-04-20T10:35:45.906 INFO:tasks.workunit.client.0.smithi042.stderr: 51100K .......... .......... .......... .......... .......... 37% 42.3M 4s 2024-04-20T10:35:45.906 INFO:tasks.workunit.client.0.smithi042.stderr: 51150K .......... .......... .......... .......... .......... 37% 27.0M 4s 2024-04-20T10:35:45.907 INFO:tasks.workunit.client.0.smithi042.stderr: 51200K .......... .......... .......... .......... .......... 37% 23.0M 4s 2024-04-20T10:35:45.909 INFO:tasks.workunit.client.0.smithi042.stderr: 51250K .......... .......... .......... .......... .......... 37% 23.0M 4s 2024-04-20T10:35:45.913 INFO:tasks.workunit.client.0.smithi042.stderr: 51300K .......... .......... .......... .......... .......... 37% 50.2M 4s 2024-04-20T10:35:45.913 INFO:tasks.workunit.client.0.smithi042.stderr: 51350K .......... .......... .......... .......... .......... 37% 18.5M 4s 2024-04-20T10:35:45.913 INFO:tasks.workunit.client.0.smithi042.stderr: 51400K .......... .......... .......... .......... .......... 37% 53.5M 4s 2024-04-20T10:35:45.919 INFO:tasks.workunit.client.0.smithi042.stderr: 51450K .......... .......... .......... .......... .......... 37% 22.8M 4s 2024-04-20T10:35:45.919 INFO:tasks.workunit.client.0.smithi042.stderr: 51500K .......... .......... .......... .......... .......... 37% 26.3M 4s 2024-04-20T10:35:45.920 INFO:tasks.workunit.client.0.smithi042.stderr: 51550K .......... .......... .......... .......... .......... 38% 47.7M 4s 2024-04-20T10:35:45.923 INFO:tasks.workunit.client.0.smithi042.stderr: 51600K .......... .......... .......... .......... .......... 38% 19.8M 4s 2024-04-20T10:35:45.923 INFO:tasks.workunit.client.0.smithi042.stderr: 51650K .......... .......... .......... .......... .......... 38% 36.4M 4s 2024-04-20T10:35:45.924 INFO:tasks.workunit.client.0.smithi042.stderr: 51700K .......... .......... .......... .......... .......... 38% 26.5M 4s 2024-04-20T10:35:45.925 INFO:tasks.workunit.client.0.smithi042.stderr: 51750K .......... .......... .......... .......... .......... 38% 33.6M 4s 2024-04-20T10:35:45.926 INFO:tasks.workunit.client.0.smithi042.stderr: 51800K .......... .......... .......... .......... .......... 38% 25.8M 4s 2024-04-20T10:35:45.928 INFO:tasks.workunit.client.0.smithi042.stderr: 51850K .......... .......... .......... .......... .......... 38% 30.8M 4s 2024-04-20T10:35:45.930 INFO:tasks.workunit.client.0.smithi042.stderr: 51900K .......... .......... .......... .......... .......... 38% 28.0M 4s 2024-04-20T10:35:45.932 INFO:tasks.workunit.client.0.smithi042.stderr: 51950K .......... .......... .......... .......... .......... 38% 29.4M 4s 2024-04-20T10:35:45.934 INFO:tasks.workunit.client.0.smithi042.stderr: 52000K .......... .......... .......... .......... .......... 38% 25.7M 4s 2024-04-20T10:35:45.936 INFO:tasks.workunit.client.0.smithi042.stderr: 52050K .......... .......... .......... .......... .......... 38% 25.1M 4s 2024-04-20T10:35:45.938 INFO:tasks.workunit.client.0.smithi042.stderr: 52100K .......... .......... .......... .......... .......... 38% 27.2M 4s 2024-04-20T10:35:45.938 INFO:tasks.workunit.client.0.smithi042.stderr: 52150K .......... .......... .......... .......... .......... 38% 38.2M 4s 2024-04-20T10:35:45.940 INFO:tasks.workunit.client.0.smithi042.stderr: 52200K .......... .......... .......... .......... .......... 38% 27.3M 4s 2024-04-20T10:35:45.942 INFO:tasks.workunit.client.0.smithi042.stderr: 52250K .......... .......... .......... .......... .......... 38% 30.1M 4s 2024-04-20T10:35:45.943 INFO:tasks.workunit.client.0.smithi042.stderr: 52300K .......... .......... .......... .......... .......... 38% 27.4M 4s 2024-04-20T10:35:45.947 INFO:tasks.workunit.client.0.smithi042.stderr: 52350K .......... .......... .......... .......... .......... 38% 22.5M 4s 2024-04-20T10:35:45.947 INFO:tasks.workunit.client.0.smithi042.stderr: 52400K .......... .......... .......... .......... .......... 38% 47.4M 4s 2024-04-20T10:35:45.949 INFO:tasks.workunit.client.0.smithi042.stderr: 52450K .......... .......... .......... .......... .......... 38% 22.3M 4s 2024-04-20T10:35:45.950 INFO:tasks.workunit.client.0.smithi042.stderr: 52500K .......... .......... .......... .......... .......... 38% 32.6M 4s 2024-04-20T10:35:45.952 INFO:tasks.workunit.client.0.smithi042.stderr: 52550K .......... .......... .......... .......... .......... 38% 24.0M 4s 2024-04-20T10:35:45.955 INFO:tasks.workunit.client.0.smithi042.stderr: 52600K .......... .......... .......... .......... .......... 38% 26.8M 4s 2024-04-20T10:35:45.955 INFO:tasks.workunit.client.0.smithi042.stderr: 52650K .......... .......... .......... .......... .......... 38% 37.4M 4s 2024-04-20T10:35:45.958 INFO:tasks.workunit.client.0.smithi042.stderr: 52700K .......... .......... .......... .......... .......... 38% 23.9M 4s 2024-04-20T10:35:45.959 INFO:tasks.workunit.client.0.smithi042.stderr: 52750K .......... .......... .......... .......... .......... 38% 35.3M 4s 2024-04-20T10:35:45.961 INFO:tasks.workunit.client.0.smithi042.stderr: 52800K .......... .......... .......... .......... .......... 38% 27.0M 4s 2024-04-20T10:35:45.963 INFO:tasks.workunit.client.0.smithi042.stderr: 52850K .......... .......... .......... .......... .......... 38% 23.6M 4s 2024-04-20T10:35:45.964 INFO:tasks.workunit.client.0.smithi042.stderr: 52900K .......... .......... .......... .......... .......... 39% 39.0M 4s 2024-04-20T10:35:45.966 INFO:tasks.workunit.client.0.smithi042.stderr: 52950K .......... .......... .......... .......... .......... 39% 23.4M 4s 2024-04-20T10:35:45.968 INFO:tasks.workunit.client.0.smithi042.stderr: 53000K .......... .......... .......... .......... .......... 39% 37.7M 4s 2024-04-20T10:35:45.969 INFO:tasks.workunit.client.0.smithi042.stderr: 53050K .......... .......... .......... .......... .......... 39% 25.2M 4s 2024-04-20T10:35:45.973 INFO:tasks.workunit.client.0.smithi042.stderr: 53100K .......... .......... .......... .......... .......... 39% 26.6M 4s 2024-04-20T10:35:45.973 INFO:tasks.workunit.client.0.smithi042.stderr: 53150K .......... .......... .......... .......... .......... 39% 40.7M 4s 2024-04-20T10:35:45.975 INFO:tasks.workunit.client.0.smithi042.stderr: 53200K .......... .......... .......... .......... .......... 39% 23.8M 4s 2024-04-20T10:35:45.977 INFO:tasks.workunit.client.0.smithi042.stderr: 53250K .......... .......... .......... .......... .......... 39% 37.9M 4s 2024-04-20T10:35:45.977 INFO:tasks.workunit.client.0.smithi042.stderr: 53300K .......... .......... .......... .......... .......... 39% 27.4M 4s 2024-04-20T10:35:45.980 INFO:tasks.workunit.client.0.smithi042.stderr: 53350K .......... .......... .......... .......... .......... 39% 24.2M 4s 2024-04-20T10:35:45.981 INFO:tasks.workunit.client.0.smithi042.stderr: 53400K .......... .......... .......... .......... .......... 39% 33.8M 4s 2024-04-20T10:35:45.983 INFO:tasks.workunit.client.0.smithi042.stderr: 53450K .......... .......... .......... .......... .......... 39% 24.4M 4s 2024-04-20T10:35:45.984 INFO:tasks.workunit.client.0.smithi042.stderr: 53500K .......... .......... .......... .......... .......... 39% 34.7M 4s 2024-04-20T10:35:45.987 INFO:tasks.workunit.client.0.smithi042.stderr: 53550K .......... .......... .......... .......... .......... 39% 24.8M 4s 2024-04-20T10:35:45.988 INFO:tasks.workunit.client.0.smithi042.stderr: 53600K .......... .......... .......... .......... .......... 39% 25.7M 4s 2024-04-20T10:35:45.989 INFO:tasks.workunit.client.0.smithi042.stderr: 53650K .......... .......... .......... .......... .......... 39% 39.6M 4s 2024-04-20T10:35:45.991 INFO:tasks.workunit.client.0.smithi042.stderr: 53700K .......... .......... .......... .......... .......... 39% 26.7M 4s 2024-04-20T10:35:45.994 INFO:tasks.workunit.client.0.smithi042.stderr: 53750K .......... .......... .......... .......... .......... 39% 22.8M 4s 2024-04-20T10:35:45.995 INFO:tasks.workunit.client.0.smithi042.stderr: 53800K .......... .......... .......... .......... .......... 39% 38.9M 4s 2024-04-20T10:35:45.997 INFO:tasks.workunit.client.0.smithi042.stderr: 53850K .......... .......... .......... .......... .......... 39% 24.2M 4s 2024-04-20T10:35:45.998 INFO:tasks.workunit.client.0.smithi042.stderr: 53900K .......... .......... .......... .......... .......... 39% 39.9M 4s 2024-04-20T10:35:46.000 INFO:tasks.workunit.client.0.smithi042.stderr: 53950K .......... .......... .......... .......... .......... 39% 26.9M 4s 2024-04-20T10:35:46.002 INFO:tasks.workunit.client.0.smithi042.stderr: 54000K .......... .......... .......... .......... .......... 39% 24.7M 4s 2024-04-20T10:35:46.003 INFO:tasks.workunit.client.0.smithi042.stderr: 54050K .......... .......... .......... .......... .......... 39% 35.1M 4s 2024-04-20T10:35:46.006 INFO:tasks.workunit.client.0.smithi042.stderr: 54100K .......... .......... .......... .......... .......... 39% 20.2M 4s 2024-04-20T10:35:46.007 INFO:tasks.workunit.client.0.smithi042.stderr: 54150K .......... .......... .......... .......... .......... 39% 41.4M 4s 2024-04-20T10:35:46.024 INFO:tasks.workunit.client.0.smithi042.stderr: 54200K .......... .......... .......... .......... .......... 39% 26.3M 4s 2024-04-20T10:35:46.024 INFO:tasks.workunit.client.0.smithi042.stderr: 54250K .......... .......... .......... .......... .......... 39% 25.0M 4s 2024-04-20T10:35:46.024 INFO:tasks.workunit.client.0.smithi042.stderr: 54300K .......... .......... .......... .......... .......... 40% 33.9M 4s 2024-04-20T10:35:46.024 INFO:tasks.workunit.client.0.smithi042.stderr: 54350K .......... .......... .......... .......... .......... 40% 22.4M 4s 2024-04-20T10:35:46.024 INFO:tasks.workunit.client.0.smithi042.stderr: 54400K .......... .......... .......... .......... .......... 40% 42.3M 4s 2024-04-20T10:35:46.025 INFO:tasks.workunit.client.0.smithi042.stderr: 54450K .......... .......... .......... .......... .......... 40% 27.3M 4s 2024-04-20T10:35:46.025 INFO:tasks.workunit.client.0.smithi042.stderr: 54500K .......... .......... .......... .......... .......... 40% 24.8M 4s 2024-04-20T10:35:46.025 INFO:tasks.workunit.client.0.smithi042.stderr: 54550K .......... .......... .......... .......... .......... 40% 34.2M 4s 2024-04-20T10:35:46.025 INFO:tasks.workunit.client.0.smithi042.stderr: 54600K .......... .......... .......... .......... .......... 40% 21.3M 4s 2024-04-20T10:35:46.025 INFO:tasks.workunit.client.0.smithi042.stderr: 54650K .......... .......... .......... .......... .......... 40% 41.9M 4s 2024-04-20T10:35:46.027 INFO:tasks.workunit.client.0.smithi042.stderr: 54700K .......... .......... .......... .......... .......... 40% 23.5M 4s 2024-04-20T10:35:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.026+0000 7fee07e00640 1 -- 172.21.15.42:0/3747393009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08103950 msgr2=0x7fee0810be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.026+0000 7fee07e00640 1 --2- 172.21.15.42:0/3747393009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08103950 0x7fee0810be30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009860 tx=0x7fedf402bc20 comp rx=0 tx=0).stop 2024-04-20T10:35:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.026+0000 7fee07e00640 1 -- 172.21.15.42:0/3747393009 shutdown_connections 2024-04-20T10:35:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.026+0000 7fee07e00640 1 --2- 172.21.15.42:0/3747393009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0810c450 0x7fee0810e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.026+0000 7fee07e00640 1 --2- 172.21.15.42:0/3747393009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08103950 0x7fee0810be30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.026+0000 7fee07e00640 1 --2- 172.21.15.42:0/3747393009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee08103030 0x7fee08103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.026+0000 7fee07e00640 1 -- 172.21.15.42:0/3747393009 >> 172.21.15.42:0/3747393009 conn(0x7fee080fc870 msgr2=0x7fee080fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.026+0000 7fee07e00640 1 -- 172.21.15.42:0/3747393009 shutdown_connections 2024-04-20T10:35:46.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.026+0000 7fee07e00640 1 -- 172.21.15.42:0/3747393009 wait complete. 2024-04-20T10:35:46.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.028+0000 7fee07e00640 1 Processor -- start 2024-04-20T10:35:46.030 INFO:tasks.workunit.client.0.smithi042.stderr: 54750K .......... .......... .......... .......... .......... 40% 25.6M 4s 2024-04-20T10:35:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.028+0000 7fee07e00640 1 -- start start 2024-04-20T10:35:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.028+0000 7fee07e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee08103030 0x7fee081a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.028+0000 7fee07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08103950 0x7fee081a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.028+0000 7fee07e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0810c450 0x7fee0819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.028+0000 7fee07e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fee08112230 con 0x7fee0810c450 2024-04-20T10:35:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.028+0000 7fee07e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee081120b0 con 0x7fee08103950 2024-04-20T10:35:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.028+0000 7fee07e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fee081123b0 con 0x7fee08103030 2024-04-20T10:35:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.029+0000 7fee07400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0810c450 0x7fee0819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.029+0000 7fee07400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0810c450 0x7fee0819a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54590/0 (socket says 172.21.15.42:54590) 2024-04-20T10:35:46.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.029+0000 7fee07400640 1 -- 172.21.15.42:0/908875466 learned_addr learned my addr 172.21.15.42:0/908875466 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.029+0000 7fee06a00640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee08103030 0x7fee081a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.029+0000 7fee06000640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08103950 0x7fee081a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.033 INFO:tasks.workunit.client.0.smithi042.stderr: 54800K .......... .......... .......... .......... .......... 40% 33.6M 4s 2024-04-20T10:35:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.029+0000 7fee07400640 1 -- 172.21.15.42:0/908875466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee08103030 msgr2=0x7fee081a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.029+0000 7fee07400640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee08103030 0x7fee081a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.029+0000 7fee07400640 1 -- 172.21.15.42:0/908875466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08103950 msgr2=0x7fee081a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.029+0000 7fee07400640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08103950 0x7fee081a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.029+0000 7fee07400640 1 -- 172.21.15.42:0/908875466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0819af80 con 0x7fee0810c450 2024-04-20T10:35:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.029+0000 7fee06a00640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee08103030 0x7fee081a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:35:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.029+0000 7fee07400640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0810c450 0x7fee0819a7b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fee0000eb10 tx=0x7fee0000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.030+0000 7fedff400640 1 -- 172.21.15.42:0/908875466 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00004b20 con 0x7fee0810c450 2024-04-20T10:35:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.030+0000 7fee07e00640 1 -- 172.21.15.42:0/908875466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0819b270 con 0x7fee0810c450 2024-04-20T10:35:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.030+0000 7fedff400640 1 -- 172.21.15.42:0/908875466 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee00004cc0 con 0x7fee0810c450 2024-04-20T10:35:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.030+0000 7fedff400640 1 -- 172.21.15.42:0/908875466 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00010760 con 0x7fee0810c450 2024-04-20T10:35:46.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.030+0000 7fee07e00640 1 -- 172.21.15.42:0/908875466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee081a7490 con 0x7fee0810c450 2024-04-20T10:35:46.038 INFO:tasks.workunit.client.0.smithi042.stderr: 54850K .......... .......... .......... .......... .......... 40% 22.8M 4s 2024-04-20T10:35:46.039 INFO:tasks.workunit.client.0.smithi042.stderr: 54900K .......... .......... .......... .......... .......... 40% 51.2M 4s 2024-04-20T10:35:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.033+0000 7fee107e5740 1 Processor -- start 2024-04-20T10:35:46.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.034+0000 7fedff400640 1 -- 172.21.15.42:0/908875466 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fee00005020 con 0x7fee0810c450 2024-04-20T10:35:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.034+0000 7fee107e5740 1 -- start start 2024-04-20T10:35:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.034+0000 7fee107e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26100f0 0x5566c26104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.034+0000 7fee107e5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c2610a10 0x5566c26190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.034+0000 7fee107e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c261b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.034+0000 7fee107e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5566c2574d20 con 0x5566c26100f0 2024-04-20T10:35:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.034+0000 7fee107e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5566c251b3e0 con 0x5566c26195f0 2024-04-20T10:35:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.034+0000 7fee107e5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5566c2570ce0 con 0x5566c2610a10 2024-04-20T10:35:46.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.035+0000 7fee06000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26100f0 0x5566c26104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.035+0000 7fee06a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c2610a10 0x5566c26190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.035+0000 7fee06a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c2610a10 0x5566c26190b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:51278/0 (socket says 172.21.15.42:51278) 2024-04-20T10:35:46.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.035+0000 7fee06a00640 1 -- 172.21.15.42:0/3120847305 learned_addr learned my addr 172.21.15.42:0/3120847305 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:46.043 INFO:tasks.workunit.client.0.smithi042.stderr: 54950K .......... .......... .......... .......... .......... 40% 17.9M 4s 2024-04-20T10:35:46.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.035+0000 7fee06000640 1 --2- 172.21.15.42:0/3120847305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c261b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.035+0000 7fee06a00640 1 -- 172.21.15.42:0/3120847305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 msgr2=0x5566c261b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.035+0000 7fee06a00640 1 --2- 172.21.15.42:0/3120847305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c261b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.035+0000 7fee06a00640 1 -- 172.21.15.42:0/3120847305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26100f0 msgr2=0x5566c26104d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.035+0000 7fee06a00640 1 --2- 172.21.15.42:0/3120847305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26100f0 0x5566c26104d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.035+0000 7fee06a00640 1 -- 172.21.15.42:0/3120847305 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c253a980 con 0x5566c2610a10 2024-04-20T10:35:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.036+0000 7fedff400640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fedec0a3d20 0x7fedec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.036+0000 7fee06a00640 1 --2- 172.21.15.42:0/3120847305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c2610a10 0x5566c26190b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009d00 tx=0x7fedf80117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=156f7074b8c2b876 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.036+0000 7fedff400640 1 -- 172.21.15.42:0/908875466 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fee000d0dc0 con 0x7fee0810c450 2024-04-20T10:35:46.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.036+0000 7fee06000640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fedec0a3d20 0x7fedec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.036+0000 7fedf3e00640 1 -- 172.21.15.42:0/3120847305 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8012450 con 0x5566c2610a10 2024-04-20T10:35:46.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.036+0000 7fedf3e00640 1 -- 172.21.15.42:0/3120847305 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedf8012ad0 con 0x5566c2610a10 2024-04-20T10:35:46.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.036+0000 7fedf3e00640 1 -- 172.21.15.42:0/3120847305 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf800fc40 con 0x5566c2610a10 2024-04-20T10:35:46.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.037+0000 7fee06000640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fedec0a3d20 0x7fedec0a61e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fedf402c230 tx=0x7fedf4000bc0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:46.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.037+0000 7fee107e5740 1 -- 172.21.15.42:0/3120847305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c2610a10 msgr2=0x5566c26190b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.037+0000 7fee107e5740 1 --2- 172.21.15.42:0/3120847305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c2610a10 0x5566c26190b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009d00 tx=0x7fedf80117b0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.048 INFO:tasks.workunit.client.0.smithi042.stderr: 55000K .......... .......... .......... .......... .......... 40% 23.4M 4s 2024-04-20T10:35:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.037+0000 7fee107e5740 1 -- 172.21.15.42:0/3120847305 shutdown_connections 2024-04-20T10:35:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.037+0000 7fee107e5740 1 --2- 172.21.15.42:0/3120847305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c261b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.037+0000 7fee107e5740 1 --2- 172.21.15.42:0/3120847305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c2610a10 0x5566c26190b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.037+0000 7fee107e5740 1 --2- 172.21.15.42:0/3120847305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26100f0 0x5566c26104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.037+0000 7fee107e5740 1 -- 172.21.15.42:0/3120847305 >> 172.21.15.42:0/3120847305 conn(0x5566c260a820 msgr2=0x5566c260ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:46.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.037+0000 7fee107e5740 1 -- 172.21.15.42:0/3120847305 shutdown_connections 2024-04-20T10:35:46.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.037+0000 7fee107e5740 1 -- 172.21.15.42:0/3120847305 wait complete. 2024-04-20T10:35:46.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.038+0000 7fee107e5740 1 -- start start 2024-04-20T10:35:46.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.038+0000 7fee107e5740 1 Processor -- start 2024-04-20T10:35:46.051 INFO:tasks.workunit.client.0.smithi042.stderr: 55050K .......... .......... .......... .......... .......... 40% 38.3M 4s 2024-04-20T10:35:46.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.038+0000 7fee107e5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c26100f0 0x5566c2616610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.038+0000 7fee107e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c2610a10 0x5566c2616b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.038+0000 7fee107e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26195f0 0x5566c2617090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.038+0000 7fee107e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5566c261c3e0 con 0x5566c26195f0 2024-04-20T10:35:46.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.038+0000 7fee107e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5566c252e7a0 con 0x5566c2610a10 2024-04-20T10:35:46.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.038+0000 7fee107e5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5566c261c560 con 0x5566c26100f0 2024-04-20T10:35:46.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.038+0000 7fee107e5740 10 client.0 _ll_register_callbacks cb 0x5566c2530b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:46.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.039+0000 7fee06000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c2610a10 0x5566c2616b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.039+0000 7fee06a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c26100f0 0x5566c2616610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.039+0000 7fee06a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c26100f0 0x5566c2616610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:51292/0 (socket says 172.21.15.42:51292) 2024-04-20T10:35:46.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.039+0000 7fee06000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c2610a10 0x5566c2616b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:48798/0 (socket says 172.21.15.42:48798) 2024-04-20T10:35:46.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.039+0000 7fee06a00640 1 -- 172.21.15.42:0/175978349 learned_addr learned my addr 172.21.15.42:0/175978349 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:46.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.039+0000 7fee07400640 1 --2- 172.21.15.42:0/175978349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26195f0 0x5566c2617090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.039+0000 7fee06000640 1 -- 172.21.15.42:0/175978349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c26100f0 msgr2=0x5566c2616610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.039+0000 7fee06000640 1 --2- 172.21.15.42:0/175978349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c26100f0 0x5566c2616610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.039+0000 7fee06000640 1 -- 172.21.15.42:0/175978349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26195f0 msgr2=0x5566c2617090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.039+0000 7fee06000640 1 --2- 172.21.15.42:0/175978349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26195f0 0x5566c2617090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.039+0000 7fee06000640 1 -- 172.21.15.42:0/175978349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c234f7e0 con 0x5566c2610a10 2024-04-20T10:35:46.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.040+0000 7fee06000640 1 --2- 172.21.15.42:0/175978349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c2610a10 0x5566c2616b50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fedf4037040 tx=0x7fedf40056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0d403641b07e97a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:46.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.040+0000 7fedf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:35:46.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.040+0000 7fedf0c00640 1 -- 172.21.15.42:0/175978349 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4052020 con 0x5566c2610a10 2024-04-20T10:35:46.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.040+0000 7fee107e5740 1 -- 172.21.15.42:0/175978349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedf8012ad0 con 0x5566c2610a10 2024-04-20T10:35:46.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.040+0000 7fedf0c00640 1 -- 172.21.15.42:0/175978349 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fedf402c7c0 con 0x5566c2610a10 2024-04-20T10:35:46.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.040+0000 7fedf0c00640 1 -- 172.21.15.42:0/175978349 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fedf4039040 con 0x5566c2610a10 2024-04-20T10:35:46.057 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:45 smithi046 ceph-mon[21574]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-20T10:35:46.058 INFO:tasks.workunit.client.0.smithi042.stderr: 55100K .......... .......... .......... .......... .......... 40% 23.6M 4s 2024-04-20T10:35:46.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.040+0000 7fee107e5740 10 client.25568 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:46.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.040+0000 7fee107e5740 1 -- 172.21.15.42:0/175978349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5566c23aff70 con 0x5566c2610a10 2024-04-20T10:35:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.041+0000 7fedf0c00640 1 -- 172.21.15.42:0/175978349 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedf8012ad0 con 0x5566c2610a10 2024-04-20T10:35:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.041+0000 7fedf0c00640 1 -- 172.21.15.42:0/175978349 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fedf404d480 con 0x5566c2610a10 2024-04-20T10:35:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.041+0000 7fee107e5740 10 client.25568 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.041+0000 7fee107e5740 10 client.25568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.041+0000 7fee107e5740 20 client.25568 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.041+0000 7fee107e5740 1 --2- 172.21.15.42:0/175978349 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5566c2629850 0x5566c262bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.041+0000 7fee107e5740 4 client.25568 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:35:46.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.041+0000 7fee107e5740 1 -- 172.21.15.42:0/175978349 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566c2582740 con 0x5566c2629850 2024-04-20T10:35:46.061 INFO:tasks.workunit.client.0.smithi042.stderr: 55150K .......... .......... .......... .......... .......... 40% 45.6M 4s 2024-04-20T10:35:46.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.041+0000 7fee06a00640 1 --2- 172.21.15.42:0/175978349 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5566c2629850 0x5566c262bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.042+0000 7fee06a00640 1 --2- 172.21.15.42:0/175978349 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5566c2629850 0x5566c262bc90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6646713f1c51aa5d server_cookie=4c6dc94ec5c0b12f in_seq=0 out_seq=0 2024-04-20T10:35:46.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.042+0000 7fedf0c00640 10 client.25568 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:46.063 INFO:tasks.workunit.client.0.smithi042.stderr: 55200K .......... .......... .......... .......... .......... 40% 23.4M 4s 2024-04-20T10:35:46.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fedf0c00640 1 -- 172.21.15.42:0/175978349 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566c2582740 con 0x5566c2629850 2024-04-20T10:35:46.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fedf0c00640 10 client.25568 handle_command_reply: tid=0 2024-04-20T10:35:46.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fee107e5740 1 client.25568 shutdown 2024-04-20T10:35:46.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fee107e5740 1 -- 172.21.15.42:0/175978349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c2610a10 msgr2=0x5566c2616b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fee107e5740 1 --2- 172.21.15.42:0/175978349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c2610a10 0x5566c2616b50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fedf4037040 tx=0x7fedf40056d0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fee107e5740 1 -- 172.21.15.42:0/175978349 shutdown_connections 2024-04-20T10:35:46.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fee107e5740 1 --2- 172.21.15.42:0/175978349 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5566c2629850 0x5566c262bc90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fee107e5740 1 --2- 172.21.15.42:0/175978349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26195f0 0x5566c2617090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fee107e5740 1 --2- 172.21.15.42:0/175978349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c2610a10 0x5566c2616b50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fee107e5740 1 --2- 172.21.15.42:0/175978349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c26100f0 0x5566c2616610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fee107e5740 1 -- 172.21.15.42:0/175978349 >> 172.21.15.42:0/175978349 conn(0x5566c260a820 msgr2=0x5566c260c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:46.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fedf0c00640 0 client.25568 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:46.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fee107e5740 1 -- 172.21.15.42:0/175978349 shutdown_connections 2024-04-20T10:35:46.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fee107e5740 1 -- 172.21.15.42:0/175978349 wait complete. 2024-04-20T10:35:46.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.044+0000 7fee107e5740 20 client.25568 trim_cache size 0 max 16384 2024-04-20T10:35:46.075 INFO:tasks.workunit.client.0.smithi042.stderr: 55250K .......... .......... .......... .......... .......... 40% 26.7M 4s 2024-04-20T10:35:46.075 INFO:tasks.workunit.client.0.smithi042.stderr: 55300K .......... .......... .......... .......... .......... 40% 34.7M 4s 2024-04-20T10:35:46.075 INFO:tasks.workunit.client.0.smithi042.stderr: 55350K .......... .......... .......... .......... .......... 40% 25.1M 4s 2024-04-20T10:35:46.076 INFO:tasks.workunit.client.0.smithi042.stderr: 55400K .......... .......... .......... .......... .......... 40% 30.0M 4s 2024-04-20T10:35:46.076 INFO:tasks.workunit.client.0.smithi042.stderr: 55450K .......... .......... .......... .......... .......... 40% 23.0M 4s 2024-04-20T10:35:46.076 INFO:tasks.workunit.client.0.smithi042.stderr: 55500K .......... .......... .......... .......... .......... 40% 20.9M 4s 2024-04-20T10:35:46.076 INFO:tasks.workunit.client.0.smithi042.stderr: 55550K .......... .......... .......... .......... .......... 40% 27.5M 4s 2024-04-20T10:35:46.076 INFO:tasks.workunit.client.0.smithi042.stderr: 55600K .......... .......... .......... .......... .......... 40% 21.3M 4s 2024-04-20T10:35:46.076 INFO:tasks.workunit.client.0.smithi042.stderr: 55650K .......... .......... .......... .......... .......... 41% 39.4M 4s 2024-04-20T10:35:46.076 INFO:tasks.workunit.client.0.smithi042.stderr: 55700K .......... .......... .......... .......... .......... 41% 24.5M 4s 2024-04-20T10:35:46.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.064+0000 7fee107e5740 1 Processor -- start 2024-04-20T10:35:46.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.064+0000 7fee107e5740 1 -- start start 2024-04-20T10:35:46.077 INFO:tasks.workunit.client.0.smithi042.stderr: 55750K .......... .......... .......... .......... .......... 41% 23.9M 4s 2024-04-20T10:35:46.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.064+0000 7fee107e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26100f0 0x5566c2618d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.064+0000 7fee107e5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c2610a10 0x5566c26252a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.064+0000 7fee107e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c2631560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.064+0000 7fee107e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fee081add50 con 0x5566c26100f0 2024-04-20T10:35:46.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.064+0000 7fee107e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee081b05d0 con 0x5566c26195f0 2024-04-20T10:35:46.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.064+0000 7fee107e5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fee081ae3c0 con 0x5566c2610a10 2024-04-20T10:35:46.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.065+0000 7fee06a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26100f0 0x5566c2618d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.065+0000 7fee06a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26100f0 0x5566c2618d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54630/0 (socket says 172.21.15.42:54630) 2024-04-20T10:35:46.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.065+0000 7fee06a00640 1 -- 172.21.15.42:0/585459493 learned_addr learned my addr 172.21.15.42:0/585459493 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:46.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.065+0000 7fee06000640 1 --2- 172.21.15.42:0/585459493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c2610a10 0x5566c26252a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.065+0000 7fee07400640 1 --2- 172.21.15.42:0/585459493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c2631560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.065+0000 7fee06a00640 1 -- 172.21.15.42:0/585459493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c2610a10 msgr2=0x5566c26252a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.065+0000 7fee06a00640 1 --2- 172.21.15.42:0/585459493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c2610a10 0x5566c26252a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.065+0000 7fee06a00640 1 -- 172.21.15.42:0/585459493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 msgr2=0x5566c2631560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.065+0000 7fee06a00640 1 --2- 172.21.15.42:0/585459493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c2631560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.065+0000 7fee06a00640 1 -- 172.21.15.42:0/585459493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c2610840 con 0x5566c26100f0 2024-04-20T10:35:46.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.065+0000 7fee06a00640 1 --2- 172.21.15.42:0/585459493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26100f0 0x5566c2618d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fedf800f850 tx=0x7fedf8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b68250d1711b0b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:46.081 INFO:tasks.workunit.client.0.smithi042.stderr: 55800K .......... .......... .......... .......... .......... 41% 35.9M 4s 2024-04-20T10:35:46.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.066+0000 7fedf3400640 1 -- 172.21.15.42:0/585459493 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8032070 con 0x5566c26100f0 2024-04-20T10:35:46.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.066+0000 7fedf3400640 1 -- 172.21.15.42:0/585459493 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedf800f590 con 0x5566c26100f0 2024-04-20T10:35:46.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.066+0000 7fedf3400640 1 -- 172.21.15.42:0/585459493 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf802d5e0 con 0x5566c26100f0 2024-04-20T10:35:46.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.066+0000 7fee107e5740 1 -- 172.21.15.42:0/585459493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26100f0 msgr2=0x5566c2618d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.066+0000 7fee107e5740 1 --2- 172.21.15.42:0/585459493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26100f0 0x5566c2618d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fedf800f850 tx=0x7fedf8014040 comp rx=0 tx=0).stop 2024-04-20T10:35:46.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.066+0000 7fee107e5740 1 -- 172.21.15.42:0/585459493 shutdown_connections 2024-04-20T10:35:46.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.066+0000 7fee107e5740 1 --2- 172.21.15.42:0/585459493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c2631560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.066+0000 7fee107e5740 1 --2- 172.21.15.42:0/585459493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c2610a10 0x5566c26252a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.066+0000 7fee107e5740 1 --2- 172.21.15.42:0/585459493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c26100f0 0x5566c2618d40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.066+0000 7fee107e5740 1 -- 172.21.15.42:0/585459493 >> 172.21.15.42:0/585459493 conn(0x5566c260a820 msgr2=0x5566c2609a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:46.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.067+0000 7fee107e5740 1 -- 172.21.15.42:0/585459493 shutdown_connections 2024-04-20T10:35:46.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.067+0000 7fee107e5740 1 -- 172.21.15.42:0/585459493 wait complete. 2024-04-20T10:35:46.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.067+0000 7fee107e5740 1 -- start start 2024-04-20T10:35:46.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.067+0000 7fee107e5740 1 Processor -- start 2024-04-20T10:35:46.085 INFO:tasks.workunit.client.0.smithi042.stderr: 55850K .......... .......... .......... .......... .......... 41% 24.6M 4s 2024-04-20T10:35:46.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee107e5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c26100f0 0x5566c26217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee107e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c2610a10 0x5566c2621cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee107e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c261e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee107e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fee081aea90 con 0x5566c2610a10 2024-04-20T10:35:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee107e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee081aff00 con 0x5566c26195f0 2024-04-20T10:35:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee107e5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fee081af160 con 0x5566c26100f0 2024-04-20T10:35:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee107e5740 10 client.0 _ll_register_callbacks cb 0x7fee000093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee06000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c2610a10 0x5566c2621cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee06000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c2610a10 0x5566c2621cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54636/0 (socket says 172.21.15.42:54636) 2024-04-20T10:35:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee06000640 1 -- 172.21.15.42:0/2709976820 learned_addr learned my addr 172.21.15.42:0/2709976820 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c261e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee06a00640 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c26100f0 0x5566c26217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee06000640 1 -- 172.21.15.42:0/2709976820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c26100f0 msgr2=0x5566c26217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee06000640 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c26100f0 0x5566c26217b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee06000640 1 -- 172.21.15.42:0/2709976820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 msgr2=0x5566c261e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee06000640 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c261e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee06000640 1 -- 172.21.15.42:0/2709976820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c2247170 con 0x5566c2610a10 2024-04-20T10:35:46.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.068+0000 7fee07400640 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c261e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:46.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.069+0000 7fee06000640 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c2610a10 0x5566c2621cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fedf404d940 tx=0x7fedf404d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8ce493c5b33243 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:46.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.069+0000 7fedf0c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:35:46.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.069+0000 7fedf0c00640 1 -- 172.21.15.42:0/2709976820 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4067060 con 0x5566c2610a10 2024-04-20T10:35:46.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.069+0000 7fedf0c00640 1 -- 172.21.15.42:0/2709976820 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fedf402c5c0 con 0x5566c2610a10 2024-04-20T10:35:46.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.069+0000 7fee107e5740 1 -- 172.21.15.42:0/2709976820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedf800f590 con 0x5566c2610a10 2024-04-20T10:35:46.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.069+0000 7fedf0c00640 1 -- 172.21.15.42:0/2709976820 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fedf4052040 con 0x5566c2610a10 2024-04-20T10:35:46.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.069+0000 7fee107e5740 10 client.15462 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:46.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.069+0000 7fee107e5740 1 -- 172.21.15.42:0/2709976820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5566c2690820 con 0x5566c2610a10 2024-04-20T10:35:46.092 INFO:tasks.workunit.client.0.smithi042.stderr: 55900K .......... .......... .......... .......... .......... 41% 24.5M 4s 2024-04-20T10:35:46.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.070+0000 7fedf0c00640 1 -- 172.21.15.42:0/2709976820 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedf800f590 con 0x5566c2610a10 2024-04-20T10:35:46.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.070+0000 7fedf0c00640 1 -- 172.21.15.42:0/2709976820 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fedf40619f0 con 0x5566c2610a10 2024-04-20T10:35:46.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.070+0000 7fee107e5740 10 client.15462 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:46.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.070+0000 7fee107e5740 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:46.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.070+0000 7fee107e5740 20 client.15462 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:46.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.070+0000 7fee107e5740 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5566c2690fd0 0x5566c26933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.070+0000 7fee107e5740 4 client.15462 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:35:46.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.070+0000 7fee107e5740 1 -- 172.21.15.42:0/2709976820 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5566c257c5b0 con 0x5566c2690fd0 2024-04-20T10:35:46.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.070+0000 7fee06a00640 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5566c2690fd0 0x5566c26933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.095 INFO:tasks.workunit.client.0.smithi042.stderr: 55950K .......... .......... .......... .......... .......... 41% 46.0M 4s 2024-04-20T10:35:46.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.071+0000 7fee06a00640 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5566c2690fd0 0x5566c26933f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfed3eede1cde0b server_cookie=de9d3e5d5061545d in_seq=0 out_seq=0 2024-04-20T10:35:46.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.071+0000 7fedf0c00640 10 client.15462 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:46.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.072+0000 7fedf0c00640 1 -- 172.21.15.42:0/2709976820 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5566c257c5b0 con 0x5566c2690fd0 2024-04-20T10:35:46.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.072+0000 7fedf0c00640 10 client.15462 handle_command_reply: tid=0 2024-04-20T10:35:46.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.072+0000 7fee107e5740 1 client.15462 shutdown 2024-04-20T10:35:46.097 INFO:tasks.workunit.client.0.smithi042.stderr: 56000K .......... .......... .......... .......... .......... 41% 24.5M 4s 2024-04-20T10:35:46.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.073+0000 7fee107e5740 1 -- 172.21.15.42:0/2709976820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c2610a10 msgr2=0x5566c2621cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.073+0000 7fee107e5740 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c2610a10 0x5566c2621cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fedf404d940 tx=0x7fedf404d6b0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.073+0000 7fee107e5740 1 -- 172.21.15.42:0/2709976820 shutdown_connections 2024-04-20T10:35:46.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.073+0000 7fee107e5740 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5566c2690fd0 0x5566c26933f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.073+0000 7fee107e5740 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5566c26195f0 0x5566c261e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.073+0000 7fee107e5740 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5566c2610a10 0x5566c2621cf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.073+0000 7fedf0c00640 0 client.15462 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:46.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.073+0000 7fee107e5740 1 --2- 172.21.15.42:0/2709976820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5566c26100f0 0x5566c26217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.073+0000 7fee107e5740 1 -- 172.21.15.42:0/2709976820 >> 172.21.15.42:0/2709976820 conn(0x5566c260a820 msgr2=0x5566c26165d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:46.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.073+0000 7fee107e5740 1 -- 172.21.15.42:0/2709976820 shutdown_connections 2024-04-20T10:35:46.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.073+0000 7fee107e5740 1 -- 172.21.15.42:0/2709976820 wait complete. 2024-04-20T10:35:46.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.073+0000 7fee107e5740 20 client.15462 trim_cache size 0 max 16384 2024-04-20T10:35:46.100 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:35:46.100 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:35:46.100 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "aaffe980-3a4b-4166-8f0a-a2394b20ddb1", 2024-04-20T10:35:46.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:35:46.100 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:35:46.101 INFO:tasks.workunit.client.0.smithi042.stderr: 56050K .......... .......... .......... .......... .......... 41% 36.9M 4s 2024-04-20T10:35:46.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.074+0000 7fee07e00640 1 -- 172.21.15.42:0/908875466 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fedec0a3d20 msgr2=0x7fedec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.074+0000 7fee07e00640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fedec0a3d20 0x7fedec0a61e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fedf402c230 tx=0x7fedf4000bc0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.074+0000 7fee07e00640 1 -- 172.21.15.42:0/908875466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0810c450 msgr2=0x7fee0819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.074+0000 7fee07e00640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0810c450 0x7fee0819a7b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fee0000eb10 tx=0x7fee0000c2d0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.074+0000 7fee07e00640 1 -- 172.21.15.42:0/908875466 shutdown_connections 2024-04-20T10:35:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.074+0000 7fee07e00640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fedec0a3d20 0x7fedec0a61e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.074+0000 7fee07e00640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0810c450 0x7fee0819a7b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.074+0000 7fee07e00640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08103950 0x7fee081a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.074+0000 7fee07e00640 1 --2- 172.21.15.42:0/908875466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee08103030 0x7fee081a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.074+0000 7fee07e00640 1 -- 172.21.15.42:0/908875466 >> 172.21.15.42:0/908875466 conn(0x7fee080fc870 msgr2=0x7fee08100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.074+0000 7fee07e00640 1 -- 172.21.15.42:0/908875466 shutdown_connections 2024-04-20T10:35:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.074+0000 7fee07e00640 1 -- 172.21.15.42:0/908875466 wait complete. 2024-04-20T10:35:46.114 INFO:tasks.workunit.client.0.smithi042.stderr: 56100K .......... .......... .......... .......... .......... 41% 21.9M 4s 2024-04-20T10:35:46.114 INFO:tasks.workunit.client.0.smithi042.stderr: 56150K .......... .......... .......... .......... .......... 41% 26.4M 4s 2024-04-20T10:35:46.114 INFO:tasks.workunit.client.0.smithi042.stderr: 56200K .......... .......... .......... .......... .......... 41% 30.1M 4s 2024-04-20T10:35:46.114 INFO:tasks.workunit.client.0.smithi042.stderr: 56250K .......... .......... .......... .......... .......... 41% 22.6M 4s 2024-04-20T10:35:46.114 INFO:tasks.workunit.client.0.smithi042.stderr: 56300K .......... .......... .......... .......... .......... 41% 37.0M 4s 2024-04-20T10:35:46.114 INFO:tasks.workunit.client.0.smithi042.stderr: 56350K .......... .......... .......... .......... .......... 41% 23.8M 4s 2024-04-20T10:35:46.114 INFO:tasks.workunit.client.0.smithi042.stderr: 56400K .......... .......... .......... .......... .......... 41% 25.0M 4s 2024-04-20T10:35:46.114 INFO:tasks.workunit.client.0.smithi042.stderr: 56450K .......... .......... .......... .......... .......... 41% 49.5M 4s 2024-04-20T10:35:46.114 INFO:tasks.workunit.client.0.smithi042.stderr: 56500K .......... .......... .......... .......... .......... 41% 23.5M 4s 2024-04-20T10:35:46.114 INFO:tasks.workunit.client.0.smithi042.stderr: 56550K .......... .......... .......... .......... .......... 41% 25.8M 4s 2024-04-20T10:35:46.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:35:46.116 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:35:46.134 INFO:tasks.workunit.client.0.smithi042.stderr: 56600K .......... .......... .......... .......... .......... 41% 28.7M 4s 2024-04-20T10:35:46.134 INFO:tasks.workunit.client.0.smithi042.stderr: 56650K .......... .......... .......... .......... .......... 41% 26.8M 4s 2024-04-20T10:35:46.134 INFO:tasks.workunit.client.0.smithi042.stderr: 56700K .......... .......... .......... .......... .......... 41% 40.0M 4s 2024-04-20T10:35:46.134 INFO:tasks.workunit.client.0.smithi042.stderr: 56750K .......... .......... .......... .......... .......... 41% 27.3M 4s 2024-04-20T10:35:46.134 INFO:tasks.workunit.client.0.smithi042.stderr: 56800K .......... .......... .......... .......... .......... 41% 27.1M 4s 2024-04-20T10:35:46.134 INFO:tasks.workunit.client.0.smithi042.stderr: 56850K .......... .......... .......... .......... .......... 41% 29.0M 4s 2024-04-20T10:35:46.134 INFO:tasks.workunit.client.0.smithi042.stderr: 56900K .......... .......... .......... .......... .......... 41% 25.2M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 56950K .......... .......... .......... .......... .......... 41% 39.3M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 57000K .......... .......... .......... .......... .......... 42% 25.8M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 57050K .......... .......... .......... .......... .......... 42% 26.6M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 57100K .......... .......... .......... .......... .......... 42% 30.6M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 57150K .......... .......... .......... .......... .......... 42% 24.1M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 57200K .......... .......... .......... .......... .......... 42% 36.2M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 57250K .......... .......... .......... .......... .......... 42% 22.9M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 57300K .......... .......... .......... .......... .......... 42% 39.2M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 57350K .......... .......... .......... .......... .......... 42% 25.4M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 57400K .......... .......... .......... .......... .......... 42% 31.6M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 57450K .......... .......... .......... .......... .......... 42% 26.5M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 57500K .......... .......... .......... .......... .......... 42% 30.3M 4s 2024-04-20T10:35:46.135 INFO:tasks.workunit.client.0.smithi042.stderr: 57550K .......... .......... .......... .......... .......... 42% 39.5M 4s 2024-04-20T10:35:46.137 INFO:tasks.workunit.client.0.smithi042.stderr: 57600K .......... .......... .......... .......... .......... 42% 13.6M 4s 2024-04-20T10:35:46.137 INFO:tasks.workunit.client.0.smithi042.stderr: 57650K .......... .......... .......... .......... .......... 42% 141M 4s 2024-04-20T10:35:46.137 INFO:tasks.workunit.client.0.smithi042.stderr: 57700K .......... .......... .......... .......... .......... 42% 48.6M 4s 2024-04-20T10:35:46.138 INFO:tasks.workunit.client.0.smithi042.stderr: 57750K .......... .......... .......... .......... .......... 42% 29.0M 4s 2024-04-20T10:35:46.138 INFO:tasks.workunit.client.0.smithi042.stderr: 57800K .......... .......... .......... .......... .......... 42% 28.6M 4s 2024-04-20T10:35:46.138 INFO:tasks.workunit.client.0.smithi042.stderr: 57850K .......... .......... .......... .......... .......... 42% 24.1M 4s 2024-04-20T10:35:46.140 INFO:tasks.workunit.client.0.smithi042.stderr: 57900K .......... .......... .......... .......... .......... 42% 24.9M 4s 2024-04-20T10:35:46.140 INFO:tasks.workunit.client.0.smithi042.stderr: 57950K .......... .......... .......... .......... .......... 42% 45.8M 4s 2024-04-20T10:35:46.142 INFO:tasks.workunit.client.0.smithi042.stderr: 58000K .......... .......... .......... .......... .......... 42% 19.6M 4s 2024-04-20T10:35:46.143 INFO:tasks.workunit.client.0.smithi042.stderr: 58050K .......... .......... .......... .......... .......... 42% 35.0M 4s 2024-04-20T10:35:46.146 INFO:tasks.workunit.client.0.smithi042.stderr: 58100K .......... .......... .......... .......... .......... 42% 25.7M 4s 2024-04-20T10:35:46.147 INFO:tasks.workunit.client.0.smithi042.stderr: 58150K .......... .......... .......... .......... .......... 42% 25.9M 4s 2024-04-20T10:35:46.148 INFO:tasks.workunit.client.0.smithi042.stderr: 58200K .......... .......... .......... .......... .......... 42% 35.4M 4s 2024-04-20T10:35:46.150 INFO:tasks.workunit.client.0.smithi042.stderr: 58250K .......... .......... .......... .......... .......... 42% 24.8M 4s 2024-04-20T10:35:46.152 INFO:tasks.workunit.client.0.smithi042.stderr: 58300K .......... .......... .......... .......... .......... 42% 31.2M 4s 2024-04-20T10:35:46.153 INFO:tasks.workunit.client.0.smithi042.stderr: 58350K .......... .......... .......... .......... .......... 43% 25.6M 4s 2024-04-20T10:35:46.155 INFO:tasks.workunit.client.0.smithi042.stderr: 58400K .......... .......... .......... .......... .......... 43% 27.9M 4s 2024-04-20T10:35:46.157 INFO:tasks.workunit.client.0.smithi042.stderr: 58450K .......... .......... .......... .......... .......... 43% 31.0M 4s 2024-04-20T10:35:46.159 INFO:tasks.workunit.client.0.smithi042.stderr: 58500K .......... .......... .......... .......... .......... 43% 30.2M 4s 2024-04-20T10:35:46.161 INFO:tasks.workunit.client.0.smithi042.stderr: 58550K .......... .......... .......... .......... .......... 43% 31.5M 4s 2024-04-20T10:35:46.162 INFO:tasks.workunit.client.0.smithi042.stderr: 58600K .......... .......... .......... .......... .......... 43% 24.3M 4s 2024-04-20T10:35:46.169 INFO:tasks.workunit.client.0.smithi042.stderr: 58650K .......... .......... .......... .......... .......... 43% 28.5M 4s 2024-04-20T10:35:46.170 INFO:tasks.workunit.client.0.smithi042.stderr: 58700K .......... .......... .......... .......... .......... 43% 34.1M 4s 2024-04-20T10:35:46.170 INFO:tasks.workunit.client.0.smithi042.stderr: 58750K .......... .......... .......... .......... .......... 43% 29.5M 4s 2024-04-20T10:35:46.170 INFO:tasks.workunit.client.0.smithi042.stderr: 58800K .......... .......... .......... .......... .......... 43% 25.9M 4s 2024-04-20T10:35:46.171 INFO:tasks.workunit.client.0.smithi042.stderr: 58850K .......... .......... .......... .......... .......... 43% 28.6M 4s 2024-04-20T10:35:46.173 INFO:tasks.workunit.client.0.smithi042.stderr: 58900K .......... .......... .......... .......... .......... 43% 26.6M 4s 2024-04-20T10:35:46.175 INFO:tasks.workunit.client.0.smithi042.stderr: 58950K .......... .......... .......... .......... .......... 43% 29.7M 4s 2024-04-20T10:35:46.175 INFO:tasks.workunit.client.0.smithi042.stderr: 59000K .......... .......... .......... .......... .......... 43% 30.7M 4s 2024-04-20T10:35:46.178 INFO:tasks.workunit.client.0.smithi042.stderr: 59050K .......... .......... .......... .......... .......... 43% 24.7M 4s 2024-04-20T10:35:46.179 INFO:tasks.workunit.client.0.smithi042.stderr: 59100K .......... .......... .......... .......... .......... 43% 33.7M 4s 2024-04-20T10:35:46.181 INFO:tasks.workunit.client.0.smithi042.stderr: 59150K .......... .......... .......... .......... .......... 43% 27.1M 4s 2024-04-20T10:35:46.182 INFO:tasks.workunit.client.0.smithi042.stderr: 59200K .......... .......... .......... .......... .......... 43% 31.5M 4s 2024-04-20T10:35:46.184 INFO:tasks.workunit.client.0.smithi042.stderr: 59250K .......... .......... .......... .......... .......... 43% 29.5M 4s 2024-04-20T10:35:46.186 INFO:tasks.workunit.client.0.smithi042.stderr: 59300K .......... .......... .......... .......... .......... 43% 29.5M 4s 2024-04-20T10:35:46.187 INFO:tasks.workunit.client.0.smithi042.stderr: 59350K .......... .......... .......... .......... .......... 43% 28.8M 4s 2024-04-20T10:35:46.190 INFO:tasks.workunit.client.0.smithi042.stderr: 59400K .......... .......... .......... .......... .......... 43% 20.6M 4s 2024-04-20T10:35:46.191 INFO:tasks.workunit.client.0.smithi042.stderr: 59450K .......... .......... .......... .......... .......... 43% 38.6M 4s 2024-04-20T10:35:46.193 INFO:tasks.workunit.client.0.smithi042.stderr: 59500K .......... .......... .......... .......... .......... 43% 24.3M 4s 2024-04-20T10:35:46.195 INFO:tasks.workunit.client.0.smithi042.stderr: 59550K .......... .......... .......... .......... .......... 43% 24.8M 4s 2024-04-20T10:35:46.196 INFO:tasks.workunit.client.0.smithi042.stderr: 59600K .......... .......... .......... .......... .......... 43% 31.9M 4s 2024-04-20T10:35:46.198 INFO:tasks.workunit.client.0.smithi042.stderr: 59650K .......... .......... .......... .......... .......... 43% 24.9M 4s 2024-04-20T10:35:46.200 INFO:tasks.workunit.client.0.smithi042.stderr: 59700K .......... .......... .......... .......... .......... 44% 35.6M 4s 2024-04-20T10:35:46.201 INFO:tasks.workunit.client.0.smithi042.stderr: 59750K .......... .......... .......... .......... .......... 44% 27.6M 4s 2024-04-20T10:35:46.203 INFO:tasks.workunit.client.0.smithi042.stderr: 59800K .......... .......... .......... .......... .......... 44% 24.7M 4s 2024-04-20T10:35:46.205 INFO:tasks.workunit.client.0.smithi042.stderr: 59850K .......... .......... .......... .......... .......... 44% 29.3M 4s 2024-04-20T10:35:46.210 INFO:tasks.workunit.client.0.smithi042.stderr: 59900K .......... .......... .......... .......... .......... 44% 25.7M 4s 2024-04-20T10:35:46.210 INFO:tasks.workunit.client.0.smithi042.stderr: 59950K .......... .......... .......... .......... .......... 44% 34.0M 4s 2024-04-20T10:35:46.211 INFO:tasks.workunit.client.0.smithi042.stderr: 60000K .......... .......... .......... .......... .......... 44% 30.9M 4s 2024-04-20T10:35:46.212 INFO:tasks.workunit.client.0.smithi042.stderr: 60050K .......... .......... .......... .......... .......... 44% 25.6M 4s 2024-04-20T10:35:46.214 INFO:tasks.workunit.client.0.smithi042.stderr: 60100K .......... .......... .......... .......... .......... 44% 31.2M 4s 2024-04-20T10:35:46.215 INFO:tasks.workunit.client.0.smithi042.stderr: 60150K .......... .......... .......... .......... .......... 44% 25.1M 4s 2024-04-20T10:35:46.217 INFO:tasks.workunit.client.0.smithi042.stderr: 60200K .......... .......... .......... .......... .......... 44% 37.6M 4s 2024-04-20T10:35:46.219 INFO:tasks.workunit.client.0.smithi042.stderr: 60250K .......... .......... .......... .......... .......... 44% 21.2M 4s 2024-04-20T10:35:46.221 INFO:tasks.workunit.client.0.smithi042.stderr: 60300K .......... .......... .......... .......... .......... 44% 25.7M 4s 2024-04-20T10:35:46.223 INFO:tasks.workunit.client.0.smithi042.stderr: 60350K .......... .......... .......... .......... .......... 44% 24.2M 4s 2024-04-20T10:35:46.225 INFO:tasks.workunit.client.0.smithi042.stderr: 60400K .......... .......... .......... .......... .......... 44% 24.9M 4s 2024-04-20T10:35:46.227 INFO:tasks.workunit.client.0.smithi042.stderr: 60450K .......... .......... .......... .......... .......... 44% 28.6M 4s 2024-04-20T10:35:46.229 INFO:tasks.workunit.client.0.smithi042.stderr: 60500K .......... .......... .......... .......... .......... 44% 23.3M 4s 2024-04-20T10:35:46.231 INFO:tasks.workunit.client.0.smithi042.stderr: 60550K .......... .......... .......... .......... .......... 44% 24.9M 4s 2024-04-20T10:35:46.233 INFO:tasks.workunit.client.0.smithi042.stderr: 60600K .......... .......... .......... .......... .......... 44% 26.9M 4s 2024-04-20T10:35:46.234 INFO:tasks.workunit.client.0.smithi042.stderr: 60650K .......... .......... .......... .......... .......... 44% 29.3M 4s 2024-04-20T10:35:46.236 INFO:tasks.workunit.client.0.smithi042.stderr: 60700K .......... .......... .......... .......... .......... 44% 35.6M 4s 2024-04-20T10:35:46.237 INFO:tasks.workunit.client.0.smithi042.stderr: 60750K .......... .......... .......... .......... .......... 44% 28.7M 4s 2024-04-20T10:35:46.239 INFO:tasks.workunit.client.0.smithi042.stderr: 60800K .......... .......... .......... .......... .......... 44% 24.6M 4s 2024-04-20T10:35:46.241 INFO:tasks.workunit.client.0.smithi042.stderr: 60850K .......... .......... .......... .......... .......... 44% 30.1M 4s 2024-04-20T10:35:46.242 INFO:tasks.workunit.client.0.smithi042.stderr: 60900K .......... .......... .......... .......... .......... 44% 35.5M 3s 2024-04-20T10:35:46.245 INFO:tasks.workunit.client.0.smithi042.stderr: 60950K .......... .......... .......... .......... .......... 44% 21.0M 3s 2024-04-20T10:35:46.246 INFO:tasks.workunit.client.0.smithi042.stderr: 61000K .......... .......... .......... .......... .......... 44% 36.9M 3s 2024-04-20T10:35:46.248 INFO:tasks.workunit.client.0.smithi042.stderr: 61050K .......... .......... .......... .......... .......... 45% 22.3M 3s 2024-04-20T10:35:46.251 INFO:tasks.workunit.client.0.smithi042.stderr: 61100K .......... .......... .......... .......... .......... 45% 22.8M 3s 2024-04-20T10:35:46.252 INFO:tasks.workunit.client.0.smithi042.stderr: 61150K .......... .......... .......... .......... .......... 45% 27.9M 3s 2024-04-20T10:35:46.253 INFO:tasks.workunit.client.0.smithi042.stderr: 61200K .......... .......... .......... .......... .......... 45% 37.0M 3s 2024-04-20T10:35:46.255 INFO:tasks.workunit.client.0.smithi042.stderr: 61250K .......... .......... .......... .......... .......... 45% 26.2M 3s 2024-04-20T10:35:46.257 INFO:tasks.workunit.client.0.smithi042.stderr: 61300K .......... .......... .......... .......... .......... 45% 25.5M 3s 2024-04-20T10:35:46.259 INFO:tasks.workunit.client.0.smithi042.stderr: 61350K .......... .......... .......... .......... .......... 45% 30.0M 3s 2024-04-20T10:35:46.262 INFO:tasks.workunit.client.0.smithi042.stderr: 61400K .......... .......... .......... .......... .......... 45% 24.6M 3s 2024-04-20T10:35:46.262 INFO:tasks.workunit.client.0.smithi042.stderr: 61450K .......... .......... .......... .......... .......... 45% 39.7M 3s 2024-04-20T10:35:46.263 INFO:tasks.workunit.client.0.smithi042.stderr: 61500K .......... .......... .......... .......... .......... 45% 29.5M 3s 2024-04-20T10:35:46.265 INFO:tasks.workunit.client.0.smithi042.stderr: 61550K .......... .......... .......... .......... .......... 45% 25.7M 3s 2024-04-20T10:35:46.268 INFO:tasks.workunit.client.0.smithi042.stderr: 61600K .......... .......... .......... .......... .......... 45% 24.3M 3s 2024-04-20T10:35:46.269 INFO:tasks.workunit.client.0.smithi042.stderr: 61650K .......... .......... .......... .......... .......... 45% 28.8M 3s 2024-04-20T10:35:46.271 INFO:tasks.workunit.client.0.smithi042.stderr: 61700K .......... .......... .......... .......... .......... 45% 28.6M 3s 2024-04-20T10:35:46.272 INFO:tasks.workunit.client.0.smithi042.stderr: 61750K .......... .......... .......... .......... .......... 45% 34.9M 3s 2024-04-20T10:35:46.274 INFO:tasks.workunit.client.0.smithi042.stderr: 61800K .......... .......... .......... .......... .......... 45% 27.3M 3s 2024-04-20T10:35:46.276 INFO:tasks.workunit.client.0.smithi042.stderr: 61850K .......... .......... .......... .......... .......... 45% 27.3M 3s 2024-04-20T10:35:46.278 INFO:tasks.workunit.client.0.smithi042.stderr: 61900K .......... .......... .......... .......... .......... 45% 26.1M 3s 2024-04-20T10:35:46.280 INFO:tasks.workunit.client.0.smithi042.stderr: 61950K .......... .......... .......... .......... .......... 45% 29.8M 3s 2024-04-20T10:35:46.281 INFO:tasks.workunit.client.0.smithi042.stderr: 62000K .......... .......... .......... .......... .......... 45% 36.6M 3s 2024-04-20T10:35:46.283 INFO:tasks.workunit.client.0.smithi042.stderr: 62050K .......... .......... .......... .......... .......... 45% 26.1M 3s 2024-04-20T10:35:46.284 INFO:tasks.workunit.client.0.smithi042.stderr: 62100K .......... .......... .......... .......... .......... 45% 28.9M 3s 2024-04-20T10:35:46.285 INFO:tasks.workunit.client.0.smithi042.stderr: 62150K .......... .......... .......... .......... .......... 45% 34.5M 3s 2024-04-20T10:35:46.288 INFO:tasks.workunit.client.0.smithi042.stderr: 62200K .......... .......... .......... .......... .......... 45% 24.3M 3s 2024-04-20T10:35:46.289 INFO:tasks.workunit.client.0.smithi042.stderr: 62250K .......... .......... .......... .......... .......... 45% 29.2M 3s 2024-04-20T10:35:46.291 INFO:tasks.workunit.client.0.smithi042.stderr: 62300K .......... .......... .......... .......... .......... 45% 27.1M 3s 2024-04-20T10:35:46.294 INFO:tasks.workunit.client.0.smithi042.stderr: 62350K .......... .......... .......... .......... .......... 45% 25.4M 3s 2024-04-20T10:35:46.295 INFO:tasks.workunit.client.0.smithi042.stderr: 62400K .......... .......... .......... .......... .......... 46% 41.3M 3s 2024-04-20T10:35:46.296 INFO:tasks.workunit.client.0.smithi042.stderr: 62450K .......... .......... .......... .......... .......... 46% 21.7M 3s 2024-04-20T10:35:46.298 INFO:tasks.workunit.client.0.smithi042.stderr: 62500K .......... .......... .......... .......... .......... 46% 35.8M 3s 2024-04-20T10:35:46.300 INFO:tasks.workunit.client.0.smithi042.stderr: 62550K .......... .......... .......... .......... .......... 46% 26.9M 3s 2024-04-20T10:35:46.302 INFO:tasks.workunit.client.0.smithi042.stderr: 62600K .......... .......... .......... .......... .......... 46% 25.4M 3s 2024-04-20T10:35:46.303 INFO:tasks.workunit.client.0.smithi042.stderr: 62650K .......... .......... .......... .......... .......... 46% 33.9M 3s 2024-04-20T10:35:46.305 INFO:tasks.workunit.client.0.smithi042.stderr: 62700K .......... .......... .......... .......... .......... 46% 27.2M 3s 2024-04-20T10:35:46.306 INFO:tasks.workunit.client.0.smithi042.stderr: 62750K .......... .......... .......... .......... .......... 46% 36.8M 3s 2024-04-20T10:35:46.308 INFO:tasks.workunit.client.0.smithi042.stderr: 62800K .......... .......... .......... .......... .......... 46% 24.8M 3s 2024-04-20T10:35:46.310 INFO:tasks.workunit.client.0.smithi042.stderr: 62850K .......... .......... .......... .......... .......... 46% 34.0M 3s 2024-04-20T10:35:46.312 INFO:tasks.workunit.client.0.smithi042.stderr: 62900K .......... .......... .......... .......... .......... 46% 22.7M 3s 2024-04-20T10:35:46.313 INFO:tasks.workunit.client.0.smithi042.stderr: 62950K .......... .......... .......... .......... .......... 46% 30.5M 3s 2024-04-20T10:35:46.315 INFO:tasks.workunit.client.0.smithi042.stderr: 63000K .......... .......... .......... .......... .......... 46% 34.7M 3s 2024-04-20T10:35:46.317 INFO:tasks.workunit.client.0.smithi042.stderr: 63050K .......... .......... .......... .......... .......... 46% 23.1M 3s 2024-04-20T10:35:46.318 INFO:tasks.workunit.client.0.smithi042.stderr: 63100K .......... .......... .......... .......... .......... 46% 35.5M 3s 2024-04-20T10:35:46.320 INFO:tasks.workunit.client.0.smithi042.stderr: 63150K .......... .......... .......... .......... .......... 46% 25.8M 3s 2024-04-20T10:35:46.323 INFO:tasks.workunit.client.0.smithi042.stderr: 63200K .......... .......... .......... .......... .......... 46% 26.5M 3s 2024-04-20T10:35:46.323 INFO:tasks.workunit.client.0.smithi042.stderr: 63250K .......... .......... .......... .......... .......... 46% 40.6M 3s 2024-04-20T10:35:46.325 INFO:tasks.workunit.client.0.smithi042.stderr: 63300K .......... .......... .......... .......... .......... 46% 24.7M 3s 2024-04-20T10:35:46.327 INFO:tasks.workunit.client.0.smithi042.stderr: 63350K .......... .......... .......... .......... .......... 46% 35.6M 3s 2024-04-20T10:35:46.329 INFO:tasks.workunit.client.0.smithi042.stderr: 63400K .......... .......... .......... .......... .......... 46% 24.6M 3s 2024-04-20T10:35:46.331 INFO:tasks.workunit.client.0.smithi042.stderr: 63450K .......... .......... .......... .......... .......... 46% 26.8M 3s 2024-04-20T10:35:46.332 INFO:tasks.workunit.client.0.smithi042.stderr: 63500K .......... .......... .......... .......... .......... 46% 40.2M 3s 2024-04-20T10:35:46.334 INFO:tasks.workunit.client.0.smithi042.stderr: 63550K .......... .......... .......... .......... .......... 46% 24.4M 3s 2024-04-20T10:35:46.335 INFO:tasks.workunit.client.0.smithi042.stderr: 63600K .......... .......... .......... .......... .......... 46% 32.0M 3s 2024-04-20T10:35:46.337 INFO:tasks.workunit.client.0.smithi042.stderr: 63650K .......... .......... .......... .......... .......... 46% 24.3M 3s 2024-04-20T10:35:46.339 INFO:tasks.workunit.client.0.smithi042.stderr: 63700K .......... .......... .......... .......... .......... 46% 30.9M 3s 2024-04-20T10:35:46.340 INFO:tasks.workunit.client.0.smithi042.stderr: 63750K .......... .......... .......... .......... .......... 46% 32.0M 3s 2024-04-20T10:35:46.342 INFO:tasks.workunit.client.0.smithi042.stderr: 63800K .......... .......... .......... .......... .......... 47% 29.5M 3s 2024-04-20T10:35:46.344 INFO:tasks.workunit.client.0.smithi042.stderr: 63850K .......... .......... .......... .......... .......... 47% 33.1M 3s 2024-04-20T10:35:46.345 INFO:tasks.workunit.client.0.smithi042.stderr: 63900K .......... .......... .......... .......... .......... 47% 25.2M 3s 2024-04-20T10:35:46.347 INFO:tasks.workunit.client.0.smithi042.stderr: 63950K .......... .......... .......... .......... .......... 47% 24.5M 3s 2024-04-20T10:35:46.348 INFO:tasks.workunit.client.0.smithi042.stderr: 64000K .......... .......... .......... .......... .......... 47% 42.0M 3s 2024-04-20T10:35:46.350 INFO:tasks.workunit.client.0.smithi042.stderr: 64050K .......... .......... .......... .......... .......... 47% 24.3M 3s 2024-04-20T10:35:46.353 INFO:tasks.workunit.client.0.smithi042.stderr: 64100K .......... .......... .......... .......... .......... 47% 36.0M 3s 2024-04-20T10:35:46.355 INFO:tasks.workunit.client.0.smithi042.stderr: 64150K .......... .......... .......... .......... .......... 47% 25.1M 3s 2024-04-20T10:35:46.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.355+0000 7efe7ee00640 1 -- 172.21.15.42:0/1056167449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe78074c30 msgr2=0x7efe7810e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.355+0000 7efe7ee00640 1 --2- 172.21.15.42:0/1056167449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe78074c30 0x7efe7810e180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe64009900 tx=0x7efe6402bcd0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.356 INFO:tasks.workunit.client.0.smithi042.stderr: 64200K .......... .......... .......... .......... .......... 47% 24.7M 3s 2024-04-20T10:35:46.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.356+0000 7efe7ee00640 1 -- 172.21.15.42:0/1056167449 shutdown_connections 2024-04-20T10:35:46.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.356+0000 7efe7ee00640 1 --2- 172.21.15.42:0/1056167449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe7810e6c0 0x7efe78110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.356+0000 7efe7ee00640 1 --2- 172.21.15.42:0/1056167449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe78074c30 0x7efe7810e180 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.356+0000 7efe7ee00640 1 --2- 172.21.15.42:0/1056167449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe78074310 0x7efe780746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.356+0000 7efe7ee00640 1 -- 172.21.15.42:0/1056167449 >> 172.21.15.42:0/1056167449 conn(0x7efe780fd930 msgr2=0x7efe780ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:46.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.356+0000 7efe7ee00640 1 -- 172.21.15.42:0/1056167449 shutdown_connections 2024-04-20T10:35:46.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.356+0000 7efe7ee00640 1 -- 172.21.15.42:0/1056167449 wait complete. 2024-04-20T10:35:46.359 INFO:tasks.workunit.client.0.smithi042.stderr: 64250K .......... .......... .......... .......... .......... 47% 39.8M 3s 2024-04-20T10:35:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.357+0000 7efe7ee00640 1 Processor -- start 2024-04-20T10:35:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.357+0000 7efe7ee00640 1 -- start start 2024-04-20T10:35:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.357+0000 7efe7ee00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe78074310 0x7efe781a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.357+0000 7efe7ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe78074c30 0x7efe781a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.357+0000 7efe7ee00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe7810e6c0 0x7efe7819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.357+0000 7efe7ee00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efe78114420 con 0x7efe78074310 2024-04-20T10:35:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.357+0000 7efe7ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efe781142a0 con 0x7efe78074c30 2024-04-20T10:35:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.357+0000 7efe7ee00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efe781145a0 con 0x7efe7810e6c0 2024-04-20T10:35:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.358+0000 7efe7da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe78074310 0x7efe781a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.358+0000 7efe7da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe78074310 0x7efe781a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54652/0 (socket says 172.21.15.42:54652) 2024-04-20T10:35:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.358+0000 7efe7da00640 1 -- 172.21.15.42:0/701658850 learned_addr learned my addr 172.21.15.42:0/701658850 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.358+0000 7efe7e400640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe7810e6c0 0x7efe7819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.358+0000 7efe7d000640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe78074c30 0x7efe781a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.358+0000 7efe7da00640 1 -- 172.21.15.42:0/701658850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe7810e6c0 msgr2=0x7efe7819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.358+0000 7efe7da00640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe7810e6c0 0x7efe7819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.358+0000 7efe7da00640 1 -- 172.21.15.42:0/701658850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe78074c30 msgr2=0x7efe781a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.358+0000 7efe7da00640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe78074c30 0x7efe781a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.358+0000 7efe7da00640 1 -- 172.21.15.42:0/701658850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7819b020 con 0x7efe78074310 2024-04-20T10:35:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.358+0000 7efe7e400640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe7810e6c0 0x7efe7819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.359+0000 7efe7da00640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe78074310 0x7efe781a06a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00d770 tx=0x7efe6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:46.365 INFO:tasks.workunit.client.0.smithi042.stderr: 64300K .......... .......... .......... .......... .......... 47% 21.7M 3s 2024-04-20T10:35:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.359+0000 7efe76000640 1 -- 172.21.15.42:0/701658850 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c017070 con 0x7efe78074310 2024-04-20T10:35:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.359+0000 7efe76000640 1 -- 172.21.15.42:0/701658850 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe6c005500 con 0x7efe78074310 2024-04-20T10:35:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.359+0000 7efe7ee00640 1 -- 172.21.15.42:0/701658850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7819b310 con 0x7efe78074310 2024-04-20T10:35:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.359+0000 7efe76000640 1 -- 172.21.15.42:0/701658850 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c004e00 con 0x7efe78074310 2024-04-20T10:35:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.359+0000 7efe7ee00640 1 -- 172.21.15.42:0/701658850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe781a7620 con 0x7efe78074310 2024-04-20T10:35:46.367 INFO:tasks.workunit.client.0.smithi042.stderr: 64350K .......... .......... .......... .......... .......... 47% 36.8M 3s 2024-04-20T10:35:46.368 INFO:tasks.workunit.client.0.smithi042.stderr: 64400K .......... .......... .......... .......... .......... 47% 22.9M 3s 2024-04-20T10:35:46.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.363+0000 7efe831a1740 1 Processor -- start 2024-04-20T10:35:46.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.363+0000 7efe831a1740 1 -- start start 2024-04-20T10:35:46.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.363+0000 7efe76000640 1 -- 172.21.15.42:0/701658850 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7efe6c004fa0 con 0x7efe78074310 2024-04-20T10:35:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.363+0000 7efe831a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 0x5565a69b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.363+0000 7efe831a1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2ca0 0x5565a69bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe831a1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 0x5565a69bdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe831a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5565a6918150 con 0x5565a69b2380 2024-04-20T10:35:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe831a1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5565a68be370 con 0x5565a69b2ca0 2024-04-20T10:35:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe831a1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5565a6914110 con 0x5565a69bb880 2024-04-20T10:35:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2ca0 0x5565a69bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 0x5565a69b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2ca0 0x5565a69bb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:48844/0 (socket says 172.21.15.42:48844) 2024-04-20T10:35:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 0x5565a69b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54654/0 (socket says 172.21.15.42:54654) 2024-04-20T10:35:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7da00640 1 -- 172.21.15.42:0/1273873773 learned_addr learned my addr 172.21.15.42:0/1273873773 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7d000640 1 --2- 172.21.15.42:0/1273873773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 0x5565a69bdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.371 INFO:tasks.workunit.client.0.smithi042.stderr: 64450K .......... .......... .......... .......... .......... 47% 23.6M 3s 2024-04-20T10:35:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7da00640 1 -- 172.21.15.42:0/1273873773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 msgr2=0x5565a69bdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7da00640 1 --2- 172.21.15.42:0/1273873773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 0x5565a69bdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7da00640 1 -- 172.21.15.42:0/1273873773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 msgr2=0x5565a69b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7da00640 1 --2- 172.21.15.42:0/1273873773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 0x5565a69b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7da00640 1 -- 172.21.15.42:0/1273873773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a66f47d0 con 0x5565a69b2ca0 2024-04-20T10:35:46.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7d000640 1 --2- 172.21.15.42:0/1273873773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 0x5565a69bdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:46.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.364+0000 7efe7e400640 1 --2- 172.21.15.42:0/1273873773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 0x5565a69b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:46.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.365+0000 7efe76000640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7efe600a3da0 0x7efe600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.365+0000 7efe7da00640 1 --2- 172.21.15.42:0/1273873773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2ca0 0x5565a69bb340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0cd7a0 tx=0x7efe6c0d8910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=628789e3d6bef392 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:46.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.365+0000 7efe7e400640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7efe600a3da0 0x7efe600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.365+0000 7efe76000640 1 -- 172.21.15.42:0/701658850 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7efe6c0d0170 con 0x7efe78074310 2024-04-20T10:35:46.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.365+0000 7efe6aa00640 1 -- 172.21.15.42:0/1273873773 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c098400 con 0x5565a69b2ca0 2024-04-20T10:35:46.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.365+0000 7efe6aa00640 1 -- 172.21.15.42:0/1273873773 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe6c0046c0 con 0x5565a69b2ca0 2024-04-20T10:35:46.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.365+0000 7efe6aa00640 1 -- 172.21.15.42:0/1273873773 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0d3d10 con 0x5565a69b2ca0 2024-04-20T10:35:46.375 INFO:tasks.workunit.client.0.smithi042.stderr: 64500K .......... .......... .......... .......... .......... 47% 42.8M 3s 2024-04-20T10:35:46.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.366+0000 7efe7e400640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7efe600a3da0 0x7efe600a6260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efe70009fd0 tx=0x7efe70009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:46.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.366+0000 7efe831a1740 1 -- 172.21.15.42:0/1273873773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2ca0 msgr2=0x5565a69bb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.366+0000 7efe831a1740 1 --2- 172.21.15.42:0/1273873773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2ca0 0x5565a69bb340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0cd7a0 tx=0x7efe6c0d8910 comp rx=0 tx=0).stop 2024-04-20T10:35:46.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.366+0000 7efe831a1740 1 -- 172.21.15.42:0/1273873773 shutdown_connections 2024-04-20T10:35:46.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.366+0000 7efe831a1740 1 --2- 172.21.15.42:0/1273873773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 0x5565a69bdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.366+0000 7efe831a1740 1 --2- 172.21.15.42:0/1273873773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2ca0 0x5565a69bb340 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.366+0000 7efe831a1740 1 --2- 172.21.15.42:0/1273873773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 0x5565a69b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.366+0000 7efe831a1740 1 -- 172.21.15.42:0/1273873773 >> 172.21.15.42:0/1273873773 conn(0x5565a69acab0 msgr2=0x5565a69ace90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:46.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.367+0000 7efe831a1740 1 -- 172.21.15.42:0/1273873773 shutdown_connections 2024-04-20T10:35:46.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.367+0000 7efe831a1740 1 -- 172.21.15.42:0/1273873773 wait complete. 2024-04-20T10:35:46.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.367+0000 7efe831a1740 1 -- start start 2024-04-20T10:35:46.378 INFO:tasks.workunit.client.0.smithi042.stderr: 64550K .......... .......... .......... .......... .......... 47% 25.1M 3s 2024-04-20T10:35:46.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe831a1740 1 Processor -- start 2024-04-20T10:35:46.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe831a1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 0x5565a69b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe831a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2ca0 0x5565a69b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe831a1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 0x5565a69b9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe831a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5565a69be610 con 0x5565a69b2ca0 2024-04-20T10:35:46.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe831a1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5565a68d1740 con 0x5565a69b2380 2024-04-20T10:35:46.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe831a1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5565a69be790 con 0x5565a69bb880 2024-04-20T10:35:46.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe831a1740 10 client.0 _ll_register_callbacks cb 0x5565a68ccb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:46.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe7da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 0x5565a69b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe7e400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 0x5565a69b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe7da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 0x5565a69b89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:48848/0 (socket says 172.21.15.42:48848) 2024-04-20T10:35:46.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe7da00640 1 -- 172.21.15.42:0/2517402690 learned_addr learned my addr 172.21.15.42:0/2517402690 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:46.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe7e400640 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 0x5565a69b9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:51364/0 (socket says 172.21.15.42:51364) 2024-04-20T10:35:46.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.368+0000 7efe7d000640 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2ca0 0x5565a69b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.369+0000 7efe7da00640 1 -- 172.21.15.42:0/2517402690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 msgr2=0x5565a69b9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.369+0000 7efe7da00640 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 0x5565a69b9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.369+0000 7efe7da00640 1 -- 172.21.15.42:0/2517402690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2ca0 msgr2=0x5565a69b8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.369+0000 7efe7da00640 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2ca0 0x5565a69b8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.369+0000 7efe7da00640 1 -- 172.21.15.42:0/2517402690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a662a3f0 con 0x5565a69b2380 2024-04-20T10:35:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.369+0000 7efe7e400640 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 0x5565a69b9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:35:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.369+0000 7efe7da00640 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 0x5565a69b89f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efe6c098b30 tx=0x7efe6c0d3750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e488498dcd30d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.369+0000 7efe5f400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:35:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.369+0000 7efe5f400640 1 -- 172.21.15.42:0/2517402690 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c020070 con 0x5565a69b2380 2024-04-20T10:35:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.369+0000 7efe5f400640 1 -- 172.21.15.42:0/2517402690 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7efe6c009cc0 con 0x5565a69b2380 2024-04-20T10:35:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.369+0000 7efe831a1740 1 -- 172.21.15.42:0/2517402690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5565a6925b70 con 0x5565a69b2380 2024-04-20T10:35:46.383 INFO:tasks.workunit.client.0.smithi042.stderr: 64600K .......... .......... .......... .......... .......... 47% 36.2M 3s 2024-04-20T10:35:46.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.370+0000 7efe5f400640 1 -- 172.21.15.42:0/2517402690 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efe6c09bac0 con 0x5565a69b2380 2024-04-20T10:35:46.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.370+0000 7efe831a1740 10 client.25586 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:46.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.370+0000 7efe831a1740 1 -- 172.21.15.42:0/2517402690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5565a65ea0f0 con 0x5565a69b2380 2024-04-20T10:35:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.370+0000 7efe5f400640 1 -- 172.21.15.42:0/2517402690 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5565a6925b70 con 0x5565a69b2380 2024-04-20T10:35:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.370+0000 7efe5f400640 1 -- 172.21.15.42:0/2517402690 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7efe6c0987d0 con 0x5565a69b2380 2024-04-20T10:35:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.370+0000 7efe831a1740 10 client.25586 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.370+0000 7efe831a1740 10 client.25586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.370+0000 7efe831a1740 20 client.25586 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.370+0000 7efe831a1740 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5565a69cbb70 0x5565a69cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.370+0000 7efe831a1740 4 client.25586 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:35:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.370+0000 7efe831a1740 1 -- 172.21.15.42:0/2517402690 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565a691f9e0 con 0x5565a69cbb70 2024-04-20T10:35:46.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.371+0000 7efe7d000640 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5565a69cbb70 0x5565a69cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.386 INFO:tasks.workunit.client.0.smithi042.stderr: 64650K .......... .......... .......... .......... .......... 47% 24.8M 3s 2024-04-20T10:35:46.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.371+0000 7efe7d000640 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5565a69cbb70 0x5565a69cdf60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797247a815b57361 server_cookie=89d2efb147a33514 in_seq=0 out_seq=0 2024-04-20T10:35:46.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.371+0000 7efe5f400640 10 client.25586 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:46.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.372+0000 7efe5f400640 1 -- 172.21.15.42:0/2517402690 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565a691f9e0 con 0x5565a69cbb70 2024-04-20T10:35:46.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.372+0000 7efe5f400640 10 client.25586 handle_command_reply: tid=0 2024-04-20T10:35:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.372+0000 7efe831a1740 1 client.25586 shutdown 2024-04-20T10:35:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.372+0000 7efe831a1740 1 -- 172.21.15.42:0/2517402690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 msgr2=0x5565a69b89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.372+0000 7efe831a1740 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 0x5565a69b89f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efe6c098b30 tx=0x7efe6c0d3750 comp rx=0 tx=0).stop 2024-04-20T10:35:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.372+0000 7efe831a1740 1 -- 172.21.15.42:0/2517402690 shutdown_connections 2024-04-20T10:35:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.372+0000 7efe831a1740 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5565a69cbb70 0x5565a69cdf60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.372+0000 7efe831a1740 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69bb880 0x5565a69b9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.372+0000 7efe831a1740 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2ca0 0x5565a69b8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.372+0000 7efe831a1740 1 --2- 172.21.15.42:0/2517402690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 0x5565a69b89f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.372+0000 7efe5f400640 0 client.25586 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.372+0000 7efe831a1740 1 -- 172.21.15.42:0/2517402690 >> 172.21.15.42:0/2517402690 conn(0x5565a69acab0 msgr2=0x5565a69ae800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:46.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.373+0000 7efe831a1740 1 -- 172.21.15.42:0/2517402690 shutdown_connections 2024-04-20T10:35:46.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.373+0000 7efe831a1740 1 -- 172.21.15.42:0/2517402690 wait complete. 2024-04-20T10:35:46.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.373+0000 7efe831a1740 20 client.25586 trim_cache size 0 max 16384 2024-04-20T10:35:46.400 INFO:tasks.workunit.client.0.smithi042.stderr: 64700K .......... .......... .......... .......... .......... 47% 19.7M 3s 2024-04-20T10:35:46.401 INFO:tasks.workunit.client.0.smithi042.stderr: 64750K .......... .......... .......... .......... .......... 47% 48.6M 3s 2024-04-20T10:35:46.401 INFO:tasks.workunit.client.0.smithi042.stderr: 64800K .......... .......... .......... .......... .......... 47% 23.7M 3s 2024-04-20T10:35:46.401 INFO:tasks.workunit.client.0.smithi042.stderr: 64850K .......... .......... .......... .......... .......... 47% 24.6M 3s 2024-04-20T10:35:46.401 INFO:tasks.workunit.client.0.smithi042.stderr: 64900K .......... .......... .......... .......... .......... 47% 37.9M 3s 2024-04-20T10:35:46.401 INFO:tasks.workunit.client.0.smithi042.stderr: 64950K .......... .......... .......... .......... .......... 47% 24.4M 3s 2024-04-20T10:35:46.401 INFO:tasks.workunit.client.0.smithi042.stderr: 65000K .......... .......... .......... .......... .......... 47% 46.3M 3s 2024-04-20T10:35:46.401 INFO:tasks.workunit.client.0.smithi042.stderr: 65050K .......... .......... .......... .......... .......... 47% 23.5M 3s 2024-04-20T10:35:46.401 INFO:tasks.workunit.client.0.smithi042.stderr: 65100K .......... .......... .......... .......... .......... 47% 20.9M 3s 2024-04-20T10:35:46.401 INFO:tasks.workunit.client.0.smithi042.stderr: 65150K .......... .......... .......... .......... .......... 48% 33.8M 3s 2024-04-20T10:35:46.401 INFO:tasks.workunit.client.0.smithi042.stderr: 65200K .......... .......... .......... .......... .......... 48% 24.3M 3s 2024-04-20T10:35:46.401 INFO:tasks.workunit.client.0.smithi042.stderr: 65250K .......... .......... .......... .......... .......... 48% 34.8M 3s 2024-04-20T10:35:46.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.393+0000 7efe831a1740 1 Processor -- start 2024-04-20T10:35:46.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.393+0000 7efe831a1740 1 -- start start 2024-04-20T10:35:46.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.394+0000 7efe831a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 0x5565a69bb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.394+0000 7efe831a1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69b2ca0 0x5565a69b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.394+0000 7efe831a1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69bb880 0x5565a69ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.394+0000 7efe831a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efe781addd0 con 0x5565a69b2380 2024-04-20T10:35:46.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.394+0000 7efe831a1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efe781b0650 con 0x5565a69bb880 2024-04-20T10:35:46.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.394+0000 7efe831a1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efe781ae440 con 0x5565a69b2ca0 2024-04-20T10:35:46.403 INFO:tasks.workunit.client.0.smithi042.stderr: 65300K .......... .......... .......... .......... .......... 48% 22.5M 3s 2024-04-20T10:35:46.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.394+0000 7efe7da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 0x5565a69bb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.394+0000 7efe7da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 0x5565a69bb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54668/0 (socket says 172.21.15.42:54668) 2024-04-20T10:35:46.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.394+0000 7efe7da00640 1 -- 172.21.15.42:0/2969660853 learned_addr learned my addr 172.21.15.42:0/2969660853 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:46.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.394+0000 7efe7e400640 1 --2- 172.21.15.42:0/2969660853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69bb880 0x5565a69ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.394+0000 7efe7d000640 1 --2- 172.21.15.42:0/2969660853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69b2ca0 0x5565a69b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.395+0000 7efe7da00640 1 -- 172.21.15.42:0/2969660853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69b2ca0 msgr2=0x5565a69b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.395+0000 7efe7da00640 1 --2- 172.21.15.42:0/2969660853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69b2ca0 0x5565a69b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.395+0000 7efe7da00640 1 -- 172.21.15.42:0/2969660853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69bb880 msgr2=0x5565a69ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.395+0000 7efe7da00640 1 --2- 172.21.15.42:0/2969660853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69bb880 0x5565a69ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.395+0000 7efe7da00640 1 -- 172.21.15.42:0/2969660853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a69b79a0 con 0x5565a69b2380 2024-04-20T10:35:46.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.395+0000 7efe7da00640 1 --2- 172.21.15.42:0/2969660853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 0x5565a69bb120 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efe6c09a410 tx=0x7efe6c0d8910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770f86f5dc8ea77b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:46.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.395+0000 7efe6a000640 1 -- 172.21.15.42:0/2969660853 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c020070 con 0x5565a69b2380 2024-04-20T10:35:46.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.395+0000 7efe6a000640 1 -- 172.21.15.42:0/2969660853 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe6c09c160 con 0x5565a69b2380 2024-04-20T10:35:46.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.395+0000 7efe6a000640 1 -- 172.21.15.42:0/2969660853 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c09bca0 con 0x5565a69b2380 2024-04-20T10:35:46.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.396+0000 7efe831a1740 1 -- 172.21.15.42:0/2969660853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 msgr2=0x5565a69bb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.396+0000 7efe831a1740 1 --2- 172.21.15.42:0/2969660853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 0x5565a69bb120 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efe6c09a410 tx=0x7efe6c0d8910 comp rx=0 tx=0).stop 2024-04-20T10:35:46.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.396+0000 7efe831a1740 1 -- 172.21.15.42:0/2969660853 shutdown_connections 2024-04-20T10:35:46.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.396+0000 7efe831a1740 1 --2- 172.21.15.42:0/2969660853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69bb880 0x5565a69ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.396+0000 7efe831a1740 1 --2- 172.21.15.42:0/2969660853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69b2ca0 0x5565a69b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.396+0000 7efe831a1740 1 --2- 172.21.15.42:0/2969660853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69b2380 0x5565a69bb120 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.396+0000 7efe831a1740 1 -- 172.21.15.42:0/2969660853 >> 172.21.15.42:0/2969660853 conn(0x5565a69acab0 msgr2=0x5565a69adee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:46.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.396+0000 7efe831a1740 1 -- 172.21.15.42:0/2969660853 shutdown_connections 2024-04-20T10:35:46.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.396+0000 7efe831a1740 1 -- 172.21.15.42:0/2969660853 wait complete. 2024-04-20T10:35:46.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.396+0000 7efe831a1740 1 -- start start 2024-04-20T10:35:46.408 INFO:tasks.workunit.client.0.smithi042.stderr: 65350K .......... .......... .......... .......... .......... 48% 19.9M 3s 2024-04-20T10:35:46.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.397+0000 7efe831a1740 1 Processor -- start 2024-04-20T10:35:46.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.397+0000 7efe831a1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 0x5565a69d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.397+0000 7efe831a1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69b2ca0 0x5565a69ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.397+0000 7efe831a1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69bb880 0x5565a69ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.397+0000 7efe831a1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efe781aeb10 con 0x5565a69bb880 2024-04-20T10:35:46.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.397+0000 7efe831a1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efe781aff80 con 0x5565a69b2380 2024-04-20T10:35:46.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.397+0000 7efe831a1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efe781af1e0 con 0x5565a69b2ca0 2024-04-20T10:35:46.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.397+0000 7efe831a1740 10 client.0 _ll_register_callbacks cb 0x7efe70009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:35:46.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.397+0000 7efe7da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 0x5565a69d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.397+0000 7efe7e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69bb880 0x5565a69ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.397+0000 7efe7da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 0x5565a69d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:48862/0 (socket says 172.21.15.42:48862) 2024-04-20T10:35:46.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.398+0000 7efe7da00640 1 -- 172.21.15.42:0/2426294816 learned_addr learned my addr 172.21.15.42:0/2426294816 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:35:46.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.398+0000 7efe7d000640 1 --2- 172.21.15.42:0/2426294816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69b2ca0 0x5565a69ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.398+0000 7efe7da00640 1 -- 172.21.15.42:0/2426294816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69b2ca0 msgr2=0x5565a69ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.398+0000 7efe7da00640 1 --2- 172.21.15.42:0/2426294816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69b2ca0 0x5565a69ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.398+0000 7efe7da00640 1 -- 172.21.15.42:0/2426294816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69bb880 msgr2=0x5565a69ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:35:46.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.398+0000 7efe7da00640 1 --2- 172.21.15.42:0/2426294816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69bb880 0x5565a69ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.398+0000 7efe7da00640 1 -- 172.21.15.42:0/2426294816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a69b2ad0 con 0x5565a69b2380 2024-04-20T10:35:46.412 INFO:tasks.workunit.client.0.smithi042.stderr: 65400K .......... .......... .......... .......... .......... 48% 36.0M 3s 2024-04-20T10:35:46.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.398+0000 7efe7da00640 1 --2- 172.21.15.42:0/2426294816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 0x5565a69d1260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0985c0 tx=0x7efe6c0d8370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aef5d4dfdf00e968 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:35:46.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.398+0000 7efe5f400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:35:46.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.398+0000 7efe5f400640 1 -- 172.21.15.42:0/2426294816 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0b9030 con 0x5565a69b2380 2024-04-20T10:35:46.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.398+0000 7efe5f400640 1 -- 172.21.15.42:0/2426294816 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7efe6c0987d0 con 0x5565a69b2380 2024-04-20T10:35:46.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.399+0000 7efe831a1740 1 -- 172.21.15.42:0/2426294816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe6c09c160 con 0x5565a69b2380 2024-04-20T10:35:46.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.399+0000 7efe5f400640 1 -- 172.21.15.42:0/2426294816 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efe6c014030 con 0x5565a69b2380 2024-04-20T10:35:46.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.399+0000 7efe831a1740 10 client.25592 fetch_fsmap learned FSMap version 19 2024-04-20T10:35:46.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.399+0000 7efe831a1740 1 -- 172.21.15.42:0/2426294816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5565a69b2ad0 con 0x5565a69b2380 2024-04-20T10:35:46.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.399+0000 7efe5f400640 1 -- 172.21.15.42:0/2426294816 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe6c09c160 con 0x5565a69b2380 2024-04-20T10:35:46.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.399+0000 7efe5f400640 1 -- 172.21.15.42:0/2426294816 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7efe6c09c3c0 con 0x5565a69b2380 2024-04-20T10:35:46.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.399+0000 7efe831a1740 10 client.25592 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:35:46.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.399+0000 7efe831a1740 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:35:46.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.399+0000 7efe831a1740 20 client.25592 populate_metadata read hostname 'smithi042' 2024-04-20T10:35:46.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.400+0000 7efe831a1740 1 --2- 172.21.15.42:0/2426294816 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5565a6a31d40 0x5565a6a34130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:35:46.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.400+0000 7efe831a1740 4 client.25592 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:35:46.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.400+0000 7efe831a1740 1 -- 172.21.15.42:0/2426294816 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5565a689b450 con 0x5565a6a31d40 2024-04-20T10:35:46.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.400+0000 7efe7d000640 1 --2- 172.21.15.42:0/2426294816 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5565a6a31d40 0x5565a6a34130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:35:46.417 INFO:tasks.workunit.client.0.smithi042.stderr: 65450K .......... .......... .......... .......... .......... 48% 21.5M 3s 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.400+0000 7efe7d000640 1 --2- 172.21.15.42:0/2426294816 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5565a6a31d40 0x5565a6a34130 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f021f9104c4938b9 server_cookie=3f05ef76c141e158 in_seq=0 out_seq=0 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.400+0000 7efe5f400640 10 client.25592 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe5f400640 1 -- 172.21.15.42:0/2426294816 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5565a689b450 con 0x5565a6a31d40 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe5f400640 10 client.25592 handle_command_reply: tid=0 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe831a1740 1 client.25592 shutdown 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe831a1740 1 -- 172.21.15.42:0/2426294816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 msgr2=0x5565a69d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe831a1740 1 --2- 172.21.15.42:0/2426294816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 0x5565a69d1260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0985c0 tx=0x7efe6c0d8370 comp rx=0 tx=0).stop 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe831a1740 1 -- 172.21.15.42:0/2426294816 shutdown_connections 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe831a1740 1 --2- 172.21.15.42:0/2426294816 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5565a6a31d40 0x5565a6a34130 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe831a1740 1 --2- 172.21.15.42:0/2426294816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5565a69bb880 0x5565a69ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe831a1740 1 --2- 172.21.15.42:0/2426294816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565a69b2ca0 0x5565a69ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe831a1740 1 --2- 172.21.15.42:0/2426294816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5565a69b2380 0x5565a69d1260 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe5f400640 0 client.25592 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:35:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe831a1740 1 -- 172.21.15.42:0/2426294816 >> 172.21.15.42:0/2426294816 conn(0x5565a69acab0 msgr2=0x5565a69c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe831a1740 1 -- 172.21.15.42:0/2426294816 shutdown_connections 2024-04-20T10:35:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe831a1740 1 -- 172.21.15.42:0/2426294816 wait complete. 2024-04-20T10:35:46.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.401+0000 7efe831a1740 20 client.25592 trim_cache size 0 max 16384 2024-04-20T10:35:46.420 INFO:tasks.workunit.client.0.smithi042.stderr: 65500K .......... .......... .......... .......... .......... 48% 39.3M 3s 2024-04-20T10:35:46.420 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:35:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T10:35:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:35:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "aaffe980-3a4b-4166-8f0a-a2394b20ddb1": { 2024-04-20T10:35:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:35:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "aaffe980-3a4b-4166-8f0a-a2394b20ddb1", 2024-04-20T10:35:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:35:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:35:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:35:46.421 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:35:46.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.402+0000 7efe7ee00640 1 -- 172.21.15.42:0/701658850 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7efe600a3da0 msgr2=0x7efe600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.402+0000 7efe7ee00640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7efe600a3da0 0x7efe600a6260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efe70009fd0 tx=0x7efe70009500 comp rx=0 tx=0).stop 2024-04-20T10:35:46.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.402+0000 7efe7ee00640 1 -- 172.21.15.42:0/701658850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe78074310 msgr2=0x7efe781a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:35:46.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.402+0000 7efe7ee00640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe78074310 0x7efe781a06a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00d770 tx=0x7efe6c00dc30 comp rx=0 tx=0).stop 2024-04-20T10:35:46.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.402+0000 7efe7ee00640 1 -- 172.21.15.42:0/701658850 shutdown_connections 2024-04-20T10:35:46.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.402+0000 7efe7ee00640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7efe600a3da0 0x7efe600a6260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.402+0000 7efe7ee00640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe7810e6c0 0x7efe7819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.402+0000 7efe7ee00640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe78074c30 0x7efe781a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.402+0000 7efe7ee00640 1 --2- 172.21.15.42:0/701658850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe78074310 0x7efe781a06a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:35:46.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.402+0000 7efe7ee00640 1 -- 172.21.15.42:0/701658850 >> 172.21.15.42:0/701658850 conn(0x7efe780fd930 msgr2=0x7efe780fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:35:46.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.402+0000 7efe7ee00640 1 -- 172.21.15.42:0/701658850 shutdown_connections 2024-04-20T10:35:46.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:35:46.402+0000 7efe7ee00640 1 -- 172.21.15.42:0/701658850 wait complete. 2024-04-20T10:35:46.431 INFO:tasks.workunit.client.0.smithi042.stderr: 65550K .......... .......... .......... .......... .......... 48% 24.3M 3s 2024-04-20T10:35:46.431 INFO:tasks.workunit.client.0.smithi042.stderr: 65600K .......... .......... .......... .......... .......... 48% 23.4M 3s 2024-04-20T10:35:46.431 INFO:tasks.workunit.client.0.smithi042.stderr: 65650K .......... .......... .......... .......... .......... 48% 37.1M 3s 2024-04-20T10:35:46.431 INFO:tasks.workunit.client.0.smithi042.stderr: 65700K .......... .......... .......... .......... .......... 48% 25.0M 3s 2024-04-20T10:35:46.431 INFO:tasks.workunit.client.0.smithi042.stderr: 65750K .......... .......... .......... .......... .......... 48% 44.5M 3s 2024-04-20T10:35:46.431 INFO:tasks.workunit.client.0.smithi042.stderr: 65800K .......... .......... .......... .......... .......... 48% 24.7M 3s 2024-04-20T10:35:46.431 INFO:tasks.workunit.client.0.smithi042.stderr: 65850K .......... .......... .......... .......... .......... 48% 23.9M 3s 2024-04-20T10:35:46.431 INFO:tasks.workunit.client.0.smithi042.stderr: 65900K .......... .......... .......... .......... .......... 48% 36.6M 3s 2024-04-20T10:35:46.431 INFO:tasks.workunit.client.0.smithi042.stderr: 65950K .......... .......... .......... .......... .......... 48% 21.3M 3s 2024-04-20T10:35:46.432 INFO:tasks.cephfs.filesystem:scrub status for tag:aaffe980-3a4b-4166-8f0a-a2394b20ddb1 - {'path': '/', 'tag': 'aaffe980-3a4b-4166-8f0a-a2394b20ddb1', 'options': 'recursive,force'} 2024-04-20T10:35:46.443 INFO:tasks.workunit.client.0.smithi042.stderr: 66000K .......... .......... .......... .......... .......... 48% 45.5M 3s 2024-04-20T10:35:46.443 INFO:tasks.workunit.client.0.smithi042.stderr: 66050K .......... .......... .......... .......... .......... 48% 21.7M 3s 2024-04-20T10:35:46.443 INFO:tasks.workunit.client.0.smithi042.stderr: 66100K .......... .......... .......... .......... .......... 48% 27.1M 3s 2024-04-20T10:35:46.443 INFO:tasks.workunit.client.0.smithi042.stderr: 66150K .......... .......... .......... .......... .......... 48% 28.2M 3s 2024-04-20T10:35:46.443 INFO:tasks.workunit.client.0.smithi042.stderr: 66200K .......... .......... .......... .......... .......... 48% 31.6M 3s 2024-04-20T10:35:46.444 INFO:tasks.workunit.client.0.smithi042.stderr: 66250K .......... .......... .......... .......... .......... 48% 19.3M 3s 2024-04-20T10:35:46.444 INFO:tasks.workunit.client.0.smithi042.stderr: 66300K .......... .......... .......... .......... .......... 48% 37.9M 3s 2024-04-20T10:35:46.444 INFO:tasks.workunit.client.0.smithi042.stderr: 66350K .......... .......... .......... .......... .......... 48% 28.6M 3s 2024-04-20T10:35:46.444 INFO:tasks.workunit.client.0.smithi042.stderr: 66400K .......... .......... .......... .......... .......... 48% 35.5M 3s 2024-04-20T10:35:46.444 INFO:tasks.workunit.client.0.smithi042.stderr: 66450K .......... .......... .......... .......... .......... 48% 23.9M 3s 2024-04-20T10:35:46.444 INFO:tasks.workunit.client.0.smithi042.stderr: 66500K .......... .......... .......... .......... .......... 49% 26.4M 3s 2024-04-20T10:35:46.444 INFO:tasks.workunit.client.0.smithi042.stderr: 66550K .......... .......... .......... .......... .......... 49% 37.8M 3s 2024-04-20T10:35:46.444 INFO:tasks.workunit.client.0.smithi042.stderr: 66600K .......... .......... .......... .......... .......... 49% 29.8M 3s 2024-04-20T10:35:46.444 INFO:tasks.workunit.client.0.smithi042.stderr: 66650K .......... .......... .......... .......... .......... 49% 26.7M 3s 2024-04-20T10:35:46.447 INFO:tasks.workunit.client.0.smithi042.stderr: 66700K .......... .......... .......... .......... .......... 49% 23.8M 3s 2024-04-20T10:35:46.447 INFO:tasks.workunit.client.0.smithi042.stderr: 66750K .......... .......... .......... .......... .......... 49% 26.4M 3s 2024-04-20T10:35:46.447 INFO:tasks.workunit.client.0.smithi042.stderr: 66800K .......... .......... .......... .......... .......... 49% 51.6M 3s 2024-04-20T10:35:46.449 INFO:tasks.workunit.client.0.smithi042.stderr: 66850K .......... .......... .......... .......... .......... 49% 34.9M 3s 2024-04-20T10:35:46.451 INFO:tasks.workunit.client.0.smithi042.stderr: 66900K .......... .......... .......... .......... .......... 49% 23.4M 3s 2024-04-20T10:35:46.453 INFO:tasks.workunit.client.0.smithi042.stderr: 66950K .......... .......... .......... .......... .......... 49% 25.9M 3s 2024-04-20T10:35:46.454 INFO:tasks.workunit.client.0.smithi042.stderr: 67000K .......... .......... .......... .......... .......... 49% 37.5M 3s 2024-04-20T10:35:46.476 INFO:tasks.workunit.client.0.smithi042.stderr: 67050K .......... .......... .......... .......... .......... 49% 26.5M 3s 2024-04-20T10:35:46.476 INFO:tasks.workunit.client.0.smithi042.stderr: 67100K .......... .......... .......... .......... .......... 49% 23.3M 3s 2024-04-20T10:35:46.476 INFO:tasks.workunit.client.0.smithi042.stderr: 67150K .......... .......... .......... .......... .......... 49% 35.3M 3s 2024-04-20T10:35:46.476 INFO:tasks.workunit.client.0.smithi042.stderr: 67200K .......... .......... .......... .......... .......... 49% 26.7M 3s 2024-04-20T10:35:46.476 INFO:tasks.workunit.client.0.smithi042.stderr: 67250K .......... .......... .......... .......... .......... 49% 23.9M 3s 2024-04-20T10:35:46.476 INFO:tasks.workunit.client.0.smithi042.stderr: 67300K .......... .......... .......... .......... .......... 49% 48.3M 3s 2024-04-20T10:35:46.476 INFO:tasks.workunit.client.0.smithi042.stderr: 67350K .......... .......... .......... .......... .......... 49% 27.0M 3s 2024-04-20T10:35:46.476 INFO:tasks.workunit.client.0.smithi042.stderr: 67400K .......... .......... .......... .......... .......... 49% 28.8M 3s 2024-04-20T10:35:46.476 INFO:tasks.workunit.client.0.smithi042.stderr: 67450K .......... .......... .......... .......... .......... 49% 24.0M 3s 2024-04-20T10:35:46.476 INFO:tasks.workunit.client.0.smithi042.stderr: 67500K .......... .......... .......... .......... .......... 49% 25.8M 3s 2024-04-20T10:35:46.477 INFO:tasks.workunit.client.0.smithi042.stderr: 67550K .......... .......... .......... .......... .......... 49% 41.6M 3s 2024-04-20T10:35:46.477 INFO:tasks.workunit.client.0.smithi042.stderr: 67600K .......... .......... .......... .......... .......... 49% 29.4M 3s 2024-04-20T10:35:46.477 INFO:tasks.workunit.client.0.smithi042.stderr: 67650K .......... .......... .......... .......... .......... 49% 27.3M 3s 2024-04-20T10:35:46.479 INFO:tasks.workunit.client.0.smithi042.stderr: 67700K .......... .......... .......... .......... .......... 49% 21.5M 3s 2024-04-20T10:35:46.481 INFO:tasks.workunit.client.0.smithi042.stderr: 67750K .......... .......... .......... .......... .......... 49% 28.5M 3s 2024-04-20T10:35:46.481 INFO:tasks.workunit.client.0.smithi042.stderr: 67800K .......... .......... .......... .......... .......... 49% 48.4M 3s 2024-04-20T10:35:46.483 INFO:tasks.workunit.client.0.smithi042.stderr: 67850K .......... .......... .......... .......... .......... 50% 19.4M 3s 2024-04-20T10:35:46.484 INFO:tasks.workunit.client.0.smithi042.stderr: 67900K .......... .......... .......... .......... .......... 50% 42.1M 3s 2024-04-20T10:35:46.486 INFO:tasks.workunit.client.0.smithi042.stderr: 67950K .......... .......... .......... .......... .......... 50% 24.8M 3s 2024-04-20T10:35:46.488 INFO:tasks.workunit.client.0.smithi042.stderr: 68000K .......... .......... .......... .......... .......... 50% 26.4M 3s 2024-04-20T10:35:46.490 INFO:tasks.workunit.client.0.smithi042.stderr: 68050K .......... .......... .......... .......... .......... 50% 37.6M 3s 2024-04-20T10:35:46.491 INFO:tasks.workunit.client.0.smithi042.stderr: 68100K .......... .......... .......... .......... .......... 50% 26.2M 3s 2024-04-20T10:35:46.493 INFO:tasks.workunit.client.0.smithi042.stderr: 68150K .......... .......... .......... .......... .......... 50% 31.0M 3s 2024-04-20T10:35:46.495 INFO:tasks.workunit.client.0.smithi042.stderr: 68200K .......... .......... .......... .......... .......... 50% 25.3M 3s 2024-04-20T10:35:46.497 INFO:tasks.workunit.client.0.smithi042.stderr: 68250K .......... .......... .......... .......... .......... 50% 26.3M 3s 2024-04-20T10:35:46.498 INFO:tasks.workunit.client.0.smithi042.stderr: 68300K .......... .......... .......... .......... .......... 50% 38.2M 3s 2024-04-20T10:35:46.500 INFO:tasks.workunit.client.0.smithi042.stderr: 68350K .......... .......... .......... .......... .......... 50% 23.3M 3s 2024-04-20T10:35:46.501 INFO:tasks.workunit.client.0.smithi042.stderr: 68400K .......... .......... .......... .......... .......... 50% 36.2M 3s 2024-04-20T10:35:46.503 INFO:tasks.workunit.client.0.smithi042.stderr: 68450K .......... .......... .......... .......... .......... 50% 26.2M 3s 2024-04-20T10:35:46.506 INFO:tasks.workunit.client.0.smithi042.stderr: 68500K .......... .......... .......... .......... .......... 50% 25.2M 3s 2024-04-20T10:35:46.507 INFO:tasks.workunit.client.0.smithi042.stderr: 68550K .......... .......... .......... .......... .......... 50% 36.1M 3s 2024-04-20T10:35:46.509 INFO:tasks.workunit.client.0.smithi042.stderr: 68600K .......... .......... .......... .......... .......... 50% 25.3M 3s 2024-04-20T10:35:46.510 INFO:tasks.workunit.client.0.smithi042.stderr: 68650K .......... .......... .......... .......... .......... 50% 36.4M 3s 2024-04-20T10:35:46.512 INFO:tasks.workunit.client.0.smithi042.stderr: 68700K .......... .......... .......... .......... .......... 50% 25.0M 3s 2024-04-20T10:35:46.514 INFO:tasks.workunit.client.0.smithi042.stderr: 68750K .......... .......... .......... .......... .......... 50% 26.3M 3s 2024-04-20T10:35:46.515 INFO:tasks.workunit.client.0.smithi042.stderr: 68800K .......... .......... .......... .......... .......... 50% 36.0M 3s 2024-04-20T10:35:46.517 INFO:tasks.workunit.client.0.smithi042.stderr: 68850K .......... .......... .......... .......... .......... 50% 26.3M 3s 2024-04-20T10:35:46.518 INFO:tasks.workunit.client.0.smithi042.stderr: 68900K .......... .......... .......... .......... .......... 50% 35.5M 3s 2024-04-20T10:35:46.520 INFO:tasks.workunit.client.0.smithi042.stderr: 68950K .......... .......... .......... .......... .......... 50% 24.0M 3s 2024-04-20T10:35:46.522 INFO:tasks.workunit.client.0.smithi042.stderr: 69000K .......... .......... .......... .......... .......... 50% 26.8M 3s 2024-04-20T10:35:46.523 INFO:tasks.workunit.client.0.smithi042.stderr: 69050K .......... .......... .......... .......... .......... 50% 38.3M 3s 2024-04-20T10:35:46.525 INFO:tasks.workunit.client.0.smithi042.stderr: 69100K .......... .......... .......... .......... .......... 50% 24.0M 3s 2024-04-20T10:35:46.527 INFO:tasks.workunit.client.0.smithi042.stderr: 69150K .......... .......... .......... .......... .......... 50% 32.1M 3s 2024-04-20T10:35:46.529 INFO:tasks.workunit.client.0.smithi042.stderr: 69200K .......... .......... .......... .......... .......... 51% 22.3M 3s 2024-04-20T10:35:46.531 INFO:tasks.workunit.client.0.smithi042.stderr: 69250K .......... .......... .......... .......... .......... 51% 28.4M 3s 2024-04-20T10:35:46.532 INFO:tasks.workunit.client.0.smithi042.stderr: 69300K .......... .......... .......... .......... .......... 51% 42.3M 3s 2024-04-20T10:35:46.534 INFO:tasks.workunit.client.0.smithi042.stderr: 69350K .......... .......... .......... .......... .......... 51% 23.3M 3s 2024-04-20T10:35:46.535 INFO:tasks.workunit.client.0.smithi042.stderr: 69400K .......... .......... .......... .......... .......... 51% 33.4M 3s 2024-04-20T10:35:46.537 INFO:tasks.workunit.client.0.smithi042.stderr: 69450K .......... .......... .......... .......... .......... 51% 27.8M 3s 2024-04-20T10:35:46.540 INFO:tasks.workunit.client.0.smithi042.stderr: 69500K .......... .......... .......... .......... .......... 51% 21.5M 3s 2024-04-20T10:35:46.540 INFO:tasks.workunit.client.0.smithi042.stderr: 69550K .......... .......... .......... .......... .......... 51% 42.2M 3s 2024-04-20T10:35:46.543 INFO:tasks.workunit.client.0.smithi042.stderr: 69600K .......... .......... .......... .......... .......... 51% 25.2M 3s 2024-04-20T10:35:46.544 INFO:tasks.workunit.client.0.smithi042.stderr: 69650K .......... .......... .......... .......... .......... 51% 31.9M 3s 2024-04-20T10:35:46.547 INFO:tasks.workunit.client.0.smithi042.stderr: 69700K .......... .......... .......... .......... .......... 51% 21.9M 3s 2024-04-20T10:35:46.549 INFO:tasks.workunit.client.0.smithi042.stderr: 69750K .......... .......... .......... .......... .......... 51% 23.3M 3s 2024-04-20T10:35:46.550 INFO:tasks.workunit.client.0.smithi042.stderr: 69800K .......... .......... .......... .......... .......... 51% 43.6M 3s 2024-04-20T10:35:46.552 INFO:tasks.workunit.client.0.smithi042.stderr: 69850K .......... .......... .......... .......... .......... 51% 26.7M 3s 2024-04-20T10:35:46.553 INFO:tasks.workunit.client.0.smithi042.stderr: 69900K .......... .......... .......... .......... .......... 51% 32.5M 3s 2024-04-20T10:35:46.555 INFO:tasks.workunit.client.0.smithi042.stderr: 69950K .......... .......... .......... .......... .......... 51% 24.5M 3s 2024-04-20T10:35:46.557 INFO:tasks.workunit.client.0.smithi042.stderr: 70000K .......... .......... .......... .......... .......... 51% 26.2M 3s 2024-04-20T10:35:46.558 INFO:tasks.workunit.client.0.smithi042.stderr: 70050K .......... .......... .......... .......... .......... 51% 31.3M 3s 2024-04-20T10:35:46.561 INFO:tasks.workunit.client.0.smithi042.stderr: 70100K .......... .......... .......... .......... .......... 51% 23.8M 3s 2024-04-20T10:35:46.563 INFO:tasks.workunit.client.0.smithi042.stderr: 70150K .......... .......... .......... .......... .......... 51% 20.4M 3s 2024-04-20T10:35:46.564 INFO:tasks.workunit.client.0.smithi042.stderr: 70200K .......... .......... .......... .......... .......... 51% 33.4M 3s 2024-04-20T10:35:46.566 INFO:tasks.workunit.client.0.smithi042.stderr: 70250K .......... .......... .......... .......... .......... 51% 23.7M 3s 2024-04-20T10:35:46.568 INFO:tasks.workunit.client.0.smithi042.stderr: 70300K .......... .......... .......... .......... .......... 51% 30.7M 3s 2024-04-20T10:35:46.570 INFO:tasks.workunit.client.0.smithi042.stderr: 70350K .......... .......... .......... .......... .......... 51% 24.5M 3s 2024-04-20T10:35:46.572 INFO:tasks.workunit.client.0.smithi042.stderr: 70400K .......... .......... .......... .......... .......... 51% 23.5M 3s 2024-04-20T10:35:46.574 INFO:tasks.workunit.client.0.smithi042.stderr: 70450K .......... .......... .......... .......... .......... 51% 29.1M 3s 2024-04-20T10:35:46.576 INFO:tasks.workunit.client.0.smithi042.stderr: 70500K .......... .......... .......... .......... .......... 51% 25.6M 3s 2024-04-20T10:35:46.577 INFO:tasks.workunit.client.0.smithi042.stderr: 70550K .......... .......... .......... .......... .......... 52% 36.5M 3s 2024-04-20T10:35:46.579 INFO:tasks.workunit.client.0.smithi042.stderr: 70600K .......... .......... .......... .......... .......... 52% 27.1M 3s 2024-04-20T10:35:46.581 INFO:tasks.workunit.client.0.smithi042.stderr: 70650K .......... .......... .......... .......... .......... 52% 24.7M 3s 2024-04-20T10:35:46.582 INFO:tasks.workunit.client.0.smithi042.stderr: 70700K .......... .......... .......... .......... .......... 52% 34.9M 3s 2024-04-20T10:35:46.584 INFO:tasks.workunit.client.0.smithi042.stderr: 70750K .......... .......... .......... .......... .......... 52% 23.6M 3s 2024-04-20T10:35:46.585 INFO:tasks.workunit.client.0.smithi042.stderr: 70800K .......... .......... .......... .......... .......... 52% 34.6M 3s 2024-04-20T10:35:46.587 INFO:tasks.workunit.client.0.smithi042.stderr: 70850K .......... .......... .......... .......... .......... 52% 30.6M 3s 2024-04-20T10:35:46.590 INFO:tasks.workunit.client.0.smithi042.stderr: 70900K .......... .......... .......... .......... .......... 52% 19.7M 3s 2024-04-20T10:35:46.591 INFO:tasks.workunit.client.0.smithi042.stderr: 70950K .......... .......... .......... .......... .......... 52% 32.9M 3s 2024-04-20T10:35:46.593 INFO:tasks.workunit.client.0.smithi042.stderr: 71000K .......... .......... .......... .......... .......... 52% 22.7M 3s 2024-04-20T10:35:46.596 INFO:tasks.workunit.client.0.smithi042.stderr: 71050K .......... .......... .......... .......... .......... 52% 43.6M 3s 2024-04-20T10:35:46.597 INFO:tasks.workunit.client.0.smithi042.stderr: 71100K .......... .......... .......... .......... .......... 52% 25.5M 3s 2024-04-20T10:35:46.599 INFO:tasks.workunit.client.0.smithi042.stderr: 71150K .......... .......... .......... .......... .......... 52% 24.2M 3s 2024-04-20T10:35:46.600 INFO:tasks.workunit.client.0.smithi042.stderr: 71200K .......... .......... .......... .......... .......... 52% 31.1M 3s 2024-04-20T10:35:46.602 INFO:tasks.workunit.client.0.smithi042.stderr: 71250K .......... .......... .......... .......... .......... 52% 27.4M 3s 2024-04-20T10:35:46.604 INFO:tasks.workunit.client.0.smithi042.stderr: 71300K .......... .......... .......... .......... .......... 52% 37.5M 3s 2024-04-20T10:35:46.605 INFO:tasks.workunit.client.0.smithi042.stderr: 71350K .......... .......... .......... .......... .......... 52% 26.5M 3s 2024-04-20T10:35:46.607 INFO:tasks.workunit.client.0.smithi042.stderr: 71400K .......... .......... .......... .......... .......... 52% 24.3M 3s 2024-04-20T10:35:46.609 INFO:tasks.workunit.client.0.smithi042.stderr: 71450K .......... .......... .......... .......... .......... 52% 30.7M 3s 2024-04-20T10:35:46.611 INFO:tasks.workunit.client.0.smithi042.stderr: 71500K .......... .......... .......... .......... .......... 52% 26.0M 3s 2024-04-20T10:35:46.613 INFO:tasks.workunit.client.0.smithi042.stderr: 71550K .......... .......... .......... .......... .......... 52% 25.4M 3s 2024-04-20T10:35:46.613 INFO:tasks.workunit.client.0.smithi042.stderr: 71600K .......... .......... .......... .......... .......... 52% 42.7M 3s 2024-04-20T10:35:46.615 INFO:tasks.workunit.client.0.smithi042.stderr: 71650K .......... .......... .......... .......... .......... 52% 25.5M 3s 2024-04-20T10:35:46.617 INFO:tasks.workunit.client.0.smithi042.stderr: 71700K .......... .......... .......... .......... .......... 52% 33.2M 3s 2024-04-20T10:35:46.619 INFO:tasks.workunit.client.0.smithi042.stderr: 71750K .......... .......... .......... .......... .......... 52% 25.3M 3s 2024-04-20T10:35:46.621 INFO:tasks.workunit.client.0.smithi042.stderr: 71800K .......... .......... .......... .......... .......... 52% 23.8M 3s 2024-04-20T10:35:46.622 INFO:tasks.workunit.client.0.smithi042.stderr: 71850K .......... .......... .......... .......... .......... 52% 43.7M 3s 2024-04-20T10:35:46.624 INFO:tasks.workunit.client.0.smithi042.stderr: 71900K .......... .......... .......... .......... .......... 53% 25.1M 3s 2024-04-20T10:35:46.625 INFO:tasks.workunit.client.0.smithi042.stderr: 71950K .......... .......... .......... .......... .......... 53% 30.3M 3s 2024-04-20T10:35:46.628 INFO:tasks.workunit.client.0.smithi042.stderr: 72000K .......... .......... .......... .......... .......... 53% 26.5M 3s 2024-04-20T10:35:46.630 INFO:tasks.workunit.client.0.smithi042.stderr: 72050K .......... .......... .......... .......... .......... 53% 24.2M 3s 2024-04-20T10:35:46.631 INFO:tasks.workunit.client.0.smithi042.stderr: 72100K .......... .......... .......... .......... .......... 53% 41.8M 3s 2024-04-20T10:35:46.632 INFO:tasks.workunit.client.0.smithi042.stderr: 72150K .......... .......... .......... .......... .......... 53% 27.1M 3s 2024-04-20T10:35:46.634 INFO:tasks.workunit.client.0.smithi042.stderr: 72200K .......... .......... .......... .......... .......... 53% 34.2M 3s 2024-04-20T10:35:46.636 INFO:tasks.workunit.client.0.smithi042.stderr: 72250K .......... .......... .......... .......... .......... 53% 23.8M 3s 2024-04-20T10:35:46.638 INFO:tasks.workunit.client.0.smithi042.stderr: 72300K .......... .......... .......... .......... .......... 53% 25.4M 3s 2024-04-20T10:35:46.639 INFO:tasks.workunit.client.0.smithi042.stderr: 72350K .......... .......... .......... .......... .......... 53% 43.5M 3s 2024-04-20T10:35:46.641 INFO:tasks.workunit.client.0.smithi042.stderr: 72400K .......... .......... .......... .......... .......... 53% 21.9M 3s 2024-04-20T10:35:46.643 INFO:tasks.workunit.client.0.smithi042.stderr: 72450K .......... .......... .......... .......... .......... 53% 29.5M 3s 2024-04-20T10:35:46.644 INFO:tasks.workunit.client.0.smithi042.stderr: 72500K .......... .......... .......... .......... .......... 53% 32.7M 3s 2024-04-20T10:35:46.646 INFO:tasks.workunit.client.0.smithi042.stderr: 72550K .......... .......... .......... .......... .......... 53% 25.7M 3s 2024-04-20T10:35:46.648 INFO:tasks.workunit.client.0.smithi042.stderr: 72600K .......... .......... .......... .......... .......... 53% 33.7M 3s 2024-04-20T10:35:46.650 INFO:tasks.workunit.client.0.smithi042.stderr: 72650K .......... .......... .......... .......... .......... 53% 26.8M 3s 2024-04-20T10:35:46.651 INFO:tasks.workunit.client.0.smithi042.stderr: 72700K .......... .......... .......... .......... .......... 53% 31.7M 3s 2024-04-20T10:35:46.653 INFO:tasks.workunit.client.0.smithi042.stderr: 72750K .......... .......... .......... .......... .......... 53% 26.5M 3s 2024-04-20T10:35:46.654 INFO:tasks.workunit.client.0.smithi042.stderr: 72800K .......... .......... .......... .......... .......... 53% 31.9M 3s 2024-04-20T10:35:46.656 INFO:tasks.workunit.client.0.smithi042.stderr: 72850K .......... .......... .......... .......... .......... 53% 35.1M 3s 2024-04-20T10:35:46.658 INFO:tasks.workunit.client.0.smithi042.stderr: 72900K .......... .......... .......... .......... .......... 53% 23.8M 3s 2024-04-20T10:35:46.659 INFO:tasks.workunit.client.0.smithi042.stderr: 72950K .......... .......... .......... .......... .......... 53% 33.6M 3s 2024-04-20T10:35:46.661 INFO:tasks.workunit.client.0.smithi042.stderr: 73000K .......... .......... .......... .......... .......... 53% 23.7M 3s 2024-04-20T10:35:46.663 INFO:tasks.workunit.client.0.smithi042.stderr: 73050K .......... .......... .......... .......... .......... 53% 35.5M 3s 2024-04-20T10:35:46.664 INFO:tasks.workunit.client.0.smithi042.stderr: 73100K .......... .......... .......... .......... .......... 53% 28.9M 3s 2024-04-20T10:35:46.666 INFO:tasks.workunit.client.0.smithi042.stderr: 73150K .......... .......... .......... .......... .......... 53% 25.2M 3s 2024-04-20T10:35:46.668 INFO:tasks.workunit.client.0.smithi042.stderr: 73200K .......... .......... .......... .......... .......... 53% 24.6M 3s 2024-04-20T10:35:46.670 INFO:tasks.workunit.client.0.smithi042.stderr: 73250K .......... .......... .......... .......... .......... 53% 36.2M 3s 2024-04-20T10:35:46.672 INFO:tasks.workunit.client.0.smithi042.stderr: 73300K .......... .......... .......... .......... .......... 54% 26.3M 3s 2024-04-20T10:35:46.673 INFO:tasks.workunit.client.0.smithi042.stderr: 73350K .......... .......... .......... .......... .......... 54% 34.2M 3s 2024-04-20T10:35:46.675 INFO:tasks.workunit.client.0.smithi042.stderr: 73400K .......... .......... .......... .......... .......... 54% 24.7M 3s 2024-04-20T10:35:46.676 INFO:tasks.workunit.client.0.smithi042.stderr: 73450K .......... .......... .......... .......... .......... 54% 36.0M 3s 2024-04-20T10:35:46.678 INFO:tasks.workunit.client.0.smithi042.stderr: 73500K .......... .......... .......... .......... .......... 54% 25.6M 3s 2024-04-20T10:35:46.680 INFO:tasks.workunit.client.0.smithi042.stderr: 73550K .......... .......... .......... .......... .......... 54% 24.2M 3s 2024-04-20T10:35:46.682 INFO:tasks.workunit.client.0.smithi042.stderr: 73600K .......... .......... .......... .......... .......... 54% 38.9M 3s 2024-04-20T10:35:46.683 INFO:tasks.workunit.client.0.smithi042.stderr: 73650K .......... .......... .......... .......... .......... 54% 26.2M 3s 2024-04-20T10:35:46.685 INFO:tasks.workunit.client.0.smithi042.stderr: 73700K .......... .......... .......... .......... .......... 54% 33.3M 3s 2024-04-20T10:35:46.687 INFO:tasks.workunit.client.0.smithi042.stderr: 73750K .......... .......... .......... .......... .......... 54% 23.6M 3s 2024-04-20T10:35:46.688 INFO:tasks.workunit.client.0.smithi042.stderr: 73800K .......... .......... .......... .......... .......... 54% 26.8M 3s 2024-04-20T10:35:46.690 INFO:tasks.workunit.client.0.smithi042.stderr: 73850K .......... .......... .......... .......... .......... 54% 38.2M 3s 2024-04-20T10:35:46.692 INFO:tasks.workunit.client.0.smithi042.stderr: 73900K .......... .......... .......... .......... .......... 54% 22.4M 3s 2024-04-20T10:35:46.693 INFO:tasks.workunit.client.0.smithi042.stderr: 73950K .......... .......... .......... .......... .......... 54% 37.7M 3s 2024-04-20T10:35:46.696 INFO:tasks.workunit.client.0.smithi042.stderr: 74000K .......... .......... .......... .......... .......... 54% 23.1M 3s 2024-04-20T10:35:46.697 INFO:tasks.workunit.client.0.smithi042.stderr: 74050K .......... .......... .......... .......... .......... 54% 27.3M 3s 2024-04-20T10:35:46.699 INFO:tasks.workunit.client.0.smithi042.stderr: 74100K .......... .......... .......... .......... .......... 54% 33.0M 3s 2024-04-20T10:35:46.701 INFO:tasks.workunit.client.0.smithi042.stderr: 74150K .......... .......... .......... .......... .......... 54% 23.5M 3s 2024-04-20T10:35:46.703 INFO:tasks.workunit.client.0.smithi042.stderr: 74200K .......... .......... .......... .......... .......... 54% 23.1M 3s 2024-04-20T10:35:46.705 INFO:tasks.workunit.client.0.smithi042.stderr: 74250K .......... .......... .......... .......... .......... 54% 28.7M 3s 2024-04-20T10:35:46.707 INFO:tasks.workunit.client.0.smithi042.stderr: 74300K .......... .......... .......... .......... .......... 54% 32.3M 3s 2024-04-20T10:35:46.707 INFO:tasks.workunit.client.0.smithi042.stderr: 74350K .......... .......... .......... .......... .......... 54% 39.9M 3s 2024-04-20T10:35:46.709 INFO:tasks.workunit.client.0.smithi042.stderr: 74400K .......... .......... .......... .......... .......... 54% 24.9M 3s 2024-04-20T10:35:46.711 INFO:tasks.workunit.client.0.smithi042.stderr: 74450K .......... .......... .......... .......... .......... 54% 30.5M 3s 2024-04-20T10:35:46.713 INFO:tasks.workunit.client.0.smithi042.stderr: 74500K .......... .......... .......... .......... .......... 54% 25.8M 3s 2024-04-20T10:35:46.714 INFO:tasks.workunit.client.0.smithi042.stderr: 74550K .......... .......... .......... .......... .......... 54% 40.1M 3s 2024-04-20T10:35:46.716 INFO:tasks.workunit.client.0.smithi042.stderr: 74600K .......... .......... .......... .......... .......... 54% 23.7M 3s 2024-04-20T10:35:46.718 INFO:tasks.workunit.client.0.smithi042.stderr: 74650K .......... .......... .......... .......... .......... 55% 24.0M 3s 2024-04-20T10:35:46.719 INFO:tasks.workunit.client.0.smithi042.stderr: 74700K .......... .......... .......... .......... .......... 55% 32.7M 3s 2024-04-20T10:35:46.722 INFO:tasks.workunit.client.0.smithi042.stderr: 74750K .......... .......... .......... .......... .......... 55% 26.2M 3s 2024-04-20T10:35:46.724 INFO:tasks.workunit.client.0.smithi042.stderr: 74800K .......... .......... .......... .......... .......... 55% 23.5M 3s 2024-04-20T10:35:46.724 INFO:tasks.workunit.client.0.smithi042.stderr: 74850K .......... .......... .......... .......... .......... 55% 43.0M 3s 2024-04-20T10:35:46.726 INFO:tasks.workunit.client.0.smithi042.stderr: 74900K .......... .......... .......... .......... .......... 55% 27.6M 3s 2024-04-20T10:35:46.728 INFO:tasks.workunit.client.0.smithi042.stderr: 74950K .......... .......... .......... .......... .......... 55% 29.7M 3s 2024-04-20T10:35:46.730 INFO:tasks.workunit.client.0.smithi042.stderr: 75000K .......... .......... .......... .......... .......... 55% 22.2M 3s 2024-04-20T10:35:46.733 INFO:tasks.workunit.client.0.smithi042.stderr: 75050K .......... .......... .......... .......... .......... 55% 23.2M 3s 2024-04-20T10:35:46.733 INFO:tasks.workunit.client.0.smithi042.stderr: 75100K .......... .......... .......... .......... .......... 55% 44.0M 3s 2024-04-20T10:35:46.736 INFO:tasks.workunit.client.0.smithi042.stderr: 75150K .......... .......... .......... .......... .......... 55% 21.1M 3s 2024-04-20T10:35:46.738 INFO:tasks.workunit.client.0.smithi042.stderr: 75200K .......... .......... .......... .......... .......... 55% 22.5M 3s 2024-04-20T10:35:46.740 INFO:tasks.workunit.client.0.smithi042.stderr: 75250K .......... .......... .......... .......... .......... 55% 31.3M 3s 2024-04-20T10:35:46.742 INFO:tasks.workunit.client.0.smithi042.stderr: 75300K .......... .......... .......... .......... .......... 55% 20.6M 3s 2024-04-20T10:35:46.743 INFO:tasks.workunit.client.0.smithi042.stderr: 75350K .......... .......... .......... .......... .......... 55% 40.0M 3s 2024-04-20T10:35:46.745 INFO:tasks.workunit.client.0.smithi042.stderr: 75400K .......... .......... .......... .......... .......... 55% 22.9M 3s 2024-04-20T10:35:46.747 INFO:tasks.workunit.client.0.smithi042.stderr: 75450K .......... .......... .......... .......... .......... 55% 25.1M 3s 2024-04-20T10:35:46.749 INFO:tasks.workunit.client.0.smithi042.stderr: 75500K .......... .......... .......... .......... .......... 55% 34.3M 3s 2024-04-20T10:35:46.751 INFO:tasks.workunit.client.0.smithi042.stderr: 75550K .......... .......... .......... .......... .......... 55% 23.8M 3s 2024-04-20T10:35:46.752 INFO:tasks.workunit.client.0.smithi042.stderr: 75600K .......... .......... .......... .......... .......... 55% 45.6M 3s 2024-04-20T10:35:46.754 INFO:tasks.workunit.client.0.smithi042.stderr: 75650K .......... .......... .......... .......... .......... 55% 24.9M 3s 2024-04-20T10:35:46.756 INFO:tasks.workunit.client.0.smithi042.stderr: 75700K .......... .......... .......... .......... .......... 55% 25.6M 3s 2024-04-20T10:35:46.757 INFO:tasks.workunit.client.0.smithi042.stderr: 75750K .......... .......... .......... .......... .......... 55% 32.9M 3s 2024-04-20T10:35:46.759 INFO:tasks.workunit.client.0.smithi042.stderr: 75800K .......... .......... .......... .......... .......... 55% 24.4M 3s 2024-04-20T10:35:46.760 INFO:tasks.workunit.client.0.smithi042.stderr: 75850K .......... .......... .......... .......... .......... 55% 45.2M 3s 2024-04-20T10:35:46.762 INFO:tasks.workunit.client.0.smithi042.stderr: 75900K .......... .......... .......... .......... .......... 55% 22.2M 3s 2024-04-20T10:35:46.765 INFO:tasks.workunit.client.0.smithi042.stderr: 75950K .......... .......... .......... .......... .......... 55% 21.9M 3s 2024-04-20T10:35:46.766 INFO:tasks.workunit.client.0.smithi042.stderr: 76000K .......... .......... .......... .......... .......... 56% 33.0M 3s 2024-04-20T10:35:46.768 INFO:tasks.workunit.client.0.smithi042.stderr: 76050K .......... .......... .......... .......... .......... 56% 23.2M 3s 2024-04-20T10:35:46.769 INFO:tasks.workunit.client.0.smithi042.stderr: 76100K .......... .......... .......... .......... .......... 56% 44.1M 3s 2024-04-20T10:35:46.771 INFO:tasks.workunit.client.0.smithi042.stderr: 76150K .......... .......... .......... .......... .......... 56% 26.1M 3s 2024-04-20T10:35:46.773 INFO:tasks.workunit.client.0.smithi042.stderr: 76200K .......... .......... .......... .......... .......... 56% 24.2M 3s 2024-04-20T10:35:46.780 INFO:tasks.workunit.client.0.smithi042.stderr: 76250K .......... .......... .......... .......... .......... 56% 35.5M 3s 2024-04-20T10:35:46.780 INFO:tasks.workunit.client.0.smithi042.stderr: 76300K .......... .......... .......... .......... .......... 56% 24.6M 3s 2024-04-20T10:35:46.780 INFO:tasks.workunit.client.0.smithi042.stderr: 76350K .......... .......... .......... .......... .......... 56% 43.7M 3s 2024-04-20T10:35:46.780 INFO:tasks.workunit.client.0.smithi042.stderr: 76400K .......... .......... .......... .......... .......... 56% 24.4M 3s 2024-04-20T10:35:46.782 INFO:tasks.workunit.client.0.smithi042.stderr: 76450K .......... .......... .......... .......... .......... 56% 26.6M 3s 2024-04-20T10:35:46.783 INFO:tasks.workunit.client.0.smithi042.stderr: 76500K .......... .......... .......... .......... .......... 56% 29.0M 3s 2024-04-20T10:35:46.785 INFO:tasks.workunit.client.0.smithi042.stderr: 76550K .......... .......... .......... .......... .......... 56% 25.5M 3s 2024-04-20T10:35:46.787 INFO:tasks.workunit.client.0.smithi042.stderr: 76600K .......... .......... .......... .......... .......... 56% 42.8M 3s 2024-04-20T10:35:46.788 INFO:tasks.workunit.client.0.smithi042.stderr: 76650K .......... .......... .......... .......... .......... 56% 27.5M 3s 2024-04-20T10:35:46.790 INFO:tasks.workunit.client.0.smithi042.stderr: 76700K .......... .......... .......... .......... .......... 56% 24.9M 3s 2024-04-20T10:35:46.792 INFO:tasks.workunit.client.0.smithi042.stderr: 76750K .......... .......... .......... .......... .......... 56% 31.5M 3s 2024-04-20T10:35:46.794 INFO:tasks.workunit.client.0.smithi042.stderr: 76800K .......... .......... .......... .......... .......... 56% 26.1M 3s 2024-04-20T10:35:46.796 INFO:tasks.workunit.client.0.smithi042.stderr: 76850K .......... .......... .......... .......... .......... 56% 24.4M 3s 2024-04-20T10:35:46.796 INFO:tasks.workunit.client.0.smithi042.stderr: 76900K .......... .......... .......... .......... .......... 56% 41.4M 3s 2024-04-20T10:35:46.799 INFO:tasks.workunit.client.0.smithi042.stderr: 76950K .......... .......... .......... .......... .......... 56% 26.2M 3s 2024-04-20T10:35:46.800 INFO:tasks.workunit.client.0.smithi042.stderr: 77000K .......... .......... .......... .......... .......... 56% 33.4M 3s 2024-04-20T10:35:46.802 INFO:tasks.workunit.client.0.smithi042.stderr: 77050K .......... .......... .......... .......... .......... 56% 23.3M 3s 2024-04-20T10:35:46.804 INFO:tasks.workunit.client.0.smithi042.stderr: 77100K .......... .......... .......... .......... .......... 56% 25.0M 3s 2024-04-20T10:35:46.805 INFO:tasks.workunit.client.0.smithi042.stderr: 77150K .......... .......... .......... .......... .......... 56% 45.1M 3s 2024-04-20T10:35:46.807 INFO:tasks.workunit.client.0.smithi042.stderr: 77200K .......... .......... .......... .......... .......... 56% 22.3M 3s 2024-04-20T10:35:46.809 INFO:tasks.workunit.client.0.smithi042.stderr: 77250K .......... .......... .......... .......... .......... 56% 32.9M 3s 2024-04-20T10:35:46.811 INFO:tasks.workunit.client.0.smithi042.stderr: 77300K .......... .......... .......... .......... .......... 56% 25.1M 3s 2024-04-20T10:35:46.813 INFO:tasks.workunit.client.0.smithi042.stderr: 77350K .......... .......... .......... .......... .......... 57% 24.1M 3s 2024-04-20T10:35:46.814 INFO:tasks.workunit.client.0.smithi042.stderr: 77400K .......... .......... .......... .......... .......... 57% 41.8M 3s 2024-04-20T10:35:46.816 INFO:tasks.workunit.client.0.smithi042.stderr: 77450K .......... .......... .......... .......... .......... 57% 24.7M 3s 2024-04-20T10:35:46.817 INFO:tasks.workunit.client.0.smithi042.stderr: 77500K .......... .......... .......... .......... .......... 57% 31.8M 3s 2024-04-20T10:35:46.819 INFO:tasks.workunit.client.0.smithi042.stderr: 77550K .......... .......... .......... .......... .......... 57% 25.2M 3s 2024-04-20T10:35:46.821 INFO:tasks.workunit.client.0.smithi042.stderr: 77600K .......... .......... .......... .......... .......... 57% 28.0M 3s 2024-04-20T10:35:46.823 INFO:tasks.workunit.client.0.smithi042.stderr: 77650K .......... .......... .......... .......... .......... 57% 35.1M 3s 2024-04-20T10:35:46.825 INFO:tasks.workunit.client.0.smithi042.stderr: 77700K .......... .......... .......... .......... .......... 57% 25.2M 3s 2024-04-20T10:35:46.827 INFO:tasks.workunit.client.0.smithi042.stderr: 77750K .......... .......... .......... .......... .......... 57% 31.1M 3s 2024-04-20T10:35:46.830 INFO:tasks.workunit.client.0.smithi042.stderr: 77800K .......... .......... .......... .......... .......... 57% 26.1M 3s 2024-04-20T10:35:46.830 INFO:tasks.workunit.client.0.smithi042.stderr: 77850K .......... .......... .......... .......... .......... 57% 27.3M 3s 2024-04-20T10:35:46.836 INFO:tasks.workunit.client.0.smithi042.stderr: 77900K .......... .......... .......... .......... .......... 57% 34.4M 3s 2024-04-20T10:35:46.836 INFO:tasks.workunit.client.0.smithi042.stderr: 77950K .......... .......... .......... .......... .......... 57% 34.1M 3s 2024-04-20T10:35:46.836 INFO:tasks.workunit.client.0.smithi042.stderr: 78000K .......... .......... .......... .......... .......... 57% 25.4M 3s 2024-04-20T10:35:46.851 INFO:tasks.workunit.client.0.smithi042.stderr: 78050K .......... .......... .......... .......... .......... 57% 27.3M 3s 2024-04-20T10:35:46.851 INFO:tasks.workunit.client.0.smithi042.stderr: 78100K .......... .......... .......... .......... .......... 57% 24.0M 3s 2024-04-20T10:35:46.851 INFO:tasks.workunit.client.0.smithi042.stderr: 78150K .......... .......... .......... .......... .......... 57% 40.3M 3s 2024-04-20T10:35:46.851 INFO:tasks.workunit.client.0.smithi042.stderr: 78200K .......... .......... .......... .......... .......... 57% 24.3M 3s 2024-04-20T10:35:46.851 INFO:tasks.workunit.client.0.smithi042.stderr: 78250K .......... .......... .......... .......... .......... 57% 37.6M 3s 2024-04-20T10:35:46.851 INFO:tasks.workunit.client.0.smithi042.stderr: 78300K .......... .......... .......... .......... .......... 57% 24.0M 3s 2024-04-20T10:35:46.852 INFO:tasks.workunit.client.0.smithi042.stderr: 78350K .......... .......... .......... .......... .......... 57% 26.5M 3s 2024-04-20T10:35:46.852 INFO:tasks.workunit.client.0.smithi042.stderr: 78400K .......... .......... .......... .......... .......... 57% 42.0M 3s 2024-04-20T10:35:46.852 INFO:tasks.workunit.client.0.smithi042.stderr: 78450K .......... .......... .......... .......... .......... 57% 28.9M 3s 2024-04-20T10:35:46.853 INFO:tasks.workunit.client.0.smithi042.stderr: 78500K .......... .......... .......... .......... .......... 57% 26.4M 3s 2024-04-20T10:35:46.854 INFO:tasks.workunit.client.0.smithi042.stderr: 78550K .......... .......... .......... .......... .......... 57% 25.2M 3s 2024-04-20T10:35:46.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:46 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:35:46.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:46 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:35:46.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:46 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:35:46.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:46 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:35:46.857 INFO:tasks.workunit.client.0.smithi042.stderr: 78600K .......... .......... .......... .......... .......... 57% 27.7M 3s 2024-04-20T10:35:46.857 INFO:tasks.workunit.client.0.smithi042.stderr: 78650K .......... .......... .......... .......... .......... 57% 44.3M 3s 2024-04-20T10:35:46.859 INFO:tasks.workunit.client.0.smithi042.stderr: 78700K .......... .......... .......... .......... .......... 58% 20.9M 3s 2024-04-20T10:35:46.860 INFO:tasks.workunit.client.0.smithi042.stderr: 78750K .......... .......... .......... .......... .......... 58% 36.9M 3s 2024-04-20T10:35:46.863 INFO:tasks.workunit.client.0.smithi042.stderr: 78800K .......... .......... .......... .......... .......... 58% 23.8M 3s 2024-04-20T10:35:46.865 INFO:tasks.workunit.client.0.smithi042.stderr: 78850K .......... .......... .......... .......... .......... 58% 27.6M 3s 2024-04-20T10:35:46.865 INFO:tasks.workunit.client.0.smithi042.stderr: 78900K .......... .......... .......... .......... .......... 58% 41.3M 2s 2024-04-20T10:35:46.867 INFO:tasks.workunit.client.0.smithi042.stderr: 78950K .......... .......... .......... .......... .......... 58% 24.4M 2s 2024-04-20T10:35:46.868 INFO:tasks.workunit.client.0.smithi042.stderr: 79000K .......... .......... .......... .......... .......... 58% 29.3M 2s 2024-04-20T10:35:46.870 INFO:tasks.workunit.client.0.smithi042.stderr: 79050K .......... .......... .......... .......... .......... 58% 29.0M 2s 2024-04-20T10:35:46.872 INFO:tasks.workunit.client.0.smithi042.stderr: 79100K .......... .......... .......... .......... .......... 58% 24.5M 2s 2024-04-20T10:35:46.873 INFO:tasks.workunit.client.0.smithi042.stderr: 79150K .......... .......... .......... .......... .......... 58% 35.5M 2s 2024-04-20T10:35:46.876 INFO:tasks.workunit.client.0.smithi042.stderr: 79200K .......... .......... .......... .......... .......... 58% 25.1M 2s 2024-04-20T10:35:46.885 INFO:tasks.workunit.client.0.smithi042.stderr: 79250K .......... .......... .......... .......... .......... 58% 28.9M 2s 2024-04-20T10:35:46.885 INFO:tasks.workunit.client.0.smithi042.stderr: 79300K .......... .......... .......... .......... .......... 58% 28.4M 2s 2024-04-20T10:35:46.885 INFO:tasks.workunit.client.0.smithi042.stderr: 79350K .......... .......... .......... .......... .......... 58% 24.9M 2s 2024-04-20T10:35:46.885 INFO:tasks.workunit.client.0.smithi042.stderr: 79400K .......... .......... .......... .......... .......... 58% 45.0M 2s 2024-04-20T10:35:46.885 INFO:tasks.workunit.client.0.smithi042.stderr: 79450K .......... .......... .......... .......... .......... 58% 23.2M 2s 2024-04-20T10:35:46.905 INFO:tasks.workunit.client.0.smithi042.stderr: 79500K .......... .......... .......... .......... .......... 58% 33.0M 2s 2024-04-20T10:35:46.905 INFO:tasks.workunit.client.0.smithi042.stderr: 79550K .......... .......... .......... .......... .......... 58% 22.3M 2s 2024-04-20T10:35:46.905 INFO:tasks.workunit.client.0.smithi042.stderr: 79600K .......... .......... .......... .......... .......... 58% 25.3M 2s 2024-04-20T10:35:46.905 INFO:tasks.workunit.client.0.smithi042.stderr: 79650K .......... .......... .......... .......... .......... 58% 45.1M 2s 2024-04-20T10:35:46.905 INFO:tasks.workunit.client.0.smithi042.stderr: 79700K .......... .......... .......... .......... .......... 58% 23.3M 2s 2024-04-20T10:35:46.905 INFO:tasks.workunit.client.0.smithi042.stderr: 79750K .......... .......... .......... .......... .......... 58% 30.8M 2s 2024-04-20T10:35:46.905 INFO:tasks.workunit.client.0.smithi042.stderr: 79800K .......... .......... .......... .......... .......... 58% 24.9M 2s 2024-04-20T10:35:46.905 INFO:tasks.workunit.client.0.smithi042.stderr: 79850K .......... .......... .......... .......... .......... 58% 28.9M 2s 2024-04-20T10:35:46.906 INFO:tasks.workunit.client.0.smithi042.stderr: 79900K .......... .......... .......... .......... .......... 58% 39.4M 2s 2024-04-20T10:35:46.906 INFO:tasks.workunit.client.0.smithi042.stderr: 79950K .......... .......... .......... .......... .......... 58% 17.1M 2s 2024-04-20T10:35:46.906 INFO:tasks.workunit.client.0.smithi042.stderr: 80000K .......... .......... .......... .......... .......... 58% 38.7M 2s 2024-04-20T10:35:46.907 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:46 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:35:46.907 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:46 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:35:46.907 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:46 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:35:46.907 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:46 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:35:46.912 INFO:tasks.workunit.client.0.smithi042.stderr: 80050K .......... .......... .......... .......... .......... 59% 23.2M 2s 2024-04-20T10:35:46.912 INFO:tasks.workunit.client.0.smithi042.stderr: 80100K .......... .......... .......... .......... .......... 59% 23.2M 2s 2024-04-20T10:35:46.912 INFO:tasks.workunit.client.0.smithi042.stderr: 80150K .......... .......... .......... .......... .......... 59% 36.9M 2s 2024-04-20T10:35:46.912 INFO:tasks.workunit.client.0.smithi042.stderr: 80200K .......... .......... .......... .......... .......... 59% 23.0M 2s 2024-04-20T10:35:46.913 INFO:tasks.workunit.client.0.smithi042.stderr: 80250K .......... .......... .......... .......... .......... 59% 30.5M 2s 2024-04-20T10:35:46.923 INFO:tasks.workunit.client.0.smithi042.stderr: 80300K .......... .......... .......... .......... .......... 59% 24.9M 2s 2024-04-20T10:35:46.923 INFO:tasks.workunit.client.0.smithi042.stderr: 80350K .......... .......... .......... .......... .......... 59% 21.6M 2s 2024-04-20T10:35:46.923 INFO:tasks.workunit.client.0.smithi042.stderr: 80400K .......... .......... .......... .......... .......... 59% 32.0M 2s 2024-04-20T10:35:46.923 INFO:tasks.workunit.client.0.smithi042.stderr: 80450K .......... .......... .......... .......... .......... 59% 27.0M 2s 2024-04-20T10:35:46.923 INFO:tasks.workunit.client.0.smithi042.stderr: 80500K .......... .......... .......... .......... .......... 59% 32.4M 2s 2024-04-20T10:35:46.927 INFO:tasks.workunit.client.0.smithi042.stderr: 80550K .......... .......... .......... .......... .......... 59% 26.1M 2s 2024-04-20T10:35:46.927 INFO:tasks.workunit.client.0.smithi042.stderr: 80600K .......... .......... .......... .......... .......... 59% 23.1M 2s 2024-04-20T10:35:46.927 INFO:tasks.workunit.client.0.smithi042.stderr: 80650K .......... .......... .......... .......... .......... 59% 48.1M 2s 2024-04-20T10:35:46.929 INFO:tasks.workunit.client.0.smithi042.stderr: 80700K .......... .......... .......... .......... .......... 59% 25.6M 2s 2024-04-20T10:35:46.931 INFO:tasks.workunit.client.0.smithi042.stderr: 80750K .......... .......... .......... .......... .......... 59% 25.1M 2s 2024-04-20T10:35:46.932 INFO:tasks.workunit.client.0.smithi042.stderr: 80800K .......... .......... .......... .......... .......... 59% 31.8M 2s 2024-04-20T10:35:46.934 INFO:tasks.workunit.client.0.smithi042.stderr: 80850K .......... .......... .......... .......... .......... 59% 25.0M 2s 2024-04-20T10:35:46.935 INFO:tasks.workunit.client.0.smithi042.stderr: 80900K .......... .......... .......... .......... .......... 59% 41.3M 2s 2024-04-20T10:35:46.938 INFO:tasks.workunit.client.0.smithi042.stderr: 80950K .......... .......... .......... .......... .......... 59% 21.5M 2s 2024-04-20T10:35:46.940 INFO:tasks.workunit.client.0.smithi042.stderr: 81000K .......... .......... .......... .......... .......... 59% 23.6M 2s 2024-04-20T10:35:46.941 INFO:tasks.workunit.client.0.smithi042.stderr: 81050K .......... .......... .......... .......... .......... 59% 30.6M 2s 2024-04-20T10:35:46.943 INFO:tasks.workunit.client.0.smithi042.stderr: 81100K .......... .......... .......... .......... .......... 59% 24.6M 2s 2024-04-20T10:35:46.945 INFO:tasks.workunit.client.0.smithi042.stderr: 81150K .......... .......... .......... .......... .......... 59% 44.6M 2s 2024-04-20T10:35:46.946 INFO:tasks.workunit.client.0.smithi042.stderr: 81200K .......... .......... .......... .......... .......... 59% 26.9M 2s 2024-04-20T10:35:46.948 INFO:tasks.workunit.client.0.smithi042.stderr: 81250K .......... .......... .......... .......... .......... 59% 23.0M 2s 2024-04-20T10:35:46.950 INFO:tasks.workunit.client.0.smithi042.stderr: 81300K .......... .......... .......... .......... .......... 59% 33.9M 2s 2024-04-20T10:35:46.952 INFO:tasks.workunit.client.0.smithi042.stderr: 81350K .......... .......... .......... .......... .......... 59% 23.7M 2s 2024-04-20T10:35:46.953 INFO:tasks.workunit.client.0.smithi042.stderr: 81400K .......... .......... .......... .......... .......... 59% 46.6M 2s 2024-04-20T10:35:46.955 INFO:tasks.workunit.client.0.smithi042.stderr: 81450K .......... .......... .......... .......... .......... 60% 24.9M 2s 2024-04-20T10:35:46.957 INFO:tasks.workunit.client.0.smithi042.stderr: 81500K .......... .......... .......... .......... .......... 60% 25.3M 2s 2024-04-20T10:35:46.959 INFO:tasks.workunit.client.0.smithi042.stderr: 81550K .......... .......... .......... .......... .......... 60% 30.5M 2s 2024-04-20T10:35:46.960 INFO:tasks.workunit.client.0.smithi042.stderr: 81600K .......... .......... .......... .......... .......... 60% 25.1M 2s 2024-04-20T10:35:46.962 INFO:tasks.workunit.client.0.smithi042.stderr: 81650K .......... .......... .......... .......... .......... 60% 46.5M 2s 2024-04-20T10:35:46.963 INFO:tasks.workunit.client.0.smithi042.stderr: 81700K .......... .......... .......... .......... .......... 60% 24.7M 2s 2024-04-20T10:35:46.965 INFO:tasks.workunit.client.0.smithi042.stderr: 81750K .......... .......... .......... .......... .......... 60% 24.5M 2s 2024-04-20T10:35:46.967 INFO:tasks.workunit.client.0.smithi042.stderr: 81800K .......... .......... .......... .......... .......... 60% 32.7M 2s 2024-04-20T10:35:46.969 INFO:tasks.workunit.client.0.smithi042.stderr: 81850K .......... .......... .......... .......... .......... 60% 25.5M 2s 2024-04-20T10:35:46.971 INFO:tasks.workunit.client.0.smithi042.stderr: 81900K .......... .......... .......... .......... .......... 60% 46.7M 2s 2024-04-20T10:35:46.972 INFO:tasks.workunit.client.0.smithi042.stderr: 81950K .......... .......... .......... .......... .......... 60% 23.5M 2s 2024-04-20T10:35:46.974 INFO:tasks.workunit.client.0.smithi042.stderr: 82000K .......... .......... .......... .......... .......... 60% 26.9M 2s 2024-04-20T10:35:46.975 INFO:tasks.workunit.client.0.smithi042.stderr: 82050K .......... .......... .......... .......... .......... 60% 29.5M 2s 2024-04-20T10:35:46.977 INFO:tasks.workunit.client.0.smithi042.stderr: 82100K .......... .......... .......... .......... .......... 60% 33.6M 2s 2024-04-20T10:35:46.979 INFO:tasks.workunit.client.0.smithi042.stderr: 82150K .......... .......... .......... .......... .......... 60% 19.6M 2s 2024-04-20T10:35:46.985 INFO:tasks.workunit.client.0.smithi042.stderr: 82200K .......... .......... .......... .......... .......... 60% 41.8M 2s 2024-04-20T10:35:46.985 INFO:tasks.workunit.client.0.smithi042.stderr: 82250K .......... .......... .......... .......... .......... 60% 26.7M 2s 2024-04-20T10:35:46.985 INFO:tasks.workunit.client.0.smithi042.stderr: 82300K .......... .......... .......... .......... .......... 60% 33.0M 2s 2024-04-20T10:35:46.989 INFO:tasks.workunit.client.0.smithi042.stderr: 82350K .......... .......... .......... .......... .......... 60% 23.4M 2s 2024-04-20T10:35:46.989 INFO:tasks.workunit.client.0.smithi042.stderr: 82400K .......... .......... .......... .......... .......... 60% 25.8M 2s 2024-04-20T10:35:46.989 INFO:tasks.workunit.client.0.smithi042.stderr: 82450K .......... .......... .......... .......... .......... 60% 40.4M 2s 2024-04-20T10:35:46.992 INFO:tasks.workunit.client.0.smithi042.stderr: 82500K .......... .......... .......... .......... .......... 60% 25.4M 2s 2024-04-20T10:35:46.992 INFO:tasks.workunit.client.0.smithi042.stderr: 82550K .......... .......... .......... .......... .......... 60% 32.6M 2s 2024-04-20T10:35:46.994 INFO:tasks.workunit.client.0.smithi042.stderr: 82600K .......... .......... .......... .......... .......... 60% 25.2M 2s 2024-04-20T10:35:46.996 INFO:tasks.workunit.client.0.smithi042.stderr: 82650K .......... .......... .......... .......... .......... 60% 26.3M 2s 2024-04-20T10:35:46.997 INFO:tasks.workunit.client.0.smithi042.stderr: 82700K .......... .......... .......... .......... .......... 60% 38.6M 2s 2024-04-20T10:35:46.999 INFO:tasks.workunit.client.0.smithi042.stderr: 82750K .......... .......... .......... .......... .......... 60% 27.0M 2s 2024-04-20T10:35:47.000 INFO:tasks.workunit.client.0.smithi042.stderr: 82800K .......... .......... .......... .......... .......... 61% 34.5M 2s 2024-04-20T10:35:47.003 INFO:tasks.workunit.client.0.smithi042.stderr: 82850K .......... .......... .......... .......... .......... 61% 26.7M 2s 2024-04-20T10:35:47.005 INFO:tasks.workunit.client.0.smithi042.stderr: 82900K .......... .......... .......... .......... .......... 61% 23.1M 2s 2024-04-20T10:35:47.005 INFO:tasks.workunit.client.0.smithi042.stderr: 82950K .......... .......... .......... .......... .......... 61% 42.0M 2s 2024-04-20T10:35:47.008 INFO:tasks.workunit.client.0.smithi042.stderr: 83000K .......... .......... .......... .......... .......... 61% 25.7M 2s 2024-04-20T10:35:47.009 INFO:tasks.workunit.client.0.smithi042.stderr: 83050K .......... .......... .......... .......... .......... 61% 34.4M 2s 2024-04-20T10:35:47.011 INFO:tasks.workunit.client.0.smithi042.stderr: 83100K .......... .......... .......... .......... .......... 61% 25.3M 2s 2024-04-20T10:35:47.013 INFO:tasks.workunit.client.0.smithi042.stderr: 83150K .......... .......... .......... .......... .......... 61% 24.3M 2s 2024-04-20T10:35:47.014 INFO:tasks.workunit.client.0.smithi042.stderr: 83200K .......... .......... .......... .......... .......... 61% 39.7M 2s 2024-04-20T10:35:47.016 INFO:tasks.workunit.client.0.smithi042.stderr: 83250K .......... .......... .......... .......... .......... 61% 23.3M 2s 2024-04-20T10:35:47.020 INFO:tasks.workunit.client.0.smithi042.stderr: 83300K .......... .......... .......... .......... .......... 61% 37.5M 2s 2024-04-20T10:35:47.020 INFO:tasks.workunit.client.0.smithi042.stderr: 83350K .......... .......... .......... .......... .......... 61% 27.9M 2s 2024-04-20T10:35:47.022 INFO:tasks.workunit.client.0.smithi042.stderr: 83400K .......... .......... .......... .......... .......... 61% 23.1M 2s 2024-04-20T10:35:47.023 INFO:tasks.workunit.client.0.smithi042.stderr: 83450K .......... .......... .......... .......... .......... 61% 39.7M 2s 2024-04-20T10:35:47.024 INFO:tasks.workunit.client.0.smithi042.stderr: 83500K .......... .......... .......... .......... .......... 61% 24.6M 2s 2024-04-20T10:35:47.026 INFO:tasks.workunit.client.0.smithi042.stderr: 83550K .......... .......... .......... .......... .......... 61% 35.9M 2s 2024-04-20T10:35:47.028 INFO:tasks.workunit.client.0.smithi042.stderr: 83600K .......... .......... .......... .......... .......... 61% 24.2M 2s 2024-04-20T10:35:47.030 INFO:tasks.workunit.client.0.smithi042.stderr: 83650K .......... .......... .......... .......... .......... 61% 27.6M 2s 2024-04-20T10:35:47.031 INFO:tasks.workunit.client.0.smithi042.stderr: 83700K .......... .......... .......... .......... .......... 61% 36.0M 2s 2024-04-20T10:35:47.033 INFO:tasks.workunit.client.0.smithi042.stderr: 83750K .......... .......... .......... .......... .......... 61% 23.3M 2s 2024-04-20T10:35:47.034 INFO:tasks.workunit.client.0.smithi042.stderr: 83800K .......... .......... .......... .......... .......... 61% 35.7M 2s 2024-04-20T10:35:47.036 INFO:tasks.workunit.client.0.smithi042.stderr: 83850K .......... .......... .......... .......... .......... 61% 25.8M 2s 2024-04-20T10:35:47.039 INFO:tasks.workunit.client.0.smithi042.stderr: 83900K .......... .......... .......... .......... .......... 61% 24.5M 2s 2024-04-20T10:35:47.040 INFO:tasks.workunit.client.0.smithi042.stderr: 83950K .......... .......... .......... .......... .......... 61% 38.8M 2s 2024-04-20T10:35:47.042 INFO:tasks.workunit.client.0.smithi042.stderr: 84000K .......... .......... .......... .......... .......... 61% 24.6M 2s 2024-04-20T10:35:47.043 INFO:tasks.workunit.client.0.smithi042.stderr: 84050K .......... .......... .......... .......... .......... 61% 30.5M 2s 2024-04-20T10:35:47.046 INFO:tasks.workunit.client.0.smithi042.stderr: 84100K .......... .......... .......... .......... .......... 61% 22.9M 2s 2024-04-20T10:35:47.048 INFO:tasks.workunit.client.0.smithi042.stderr: 84150K .......... .......... .......... .......... .......... 62% 26.1M 2s 2024-04-20T10:35:47.048 INFO:tasks.workunit.client.0.smithi042.stderr: 84200K .......... .......... .......... .......... .......... 62% 41.8M 2s 2024-04-20T10:35:47.051 INFO:tasks.workunit.client.0.smithi042.stderr: 84250K .......... .......... .......... .......... .......... 62% 23.7M 2s 2024-04-20T10:35:47.052 INFO:tasks.workunit.client.0.smithi042.stderr: 84300K .......... .......... .......... .......... .......... 62% 35.0M 2s 2024-04-20T10:35:47.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:46 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:35:47.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:46 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:35:47.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:46 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:35:47.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:46 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:35:47.054 INFO:tasks.workunit.client.0.smithi042.stderr: 84350K .......... .......... .......... .......... .......... 62% 24.8M 2s 2024-04-20T10:35:47.056 INFO:tasks.workunit.client.0.smithi042.stderr: 84400K .......... .......... .......... .......... .......... 62% 26.3M 2s 2024-04-20T10:35:47.057 INFO:tasks.workunit.client.0.smithi042.stderr: 84450K .......... .......... .......... .......... .......... 62% 41.8M 2s 2024-04-20T10:35:47.059 INFO:tasks.workunit.client.0.smithi042.stderr: 84500K .......... .......... .......... .......... .......... 62% 24.1M 2s 2024-04-20T10:35:47.060 INFO:tasks.workunit.client.0.smithi042.stderr: 84550K .......... .......... .......... .......... .......... 62% 28.9M 2s 2024-04-20T10:35:47.063 INFO:tasks.workunit.client.0.smithi042.stderr: 84600K .......... .......... .......... .......... .......... 62% 25.3M 2s 2024-04-20T10:35:47.064 INFO:tasks.workunit.client.0.smithi042.stderr: 84650K .......... .......... .......... .......... .......... 62% 29.8M 2s 2024-04-20T10:35:47.066 INFO:tasks.workunit.client.0.smithi042.stderr: 84700K .......... .......... .......... .......... .......... 62% 30.6M 2s 2024-04-20T10:35:47.068 INFO:tasks.workunit.client.0.smithi042.stderr: 84750K .......... .......... .......... .......... .......... 62% 27.4M 2s 2024-04-20T10:35:47.070 INFO:tasks.workunit.client.0.smithi042.stderr: 84800K .......... .......... .......... .......... .......... 62% 24.4M 2s 2024-04-20T10:35:47.071 INFO:tasks.workunit.client.0.smithi042.stderr: 84850K .......... .......... .......... .......... .......... 62% 36.5M 2s 2024-04-20T10:35:47.073 INFO:tasks.workunit.client.0.smithi042.stderr: 84900K .......... .......... .......... .......... .......... 62% 25.8M 2s 2024-04-20T10:35:47.074 INFO:tasks.workunit.client.0.smithi042.stderr: 84950K .......... .......... .......... .......... .......... 62% 35.3M 2s 2024-04-20T10:35:47.077 INFO:tasks.workunit.client.0.smithi042.stderr: 85000K .......... .......... .......... .......... .......... 62% 20.5M 2s 2024-04-20T10:35:47.078 INFO:tasks.workunit.client.0.smithi042.stderr: 85050K .......... .......... .......... .......... .......... 62% 27.8M 2s 2024-04-20T10:35:47.080 INFO:tasks.workunit.client.0.smithi042.stderr: 85100K .......... .......... .......... .......... .......... 62% 24.7M 2s 2024-04-20T10:35:47.082 INFO:tasks.workunit.client.0.smithi042.stderr: 85150K .......... .......... .......... .......... .......... 62% 27.7M 2s 2024-04-20T10:35:47.084 INFO:tasks.workunit.client.0.smithi042.stderr: 85200K .......... .......... .......... .......... .......... 62% 27.2M 2s 2024-04-20T10:35:47.086 INFO:tasks.workunit.client.0.smithi042.stderr: 85250K .......... .......... .......... .......... .......... 62% 24.1M 2s 2024-04-20T10:35:47.088 INFO:tasks.workunit.client.0.smithi042.stderr: 85300K .......... .......... .......... .......... .......... 62% 26.1M 2s 2024-04-20T10:35:47.089 INFO:tasks.workunit.client.0.smithi042.stderr: 85350K .......... .......... .......... .......... .......... 62% 30.5M 2s 2024-04-20T10:35:47.092 INFO:tasks.workunit.client.0.smithi042.stderr: 85400K .......... .......... .......... .......... .......... 62% 18.3M 2s 2024-04-20T10:35:47.093 INFO:tasks.workunit.client.0.smithi042.stderr: 85450K .......... .......... .......... .......... .......... 62% 51.6M 2s 2024-04-20T10:35:47.095 INFO:tasks.workunit.client.0.smithi042.stderr: 85500K .......... .......... .......... .......... .......... 63% 25.3M 2s 2024-04-20T10:35:47.097 INFO:tasks.workunit.client.0.smithi042.stderr: 85550K .......... .......... .......... .......... .......... 63% 30.0M 2s 2024-04-20T10:35:47.098 INFO:tasks.workunit.client.0.smithi042.stderr: 85600K .......... .......... .......... .......... .......... 63% 25.3M 2s 2024-04-20T10:35:47.100 INFO:tasks.workunit.client.0.smithi042.stderr: 85650K .......... .......... .......... .......... .......... 63% 27.2M 2s 2024-04-20T10:35:47.101 INFO:tasks.workunit.client.0.smithi042.stderr: 85700K .......... .......... .......... .......... .......... 63% 40.5M 2s 2024-04-20T10:35:47.104 INFO:tasks.workunit.client.0.smithi042.stderr: 85750K .......... .......... .......... .......... .......... 63% 25.8M 2s 2024-04-20T10:35:47.106 INFO:tasks.workunit.client.0.smithi042.stderr: 85800K .......... .......... .......... .......... .......... 63% 24.3M 2s 2024-04-20T10:35:47.107 INFO:tasks.workunit.client.0.smithi042.stderr: 85850K .......... .......... .......... .......... .......... 63% 33.6M 2s 2024-04-20T10:35:47.109 INFO:tasks.workunit.client.0.smithi042.stderr: 85900K .......... .......... .......... .......... .......... 63% 23.4M 2s 2024-04-20T10:35:47.110 INFO:tasks.workunit.client.0.smithi042.stderr: 85950K .......... .......... .......... .......... .......... 63% 41.5M 2s 2024-04-20T10:35:47.112 INFO:tasks.workunit.client.0.smithi042.stderr: 86000K .......... .......... .......... .......... .......... 63% 20.3M 2s 2024-04-20T10:35:47.115 INFO:tasks.workunit.client.0.smithi042.stderr: 86050K .......... .......... .......... .......... .......... 63% 23.9M 2s 2024-04-20T10:35:47.116 INFO:tasks.workunit.client.0.smithi042.stderr: 86100K .......... .......... .......... .......... .......... 63% 28.6M 2s 2024-04-20T10:35:47.119 INFO:tasks.workunit.client.0.smithi042.stderr: 86150K .......... .......... .......... .......... .......... 63% 25.2M 2s 2024-04-20T10:35:47.119 INFO:tasks.workunit.client.0.smithi042.stderr: 86200K .......... .......... .......... .......... .......... 63% 43.3M 2s 2024-04-20T10:35:47.123 INFO:tasks.workunit.client.0.smithi042.stderr: 86250K .......... .......... .......... .......... .......... 63% 26.6M 2s 2024-04-20T10:35:47.124 INFO:tasks.workunit.client.0.smithi042.stderr: 86300K .......... .......... .......... .......... .......... 63% 25.3M 2s 2024-04-20T10:35:47.126 INFO:tasks.workunit.client.0.smithi042.stderr: 86350K .......... .......... .......... .......... .......... 63% 33.9M 2s 2024-04-20T10:35:47.127 INFO:tasks.workunit.client.0.smithi042.stderr: 86400K .......... .......... .......... .......... .......... 63% 24.1M 2s 2024-04-20T10:35:47.131 INFO:tasks.workunit.client.0.smithi042.stderr: 86450K .......... .......... .......... .......... .......... 63% 16.1M 2s 2024-04-20T10:35:47.131 INFO:tasks.workunit.client.0.smithi042.stderr: 86500K .......... .......... .......... .......... .......... 63% 67.1M 2s 2024-04-20T10:35:47.132 INFO:tasks.workunit.client.0.smithi042.stderr: 86550K .......... .......... .......... .......... .......... 63% 41.9M 2s 2024-04-20T10:35:47.133 INFO:tasks.workunit.client.0.smithi042.stderr: 86600K .......... .......... .......... .......... .......... 63% 33.8M 2s 2024-04-20T10:35:47.135 INFO:tasks.workunit.client.0.smithi042.stderr: 86650K .......... .......... .......... .......... .......... 63% 24.1M 2s 2024-04-20T10:35:47.136 INFO:tasks.workunit.client.0.smithi042.stderr: 86700K .......... .......... .......... .......... .......... 63% 48.0M 2s 2024-04-20T10:35:47.138 INFO:tasks.workunit.client.0.smithi042.stderr: 86750K .......... .......... .......... .......... .......... 63% 24.0M 2s 2024-04-20T10:35:47.140 INFO:tasks.workunit.client.0.smithi042.stderr: 86800K .......... .......... .......... .......... .......... 63% 25.2M 2s 2024-04-20T10:35:47.141 INFO:tasks.workunit.client.0.smithi042.stderr: 86850K .......... .......... .......... .......... .......... 64% 31.0M 2s 2024-04-20T10:35:47.145 INFO:tasks.workunit.client.0.smithi042.stderr: 86900K .......... .......... .......... .......... .......... 64% 26.6M 2s 2024-04-20T10:35:47.145 INFO:tasks.workunit.client.0.smithi042.stderr: 86950K .......... .......... .......... .......... .......... 64% 41.9M 2s 2024-04-20T10:35:47.146 INFO:tasks.workunit.client.0.smithi042.stderr: 87000K .......... .......... .......... .......... .......... 64% 26.1M 2s 2024-04-20T10:35:47.149 INFO:tasks.workunit.client.0.smithi042.stderr: 87050K .......... .......... .......... .......... .......... 64% 18.3M 2s 2024-04-20T10:35:47.150 INFO:tasks.workunit.client.0.smithi042.stderr: 87100K .......... .......... .......... .......... .......... 64% 45.2M 2s 2024-04-20T10:35:47.152 INFO:tasks.workunit.client.0.smithi042.stderr: 87150K .......... .......... .......... .......... .......... 64% 24.4M 2s 2024-04-20T10:35:47.155 INFO:tasks.workunit.client.0.smithi042.stderr: 87200K .......... .......... .......... .......... .......... 64% 24.6M 2s 2024-04-20T10:35:47.155 INFO:tasks.workunit.client.0.smithi042.stderr: 87250K .......... .......... .......... .......... .......... 64% 41.3M 2s 2024-04-20T10:35:47.157 INFO:tasks.workunit.client.0.smithi042.stderr: 87300K .......... .......... .......... .......... .......... 64% 26.2M 2s 2024-04-20T10:35:47.160 INFO:tasks.workunit.client.0.smithi042.stderr: 87350K .......... .......... .......... .......... .......... 64% 30.5M 2s 2024-04-20T10:35:47.161 INFO:tasks.workunit.client.0.smithi042.stderr: 87400K .......... .......... .......... .......... .......... 64% 26.6M 2s 2024-04-20T10:35:47.163 INFO:tasks.workunit.client.0.smithi042.stderr: 87450K .......... .......... .......... .......... .......... 64% 24.2M 2s 2024-04-20T10:35:47.164 INFO:tasks.workunit.client.0.smithi042.stderr: 87500K .......... .......... .......... .......... .......... 64% 33.6M 2s 2024-04-20T10:35:47.166 INFO:tasks.workunit.client.0.smithi042.stderr: 87550K .......... .......... .......... .......... .......... 64% 29.2M 2s 2024-04-20T10:35:47.168 INFO:tasks.workunit.client.0.smithi042.stderr: 87600K .......... .......... .......... .......... .......... 64% 26.2M 2s 2024-04-20T10:35:47.169 INFO:tasks.workunit.client.0.smithi042.stderr: 87650K .......... .......... .......... .......... .......... 64% 30.3M 2s 2024-04-20T10:35:47.171 INFO:tasks.workunit.client.0.smithi042.stderr: 87700K .......... .......... .......... .......... .......... 64% 25.3M 2s 2024-04-20T10:35:47.172 INFO:tasks.workunit.client.0.smithi042.stderr: 87750K .......... .......... .......... .......... .......... 64% 39.0M 2s 2024-04-20T10:35:47.175 INFO:tasks.workunit.client.0.smithi042.stderr: 87800K .......... .......... .......... .......... .......... 64% 25.5M 2s 2024-04-20T10:35:47.176 INFO:tasks.workunit.client.0.smithi042.stderr: 87850K .......... .......... .......... .......... .......... 64% 32.3M 2s 2024-04-20T10:35:47.179 INFO:tasks.workunit.client.0.smithi042.stderr: 87900K .......... .......... .......... .......... .......... 64% 26.9M 2s 2024-04-20T10:35:47.180 INFO:tasks.workunit.client.0.smithi042.stderr: 87950K .......... .......... .......... .......... .......... 64% 25.6M 2s 2024-04-20T10:35:47.181 INFO:tasks.workunit.client.0.smithi042.stderr: 88000K .......... .......... .......... .......... .......... 64% 38.3M 2s 2024-04-20T10:35:47.183 INFO:tasks.workunit.client.0.smithi042.stderr: 88050K .......... .......... .......... .......... .......... 64% 26.7M 2s 2024-04-20T10:35:47.184 INFO:tasks.workunit.client.0.smithi042.stderr: 88100K .......... .......... .......... .......... .......... 64% 31.0M 2s 2024-04-20T10:35:47.186 INFO:tasks.workunit.client.0.smithi042.stderr: 88150K .......... .......... .......... .......... .......... 64% 25.7M 2s 2024-04-20T10:35:47.189 INFO:tasks.workunit.client.0.smithi042.stderr: 88200K .......... .......... .......... .......... .......... 65% 26.3M 2s 2024-04-20T10:35:47.189 INFO:tasks.workunit.client.0.smithi042.stderr: 88250K .......... .......... .......... .......... .......... 65% 37.7M 2s 2024-04-20T10:35:47.191 INFO:tasks.workunit.client.0.smithi042.stderr: 88300K .......... .......... .......... .......... .......... 65% 26.2M 2s 2024-04-20T10:35:47.193 INFO:tasks.workunit.client.0.smithi042.stderr: 88350K .......... .......... .......... .......... .......... 65% 33.4M 2s 2024-04-20T10:35:47.195 INFO:tasks.workunit.client.0.smithi042.stderr: 88400K .......... .......... .......... .......... .......... 65% 25.5M 2s 2024-04-20T10:35:47.197 INFO:tasks.workunit.client.0.smithi042.stderr: 88450K .......... .......... .......... .......... .......... 65% 25.4M 2s 2024-04-20T10:35:47.198 INFO:tasks.workunit.client.0.smithi042.stderr: 88500K .......... .......... .......... .......... .......... 65% 37.7M 2s 2024-04-20T10:35:47.200 INFO:tasks.workunit.client.0.smithi042.stderr: 88550K .......... .......... .......... .......... .......... 65% 24.5M 2s 2024-04-20T10:35:47.201 INFO:tasks.workunit.client.0.smithi042.stderr: 88600K .......... .......... .......... .......... .......... 65% 39.7M 2s 2024-04-20T10:35:47.203 INFO:tasks.workunit.client.0.smithi042.stderr: 88650K .......... .......... .......... .......... .......... 65% 26.3M 2s 2024-04-20T10:35:47.205 INFO:tasks.workunit.client.0.smithi042.stderr: 88700K .......... .......... .......... .......... .......... 65% 24.1M 2s 2024-04-20T10:35:47.207 INFO:tasks.workunit.client.0.smithi042.stderr: 88750K .......... .......... .......... .......... .......... 65% 33.7M 2s 2024-04-20T10:35:47.208 INFO:tasks.workunit.client.0.smithi042.stderr: 88800K .......... .......... .......... .......... .......... 65% 26.6M 2s 2024-04-20T10:35:47.209 INFO:tasks.workunit.client.0.smithi042.stderr: 88850K .......... .......... .......... .......... .......... 65% 37.5M 2s 2024-04-20T10:35:47.211 INFO:tasks.workunit.client.0.smithi042.stderr: 88900K .......... .......... .......... .......... .......... 65% 25.1M 2s 2024-04-20T10:35:47.213 INFO:tasks.workunit.client.0.smithi042.stderr: 88950K .......... .......... .......... .......... .......... 65% 25.1M 2s 2024-04-20T10:35:47.214 INFO:tasks.workunit.client.0.smithi042.stderr: 89000K .......... .......... .......... .......... .......... 65% 41.0M 2s 2024-04-20T10:35:47.217 INFO:tasks.workunit.client.0.smithi042.stderr: 89050K .......... .......... .......... .......... .......... 65% 22.5M 2s 2024-04-20T10:35:47.218 INFO:tasks.workunit.client.0.smithi042.stderr: 89100K .......... .......... .......... .......... .......... 65% 32.2M 2s 2024-04-20T10:35:47.220 INFO:tasks.workunit.client.0.smithi042.stderr: 89150K .......... .......... .......... .......... .......... 65% 24.3M 2s 2024-04-20T10:35:47.222 INFO:tasks.workunit.client.0.smithi042.stderr: 89200K .......... .......... .......... .......... .......... 65% 27.4M 2s 2024-04-20T10:35:47.224 INFO:tasks.workunit.client.0.smithi042.stderr: 89250K .......... .......... .......... .......... .......... 65% 34.2M 2s 2024-04-20T10:35:47.225 INFO:tasks.workunit.client.0.smithi042.stderr: 89300K .......... .......... .......... .......... .......... 65% 27.2M 2s 2024-04-20T10:35:47.227 INFO:tasks.workunit.client.0.smithi042.stderr: 89350K .......... .......... .......... .......... .......... 65% 23.4M 2s 2024-04-20T10:35:47.229 INFO:tasks.workunit.client.0.smithi042.stderr: 89400K .......... .......... .......... .......... .......... 65% 33.7M 2s 2024-04-20T10:35:47.231 INFO:tasks.workunit.client.0.smithi042.stderr: 89450K .......... .......... .......... .......... .......... 65% 26.0M 2s 2024-04-20T10:35:47.232 INFO:tasks.workunit.client.0.smithi042.stderr: 89500K .......... .......... .......... .......... .......... 65% 45.5M 2s 2024-04-20T10:35:47.234 INFO:tasks.workunit.client.0.smithi042.stderr: 89550K .......... .......... .......... .......... .......... 66% 20.3M 2s 2024-04-20T10:35:47.235 INFO:tasks.workunit.client.0.smithi042.stderr: 89600K .......... .......... .......... .......... .......... 66% 35.2M 2s 2024-04-20T10:35:47.237 INFO:tasks.workunit.client.0.smithi042.stderr: 89650K .......... .......... .......... .......... .......... 66% 25.4M 2s 2024-04-20T10:35:47.239 INFO:tasks.workunit.client.0.smithi042.stderr: 89700K .......... .......... .......... .......... .......... 66% 27.6M 2s 2024-04-20T10:35:47.240 INFO:tasks.workunit.client.0.smithi042.stderr: 89750K .......... .......... .......... .......... .......... 66% 35.6M 2s 2024-04-20T10:35:47.242 INFO:tasks.workunit.client.0.smithi042.stderr: 89800K .......... .......... .......... .......... .......... 66% 25.3M 2s 2024-04-20T10:35:47.244 INFO:tasks.workunit.client.0.smithi042.stderr: 89850K .......... .......... .......... .......... .......... 66% 35.4M 2s 2024-04-20T10:35:47.246 INFO:tasks.workunit.client.0.smithi042.stderr: 89900K .......... .......... .......... .......... .......... 66% 24.1M 2s 2024-04-20T10:35:47.248 INFO:tasks.workunit.client.0.smithi042.stderr: 89950K .......... .......... .......... .......... .......... 66% 26.9M 2s 2024-04-20T10:35:47.249 INFO:tasks.workunit.client.0.smithi042.stderr: 90000K .......... .......... .......... .......... .......... 66% 27.7M 2s 2024-04-20T10:35:47.251 INFO:tasks.workunit.client.0.smithi042.stderr: 90050K .......... .......... .......... .......... .......... 66% 22.7M 2s 2024-04-20T10:35:47.255 INFO:tasks.workunit.client.0.smithi042.stderr: 90100K .......... .......... .......... .......... .......... 66% 35.9M 2s 2024-04-20T10:35:47.255 INFO:tasks.workunit.client.0.smithi042.stderr: 90150K .......... .......... .......... .......... .......... 66% 22.5M 2s 2024-04-20T10:35:47.257 INFO:tasks.workunit.client.0.smithi042.stderr: 90200K .......... .......... .......... .......... .......... 66% 23.5M 2s 2024-04-20T10:35:47.258 INFO:tasks.workunit.client.0.smithi042.stderr: 90250K .......... .......... .......... .......... .......... 66% 39.3M 2s 2024-04-20T10:35:47.260 INFO:tasks.workunit.client.0.smithi042.stderr: 90300K .......... .......... .......... .......... .......... 66% 23.1M 2s 2024-04-20T10:35:47.262 INFO:tasks.workunit.client.0.smithi042.stderr: 90350K .......... .......... .......... .......... .......... 66% 30.1M 2s 2024-04-20T10:35:47.264 INFO:tasks.workunit.client.0.smithi042.stderr: 90400K .......... .......... .......... .......... .......... 66% 23.3M 2s 2024-04-20T10:35:47.267 INFO:tasks.workunit.client.0.smithi042.stderr: 90450K .......... .......... .......... .......... .......... 66% 22.3M 2s 2024-04-20T10:35:47.268 INFO:tasks.workunit.client.0.smithi042.stderr: 90500K .......... .......... .......... .......... .......... 66% 38.3M 2s 2024-04-20T10:35:47.271 INFO:tasks.workunit.client.0.smithi042.stderr: 90550K .......... .......... .......... .......... .......... 66% 17.6M 2s 2024-04-20T10:35:47.273 INFO:tasks.workunit.client.0.smithi042.stderr: 90600K .......... .......... .......... .......... .......... 66% 24.8M 2s 2024-04-20T10:35:47.275 INFO:tasks.workunit.client.0.smithi042.stderr: 90650K .......... .......... .......... .......... .......... 66% 24.5M 2s 2024-04-20T10:35:47.276 INFO:tasks.workunit.client.0.smithi042.stderr: 90700K .......... .......... .......... .......... .......... 66% 34.4M 2s 2024-04-20T10:35:47.277 INFO:tasks.workunit.client.0.smithi042.stderr: 90750K .......... .......... .......... .......... .......... 66% 29.6M 2s 2024-04-20T10:35:47.280 INFO:tasks.workunit.client.0.smithi042.stderr: 90800K .......... .......... .......... .......... .......... 66% 25.6M 2s 2024-04-20T10:35:47.281 INFO:tasks.workunit.client.0.smithi042.stderr: 90850K .......... .......... .......... .......... .......... 66% 29.3M 2s 2024-04-20T10:35:47.282 INFO:tasks.workunit.client.0.smithi042.stderr: 90900K .......... .......... .......... .......... .......... 66% 41.9M 2s 2024-04-20T10:35:47.285 INFO:tasks.workunit.client.0.smithi042.stderr: 90950K .......... .......... .......... .......... .......... 67% 21.0M 2s 2024-04-20T10:35:47.287 INFO:tasks.workunit.client.0.smithi042.stderr: 91000K .......... .......... .......... .......... .......... 67% 36.8M 2s 2024-04-20T10:35:47.288 INFO:tasks.workunit.client.0.smithi042.stderr: 91050K .......... .......... .......... .......... .......... 67% 26.8M 2s 2024-04-20T10:35:47.291 INFO:tasks.workunit.client.0.smithi042.stderr: 91100K .......... .......... .......... .......... .......... 67% 24.1M 2s 2024-04-20T10:35:47.291 INFO:tasks.workunit.client.0.smithi042.stderr: 91150K .......... .......... .......... .......... .......... 67% 45.6M 2s 2024-04-20T10:35:47.294 INFO:tasks.workunit.client.0.smithi042.stderr: 91200K .......... .......... .......... .......... .......... 67% 21.2M 2s 2024-04-20T10:35:47.295 INFO:tasks.workunit.client.0.smithi042.stderr: 91250K .......... .......... .......... .......... .......... 67% 36.8M 2s 2024-04-20T10:35:47.297 INFO:tasks.workunit.client.0.smithi042.stderr: 91300K .......... .......... .......... .......... .......... 67% 27.2M 2s 2024-04-20T10:35:47.298 INFO:tasks.workunit.client.0.smithi042.stderr: 91350K .......... .......... .......... .......... .......... 67% 35.0M 2s 2024-04-20T10:35:47.299 INFO:tasks.workunit.client.0.smithi042.stderr: 91400K .......... .......... .......... .......... .......... 67% 28.1M 2s 2024-04-20T10:35:47.302 INFO:tasks.workunit.client.0.smithi042.stderr: 91450K .......... .......... .......... .......... .......... 67% 23.4M 2s 2024-04-20T10:35:47.304 INFO:tasks.workunit.client.0.smithi042.stderr: 91500K .......... .......... .......... .......... .......... 67% 35.2M 2s 2024-04-20T10:35:47.305 INFO:tasks.workunit.client.0.smithi042.stderr: 91550K .......... .......... .......... .......... .......... 67% 22.2M 2s 2024-04-20T10:35:47.306 INFO:tasks.workunit.client.0.smithi042.stderr: 91600K .......... .......... .......... .......... .......... 67% 36.6M 2s 2024-04-20T10:35:47.308 INFO:tasks.workunit.client.0.smithi042.stderr: 91650K .......... .......... .......... .......... .......... 67% 32.5M 2s 2024-04-20T10:35:47.309 INFO:tasks.workunit.client.0.smithi042.stderr: 91700K .......... .......... .......... .......... .......... 67% 31.2M 2s 2024-04-20T10:35:47.312 INFO:tasks.workunit.client.0.smithi042.stderr: 91750K .......... .......... .......... .......... .......... 67% 23.8M 2s 2024-04-20T10:35:47.314 INFO:tasks.workunit.client.0.smithi042.stderr: 91800K .......... .......... .......... .......... .......... 67% 23.5M 2s 2024-04-20T10:35:47.315 INFO:tasks.workunit.client.0.smithi042.stderr: 91850K .......... .......... .......... .......... .......... 67% 33.7M 2s 2024-04-20T10:35:47.317 INFO:tasks.workunit.client.0.smithi042.stderr: 91900K .......... .......... .......... .......... .......... 67% 30.3M 2s 2024-04-20T10:35:47.318 INFO:tasks.workunit.client.0.smithi042.stderr: 91950K .......... .......... .......... .......... .......... 67% 28.4M 2s 2024-04-20T10:35:47.321 INFO:tasks.workunit.client.0.smithi042.stderr: 92000K .......... .......... .......... .......... .......... 67% 26.4M 2s 2024-04-20T10:35:47.321 INFO:tasks.workunit.client.0.smithi042.stderr: 92050K .......... .......... .......... .......... .......... 67% 44.8M 2s 2024-04-20T10:35:47.323 INFO:tasks.workunit.client.0.smithi042.stderr: 92100K .......... .......... .......... .......... .......... 67% 21.6M 2s 2024-04-20T10:35:47.326 INFO:tasks.workunit.client.0.smithi042.stderr: 92150K .......... .......... .......... .......... .......... 67% 28.5M 2s 2024-04-20T10:35:47.327 INFO:tasks.workunit.client.0.smithi042.stderr: 92200K .......... .......... .......... .......... .......... 67% 31.5M 2s 2024-04-20T10:35:47.328 INFO:tasks.workunit.client.0.smithi042.stderr: 92250K .......... .......... .......... .......... .......... 67% 33.1M 2s 2024-04-20T10:35:47.331 INFO:tasks.workunit.client.0.smithi042.stderr: 92300K .......... .......... .......... .......... .......... 68% 23.9M 2s 2024-04-20T10:35:47.332 INFO:tasks.workunit.client.0.smithi042.stderr: 92350K .......... .......... .......... .......... .......... 68% 33.1M 2s 2024-04-20T10:35:47.334 INFO:tasks.workunit.client.0.smithi042.stderr: 92400K .......... .......... .......... .......... .......... 68% 30.8M 2s 2024-04-20T10:35:47.335 INFO:tasks.workunit.client.0.smithi042.stderr: 92450K .......... .......... .......... .......... .......... 68% 29.4M 2s 2024-04-20T10:35:47.336 INFO:tasks.workunit.client.0.smithi042.stderr: 92500K .......... .......... .......... .......... .......... 68% 33.7M 2s 2024-04-20T10:35:47.340 INFO:tasks.workunit.client.0.smithi042.stderr: 92550K .......... .......... .......... .......... .......... 68% 23.4M 2s 2024-04-20T10:35:47.341 INFO:tasks.workunit.client.0.smithi042.stderr: 92600K .......... .......... .......... .......... .......... 68% 24.1M 2s 2024-04-20T10:35:47.342 INFO:tasks.workunit.client.0.smithi042.stderr: 92650K .......... .......... .......... .......... .......... 68% 49.0M 2s 2024-04-20T10:35:47.344 INFO:tasks.workunit.client.0.smithi042.stderr: 92700K .......... .......... .......... .......... .......... 68% 24.5M 2s 2024-04-20T10:35:47.346 INFO:tasks.workunit.client.0.smithi042.stderr: 92750K .......... .......... .......... .......... .......... 68% 23.8M 2s 2024-04-20T10:35:47.348 INFO:tasks.workunit.client.0.smithi042.stderr: 92800K .......... .......... .......... .......... .......... 68% 24.4M 2s 2024-04-20T10:35:47.351 INFO:tasks.workunit.client.0.smithi042.stderr: 92850K .......... .......... .......... .......... .......... 68% 24.3M 2s 2024-04-20T10:35:47.351 INFO:tasks.workunit.client.0.smithi042.stderr: 92900K .......... .......... .......... .......... .......... 68% 51.5M 2s 2024-04-20T10:35:47.353 INFO:tasks.workunit.client.0.smithi042.stderr: 92950K .......... .......... .......... .......... .......... 68% 25.1M 2s 2024-04-20T10:35:47.355 INFO:tasks.workunit.client.0.smithi042.stderr: 93000K .......... .......... .......... .......... .......... 68% 25.9M 2s 2024-04-20T10:35:47.355 INFO:tasks.workunit.client.0.smithi042.stderr: 93050K .......... .......... .......... .......... .......... 68% 46.8M 2s 2024-04-20T10:35:47.358 INFO:tasks.workunit.client.0.smithi042.stderr: 93100K .......... .......... .......... .......... .......... 68% 24.0M 2s 2024-04-20T10:35:47.359 INFO:tasks.workunit.client.0.smithi042.stderr: 93150K .......... .......... .......... .......... .......... 68% 41.2M 2s 2024-04-20T10:35:47.361 INFO:tasks.workunit.client.0.smithi042.stderr: 93200K .......... .......... .......... .......... .......... 68% 20.7M 2s 2024-04-20T10:35:47.364 INFO:tasks.workunit.client.0.smithi042.stderr: 93250K .......... .......... .......... .......... .......... 68% 27.5M 2s 2024-04-20T10:35:47.366 INFO:tasks.workunit.client.0.smithi042.stderr: 93300K .......... .......... .......... .......... .......... 68% 25.5M 2s 2024-04-20T10:35:47.367 INFO:tasks.workunit.client.0.smithi042.stderr: 93350K .......... .......... .......... .......... .......... 68% 27.9M 2s 2024-04-20T10:35:47.368 INFO:tasks.workunit.client.0.smithi042.stderr: 93400K .......... .......... .......... .......... .......... 68% 49.9M 2s 2024-04-20T10:35:47.370 INFO:tasks.workunit.client.0.smithi042.stderr: 93450K .......... .......... .......... .......... .......... 68% 24.8M 2s 2024-04-20T10:35:47.372 INFO:tasks.workunit.client.0.smithi042.stderr: 93500K .......... .......... .......... .......... .......... 68% 28.3M 2s 2024-04-20T10:35:47.374 INFO:tasks.workunit.client.0.smithi042.stderr: 93550K .......... .......... .......... .......... .......... 68% 23.9M 2s 2024-04-20T10:35:47.376 INFO:tasks.workunit.client.0.smithi042.stderr: 93600K .......... .......... .......... .......... .......... 68% 27.6M 2s 2024-04-20T10:35:47.376 INFO:tasks.workunit.client.0.smithi042.stderr: 93650K .......... .......... .......... .......... .......... 69% 52.2M 2s 2024-04-20T10:35:47.379 INFO:tasks.workunit.client.0.smithi042.stderr: 93700K .......... .......... .......... .......... .......... 69% 20.2M 2s 2024-04-20T10:35:47.380 INFO:tasks.workunit.client.0.smithi042.stderr: 93750K .......... .......... .......... .......... .......... 69% 34.3M 2s 2024-04-20T10:35:47.381 INFO:tasks.workunit.client.0.smithi042.stderr: 93800K .......... .......... .......... .......... .......... 69% 28.3M 2s 2024-04-20T10:35:47.384 INFO:tasks.workunit.client.0.smithi042.stderr: 93850K .......... .......... .......... .......... .......... 69% 27.3M 2s 2024-04-20T10:35:47.385 INFO:tasks.workunit.client.0.smithi042.stderr: 93900K .......... .......... .......... .......... .......... 69% 37.0M 2s 2024-04-20T10:35:47.387 INFO:tasks.workunit.client.0.smithi042.stderr: 93950K .......... .......... .......... .......... .......... 69% 27.8M 2s 2024-04-20T10:35:47.391 INFO:tasks.workunit.client.0.smithi042.stderr: 94000K .......... .......... .......... .......... .......... 69% 27.0M 2s 2024-04-20T10:35:47.391 INFO:tasks.workunit.client.0.smithi042.stderr: 94050K .......... .......... .......... .......... .......... 69% 22.8M 2s 2024-04-20T10:35:47.392 INFO:tasks.workunit.client.0.smithi042.stderr: 94100K .......... .......... .......... .......... .......... 69% 37.6M 2s 2024-04-20T10:35:47.394 INFO:tasks.workunit.client.0.smithi042.stderr: 94150K .......... .......... .......... .......... .......... 69% 26.6M 2s 2024-04-20T10:35:47.395 INFO:tasks.workunit.client.0.smithi042.stderr: 94200K .......... .......... .......... .......... .......... 69% 33.0M 2s 2024-04-20T10:35:47.397 INFO:tasks.workunit.client.0.smithi042.stderr: 94250K .......... .......... .......... .......... .......... 69% 27.9M 2s 2024-04-20T10:35:47.399 INFO:tasks.workunit.client.0.smithi042.stderr: 94300K .......... .......... .......... .......... .......... 69% 26.8M 2s 2024-04-20T10:35:47.402 INFO:tasks.workunit.client.0.smithi042.stderr: 94350K .......... .......... .......... .......... .......... 69% 28.9M 2s 2024-04-20T10:35:47.402 INFO:tasks.workunit.client.0.smithi042.stderr: 94400K .......... .......... .......... .......... .......... 69% 35.8M 2s 2024-04-20T10:35:47.403 INFO:tasks.workunit.client.0.smithi042.stderr: 94450K .......... .......... .......... .......... .......... 69% 26.9M 2s 2024-04-20T10:35:47.405 INFO:tasks.workunit.client.0.smithi042.stderr: 94500K .......... .......... .......... .......... .......... 69% 25.9M 2s 2024-04-20T10:35:47.407 INFO:tasks.workunit.client.0.smithi042.stderr: 94550K .......... .......... .......... .......... .......... 69% 27.8M 2s 2024-04-20T10:35:47.409 INFO:tasks.workunit.client.0.smithi042.stderr: 94600K .......... .......... .......... .......... .......... 69% 27.6M 2s 2024-04-20T10:35:47.410 INFO:tasks.workunit.client.0.smithi042.stderr: 94650K .......... .......... .......... .......... .......... 69% 33.7M 2s 2024-04-20T10:35:47.412 INFO:tasks.workunit.client.0.smithi042.stderr: 94700K .......... .......... .......... .......... .......... 69% 24.5M 2s 2024-04-20T10:35:47.414 INFO:tasks.workunit.client.0.smithi042.stderr: 94750K .......... .......... .......... .......... .......... 69% 29.3M 2s 2024-04-20T10:35:47.416 INFO:tasks.workunit.client.0.smithi042.stderr: 94800K .......... .......... .......... .......... .......... 69% 28.9M 2s 2024-04-20T10:35:47.418 INFO:tasks.workunit.client.0.smithi042.stderr: 94850K .......... .......... .......... .......... .......... 69% 26.4M 2s 2024-04-20T10:35:47.419 INFO:tasks.workunit.client.0.smithi042.stderr: 94900K .......... .......... .......... .......... .......... 69% 29.4M 2s 2024-04-20T10:35:47.421 INFO:tasks.workunit.client.0.smithi042.stderr: 94950K .......... .......... .......... .......... .......... 69% 31.6M 2s 2024-04-20T10:35:47.423 INFO:tasks.workunit.client.0.smithi042.stderr: 95000K .......... .......... .......... .......... .......... 70% 28.0M 2s 2024-04-20T10:35:47.425 INFO:tasks.workunit.client.0.smithi042.stderr: 95050K .......... .......... .......... .......... .......... 70% 22.9M 2s 2024-04-20T10:35:47.427 INFO:tasks.workunit.client.0.smithi042.stderr: 95100K .......... .......... .......... .......... .......... 70% 22.8M 2s 2024-04-20T10:35:47.428 INFO:tasks.workunit.client.0.smithi042.stderr: 95150K .......... .......... .......... .......... .......... 70% 35.4M 2s 2024-04-20T10:35:47.430 INFO:tasks.workunit.client.0.smithi042.stderr: 95200K .......... .......... .......... .......... .......... 70% 29.8M 2s 2024-04-20T10:35:47.432 INFO:tasks.workunit.client.0.smithi042.stderr: 95250K .......... .......... .......... .......... .......... 70% 28.0M 2s 2024-04-20T10:35:47.433 INFO:tasks.workunit.client.0.smithi042.stderr: 95300K .......... .......... .......... .......... .......... 70% 22.4M 2s 2024-04-20T10:35:47.435 INFO:tasks.workunit.client.0.smithi042.stderr: 95350K .......... .......... .......... .......... .......... 70% 25.2M 2s 2024-04-20T10:35:47.437 INFO:tasks.workunit.client.0.smithi042.stderr: 95400K .......... .......... .......... .......... .......... 70% 33.0M 2s 2024-04-20T10:35:47.439 INFO:tasks.workunit.client.0.smithi042.stderr: 95450K .......... .......... .......... .......... .......... 70% 26.4M 2s 2024-04-20T10:35:47.442 INFO:tasks.workunit.client.0.smithi042.stderr: 95500K .......... .......... .......... .......... .......... 70% 19.0M 2s 2024-04-20T10:35:47.443 INFO:tasks.workunit.client.0.smithi042.stderr: 95550K .......... .......... .......... .......... .......... 70% 32.7M 2s 2024-04-20T10:35:47.445 INFO:tasks.workunit.client.0.smithi042.stderr: 95600K .......... .......... .......... .......... .......... 70% 19.9M 2s 2024-04-20T10:35:47.447 INFO:tasks.workunit.client.0.smithi042.stderr: 95650K .......... .......... .......... .......... .......... 70% 29.8M 2s 2024-04-20T10:35:47.449 INFO:tasks.workunit.client.0.smithi042.stderr: 95700K .......... .......... .......... .......... .......... 70% 29.9M 2s 2024-04-20T10:35:47.451 INFO:tasks.workunit.client.0.smithi042.stderr: 95750K .......... .......... .......... .......... .......... 70% 29.1M 2s 2024-04-20T10:35:47.453 INFO:tasks.workunit.client.0.smithi042.stderr: 95800K .......... .......... .......... .......... .......... 70% 27.9M 2s 2024-04-20T10:35:47.455 INFO:tasks.workunit.client.0.smithi042.stderr: 95850K .......... .......... .......... .......... .......... 70% 23.8M 2s 2024-04-20T10:35:47.455 INFO:tasks.workunit.client.0.smithi042.stderr: 95900K .......... .......... .......... .......... .......... 70% 53.4M 2s 2024-04-20T10:35:47.457 INFO:tasks.workunit.client.0.smithi042.stderr: 95950K .......... .......... .......... .......... .......... 70% 23.9M 2s 2024-04-20T10:35:47.459 INFO:tasks.workunit.client.0.smithi042.stderr: 96000K .......... .......... .......... .......... .......... 70% 24.8M 2s 2024-04-20T10:35:47.461 INFO:tasks.workunit.client.0.smithi042.stderr: 96050K .......... .......... .......... .......... .......... 70% 33.1M 2s 2024-04-20T10:35:47.464 INFO:tasks.workunit.client.0.smithi042.stderr: 96100K .......... .......... .......... .......... .......... 70% 24.8M 2s 2024-04-20T10:35:47.464 INFO:tasks.workunit.client.0.smithi042.stderr: 96150K .......... .......... .......... .......... .......... 70% 43.6M 2s 2024-04-20T10:35:47.466 INFO:tasks.workunit.client.0.smithi042.stderr: 96200K .......... .......... .......... .......... .......... 70% 24.4M 2s 2024-04-20T10:35:47.468 INFO:tasks.workunit.client.0.smithi042.stderr: 96250K .......... .......... .......... .......... .......... 70% 27.0M 2s 2024-04-20T10:35:47.470 INFO:tasks.workunit.client.0.smithi042.stderr: 96300K .......... .......... .......... .......... .......... 70% 31.0M 2s 2024-04-20T10:35:47.472 INFO:tasks.workunit.client.0.smithi042.stderr: 96350K .......... .......... .......... .......... .......... 71% 24.6M 2s 2024-04-20T10:35:47.473 INFO:tasks.workunit.client.0.smithi042.stderr: 96400K .......... .......... .......... .......... .......... 71% 41.6M 2s 2024-04-20T10:35:47.474 INFO:tasks.workunit.client.0.smithi042.stderr: 96450K .......... .......... .......... .......... .......... 71% 25.4M 2s 2024-04-20T10:35:47.476 INFO:tasks.workunit.client.0.smithi042.stderr: 96500K .......... .......... .......... .......... .......... 71% 25.7M 2s 2024-04-20T10:35:47.478 INFO:tasks.workunit.client.0.smithi042.stderr: 96550K .......... .......... .......... .......... .......... 71% 32.0M 2s 2024-04-20T10:35:47.479 INFO:tasks.workunit.client.0.smithi042.stderr: 96600K .......... .......... .......... .......... .......... 71% 26.5M 2s 2024-04-20T10:35:47.482 INFO:tasks.workunit.client.0.smithi042.stderr: 96650K .......... .......... .......... .......... .......... 71% 36.2M 2s 2024-04-20T10:35:47.483 INFO:tasks.workunit.client.0.smithi042.stderr: 96700K .......... .......... .......... .......... .......... 71% 25.9M 2s 2024-04-20T10:35:47.485 INFO:tasks.workunit.client.0.smithi042.stderr: 96750K .......... .......... .......... .......... .......... 71% 24.0M 2s 2024-04-20T10:35:47.487 INFO:tasks.workunit.client.0.smithi042.stderr: 96800K .......... .......... .......... .......... .......... 71% 29.3M 2s 2024-04-20T10:35:47.488 INFO:tasks.workunit.client.0.smithi042.stderr: 96850K .......... .......... .......... .......... .......... 71% 27.1M 2s 2024-04-20T10:35:47.490 INFO:tasks.workunit.client.0.smithi042.stderr: 96900K .......... .......... .......... .......... .......... 71% 24.5M 2s 2024-04-20T10:35:47.491 INFO:tasks.workunit.client.0.smithi042.stderr: 96950K .......... .......... .......... .......... .......... 71% 41.3M 2s 2024-04-20T10:35:47.494 INFO:tasks.workunit.client.0.smithi042.stderr: 97000K .......... .......... .......... .......... .......... 71% 25.4M 2s 2024-04-20T10:35:47.495 INFO:tasks.workunit.client.0.smithi042.stderr: 97050K .......... .......... .......... .......... .......... 71% 30.4M 2s 2024-04-20T10:35:47.497 INFO:tasks.workunit.client.0.smithi042.stderr: 97100K .......... .......... .......... .......... .......... 71% 25.1M 2s 2024-04-20T10:35:47.500 INFO:tasks.workunit.client.0.smithi042.stderr: 97150K .......... .......... .......... .......... .......... 71% 24.9M 2s 2024-04-20T10:35:47.500 INFO:tasks.workunit.client.0.smithi042.stderr: 97200K .......... .......... .......... .......... .......... 71% 38.5M 2s 2024-04-20T10:35:47.502 INFO:tasks.workunit.client.0.smithi042.stderr: 97250K .......... .......... .......... .......... .......... 71% 27.2M 2s 2024-04-20T10:35:47.504 INFO:tasks.workunit.client.0.smithi042.stderr: 97300K .......... .......... .......... .......... .......... 71% 29.5M 2s 2024-04-20T10:35:47.506 INFO:tasks.workunit.client.0.smithi042.stderr: 97350K .......... .......... .......... .......... .......... 71% 25.6M 2s 2024-04-20T10:35:47.508 INFO:tasks.workunit.client.0.smithi042.stderr: 97400K .......... .......... .......... .......... .......... 71% 25.3M 2s 2024-04-20T10:35:47.508 INFO:tasks.workunit.client.0.smithi042.stderr: 97450K .......... .......... .......... .......... .......... 71% 41.2M 2s 2024-04-20T10:35:47.511 INFO:tasks.workunit.client.0.smithi042.stderr: 97500K .......... .......... .......... .......... .......... 71% 25.5M 2s 2024-04-20T10:35:47.512 INFO:tasks.workunit.client.0.smithi042.stderr: 97550K .......... .......... .......... .......... .......... 71% 27.8M 2s 2024-04-20T10:35:47.514 INFO:tasks.workunit.client.0.smithi042.stderr: 97600K .......... .......... .......... .......... .......... 71% 24.6M 2s 2024-04-20T10:35:47.516 INFO:tasks.workunit.client.0.smithi042.stderr: 97650K .......... .......... .......... .......... .......... 71% 25.5M 2s 2024-04-20T10:35:47.517 INFO:tasks.workunit.client.0.smithi042.stderr: 97700K .......... .......... .......... .......... .......... 72% 38.0M 2s 2024-04-20T10:35:47.520 INFO:tasks.workunit.client.0.smithi042.stderr: 97750K .......... .......... .......... .......... .......... 72% 27.5M 2s 2024-04-20T10:35:47.520 INFO:tasks.workunit.client.0.smithi042.stderr: 97800K .......... .......... .......... .......... .......... 72% 34.0M 2s 2024-04-20T10:35:47.523 INFO:tasks.workunit.client.0.smithi042.stderr: 97850K .......... .......... .......... .......... .......... 72% 24.4M 2s 2024-04-20T10:35:47.525 INFO:tasks.workunit.client.0.smithi042.stderr: 97900K .......... .......... .......... .......... .......... 72% 25.1M 2s 2024-04-20T10:35:47.526 INFO:tasks.workunit.client.0.smithi042.stderr: 97950K .......... .......... .......... .......... .......... 72% 39.1M 2s 2024-04-20T10:35:47.528 INFO:tasks.workunit.client.0.smithi042.stderr: 98000K .......... .......... .......... .......... .......... 72% 26.1M 2s 2024-04-20T10:35:47.529 INFO:tasks.workunit.client.0.smithi042.stderr: 98050K .......... .......... .......... .......... .......... 72% 32.3M 2s 2024-04-20T10:35:47.531 INFO:tasks.workunit.client.0.smithi042.stderr: 98100K .......... .......... .......... .......... .......... 72% 27.4M 2s 2024-04-20T10:35:47.533 INFO:tasks.workunit.client.0.smithi042.stderr: 98150K .......... .......... .......... .......... .......... 72% 25.2M 2s 2024-04-20T10:35:47.534 INFO:tasks.workunit.client.0.smithi042.stderr: 98200K .......... .......... .......... .......... .......... 72% 36.3M 2s 2024-04-20T10:35:47.536 INFO:tasks.workunit.client.0.smithi042.stderr: 98250K .......... .......... .......... .......... .......... 72% 23.7M 2s 2024-04-20T10:35:47.539 INFO:tasks.workunit.client.0.smithi042.stderr: 98300K .......... .......... .......... .......... .......... 72% 26.6M 2s 2024-04-20T10:35:47.539 INFO:tasks.workunit.client.0.smithi042.stderr: 98350K .......... .......... .......... .......... .......... 72% 66.1M 2s 2024-04-20T10:35:47.542 INFO:tasks.workunit.client.0.smithi042.stderr: 98400K .......... .......... .......... .......... .......... 72% 19.4M 2s 2024-04-20T10:35:47.542 INFO:tasks.workunit.client.0.smithi042.stderr: 98450K .......... .......... .......... .......... .......... 72% 40.1M 2s 2024-04-20T10:35:47.545 INFO:tasks.workunit.client.0.smithi042.stderr: 98500K .......... .......... .......... .......... .......... 72% 24.8M 2s 2024-04-20T10:35:47.547 INFO:tasks.workunit.client.0.smithi042.stderr: 98550K .......... .......... .......... .......... .......... 72% 28.3M 2s 2024-04-20T10:35:47.547 INFO:tasks.workunit.client.0.smithi042.stderr: 98600K .......... .......... .......... .......... .......... 72% 41.9M 2s 2024-04-20T10:35:47.550 INFO:tasks.workunit.client.0.smithi042.stderr: 98650K .......... .......... .......... .......... .......... 72% 20.6M 2s 2024-04-20T10:35:47.551 INFO:tasks.workunit.client.0.smithi042.stderr: 98700K .......... .......... .......... .......... .......... 72% 38.2M 2s 2024-04-20T10:35:47.553 INFO:tasks.workunit.client.0.smithi042.stderr: 98750K .......... .......... .......... .......... .......... 72% 24.6M 2s 2024-04-20T10:35:47.554 INFO:tasks.workunit.client.0.smithi042.stderr: 98800K .......... .......... .......... .......... .......... 72% 35.3M 2s 2024-04-20T10:35:47.556 INFO:tasks.workunit.client.0.smithi042.stderr: 98850K .......... .......... .......... .......... .......... 72% 25.8M 2s 2024-04-20T10:35:47.558 INFO:tasks.workunit.client.0.smithi042.stderr: 98900K .......... .......... .......... .......... .......... 72% 27.0M 2s 2024-04-20T10:35:47.559 INFO:tasks.workunit.client.0.smithi042.stderr: 98950K .......... .......... .......... .......... .......... 72% 33.2M 2s 2024-04-20T10:35:47.562 INFO:tasks.workunit.client.0.smithi042.stderr: 99000K .......... .......... .......... .......... .......... 72% 20.9M 2s 2024-04-20T10:35:47.563 INFO:tasks.workunit.client.0.smithi042.stderr: 99050K .......... .......... .......... .......... .......... 73% 35.7M 2s 2024-04-20T10:35:47.565 INFO:tasks.workunit.client.0.smithi042.stderr: 99100K .......... .......... .......... .......... .......... 73% 26.3M 2s 2024-04-20T10:35:47.568 INFO:tasks.workunit.client.0.smithi042.stderr: 99150K .......... .......... .......... .......... .......... 73% 24.8M 2s 2024-04-20T10:35:47.568 INFO:tasks.workunit.client.0.smithi042.stderr: 99200K .......... .......... .......... .......... .......... 73% 41.1M 2s 2024-04-20T10:35:47.570 INFO:tasks.workunit.client.0.smithi042.stderr: 99250K .......... .......... .......... .......... .......... 73% 26.0M 2s 2024-04-20T10:35:47.572 INFO:tasks.workunit.client.0.smithi042.stderr: 99300K .......... .......... .......... .......... .......... 73% 28.6M 2s 2024-04-20T10:35:47.574 INFO:tasks.workunit.client.0.smithi042.stderr: 99350K .......... .......... .......... .......... .......... 73% 26.1M 2s 2024-04-20T10:35:47.576 INFO:tasks.workunit.client.0.smithi042.stderr: 99400K .......... .......... .......... .......... .......... 73% 31.4M 2s 2024-04-20T10:35:47.577 INFO:tasks.workunit.client.0.smithi042.stderr: 99450K .......... .......... .......... .......... .......... 73% 25.8M 2s 2024-04-20T10:35:47.580 INFO:tasks.workunit.client.0.smithi042.stderr: 99500K .......... .......... .......... .......... .......... 73% 25.8M 2s 2024-04-20T10:35:47.580 INFO:tasks.workunit.client.0.smithi042.stderr: 99550K .......... .......... .......... .......... .......... 73% 45.0M 2s 2024-04-20T10:35:47.583 INFO:tasks.workunit.client.0.smithi042.stderr: 99600K .......... .......... .......... .......... .......... 73% 22.2M 2s 2024-04-20T10:35:47.584 INFO:tasks.workunit.client.0.smithi042.stderr: 99650K .......... .......... .......... .......... .......... 73% 26.4M 2s 2024-04-20T10:35:47.586 INFO:tasks.workunit.client.0.smithi042.stderr: 99700K .......... .......... .......... .......... .......... 73% 31.5M 2s 2024-04-20T10:35:47.588 INFO:tasks.workunit.client.0.smithi042.stderr: 99750K .......... .......... .......... .......... .......... 73% 22.7M 2s 2024-04-20T10:35:47.589 INFO:tasks.workunit.client.0.smithi042.stderr: 99800K .......... .......... .......... .......... .......... 73% 38.6M 2s 2024-04-20T10:35:47.591 INFO:tasks.workunit.client.0.smithi042.stderr: 99850K .......... .......... .......... .......... .......... 73% 24.7M 2s 2024-04-20T10:35:47.595 INFO:tasks.workunit.client.0.smithi042.stderr: 99900K .......... .......... .......... .......... .......... 73% 28.5M 2s 2024-04-20T10:35:47.595 INFO:tasks.workunit.client.0.smithi042.stderr: 99950K .......... .......... .......... .......... .......... 73% 33.5M 2s 2024-04-20T10:35:47.596 INFO:tasks.workunit.client.0.smithi042.stderr:100000K .......... .......... .......... .......... .......... 73% 26.1M 2s 2024-04-20T10:35:47.598 INFO:tasks.workunit.client.0.smithi042.stderr:100050K .......... .......... .......... .......... .......... 73% 31.1M 1s 2024-04-20T10:35:47.600 INFO:tasks.workunit.client.0.smithi042.stderr:100100K .......... .......... .......... .......... .......... 73% 21.3M 1s 2024-04-20T10:35:47.602 INFO:tasks.workunit.client.0.smithi042.stderr:100150K .......... .......... .......... .......... .......... 73% 34.7M 1s 2024-04-20T10:35:47.603 INFO:tasks.workunit.client.0.smithi042.stderr:100200K .......... .......... .......... .......... .......... 73% 22.2M 1s 2024-04-20T10:35:47.607 INFO:tasks.workunit.client.0.smithi042.stderr:100250K .......... .......... .......... .......... .......... 73% 25.2M 1s 2024-04-20T10:35:47.607 INFO:tasks.workunit.client.0.smithi042.stderr:100300K .......... .......... .......... .......... .......... 73% 31.9M 1s 2024-04-20T10:35:47.609 INFO:tasks.workunit.client.0.smithi042.stderr:100350K .......... .......... .......... .......... .......... 73% 23.5M 1s 2024-04-20T10:35:47.612 INFO:tasks.workunit.client.0.smithi042.stderr:100400K .......... .......... .......... .......... .......... 73% 27.3M 1s 2024-04-20T10:35:47.614 INFO:tasks.workunit.client.0.smithi042.stderr:100450K .......... .......... .......... .......... .......... 74% 23.4M 1s 2024-04-20T10:35:47.616 INFO:tasks.workunit.client.0.smithi042.stderr:100500K .......... .......... .......... .......... .......... 74% 25.1M 1s 2024-04-20T10:35:47.616 INFO:tasks.workunit.client.0.smithi042.stderr:100550K .......... .......... .......... .......... .......... 74% 40.4M 1s 2024-04-20T10:35:47.619 INFO:tasks.workunit.client.0.smithi042.stderr:100600K .......... .......... .......... .......... .......... 74% 20.1M 1s 2024-04-20T10:35:47.620 INFO:tasks.workunit.client.0.smithi042.stderr:100650K .......... .......... .......... .......... .......... 74% 32.6M 1s 2024-04-20T10:35:47.622 INFO:tasks.workunit.client.0.smithi042.stderr:100700K .......... .......... .......... .......... .......... 74% 24.2M 1s 2024-04-20T10:35:47.625 INFO:tasks.workunit.client.0.smithi042.stderr:100750K .......... .......... .......... .......... .......... 74% 20.4M 1s 2024-04-20T10:35:47.626 INFO:tasks.workunit.client.0.smithi042.stderr:100800K .......... .......... .......... .......... .......... 74% 44.2M 1s 2024-04-20T10:35:47.628 INFO:tasks.workunit.client.0.smithi042.stderr:100850K .......... .......... .......... .......... .......... 74% 24.7M 1s 2024-04-20T10:35:47.629 INFO:tasks.workunit.client.0.smithi042.stderr:100900K .......... .......... .......... .......... .......... 74% 30.3M 1s 2024-04-20T10:35:47.631 INFO:tasks.workunit.client.0.smithi042.stderr:100950K .......... .......... .......... .......... .......... 74% 26.5M 1s 2024-04-20T10:35:47.633 INFO:tasks.workunit.client.0.smithi042.stderr:101000K .......... .......... .......... .......... .......... 74% 24.2M 1s 2024-04-20T10:35:47.634 INFO:tasks.workunit.client.0.smithi042.stderr:101050K .......... .......... .......... .......... .......... 74% 43.5M 1s 2024-04-20T10:35:47.636 INFO:tasks.workunit.client.0.smithi042.stderr:101100K .......... .......... .......... .......... .......... 74% 24.2M 1s 2024-04-20T10:35:47.637 INFO:tasks.workunit.client.0.smithi042.stderr:101150K .......... .......... .......... .......... .......... 74% 35.1M 1s 2024-04-20T10:35:47.640 INFO:tasks.workunit.client.0.smithi042.stderr:101200K .......... .......... .......... .......... .......... 74% 24.7M 1s 2024-04-20T10:35:47.643 INFO:tasks.workunit.client.0.smithi042.stderr:101250K .......... .......... .......... .......... .......... 74% 23.6M 1s 2024-04-20T10:35:47.643 INFO:tasks.workunit.client.0.smithi042.stderr:101300K .......... .......... .......... .......... .......... 74% 49.7M 1s 2024-04-20T10:35:47.645 INFO:tasks.workunit.client.0.smithi042.stderr:101350K .......... .......... .......... .......... .......... 74% 25.4M 1s 2024-04-20T10:35:47.647 INFO:tasks.workunit.client.0.smithi042.stderr:101400K .......... .......... .......... .......... .......... 74% 25.0M 1s 2024-04-20T10:35:47.648 INFO:tasks.workunit.client.0.smithi042.stderr:101450K .......... .......... .......... .......... .......... 74% 31.5M 1s 2024-04-20T10:35:47.650 INFO:tasks.workunit.client.0.smithi042.stderr:101500K .......... .......... .......... .......... .......... 74% 23.4M 1s 2024-04-20T10:35:47.651 INFO:tasks.workunit.client.0.smithi042.stderr:101550K .......... .......... .......... .......... .......... 74% 40.3M 1s 2024-04-20T10:35:47.653 INFO:tasks.workunit.client.0.smithi042.stderr:101600K .......... .......... .......... .......... .......... 74% 27.1M 1s 2024-04-20T10:35:47.655 INFO:tasks.workunit.client.0.smithi042.stderr:101650K .......... .......... .......... .......... .......... 74% 25.6M 1s 2024-04-20T10:35:47.657 INFO:tasks.workunit.client.0.smithi042.stderr:101700K .......... .......... .......... .......... .......... 74% 32.4M 1s 2024-04-20T10:35:47.659 INFO:tasks.workunit.client.0.smithi042.stderr:101750K .......... .......... .......... .......... .......... 74% 24.7M 1s 2024-04-20T10:35:47.659 INFO:tasks.workunit.client.0.smithi042.stderr:101800K .......... .......... .......... .......... .......... 75% 65.8M 1s 2024-04-20T10:35:47.662 INFO:tasks.workunit.client.0.smithi042.stderr:101850K .......... .......... .......... .......... .......... 75% 21.3M 1s 2024-04-20T10:35:47.663 INFO:tasks.workunit.client.0.smithi042.stderr:101900K .......... .......... .......... .......... .......... 75% 26.0M 1s 2024-04-20T10:35:47.665 INFO:tasks.workunit.client.0.smithi042.stderr:101950K .......... .......... .......... .......... .......... 75% 31.1M 1s 2024-04-20T10:35:47.668 INFO:tasks.workunit.client.0.smithi042.stderr:102000K .......... .......... .......... .......... .......... 75% 24.9M 1s 2024-04-20T10:35:47.668 INFO:tasks.workunit.client.0.smithi042.stderr:102050K .......... .......... .......... .......... .......... 75% 49.5M 1s 2024-04-20T10:35:47.670 INFO:tasks.workunit.client.0.smithi042.stderr:102100K .......... .......... .......... .......... .......... 75% 24.4M 1s 2024-04-20T10:35:47.672 INFO:tasks.workunit.client.0.smithi042.stderr:102150K .......... .......... .......... .......... .......... 75% 24.4M 1s 2024-04-20T10:35:47.674 INFO:tasks.workunit.client.0.smithi042.stderr:102200K .......... .......... .......... .......... .......... 75% 34.9M 1s 2024-04-20T10:35:47.675 INFO:tasks.workunit.client.0.smithi042.stderr:102250K .......... .......... .......... .......... .......... 75% 26.6M 1s 2024-04-20T10:35:47.676 INFO:tasks.workunit.client.0.smithi042.stderr:102300K .......... .......... .......... .......... .......... 75% 34.4M 1s 2024-04-20T10:35:47.678 INFO:tasks.workunit.client.0.smithi042.stderr:102350K .......... .......... .......... .......... .......... 75% 27.4M 1s 2024-04-20T10:35:47.681 INFO:tasks.workunit.client.0.smithi042.stderr:102400K .......... .......... .......... .......... .......... 75% 24.9M 1s 2024-04-20T10:35:47.683 INFO:tasks.workunit.client.0.smithi042.stderr:102450K .......... .......... .......... .......... .......... 75% 28.5M 1s 2024-04-20T10:35:47.684 INFO:tasks.workunit.client.0.smithi042.stderr:102500K .......... .......... .......... .......... .......... 75% 25.5M 1s 2024-04-20T10:35:47.685 INFO:tasks.workunit.client.0.smithi042.stderr:102550K .......... .......... .......... .......... .......... 75% 41.1M 1s 2024-04-20T10:35:47.687 INFO:tasks.workunit.client.0.smithi042.stderr:102600K .......... .......... .......... .......... .......... 75% 27.3M 1s 2024-04-20T10:35:47.690 INFO:tasks.workunit.client.0.smithi042.stderr:102650K .......... .......... .......... .......... .......... 75% 23.8M 1s 2024-04-20T10:35:47.691 INFO:tasks.workunit.client.0.smithi042.stderr:102700K .......... .......... .......... .......... .......... 75% 32.4M 1s 2024-04-20T10:35:47.693 INFO:tasks.workunit.client.0.smithi042.stderr:102750K .......... .......... .......... .......... .......... 75% 26.0M 1s 2024-04-20T10:35:47.695 INFO:tasks.workunit.client.0.smithi042.stderr:102800K .......... .......... .......... .......... .......... 75% 38.9M 1s 2024-04-20T10:35:47.696 INFO:tasks.workunit.client.0.smithi042.stderr:102850K .......... .......... .......... .......... .......... 75% 26.2M 1s 2024-04-20T10:35:47.698 INFO:tasks.workunit.client.0.smithi042.stderr:102900K .......... .......... .......... .......... .......... 75% 23.2M 1s 2024-04-20T10:35:47.699 INFO:tasks.workunit.client.0.smithi042.stderr:102950K .......... .......... .......... .......... .......... 75% 33.8M 1s 2024-04-20T10:35:47.701 INFO:tasks.workunit.client.0.smithi042.stderr:103000K .......... .......... .......... .......... .......... 75% 26.1M 1s 2024-04-20T10:35:47.703 INFO:tasks.workunit.client.0.smithi042.stderr:103050K .......... .......... .......... .......... .......... 75% 22.7M 1s 2024-04-20T10:35:47.705 INFO:tasks.workunit.client.0.smithi042.stderr:103100K .......... .......... .......... .......... .......... 75% 37.6M 1s 2024-04-20T10:35:47.706 INFO:tasks.workunit.client.0.smithi042.stderr:103150K .......... .......... .......... .......... .......... 76% 28.5M 1s 2024-04-20T10:35:47.708 INFO:tasks.workunit.client.0.smithi042.stderr:103200K .......... .......... .......... .......... .......... 76% 31.8M 1s 2024-04-20T10:35:47.710 INFO:tasks.workunit.client.0.smithi042.stderr:103250K .......... .......... .......... .......... .......... 76% 24.6M 1s 2024-04-20T10:35:47.712 INFO:tasks.workunit.client.0.smithi042.stderr:103300K .......... .......... .......... .......... .......... 76% 25.1M 1s 2024-04-20T10:35:47.713 INFO:tasks.workunit.client.0.smithi042.stderr:103350K .......... .......... .......... .......... .......... 76% 34.4M 1s 2024-04-20T10:35:47.715 INFO:tasks.workunit.client.0.smithi042.stderr:103400K .......... .......... .......... .......... .......... 76% 27.9M 1s 2024-04-20T10:35:47.716 INFO:tasks.workunit.client.0.smithi042.stderr:103450K .......... .......... .......... .......... .......... 76% 32.3M 1s 2024-04-20T10:35:47.718 INFO:tasks.workunit.client.0.smithi042.stderr:103500K .......... .......... .......... .......... .......... 76% 25.6M 1s 2024-04-20T10:35:47.720 INFO:tasks.workunit.client.0.smithi042.stderr:103550K .......... .......... .......... .......... .......... 76% 26.9M 1s 2024-04-20T10:35:47.721 INFO:tasks.workunit.client.0.smithi042.stderr:103600K .......... .......... .......... .......... .......... 76% 39.4M 1s 2024-04-20T10:35:47.723 INFO:tasks.workunit.client.0.smithi042.stderr:103650K .......... .......... .......... .......... .......... 76% 25.4M 1s 2024-04-20T10:35:47.725 INFO:tasks.workunit.client.0.smithi042.stderr:103700K .......... .......... .......... .......... .......... 76% 36.2M 1s 2024-04-20T10:35:47.727 INFO:tasks.workunit.client.0.smithi042.stderr:103750K .......... .......... .......... .......... .......... 76% 25.8M 1s 2024-04-20T10:35:47.729 INFO:tasks.workunit.client.0.smithi042.stderr:103800K .......... .......... .......... .......... .......... 76% 26.1M 1s 2024-04-20T10:35:47.730 INFO:tasks.workunit.client.0.smithi042.stderr:103850K .......... .......... .......... .......... .......... 76% 36.0M 1s 2024-04-20T10:35:47.732 INFO:tasks.workunit.client.0.smithi042.stderr:103900K .......... .......... .......... .......... .......... 76% 26.8M 1s 2024-04-20T10:35:47.733 INFO:tasks.workunit.client.0.smithi042.stderr:103950K .......... .......... .......... .......... .......... 76% 32.2M 1s 2024-04-20T10:35:47.735 INFO:tasks.workunit.client.0.smithi042.stderr:104000K .......... .......... .......... .......... .......... 76% 24.8M 1s 2024-04-20T10:35:47.737 INFO:tasks.workunit.client.0.smithi042.stderr:104050K .......... .......... .......... .......... .......... 76% 22.6M 1s 2024-04-20T10:35:47.738 INFO:tasks.workunit.client.0.smithi042.stderr:104100K .......... .......... .......... .......... .......... 76% 39.7M 1s 2024-04-20T10:35:47.740 INFO:tasks.workunit.client.0.smithi042.stderr:104150K .......... .......... .......... .......... .......... 76% 22.7M 1s 2024-04-20T10:35:47.742 INFO:tasks.workunit.client.0.smithi042.stderr:104200K .......... .......... .......... .......... .......... 76% 38.7M 1s 2024-04-20T10:35:47.744 INFO:tasks.workunit.client.0.smithi042.stderr:104250K .......... .......... .......... .......... .......... 76% 25.2M 1s 2024-04-20T10:35:47.746 INFO:tasks.workunit.client.0.smithi042.stderr:104300K .......... .......... .......... .......... .......... 76% 25.8M 1s 2024-04-20T10:35:47.747 INFO:tasks.workunit.client.0.smithi042.stderr:104350K .......... .......... .......... .......... .......... 76% 40.1M 1s 2024-04-20T10:35:47.749 INFO:tasks.workunit.client.0.smithi042.stderr:104400K .......... .......... .......... .......... .......... 76% 25.1M 1s 2024-04-20T10:35:47.751 INFO:tasks.workunit.client.0.smithi042.stderr:104450K .......... .......... .......... .......... .......... 76% 31.3M 1s 2024-04-20T10:35:47.752 INFO:tasks.workunit.client.0.smithi042.stderr:104500K .......... .......... .......... .......... .......... 77% 26.0M 1s 2024-04-20T10:35:47.755 INFO:tasks.workunit.client.0.smithi042.stderr:104550K .......... .......... .......... .......... .......... 77% 26.5M 1s 2024-04-20T10:35:47.755 INFO:tasks.workunit.client.0.smithi042.stderr:104600K .......... .......... .......... .......... .......... 77% 42.0M 1s 2024-04-20T10:35:47.757 INFO:tasks.workunit.client.0.smithi042.stderr:104650K .......... .......... .......... .......... .......... 77% 24.0M 1s 2024-04-20T10:35:47.759 INFO:tasks.workunit.client.0.smithi042.stderr:104700K .......... .......... .......... .......... .......... 77% 30.2M 1s 2024-04-20T10:35:47.762 INFO:tasks.workunit.client.0.smithi042.stderr:104750K .......... .......... .......... .......... .......... 77% 22.4M 1s 2024-04-20T10:35:47.763 INFO:tasks.workunit.client.0.smithi042.stderr:104800K .......... .......... .......... .......... .......... 77% 27.3M 1s 2024-04-20T10:35:47.764 INFO:tasks.workunit.client.0.smithi042.stderr:104850K .......... .......... .......... .......... .......... 77% 39.2M 1s 2024-04-20T10:35:47.766 INFO:tasks.workunit.client.0.smithi042.stderr:104900K .......... .......... .......... .......... .......... 77% 25.8M 1s 2024-04-20T10:35:47.767 INFO:tasks.workunit.client.0.smithi042.stderr:104950K .......... .......... .......... .......... .......... 77% 32.7M 1s 2024-04-20T10:35:47.769 INFO:tasks.workunit.client.0.smithi042.stderr:105000K .......... .......... .......... .......... .......... 77% 25.2M 1s 2024-04-20T10:35:47.772 INFO:tasks.workunit.client.0.smithi042.stderr:105050K .......... .......... .......... .......... .......... 77% 23.3M 1s 2024-04-20T10:35:47.773 INFO:tasks.workunit.client.0.smithi042.stderr:105100K .......... .......... .......... .......... .......... 77% 46.6M 1s 2024-04-20T10:35:47.775 INFO:tasks.workunit.client.0.smithi042.stderr:105150K .......... .......... .......... .......... .......... 77% 21.0M 1s 2024-04-20T10:35:47.777 INFO:tasks.workunit.client.0.smithi042.stderr:105200K .......... .......... .......... .......... .......... 77% 32.6M 1s 2024-04-20T10:35:47.778 INFO:tasks.workunit.client.0.smithi042.stderr:105250K .......... .......... .......... .......... .......... 77% 25.2M 1s 2024-04-20T10:35:47.781 INFO:tasks.workunit.client.0.smithi042.stderr:105300K .......... .......... .......... .......... .......... 77% 23.6M 1s 2024-04-20T10:35:47.782 INFO:tasks.workunit.client.0.smithi042.stderr:105350K .......... .......... .......... .......... .......... 77% 43.1M 1s 2024-04-20T10:35:47.784 INFO:tasks.workunit.client.0.smithi042.stderr:105400K .......... .......... .......... .......... .......... 77% 23.3M 1s 2024-04-20T10:35:47.786 INFO:tasks.workunit.client.0.smithi042.stderr:105450K .......... .......... .......... .......... .......... 77% 23.7M 1s 2024-04-20T10:35:47.788 INFO:tasks.workunit.client.0.smithi042.stderr:105500K .......... .......... .......... .......... .......... 77% 26.2M 1s 2024-04-20T10:35:47.790 INFO:tasks.workunit.client.0.smithi042.stderr:105550K .......... .......... .......... .......... .......... 77% 26.0M 1s 2024-04-20T10:35:47.791 INFO:tasks.workunit.client.0.smithi042.stderr:105600K .......... .......... .......... .......... .......... 77% 41.7M 1s 2024-04-20T10:35:47.793 INFO:tasks.workunit.client.0.smithi042.stderr:105650K .......... .......... .......... .......... .......... 77% 20.7M 1s 2024-04-20T10:35:47.795 INFO:tasks.workunit.client.0.smithi042.stderr:105700K .......... .......... .......... .......... .......... 77% 34.7M 1s 2024-04-20T10:35:47.799 INFO:tasks.workunit.client.0.smithi042.stderr:105750K .......... .......... .......... .......... .......... 77% 20.1M 1s 2024-04-20T10:35:47.799 INFO:tasks.workunit.client.0.smithi042.stderr:105800K .......... .......... .......... .......... .......... 77% 27.8M 1s 2024-04-20T10:35:47.800 INFO:tasks.workunit.client.0.smithi042.stderr:105850K .......... .......... .......... .......... .......... 78% 29.7M 1s 2024-04-20T10:35:47.803 INFO:tasks.workunit.client.0.smithi042.stderr:105900K .......... .......... .......... .......... .......... 78% 22.0M 1s 2024-04-20T10:35:47.805 INFO:tasks.workunit.client.0.smithi042.stderr:105950K .......... .......... .......... .......... .......... 78% 31.1M 1s 2024-04-20T10:35:47.806 INFO:tasks.workunit.client.0.smithi042.stderr:106000K .......... .......... .......... .......... .......... 78% 24.8M 1s 2024-04-20T10:35:47.808 INFO:tasks.workunit.client.0.smithi042.stderr:106050K .......... .......... .......... .......... .......... 78% 25.9M 1s 2024-04-20T10:35:47.809 INFO:tasks.workunit.client.0.smithi042.stderr:106100K .......... .......... .......... .......... .......... 78% 39.9M 1s 2024-04-20T10:35:47.811 INFO:tasks.workunit.client.0.smithi042.stderr:106150K .......... .......... .......... .......... .......... 78% 24.6M 1s 2024-04-20T10:35:47.814 INFO:tasks.workunit.client.0.smithi042.stderr:106200K .......... .......... .......... .......... .......... 78% 32.3M 1s 2024-04-20T10:35:47.815 INFO:tasks.workunit.client.0.smithi042.stderr:106250K .......... .......... .......... .......... .......... 78% 22.5M 1s 2024-04-20T10:35:47.817 INFO:tasks.workunit.client.0.smithi042.stderr:106300K .......... .......... .......... .......... .......... 78% 27.1M 1s 2024-04-20T10:35:47.818 INFO:tasks.workunit.client.0.smithi042.stderr:106350K .......... .......... .......... .......... .......... 78% 40.7M 1s 2024-04-20T10:35:47.820 INFO:tasks.workunit.client.0.smithi042.stderr:106400K .......... .......... .......... .......... .......... 78% 25.6M 1s 2024-04-20T10:35:47.821 INFO:tasks.workunit.client.0.smithi042.stderr:106450K .......... .......... .......... .......... .......... 78% 29.2M 1s 2024-04-20T10:35:47.823 INFO:tasks.workunit.client.0.smithi042.stderr:106500K .......... .......... .......... .......... .......... 78% 25.1M 1s 2024-04-20T10:35:47.825 INFO:tasks.workunit.client.0.smithi042.stderr:106550K .......... .......... .......... .......... .......... 78% 25.4M 1s 2024-04-20T10:35:47.826 INFO:tasks.workunit.client.0.smithi042.stderr:106600K .......... .......... .......... .......... .......... 78% 38.1M 1s 2024-04-20T10:35:47.828 INFO:tasks.workunit.client.0.smithi042.stderr:106650K .......... .......... .......... .......... .......... 78% 24.9M 1s 2024-04-20T10:35:47.831 INFO:tasks.workunit.client.0.smithi042.stderr:106700K .......... .......... .......... .......... .......... 78% 24.0M 1s 2024-04-20T10:35:47.832 INFO:tasks.workunit.client.0.smithi042.stderr:106750K .......... .......... .......... .......... .......... 78% 34.9M 1s 2024-04-20T10:35:47.834 INFO:tasks.workunit.client.0.smithi042.stderr:106800K .......... .......... .......... .......... .......... 78% 24.7M 1s 2024-04-20T10:35:47.835 INFO:tasks.workunit.client.0.smithi042.stderr:106850K .......... .......... .......... .......... .......... 78% 43.0M 1s 2024-04-20T10:35:47.837 INFO:tasks.workunit.client.0.smithi042.stderr:106900K .......... .......... .......... .......... .......... 78% 25.9M 1s 2024-04-20T10:35:47.839 INFO:tasks.workunit.client.0.smithi042.stderr:106950K .......... .......... .......... .......... .......... 78% 24.8M 1s 2024-04-20T10:35:47.840 INFO:tasks.workunit.client.0.smithi042.stderr:107000K .......... .......... .......... .......... .......... 78% 33.8M 1s 2024-04-20T10:35:47.842 INFO:tasks.workunit.client.0.smithi042.stderr:107050K .......... .......... .......... .......... .......... 78% 23.3M 1s 2024-04-20T10:35:47.843 INFO:tasks.workunit.client.0.smithi042.stderr:107100K .......... .......... .......... .......... .......... 78% 53.3M 1s 2024-04-20T10:35:47.845 INFO:tasks.workunit.client.0.smithi042.stderr:107150K .......... .......... .......... .......... .......... 78% 23.7M 1s 2024-04-20T10:35:47.847 INFO:tasks.workunit.client.0.smithi042.stderr:107200K .......... .......... .......... .......... .......... 79% 24.5M 1s 2024-04-20T10:35:47.849 INFO:tasks.workunit.client.0.smithi042.stderr:107250K .......... .......... .......... .......... .......... 79% 32.6M 1s 2024-04-20T10:35:47.851 INFO:tasks.workunit.client.0.smithi042.stderr:107300K .......... .......... .......... .......... .......... 79% 23.9M 1s 2024-04-20T10:35:47.852 INFO:tasks.workunit.client.0.smithi042.stderr:107350K .......... .......... .......... .......... .......... 79% 44.5M 1s 2024-04-20T10:35:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:47 smithi042 ceph-mon[17078]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-04-20T10:35:47.854 INFO:tasks.workunit.client.0.smithi042.stderr:107400K .......... .......... .......... .......... .......... 79% 25.6M 1s 2024-04-20T10:35:47.856 INFO:tasks.workunit.client.0.smithi042.stderr:107450K .......... .......... .......... .......... .......... 79% 25.8M 1s 2024-04-20T10:35:47.857 INFO:tasks.workunit.client.0.smithi042.stderr:107500K .......... .......... .......... .......... .......... 79% 34.3M 1s 2024-04-20T10:35:47.888 INFO:tasks.workunit.client.0.smithi042.stderr:107550K .......... .......... .......... .......... .......... 79% 1.95M 1s 2024-04-20T10:35:47.889 INFO:tasks.workunit.client.0.smithi042.stderr:107600K .......... .......... .......... .......... .......... 79% 65.9M 1s 2024-04-20T10:35:47.889 INFO:tasks.workunit.client.0.smithi042.stderr:107650K .......... .......... .......... .......... .......... 79% 64.1M 1s 2024-04-20T10:35:47.889 INFO:tasks.workunit.client.0.smithi042.stderr:107700K .......... .......... .......... .......... .......... 79% 70.1M 1s 2024-04-20T10:35:47.889 INFO:tasks.workunit.client.0.smithi042.stderr:107750K .......... .......... .......... .......... .......... 79% 51.9M 1s 2024-04-20T10:35:47.889 INFO:tasks.workunit.client.0.smithi042.stderr:107800K .......... .......... .......... .......... .......... 79% 73.3M 1s 2024-04-20T10:35:47.889 INFO:tasks.workunit.client.0.smithi042.stderr:107850K .......... .......... .......... .......... .......... 79% 77.1M 1s 2024-04-20T10:35:47.889 INFO:tasks.workunit.client.0.smithi042.stderr:107900K .......... .......... .......... .......... .......... 79% 73.3M 1s 2024-04-20T10:35:47.896 INFO:tasks.workunit.client.0.smithi042.stderr:107950K .......... .......... .......... .......... .......... 79% 7.62M 1s 2024-04-20T10:35:47.896 INFO:tasks.workunit.client.0.smithi042.stderr:108000K .......... .......... .......... .......... .......... 79% 63.4M 1s 2024-04-20T10:35:47.896 INFO:tasks.workunit.client.0.smithi042.stderr:108050K .......... .......... .......... .......... .......... 79% 56.1M 1s 2024-04-20T10:35:47.903 INFO:tasks.workunit.client.0.smithi042.stderr:108100K .......... .......... .......... .......... .......... 79% 14.9M 1s 2024-04-20T10:35:47.903 INFO:tasks.workunit.client.0.smithi042.stderr:108150K .......... .......... .......... .......... .......... 79% 53.5M 1s 2024-04-20T10:35:47.903 INFO:tasks.workunit.client.0.smithi042.stderr:108200K .......... .......... .......... .......... .......... 79% 72.2M 1s 2024-04-20T10:35:47.903 INFO:tasks.workunit.client.0.smithi042.stderr:108250K .......... .......... .......... .......... .......... 79% 48.1M 1s 2024-04-20T10:35:47.903 INFO:tasks.workunit.client.0.smithi042.stderr:108300K .......... .......... .......... .......... .......... 79% 39.8M 1s 2024-04-20T10:35:47.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:47 smithi162 ceph-mon[21854]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-04-20T10:35:47.913 INFO:tasks.workunit.client.0.smithi042.stderr:108350K .......... .......... .......... .......... .......... 79% 5.24M 1s 2024-04-20T10:35:47.913 INFO:tasks.workunit.client.0.smithi042.stderr:108400K .......... .......... .......... .......... .......... 79% 71.8M 1s 2024-04-20T10:35:47.926 INFO:tasks.workunit.client.0.smithi042.stderr:108450K .......... .......... .......... .......... .......... 79% 6.62M 1s 2024-04-20T10:35:47.926 INFO:tasks.workunit.client.0.smithi042.stderr:108500K .......... .......... .......... .......... .......... 79% 66.8M 1s 2024-04-20T10:35:47.926 INFO:tasks.workunit.client.0.smithi042.stderr:108550K .......... .......... .......... .......... .......... 79% 134M 1s 2024-04-20T10:35:47.927 INFO:tasks.workunit.client.0.smithi042.stderr:108600K .......... .......... .......... .......... .......... 80% 62.8M 1s 2024-04-20T10:35:47.927 INFO:tasks.workunit.client.0.smithi042.stderr:108650K .......... .......... .......... .......... .......... 80% 72.9M 1s 2024-04-20T10:35:47.927 INFO:tasks.workunit.client.0.smithi042.stderr:108700K .......... .......... .......... .......... .......... 80% 67.9M 1s 2024-04-20T10:35:47.927 INFO:tasks.workunit.client.0.smithi042.stderr:108750K .......... .......... .......... .......... .......... 80% 65.5M 1s 2024-04-20T10:35:47.927 INFO:tasks.workunit.client.0.smithi042.stderr:108800K .......... .......... .......... .......... .......... 80% 74.1M 1s 2024-04-20T10:35:47.927 INFO:tasks.workunit.client.0.smithi042.stderr:108850K .......... .......... .......... .......... .......... 80% 58.4M 1s 2024-04-20T10:35:47.928 INFO:tasks.workunit.client.0.smithi042.stderr:108900K .......... .......... .......... .......... .......... 80% 23.9M 1s 2024-04-20T10:35:47.930 INFO:tasks.workunit.client.0.smithi042.stderr:108950K .......... .......... .......... .......... .......... 80% 25.9M 1s 2024-04-20T10:35:47.930 INFO:tasks.workunit.client.0.smithi042.stderr:109000K .......... .......... .......... .......... .......... 80% 60.8M 1s 2024-04-20T10:35:47.932 INFO:tasks.workunit.client.0.smithi042.stderr:109050K .......... .......... .......... .......... .......... 80% 26.9M 1s 2024-04-20T10:35:47.935 INFO:tasks.workunit.client.0.smithi042.stderr:109100K .......... .......... .......... .......... .......... 80% 21.2M 1s 2024-04-20T10:35:47.937 INFO:tasks.workunit.client.0.smithi042.stderr:109150K .......... .......... .......... .......... .......... 80% 24.8M 1s 2024-04-20T10:35:47.937 INFO:tasks.workunit.client.0.smithi042.stderr:109200K .......... .......... .......... .......... .......... 80% 73.7M 1s 2024-04-20T10:35:47.939 INFO:tasks.workunit.client.0.smithi042.stderr:109250K .......... .......... .......... .......... .......... 80% 24.2M 1s 2024-04-20T10:35:47.941 INFO:tasks.workunit.client.0.smithi042.stderr:109300K .......... .......... .......... .......... .......... 80% 26.9M 1s 2024-04-20T10:35:47.944 INFO:tasks.workunit.client.0.smithi042.stderr:109350K .......... .......... .......... .......... .......... 80% 20.2M 1s 2024-04-20T10:35:47.946 INFO:tasks.workunit.client.0.smithi042.stderr:109400K .......... .......... .......... .......... .......... 80% 26.1M 1s 2024-04-20T10:35:47.946 INFO:tasks.workunit.client.0.smithi042.stderr:109450K .......... .......... .......... .......... .......... 80% 69.4M 1s 2024-04-20T10:35:47.948 INFO:tasks.workunit.client.0.smithi042.stderr:109500K .......... .......... .......... .......... .......... 80% 25.2M 1s 2024-04-20T10:35:47.950 INFO:tasks.workunit.client.0.smithi042.stderr:109550K .......... .......... .......... .......... .......... 80% 24.4M 1s 2024-04-20T10:35:47.952 INFO:tasks.workunit.client.0.smithi042.stderr:109600K .......... .......... .......... .......... .......... 80% 24.0M 1s 2024-04-20T10:35:47.952 INFO:tasks.workunit.client.0.smithi042.stderr:109650K .......... .......... .......... .......... .......... 80% 69.8M 1s 2024-04-20T10:35:47.954 INFO:tasks.workunit.client.0.smithi042.stderr:109700K .......... .......... .......... .......... .......... 80% 25.4M 1s 2024-04-20T10:35:47.957 INFO:tasks.workunit.client.0.smithi042.stderr:109750K .......... .......... .......... .......... .......... 80% 22.8M 1s 2024-04-20T10:35:47.959 INFO:tasks.workunit.client.0.smithi042.stderr:109800K .......... .......... .......... .......... .......... 80% 26.1M 1s 2024-04-20T10:35:47.961 INFO:tasks.workunit.client.0.smithi042.stderr:109850K .......... .......... .......... .......... .......... 80% 27.2M 1s 2024-04-20T10:35:47.961 INFO:tasks.workunit.client.0.smithi042.stderr:109900K .......... .......... .......... .......... .......... 80% 59.5M 1s 2024-04-20T10:35:47.963 INFO:tasks.workunit.client.0.smithi042.stderr:109950K .......... .......... .......... .......... .......... 81% 25.2M 1s 2024-04-20T10:35:47.965 INFO:tasks.workunit.client.0.smithi042.stderr:110000K .......... .......... .......... .......... .......... 81% 23.6M 1s 2024-04-20T10:35:47.968 INFO:tasks.workunit.client.0.smithi042.stderr:110050K .......... .......... .......... .......... .......... 81% 25.4M 1s 2024-04-20T10:35:47.968 INFO:tasks.workunit.client.0.smithi042.stderr:110100K .......... .......... .......... .......... .......... 81% 61.1M 1s 2024-04-20T10:35:47.970 INFO:tasks.workunit.client.0.smithi042.stderr:110150K .......... .......... .......... .......... .......... 81% 24.0M 1s 2024-04-20T10:35:47.973 INFO:tasks.workunit.client.0.smithi042.stderr:110200K .......... .......... .......... .......... .......... 81% 26.7M 1s 2024-04-20T10:35:47.974 INFO:tasks.workunit.client.0.smithi042.stderr:110250K .......... .......... .......... .......... .......... 81% 25.3M 1s 2024-04-20T10:35:47.976 INFO:tasks.workunit.client.0.smithi042.stderr:110300K .......... .......... .......... .......... .......... 81% 24.9M 1s 2024-04-20T10:35:47.976 INFO:tasks.workunit.client.0.smithi042.stderr:110350K .......... .......... .......... .......... .......... 81% 67.7M 1s 2024-04-20T10:35:47.978 INFO:tasks.workunit.client.0.smithi042.stderr:110400K .......... .......... .......... .......... .......... 81% 24.4M 1s 2024-04-20T10:35:47.980 INFO:tasks.workunit.client.0.smithi042.stderr:110450K .......... .......... .......... .......... .......... 81% 27.0M 1s 2024-04-20T10:35:47.983 INFO:tasks.workunit.client.0.smithi042.stderr:110500K .......... .......... .......... .......... .......... 81% 24.5M 1s 2024-04-20T10:35:47.985 INFO:tasks.workunit.client.0.smithi042.stderr:110550K .......... .......... .......... .......... .......... 81% 25.1M 1s 2024-04-20T10:35:47.985 INFO:tasks.workunit.client.0.smithi042.stderr:110600K .......... .......... .......... .......... .......... 81% 64.2M 1s 2024-04-20T10:35:47.987 INFO:tasks.workunit.client.0.smithi042.stderr:110650K .......... .......... .......... .......... .......... 81% 24.5M 1s 2024-04-20T10:35:47.989 INFO:tasks.workunit.client.0.smithi042.stderr:110700K .......... .......... .......... .......... .......... 81% 24.8M 1s 2024-04-20T10:35:47.992 INFO:tasks.workunit.client.0.smithi042.stderr:110750K .......... .......... .......... .......... .......... 81% 21.2M 1s 2024-04-20T10:35:47.992 INFO:tasks.workunit.client.0.smithi042.stderr:110800K .......... .......... .......... .......... .......... 81% 65.7M 1s 2024-04-20T10:35:47.994 INFO:tasks.workunit.client.0.smithi042.stderr:110850K .......... .......... .......... .......... .......... 81% 26.4M 1s 2024-04-20T10:35:47.996 INFO:tasks.workunit.client.0.smithi042.stderr:110900K .......... .......... .......... .......... .......... 81% 24.9M 1s 2024-04-20T10:35:47.998 INFO:tasks.workunit.client.0.smithi042.stderr:110950K .......... .......... .......... .......... .......... 81% 24.5M 1s 2024-04-20T10:35:48.031 INFO:tasks.workunit.client.0.smithi042.stderr:111000K .......... .......... .......... .......... .......... 81% 1.53M 1s 2024-04-20T10:35:48.032 INFO:tasks.workunit.client.0.smithi042.stderr:111050K .......... .......... .......... .......... .......... 81% 65.2M 1s 2024-04-20T10:35:48.032 INFO:tasks.workunit.client.0.smithi042.stderr:111100K .......... .......... .......... .......... .......... 81% 73.0M 1s 2024-04-20T10:35:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:47 smithi046 ceph-mon[21574]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-04-20T10:35:48.066 INFO:tasks.workunit.client.0.smithi042.stderr:111150K .......... .......... .......... .......... .......... 81% 1.45M 1s 2024-04-20T10:35:48.066 INFO:tasks.workunit.client.0.smithi042.stderr:111200K .......... .......... .......... .......... .......... 81% 71.9M 1s 2024-04-20T10:35:48.066 INFO:tasks.workunit.client.0.smithi042.stderr:111250K .......... .......... .......... .......... .......... 81% 76.8M 1s 2024-04-20T10:35:48.076 INFO:tasks.workunit.client.0.smithi042.stderr:111300K .......... .......... .......... .......... .......... 82% 5.78M 1s 2024-04-20T10:35:48.076 INFO:tasks.workunit.client.0.smithi042.stderr:111350K .......... .......... .......... .......... .......... 82% 62.6M 1s 2024-04-20T10:35:48.105 INFO:tasks.workunit.client.0.smithi042.stderr:111400K .......... .......... .......... .......... .......... 82% 2.21M 1s 2024-04-20T10:35:48.105 INFO:tasks.workunit.client.0.smithi042.stderr:111450K .......... .......... .......... .......... .......... 82% 43.6M 1s 2024-04-20T10:35:48.105 INFO:tasks.workunit.client.0.smithi042.stderr:111500K .......... .......... .......... .......... .......... 82% 75.6M 1s 2024-04-20T10:35:48.105 INFO:tasks.workunit.client.0.smithi042.stderr:111550K .......... .......... .......... .......... .......... 82% 51.1M 1s 2024-04-20T10:35:48.105 INFO:tasks.workunit.client.0.smithi042.stderr:111600K .......... .......... .......... .......... .......... 82% 69.0M 1s 2024-04-20T10:35:48.106 INFO:tasks.workunit.client.0.smithi042.stderr:111650K .......... .......... .......... .......... .......... 82% 74.3M 1s 2024-04-20T10:35:48.106 INFO:tasks.workunit.client.0.smithi042.stderr:111700K .......... .......... .......... .......... .......... 82% 58.8M 1s 2024-04-20T10:35:48.106 INFO:tasks.workunit.client.0.smithi042.stderr:111750K .......... .......... .......... .......... .......... 82% 64.6M 1s 2024-04-20T10:35:48.106 INFO:tasks.workunit.client.0.smithi042.stderr:111800K .......... .......... .......... .......... .......... 82% 75.0M 1s 2024-04-20T10:35:48.106 INFO:tasks.workunit.client.0.smithi042.stderr:111850K .......... .......... .......... .......... .......... 82% 53.3M 1s 2024-04-20T10:35:48.107 INFO:tasks.workunit.client.0.smithi042.stderr:111900K .......... .......... .......... .......... .......... 82% 30.0M 1s 2024-04-20T10:35:48.107 INFO:tasks.workunit.client.0.smithi042.stderr:111950K .......... .......... .......... .......... .......... 82% 57.5M 1s 2024-04-20T10:35:48.109 INFO:tasks.workunit.client.0.smithi042.stderr:112000K .......... .......... .......... .......... .......... 82% 25.7M 1s 2024-04-20T10:35:48.111 INFO:tasks.workunit.client.0.smithi042.stderr:112050K .......... .......... .......... .......... .......... 82% 26.3M 1s 2024-04-20T10:35:48.113 INFO:tasks.workunit.client.0.smithi042.stderr:112100K .......... .......... .......... .......... .......... 82% 24.7M 1s 2024-04-20T10:35:48.143 INFO:tasks.workunit.client.0.smithi042.stderr:112150K .......... .......... .......... .......... .......... 82% 2.43M 1s 2024-04-20T10:35:48.143 INFO:tasks.workunit.client.0.smithi042.stderr:112200K .......... .......... .......... .......... .......... 82% 60.9M 1s 2024-04-20T10:35:48.143 INFO:tasks.workunit.client.0.smithi042.stderr:112250K .......... .......... .......... .......... .......... 82% 75.7M 1s 2024-04-20T10:35:48.143 INFO:tasks.workunit.client.0.smithi042.stderr:112300K .......... .......... .......... .......... .......... 82% 66.8M 1s 2024-04-20T10:35:48.143 INFO:tasks.workunit.client.0.smithi042.stderr:112350K .......... .......... .......... .......... .......... 82% 62.7M 1s 2024-04-20T10:35:48.143 INFO:tasks.workunit.client.0.smithi042.stderr:112400K .......... .......... .......... .......... .......... 82% 71.4M 1s 2024-04-20T10:35:48.143 INFO:tasks.workunit.client.0.smithi042.stderr:112450K .......... .......... .......... .......... .......... 82% 75.5M 1s 2024-04-20T10:35:48.144 INFO:tasks.workunit.client.0.smithi042.stderr:112500K .......... .......... .......... .......... .......... 82% 75.9M 1s 2024-04-20T10:35:48.144 INFO:tasks.workunit.client.0.smithi042.stderr:112550K .......... .......... .......... .......... .......... 82% 62.0M 1s 2024-04-20T10:35:48.144 INFO:tasks.workunit.client.0.smithi042.stderr:112600K .......... .......... .......... .......... .......... 82% 25.5M 1s 2024-04-20T10:35:48.144 INFO:tasks.workunit.client.0.smithi042.stderr:112650K .......... .......... .......... .......... .......... 83% 65.8M 1s 2024-04-20T10:35:48.144 INFO:tasks.workunit.client.0.smithi042.stderr:112700K .......... .......... .......... .......... .......... 83% 24.9M 1s 2024-04-20T10:35:48.145 INFO:tasks.workunit.client.0.smithi042.stderr:112750K .......... .......... .......... .......... .......... 83% 23.5M 1s 2024-04-20T10:35:48.155 INFO:tasks.workunit.client.0.smithi042.stderr:112800K .......... .......... .......... .......... .......... 83% 7.84M 1s 2024-04-20T10:35:48.155 INFO:tasks.workunit.client.0.smithi042.stderr:112850K .......... .......... .......... .......... .......... 83% 67.5M 1s 2024-04-20T10:35:48.155 INFO:tasks.workunit.client.0.smithi042.stderr:112900K .......... .......... .......... .......... .......... 83% 72.7M 1s 2024-04-20T10:35:48.155 INFO:tasks.workunit.client.0.smithi042.stderr:112950K .......... .......... .......... .......... .......... 83% 62.9M 1s 2024-04-20T10:35:48.156 INFO:tasks.workunit.client.0.smithi042.stderr:113000K .......... .......... .......... .......... .......... 83% 70.5M 1s 2024-04-20T10:35:48.157 INFO:tasks.workunit.client.0.smithi042.stderr:113050K .......... .......... .......... .......... .......... 83% 29.0M 1s 2024-04-20T10:35:48.157 INFO:tasks.workunit.client.0.smithi042.stderr:113100K .......... .......... .......... .......... .......... 83% 63.4M 1s 2024-04-20T10:35:48.159 INFO:tasks.workunit.client.0.smithi042.stderr:113150K .......... .......... .......... .......... .......... 83% 25.0M 1s 2024-04-20T10:35:48.161 INFO:tasks.workunit.client.0.smithi042.stderr:113200K .......... .......... .......... .......... .......... 83% 24.2M 1s 2024-04-20T10:35:48.163 INFO:tasks.workunit.client.0.smithi042.stderr:113250K .......... .......... .......... .......... .......... 83% 26.1M 1s 2024-04-20T10:35:48.166 INFO:tasks.workunit.client.0.smithi042.stderr:113300K .......... .......... .......... .......... .......... 83% 62.2M 1s 2024-04-20T10:35:48.166 INFO:tasks.workunit.client.0.smithi042.stderr:113350K .......... .......... .......... .......... .......... 83% 19.2M 1s 2024-04-20T10:35:48.168 INFO:tasks.workunit.client.0.smithi042.stderr:113400K .......... .......... .......... .......... .......... 83% 26.2M 1s 2024-04-20T10:35:48.170 INFO:tasks.workunit.client.0.smithi042.stderr:113450K .......... .......... .......... .......... .......... 83% 24.4M 1s 2024-04-20T10:35:48.173 INFO:tasks.workunit.client.0.smithi042.stderr:113500K .......... .......... .......... .......... .......... 83% 24.3M 1s 2024-04-20T10:35:48.173 INFO:tasks.workunit.client.0.smithi042.stderr:113550K .......... .......... .......... .......... .......... 83% 71.3M 1s 2024-04-20T10:35:48.175 INFO:tasks.workunit.client.0.smithi042.stderr:113600K .......... .......... .......... .......... .......... 83% 23.7M 1s 2024-04-20T10:35:48.177 INFO:tasks.workunit.client.0.smithi042.stderr:113650K .......... .......... .......... .......... .......... 83% 25.5M 1s 2024-04-20T10:35:48.179 INFO:tasks.workunit.client.0.smithi042.stderr:113700K .......... .......... .......... .......... .......... 83% 24.5M 1s 2024-04-20T10:35:48.181 INFO:tasks.workunit.client.0.smithi042.stderr:113750K .......... .......... .......... .......... .......... 83% 25.3M 1s 2024-04-20T10:35:48.181 INFO:tasks.workunit.client.0.smithi042.stderr:113800K .......... .......... .......... .......... .......... 83% 69.5M 1s 2024-04-20T10:35:48.183 INFO:tasks.workunit.client.0.smithi042.stderr:113850K .......... .......... .......... .......... .......... 83% 25.1M 1s 2024-04-20T10:35:48.185 INFO:tasks.workunit.client.0.smithi042.stderr:113900K .......... .......... .......... .......... .......... 83% 24.6M 1s 2024-04-20T10:35:48.188 INFO:tasks.workunit.client.0.smithi042.stderr:113950K .......... .......... .......... .......... .......... 83% 22.2M 1s 2024-04-20T10:35:48.188 INFO:tasks.workunit.client.0.smithi042.stderr:114000K .......... .......... .......... .......... .......... 84% 59.3M 1s 2024-04-20T10:35:48.190 INFO:tasks.workunit.client.0.smithi042.stderr:114050K .......... .......... .......... .......... .......... 84% 27.0M 1s 2024-04-20T10:35:48.192 INFO:tasks.workunit.client.0.smithi042.stderr:114100K .......... .......... .......... .......... .......... 84% 20.6M 1s 2024-04-20T10:35:48.195 INFO:tasks.workunit.client.0.smithi042.stderr:114150K .......... .......... .......... .......... .......... 84% 24.4M 1s 2024-04-20T10:35:48.197 INFO:tasks.workunit.client.0.smithi042.stderr:114200K .......... .......... .......... .......... .......... 84% 27.3M 1s 2024-04-20T10:35:48.197 INFO:tasks.workunit.client.0.smithi042.stderr:114250K .......... .......... .......... .......... .......... 84% 53.5M 1s 2024-04-20T10:35:48.199 INFO:tasks.workunit.client.0.smithi042.stderr:114300K .......... .......... .......... .......... .......... 84% 25.0M 1s 2024-04-20T10:35:48.201 INFO:tasks.workunit.client.0.smithi042.stderr:114350K .......... .......... .......... .......... .......... 84% 25.0M 1s 2024-04-20T10:35:48.203 INFO:tasks.workunit.client.0.smithi042.stderr:114400K .......... .......... .......... .......... .......... 84% 27.2M 1s 2024-04-20T10:35:48.203 INFO:tasks.workunit.client.0.smithi042.stderr:114450K .......... .......... .......... .......... .......... 84% 59.3M 1s 2024-04-20T10:35:48.205 INFO:tasks.workunit.client.0.smithi042.stderr:114500K .......... .......... .......... .......... .......... 84% 25.3M 1s 2024-04-20T10:35:48.208 INFO:tasks.workunit.client.0.smithi042.stderr:114550K .......... .......... .......... .......... .......... 84% 24.6M 1s 2024-04-20T10:35:48.210 INFO:tasks.workunit.client.0.smithi042.stderr:114600K .......... .......... .......... .......... .......... 84% 24.5M 1s 2024-04-20T10:35:48.212 INFO:tasks.workunit.client.0.smithi042.stderr:114650K .......... .......... .......... .......... .......... 84% 28.6M 1s 2024-04-20T10:35:48.213 INFO:tasks.workunit.client.0.smithi042.stderr:114700K .......... .......... .......... .......... .......... 84% 58.2M 1s 2024-04-20T10:35:48.214 INFO:tasks.workunit.client.0.smithi042.stderr:114750K .......... .......... .......... .......... .......... 84% 23.7M 1s 2024-04-20T10:35:48.216 INFO:tasks.workunit.client.0.smithi042.stderr:114800K .......... .......... .......... .......... .......... 84% 26.3M 1s 2024-04-20T10:35:48.219 INFO:tasks.workunit.client.0.smithi042.stderr:114850K .......... .......... .......... .......... .......... 84% 22.9M 1s 2024-04-20T10:35:48.220 INFO:tasks.workunit.client.0.smithi042.stderr:114900K .......... .......... .......... .......... .......... 84% 62.1M 1s 2024-04-20T10:35:48.221 INFO:tasks.workunit.client.0.smithi042.stderr:114950K .......... .......... .......... .......... .......... 84% 24.9M 1s 2024-04-20T10:35:48.223 INFO:tasks.workunit.client.0.smithi042.stderr:115000K .......... .......... .......... .......... .......... 84% 24.9M 1s 2024-04-20T10:35:48.225 INFO:tasks.workunit.client.0.smithi042.stderr:115050K .......... .......... .......... .......... .......... 84% 26.4M 1s 2024-04-20T10:35:48.227 INFO:tasks.workunit.client.0.smithi042.stderr:115100K .......... .......... .......... .......... .......... 84% 25.1M 1s 2024-04-20T10:35:48.227 INFO:tasks.workunit.client.0.smithi042.stderr:115150K .......... .......... .......... .......... .......... 84% 51.5M 1s 2024-04-20T10:35:48.229 INFO:tasks.workunit.client.0.smithi042.stderr:115200K .......... .......... .......... .......... .......... 84% 26.9M 1s 2024-04-20T10:35:48.231 INFO:tasks.workunit.client.0.smithi042.stderr:115250K .......... .......... .......... .......... .......... 84% 24.9M 1s 2024-04-20T10:35:48.233 INFO:tasks.workunit.client.0.smithi042.stderr:115300K .......... .......... .......... .......... .......... 84% 25.1M 1s 2024-04-20T10:35:48.236 INFO:tasks.workunit.client.0.smithi042.stderr:115350K .......... .......... .......... .......... .......... 85% 25.4M 1s 2024-04-20T10:35:48.236 INFO:tasks.workunit.client.0.smithi042.stderr:115400K .......... .......... .......... .......... .......... 85% 57.8M 1s 2024-04-20T10:35:48.238 INFO:tasks.workunit.client.0.smithi042.stderr:115450K .......... .......... .......... .......... .......... 85% 26.9M 1s 2024-04-20T10:35:48.240 INFO:tasks.workunit.client.0.smithi042.stderr:115500K .......... .......... .......... .......... .......... 85% 23.7M 1s 2024-04-20T10:35:48.242 INFO:tasks.workunit.client.0.smithi042.stderr:115550K .......... .......... .......... .......... .......... 85% 26.8M 1s 2024-04-20T10:35:48.242 INFO:tasks.workunit.client.0.smithi042.stderr:115600K .......... .......... .......... .......... .......... 85% 64.6M 1s 2024-04-20T10:35:48.245 INFO:tasks.workunit.client.0.smithi042.stderr:115650K .......... .......... .......... .......... .......... 85% 22.7M 1s 2024-04-20T10:35:48.246 INFO:tasks.workunit.client.0.smithi042.stderr:115700K .......... .......... .......... .......... .......... 85% 28.9M 1s 2024-04-20T10:35:48.249 INFO:tasks.workunit.client.0.smithi042.stderr:115750K .......... .......... .......... .......... .......... 85% 21.4M 1s 2024-04-20T10:35:48.251 INFO:tasks.workunit.client.0.smithi042.stderr:115800K .......... .......... .......... .......... .......... 85% 29.0M 1s 2024-04-20T10:35:48.251 INFO:tasks.workunit.client.0.smithi042.stderr:115850K .......... .......... .......... .......... .......... 85% 53.6M 1s 2024-04-20T10:35:48.253 INFO:tasks.workunit.client.0.smithi042.stderr:115900K .......... .......... .......... .......... .......... 85% 26.1M 1s 2024-04-20T10:35:48.255 INFO:tasks.workunit.client.0.smithi042.stderr:115950K .......... .......... .......... .......... .......... 85% 24.7M 1s 2024-04-20T10:35:48.258 INFO:tasks.workunit.client.0.smithi042.stderr:116000K .......... .......... .......... .......... .......... 85% 22.0M 1s 2024-04-20T10:35:48.258 INFO:tasks.workunit.client.0.smithi042.stderr:116050K .......... .......... .......... .......... .......... 85% 66.8M 1s 2024-04-20T10:35:48.260 INFO:tasks.workunit.client.0.smithi042.stderr:116100K .......... .......... .......... .......... .......... 85% 25.2M 1s 2024-04-20T10:35:48.262 INFO:tasks.workunit.client.0.smithi042.stderr:116150K .......... .......... .......... .......... .......... 85% 23.1M 1s 2024-04-20T10:35:48.264 INFO:tasks.workunit.client.0.smithi042.stderr:116200K .......... .......... .......... .......... .......... 85% 27.4M 1s 2024-04-20T10:35:48.266 INFO:tasks.workunit.client.0.smithi042.stderr:116250K .......... .......... .......... .......... .......... 85% 26.8M 1s 2024-04-20T10:35:48.266 INFO:tasks.workunit.client.0.smithi042.stderr:116300K .......... .......... .......... .......... .......... 85% 57.0M 1s 2024-04-20T10:35:48.268 INFO:tasks.workunit.client.0.smithi042.stderr:116350K .......... .......... .......... .......... .......... 85% 25.9M 1s 2024-04-20T10:35:48.270 INFO:tasks.workunit.client.0.smithi042.stderr:116400K .......... .......... .......... .......... .......... 85% 26.7M 1s 2024-04-20T10:35:48.273 INFO:tasks.workunit.client.0.smithi042.stderr:116450K .......... .......... .......... .......... .......... 85% 24.2M 1s 2024-04-20T10:35:48.273 INFO:tasks.workunit.client.0.smithi042.stderr:116500K .......... .......... .......... .......... .......... 85% 57.8M 1s 2024-04-20T10:35:48.274 INFO:tasks.workunit.client.0.smithi042.stderr:116550K .......... .......... .......... .......... .......... 85% 36.0M 1s 2024-04-20T10:35:48.276 INFO:tasks.workunit.client.0.smithi042.stderr:116600K .......... .......... .......... .......... .......... 85% 21.7M 1s 2024-04-20T10:35:48.279 INFO:tasks.workunit.client.0.smithi042.stderr:116650K .......... .......... .......... .......... .......... 85% 20.6M 1s 2024-04-20T10:35:48.281 INFO:tasks.workunit.client.0.smithi042.stderr:116700K .......... .......... .......... .......... .......... 86% 26.9M 1s 2024-04-20T10:35:48.281 INFO:tasks.workunit.client.0.smithi042.stderr:116750K .......... .......... .......... .......... .......... 86% 55.8M 1s 2024-04-20T10:35:48.284 INFO:tasks.workunit.client.0.smithi042.stderr:116800K .......... .......... .......... .......... .......... 86% 21.6M 1s 2024-04-20T10:35:48.286 INFO:tasks.workunit.client.0.smithi042.stderr:116850K .......... .......... .......... .......... .......... 86% 26.7M 1s 2024-04-20T10:35:48.288 INFO:tasks.workunit.client.0.smithi042.stderr:116900K .......... .......... .......... .......... .......... 86% 25.4M 1s 2024-04-20T10:35:48.290 INFO:tasks.workunit.client.0.smithi042.stderr:116950K .......... .......... .......... .......... .......... 86% 28.6M 1s 2024-04-20T10:35:48.290 INFO:tasks.workunit.client.0.smithi042.stderr:117000K .......... .......... .......... .......... .......... 86% 61.1M 1s 2024-04-20T10:35:48.293 INFO:tasks.workunit.client.0.smithi042.stderr:117050K .......... .......... .......... .......... .......... 86% 21.6M 1s 2024-04-20T10:35:48.293 INFO:tasks.workunit.client.0.smithi042.stderr:117100K .......... .......... .......... .......... .......... 86% 40.5M 1s 2024-04-20T10:35:48.297 INFO:tasks.workunit.client.0.smithi042.stderr:117150K .......... .......... .......... .......... .......... 86% 17.1M 1s 2024-04-20T10:35:48.297 INFO:tasks.workunit.client.0.smithi042.stderr:117200K .......... .......... .......... .......... .......... 86% 52.8M 1s 2024-04-20T10:35:48.299 INFO:tasks.workunit.client.0.smithi042.stderr:117250K .......... .......... .......... .......... .......... 86% 28.9M 1s 2024-04-20T10:35:48.301 INFO:tasks.workunit.client.0.smithi042.stderr:117300K .......... .......... .......... .......... .......... 86% 24.8M 1s 2024-04-20T10:35:48.302 INFO:tasks.workunit.client.0.smithi042.stderr:117350K .......... .......... .......... .......... .......... 86% 27.8M 1s 2024-04-20T10:35:48.305 INFO:tasks.workunit.client.0.smithi042.stderr:117400K .......... .......... .......... .......... .......... 86% 21.8M 1s 2024-04-20T10:35:48.306 INFO:tasks.workunit.client.0.smithi042.stderr:117450K .......... .......... .......... .......... .......... 86% 59.2M 1s 2024-04-20T10:35:48.308 INFO:tasks.workunit.client.0.smithi042.stderr:117500K .......... .......... .......... .......... .......... 86% 25.3M 1s 2024-04-20T10:35:48.310 INFO:tasks.workunit.client.0.smithi042.stderr:117550K .......... .......... .......... .......... .......... 86% 18.4M 1s 2024-04-20T10:35:48.311 INFO:tasks.workunit.client.0.smithi042.stderr:117600K .......... .......... .......... .......... .......... 86% 38.8M 1s 2024-04-20T10:35:48.313 INFO:tasks.workunit.client.0.smithi042.stderr:117650K .......... .......... .......... .......... .......... 86% 34.7M 1s 2024-04-20T10:35:48.316 INFO:tasks.workunit.client.0.smithi042.stderr:117700K .......... .......... .......... .......... .......... 86% 18.2M 1s 2024-04-20T10:35:48.318 INFO:tasks.workunit.client.0.smithi042.stderr:117750K .......... .......... .......... .......... .......... 86% 29.9M 1s 2024-04-20T10:35:48.319 INFO:tasks.workunit.client.0.smithi042.stderr:117800K .......... .......... .......... .......... .......... 86% 31.8M 1s 2024-04-20T10:35:48.320 INFO:tasks.workunit.client.0.smithi042.stderr:117850K .......... .......... .......... .......... .......... 86% 33.0M 1s 2024-04-20T10:35:48.322 INFO:tasks.workunit.client.0.smithi042.stderr:117900K .......... .......... .......... .......... .......... 86% 33.4M 1s 2024-04-20T10:35:48.324 INFO:tasks.workunit.client.0.smithi042.stderr:117950K .......... .......... .......... .......... .......... 86% 19.8M 1s 2024-04-20T10:35:48.326 INFO:tasks.workunit.client.0.smithi042.stderr:118000K .......... .......... .......... .......... .......... 86% 32.6M 1s 2024-04-20T10:35:48.327 INFO:tasks.workunit.client.0.smithi042.stderr:118050K .......... .......... .......... .......... .......... 86% 29.8M 1s 2024-04-20T10:35:48.329 INFO:tasks.workunit.client.0.smithi042.stderr:118100K .......... .......... .......... .......... .......... 87% 35.7M 1s 2024-04-20T10:35:48.330 INFO:tasks.workunit.client.0.smithi042.stderr:118150K .......... .......... .......... .......... .......... 87% 28.6M 1s 2024-04-20T10:35:48.332 INFO:tasks.workunit.client.0.smithi042.stderr:118200K .......... .......... .......... .......... .......... 87% 32.9M 1s 2024-04-20T10:35:48.334 INFO:tasks.workunit.client.0.smithi042.stderr:118250K .......... .......... .......... .......... .......... 87% 21.5M 1s 2024-04-20T10:35:48.338 INFO:tasks.workunit.client.0.smithi042.stderr:118300K .......... .......... .......... .......... .......... 87% 15.2M 1s 2024-04-20T10:35:48.339 INFO:tasks.workunit.client.0.smithi042.stderr:118350K .......... .......... .......... .......... .......... 87% 34.9M 1s 2024-04-20T10:35:48.341 INFO:tasks.workunit.client.0.smithi042.stderr:118400K .......... .......... .......... .......... .......... 87% 28.2M 1s 2024-04-20T10:35:48.343 INFO:tasks.workunit.client.0.smithi042.stderr:118450K .......... .......... .......... .......... .......... 87% 26.6M 1s 2024-04-20T10:35:48.344 INFO:tasks.workunit.client.0.smithi042.stderr:118500K .......... .......... .......... .......... .......... 87% 26.7M 1s 2024-04-20T10:35:48.346 INFO:tasks.workunit.client.0.smithi042.stderr:118550K .......... .......... .......... .......... .......... 87% 27.0M 1s 2024-04-20T10:35:48.347 INFO:tasks.workunit.client.0.smithi042.stderr:118600K .......... .......... .......... .......... .......... 87% 35.5M 1s 2024-04-20T10:35:48.349 INFO:tasks.workunit.client.0.smithi042.stderr:118650K .......... .......... .......... .......... .......... 87% 33.2M 1s 2024-04-20T10:35:48.351 INFO:tasks.workunit.client.0.smithi042.stderr:118700K .......... .......... .......... .......... .......... 87% 26.6M 1s 2024-04-20T10:35:48.353 INFO:tasks.workunit.client.0.smithi042.stderr:118750K .......... .......... .......... .......... .......... 87% 24.3M 1s 2024-04-20T10:35:48.354 INFO:tasks.workunit.client.0.smithi042.stderr:118800K .......... .......... .......... .......... .......... 87% 58.7M 1s 2024-04-20T10:35:48.358 INFO:tasks.workunit.client.0.smithi042.stderr:118850K .......... .......... .......... .......... .......... 87% 12.7M 1s 2024-04-20T10:35:48.359 INFO:tasks.workunit.client.0.smithi042.stderr:118900K .......... .......... .......... .......... .......... 87% 32.6M 1s 2024-04-20T10:35:48.361 INFO:tasks.workunit.client.0.smithi042.stderr:118950K .......... .......... .......... .......... .......... 87% 28.9M 1s 2024-04-20T10:35:48.362 INFO:tasks.workunit.client.0.smithi042.stderr:119000K .......... .......... .......... .......... .......... 87% 30.1M 1s 2024-04-20T10:35:48.364 INFO:tasks.workunit.client.0.smithi042.stderr:119050K .......... .......... .......... .......... .......... 87% 31.2M 1s 2024-04-20T10:35:48.366 INFO:tasks.workunit.client.0.smithi042.stderr:119100K .......... .......... .......... .......... .......... 87% 21.6M 1s 2024-04-20T10:35:48.368 INFO:tasks.workunit.client.0.smithi042.stderr:119150K .......... .......... .......... .......... .......... 87% 30.8M 1s 2024-04-20T10:35:48.388 INFO:tasks.workunit.client.0.smithi042.stderr:119200K .......... .......... .......... .......... .......... 87% 2.92M 1s 2024-04-20T10:35:48.388 INFO:tasks.workunit.client.0.smithi042.stderr:119250K .......... .......... .......... .......... .......... 87% 63.6M 1s 2024-04-20T10:35:48.389 INFO:tasks.workunit.client.0.smithi042.stderr:119300K .......... .......... .......... .......... .......... 87% 64.1M 1s 2024-04-20T10:35:48.389 INFO:tasks.workunit.client.0.smithi042.stderr:119350K .......... .......... .......... .......... .......... 87% 54.7M 1s 2024-04-20T10:35:48.389 INFO:tasks.workunit.client.0.smithi042.stderr:119400K .......... .......... .......... .......... .......... 87% 66.5M 1s 2024-04-20T10:35:48.410 INFO:tasks.workunit.client.0.smithi042.stderr:119450K .......... .......... .......... .......... .......... 88% 3.89M 1s 2024-04-20T10:35:48.410 INFO:tasks.workunit.client.0.smithi042.stderr:119500K .......... .......... .......... .......... .......... 88% 50.2M 1s 2024-04-20T10:35:48.411 INFO:tasks.workunit.client.0.smithi042.stderr:119550K .......... .......... .......... .......... .......... 88% 41.8M 1s 2024-04-20T10:35:48.411 INFO:tasks.workunit.client.0.smithi042.stderr:119600K .......... .......... .......... .......... .......... 88% 55.4M 1s 2024-04-20T10:35:48.411 INFO:tasks.workunit.client.0.smithi042.stderr:119650K .......... .......... .......... .......... .......... 88% 66.2M 1s 2024-04-20T10:35:48.411 INFO:tasks.workunit.client.0.smithi042.stderr:119700K .......... .......... .......... .......... .......... 88% 60.0M 1s 2024-04-20T10:35:48.411 INFO:tasks.workunit.client.0.smithi042.stderr:119750K .......... .......... .......... .......... .......... 88% 47.5M 1s 2024-04-20T10:35:48.411 INFO:tasks.workunit.client.0.smithi042.stderr:119800K .......... .......... .......... .......... .......... 88% 52.2M 1s 2024-04-20T10:35:48.411 INFO:tasks.workunit.client.0.smithi042.stderr:119850K .......... .......... .......... .......... .......... 88% 45.8M 1s 2024-04-20T10:35:48.411 INFO:tasks.workunit.client.0.smithi042.stderr:119900K .......... .......... .......... .......... .......... 88% 59.8M 1s 2024-04-20T10:35:48.411 INFO:tasks.workunit.client.0.smithi042.stderr:119950K .......... .......... .......... .......... .......... 88% 52.2M 1s 2024-04-20T10:35:48.411 INFO:tasks.workunit.client.0.smithi042.stderr:120000K .......... .......... .......... .......... .......... 88% 59.1M 1s 2024-04-20T10:35:48.413 INFO:tasks.workunit.client.0.smithi042.stderr:120050K .......... .......... .......... .......... .......... 88% 31.2M 1s 2024-04-20T10:35:48.414 INFO:tasks.workunit.client.0.smithi042.stderr:120100K .......... .......... .......... .......... .......... 88% 24.1M 1s 2024-04-20T10:35:48.418 INFO:tasks.workunit.client.0.smithi042.stderr:120150K .......... .......... .......... .......... .......... 88% 26.9M 1s 2024-04-20T10:35:48.418 INFO:tasks.workunit.client.0.smithi042.stderr:120200K .......... .......... .......... .......... .......... 88% 61.1M 1s 2024-04-20T10:35:48.421 INFO:tasks.workunit.client.0.smithi042.stderr:120250K .......... .......... .......... .......... .......... 88% 16.0M 1s 2024-04-20T10:35:48.421 INFO:tasks.workunit.client.0.smithi042.stderr:120300K .......... .......... .......... .......... .......... 88% 60.5M 1s 2024-04-20T10:35:48.423 INFO:tasks.workunit.client.0.smithi042.stderr:120350K .......... .......... .......... .......... .......... 88% 24.1M 1s 2024-04-20T10:35:48.423 INFO:tasks.workunit.client.0.smithi042.stderr:120400K .......... .......... .......... .......... .......... 88% 64.6M 1s 2024-04-20T10:35:48.425 INFO:tasks.workunit.client.0.smithi042.stderr:120450K .......... .......... .......... .......... .......... 88% 25.8M 1s 2024-04-20T10:35:48.427 INFO:tasks.workunit.client.0.smithi042.stderr:120500K .......... .......... .......... .......... .......... 88% 25.3M 1s 2024-04-20T10:35:48.429 INFO:tasks.workunit.client.0.smithi042.stderr:120550K .......... .......... .......... .......... .......... 88% 24.3M 1s 2024-04-20T10:35:48.431 INFO:tasks.workunit.client.0.smithi042.stderr:120600K .......... .......... .......... .......... .......... 88% 27.3M 1s 2024-04-20T10:35:48.431 INFO:tasks.workunit.client.0.smithi042.stderr:120650K .......... .......... .......... .......... .......... 88% 55.7M 1s 2024-04-20T10:35:48.433 INFO:tasks.workunit.client.0.smithi042.stderr:120700K .......... .......... .......... .......... .......... 88% 23.3M 1s 2024-04-20T10:35:48.464 INFO:tasks.workunit.client.0.smithi042.stderr:120750K .......... .......... .......... .......... .......... 88% 1.75M 1s 2024-04-20T10:35:48.464 INFO:tasks.workunit.client.0.smithi042.stderr:120800K .......... .......... .......... .......... .......... 89% 58.2M 1s 2024-04-20T10:35:48.464 INFO:tasks.workunit.client.0.smithi042.stderr:120850K .......... .......... .......... .......... .......... 89% 66.3M 1s 2024-04-20T10:35:48.465 INFO:tasks.workunit.client.0.smithi042.stderr:120900K .......... .......... .......... .......... .......... 89% 64.7M 1s 2024-04-20T10:35:48.465 INFO:tasks.workunit.client.0.smithi042.stderr:120950K .......... .......... .......... .......... .......... 89% 62.8M 1s 2024-04-20T10:35:48.544 INFO:tasks.workunit.client.0.smithi042.stderr:121000K .......... .......... .......... .......... .......... 89% 677K 1s 2024-04-20T10:35:48.544 INFO:tasks.workunit.client.0.smithi042.stderr:121050K .......... .......... .......... .......... .......... 89% 100M 1s 2024-04-20T10:35:48.544 INFO:tasks.workunit.client.0.smithi042.stderr:121100K .......... .......... .......... .......... .......... 89% 103M 1s 2024-04-20T10:35:48.544 INFO:tasks.workunit.client.0.smithi042.stderr:121150K .......... .......... .......... .......... .......... 89% 82.3M 1s 2024-04-20T10:35:48.544 INFO:tasks.workunit.client.0.smithi042.stderr:121200K .......... .......... .......... .......... .......... 89% 102M 1s 2024-04-20T10:35:48.611 INFO:tasks.workunit.client.0.smithi042.stderr:121250K .......... .......... .......... .......... .......... 89% 726K 1s 2024-04-20T10:35:48.611 INFO:tasks.workunit.client.0.smithi042.stderr:121300K .......... .......... .......... .......... .......... 89% 92.9M 1s 2024-04-20T10:35:48.611 INFO:tasks.workunit.client.0.smithi042.stderr:121350K .......... .......... .......... .......... .......... 89% 92.8M 1s 2024-04-20T10:35:48.614 INFO:tasks.workunit.client.0.smithi042.stderr:121400K .......... .......... .......... .......... .......... 89% 17.4M 1s 2024-04-20T10:35:48.614 INFO:tasks.workunit.client.0.smithi042.stderr:121450K .......... .......... .......... .......... .......... 89% 121M 1s 2024-04-20T10:35:48.624 INFO:tasks.workunit.client.0.smithi042.stderr:121500K .......... .......... .......... .......... .......... 89% 5.77M 1s 2024-04-20T10:35:48.624 INFO:tasks.workunit.client.0.smithi042.stderr:121550K .......... .......... .......... .......... .......... 89% 82.3M 1s 2024-04-20T10:35:48.625 INFO:tasks.workunit.client.0.smithi042.stderr:121600K .......... .......... .......... .......... .......... 89% 109M 1s 2024-04-20T10:35:48.625 INFO:tasks.workunit.client.0.smithi042.stderr:121650K .......... .......... .......... .......... .......... 89% 105M 1s 2024-04-20T10:35:48.625 INFO:tasks.workunit.client.0.smithi042.stderr:121700K .......... .......... .......... .......... .......... 89% 105M 1s 2024-04-20T10:35:48.627 INFO:tasks.workunit.client.0.smithi042.stderr:121750K .......... .......... .......... .......... .......... 89% 48.9M 1s 2024-04-20T10:35:48.627 INFO:tasks.workunit.client.0.smithi042.stderr:121800K .......... .......... .......... .......... .......... 89% 73.0M 1s 2024-04-20T10:35:48.627 INFO:tasks.workunit.client.0.smithi042.stderr:121850K .......... .......... .......... .......... .......... 89% 105M 1s 2024-04-20T10:35:48.627 INFO:tasks.workunit.client.0.smithi042.stderr:121900K .......... .......... .......... .......... .......... 89% 119M 1s 2024-04-20T10:35:48.628 INFO:tasks.workunit.client.0.smithi042.stderr:121950K .......... .......... .......... .......... .......... 89% 38.8M 1s 2024-04-20T10:35:48.628 INFO:tasks.workunit.client.0.smithi042.stderr:122000K .......... .......... .......... .......... .......... 89% 102M 1s 2024-04-20T10:35:48.630 INFO:tasks.workunit.client.0.smithi042.stderr:122050K .......... .......... .......... .......... .......... 89% 25.4M 1s 2024-04-20T10:35:48.632 INFO:tasks.workunit.client.0.smithi042.stderr:122100K .......... .......... .......... .......... .......... 89% 24.5M 1s 2024-04-20T10:35:48.634 INFO:tasks.workunit.client.0.smithi042.stderr:122150K .......... .......... .......... .......... .......... 90% 24.5M 1s 2024-04-20T10:35:48.637 INFO:tasks.workunit.client.0.smithi042.stderr:122200K .......... .......... .......... .......... .......... 90% 24.3M 1s 2024-04-20T10:35:48.637 INFO:tasks.workunit.client.0.smithi042.stderr:122250K .......... .......... .......... .......... .......... 90% 94.0M 1s 2024-04-20T10:35:48.640 INFO:tasks.workunit.client.0.smithi042.stderr:122300K .......... .......... .......... .......... .......... 90% 18.8M 1s 2024-04-20T10:35:48.642 INFO:tasks.workunit.client.0.smithi042.stderr:122350K .......... .......... .......... .......... .......... 90% 28.7M 1s 2024-04-20T10:35:48.643 INFO:tasks.workunit.client.0.smithi042.stderr:122400K .......... .......... .......... .......... .......... 90% 28.1M 1s 2024-04-20T10:35:48.645 INFO:tasks.workunit.client.0.smithi042.stderr:122450K .......... .......... .......... .......... .......... 90% 31.8M 1s 2024-04-20T10:35:48.646 INFO:tasks.workunit.client.0.smithi042.stderr:122500K .......... .......... .......... .......... .......... 90% 26.6M 1s 2024-04-20T10:35:48.649 INFO:tasks.workunit.client.0.smithi042.stderr:122550K .......... .......... .......... .......... .......... 90% 23.2M 1s 2024-04-20T10:35:48.649 INFO:tasks.workunit.client.0.smithi042.stderr:122600K .......... .......... .......... .......... .......... 90% 48.6M 1s 2024-04-20T10:35:48.657 INFO:tasks.workunit.client.0.smithi042.stderr:122650K .......... .......... .......... .......... .......... 90% 9.99M 1s 2024-04-20T10:35:48.657 INFO:tasks.workunit.client.0.smithi042.stderr:122700K .......... .......... .......... .......... .......... 90% 82.0M 1s 2024-04-20T10:35:48.657 INFO:tasks.workunit.client.0.smithi042.stderr:122750K .......... .......... .......... .......... .......... 90% 92.4M 1s 2024-04-20T10:35:48.658 INFO:tasks.workunit.client.0.smithi042.stderr:122800K .......... .......... .......... .......... .......... 90% 58.6M 1s 2024-04-20T10:35:48.659 INFO:tasks.workunit.client.0.smithi042.stderr:122850K .......... .......... .......... .......... .......... 90% 24.9M 1s 2024-04-20T10:35:48.659 INFO:tasks.workunit.client.0.smithi042.stderr:122900K .......... .......... .......... .......... .......... 90% 93.0M 1s 2024-04-20T10:35:48.661 INFO:tasks.workunit.client.0.smithi042.stderr:122950K .......... .......... .......... .......... .......... 90% 24.7M 1s 2024-04-20T10:35:48.663 INFO:tasks.workunit.client.0.smithi042.stderr:123000K .......... .......... .......... .......... .......... 90% 24.4M 1s 2024-04-20T10:35:48.666 INFO:tasks.workunit.client.0.smithi042.stderr:123050K .......... .......... .......... .......... .......... 90% 21.0M 1s 2024-04-20T10:35:48.670 INFO:tasks.workunit.client.0.smithi042.stderr:123100K .......... .......... .......... .......... .......... 90% 11.3M 1s 2024-04-20T10:35:48.670 INFO:tasks.workunit.client.0.smithi042.stderr:123150K .......... .......... .......... .......... .......... 90% 112M 1s 2024-04-20T10:35:48.678 INFO:tasks.workunit.client.0.smithi042.stderr:123200K .......... .......... .......... .......... .......... 90% 6.97M 1s 2024-04-20T10:35:48.678 INFO:tasks.workunit.client.0.smithi042.stderr:123250K .......... .......... .......... .......... .......... 90% 210M 1s 2024-04-20T10:35:48.687 INFO:tasks.workunit.client.0.smithi042.stderr:123300K .......... .......... .......... .......... .......... 90% 16.1M 1s 2024-04-20T10:35:48.687 INFO:tasks.workunit.client.0.smithi042.stderr:123350K .......... .......... .......... .......... .......... 90% 113M 1s 2024-04-20T10:35:48.687 INFO:tasks.workunit.client.0.smithi042.stderr:123400K .......... .......... .......... .......... .......... 90% 202M 1s 2024-04-20T10:35:48.687 INFO:tasks.workunit.client.0.smithi042.stderr:123450K .......... .......... .......... .......... .......... 90% 165M 1s 2024-04-20T10:35:48.687 INFO:tasks.workunit.client.0.smithi042.stderr:123500K .......... .......... .......... .......... .......... 91% 122M 1s 2024-04-20T10:35:48.688 INFO:tasks.workunit.client.0.smithi042.stderr:123550K .......... .......... .......... .......... .......... 91% 48.2M 1s 2024-04-20T10:35:48.688 INFO:tasks.workunit.client.0.smithi042.stderr:123600K .......... .......... .......... .......... .......... 91% 185M 1s 2024-04-20T10:35:48.688 INFO:tasks.workunit.client.0.smithi042.stderr:123650K .......... .......... .......... .......... .......... 91% 23.9M 1s 2024-04-20T10:35:48.688 INFO:tasks.workunit.client.0.smithi042.stderr:123700K .......... .......... .......... .......... .......... 91% 21.4M 1s 2024-04-20T10:35:48.690 INFO:tasks.workunit.client.0.smithi042.stderr:123750K .......... .......... .......... .......... .......... 91% 22.9M 1s 2024-04-20T10:35:48.693 INFO:tasks.workunit.client.0.smithi042.stderr:123800K .......... .......... .......... .......... .......... 91% 17.0M 1s 2024-04-20T10:35:48.693 INFO:tasks.workunit.client.0.smithi042.stderr:123850K .......... .......... .......... .......... .......... 91% 97.8M 1s 2024-04-20T10:35:48.695 INFO:tasks.workunit.client.0.smithi042.stderr:123900K .......... .......... .......... .......... .......... 91% 27.1M 1s 2024-04-20T10:35:48.697 INFO:tasks.workunit.client.0.smithi042.stderr:123950K .......... .......... .......... .......... .......... 91% 23.2M 1s 2024-04-20T10:35:48.699 INFO:tasks.workunit.client.0.smithi042.stderr:124000K .......... .......... .......... .......... .......... 91% 24.4M 1s 2024-04-20T10:35:48.699 INFO:tasks.workunit.client.0.smithi042.stderr:124050K .......... .......... .......... .......... .......... 91% 200M 0s 2024-04-20T10:35:48.701 INFO:tasks.workunit.client.0.smithi042.stderr:124100K .......... .......... .......... .......... .......... 91% 20.9M 0s 2024-04-20T10:35:48.704 INFO:tasks.workunit.client.0.smithi042.stderr:124150K .......... .......... .......... .......... .......... 91% 23.7M 0s 2024-04-20T10:35:48.706 INFO:tasks.workunit.client.0.smithi042.stderr:124200K .......... .......... .......... .......... .......... 91% 23.4M 0s 2024-04-20T10:35:48.707 INFO:tasks.workunit.client.0.smithi042.stderr:124250K .......... .......... .......... .......... .......... 91% 26.0M 0s 2024-04-20T10:35:48.708 INFO:tasks.workunit.client.0.smithi042.stderr:124300K .......... .......... .......... .......... .......... 91% 116M 0s 2024-04-20T10:35:48.710 INFO:tasks.workunit.client.0.smithi042.stderr:124350K .......... .......... .......... .......... .......... 91% 22.2M 0s 2024-04-20T10:35:48.714 INFO:tasks.workunit.client.0.smithi042.stderr:124400K .......... .......... .......... .......... .......... 91% 24.5M 0s 2024-04-20T10:35:48.714 INFO:tasks.workunit.client.0.smithi042.stderr:124450K .......... .......... .......... .......... .......... 91% 26.8M 0s 2024-04-20T10:35:48.716 INFO:tasks.workunit.client.0.smithi042.stderr:124500K .......... .......... .......... .......... .......... 91% 23.3M 0s 2024-04-20T10:35:48.717 INFO:tasks.workunit.client.0.smithi042.stderr:124550K .......... .......... .......... .......... .......... 91% 77.4M 0s 2024-04-20T10:35:48.718 INFO:tasks.workunit.client.0.smithi042.stderr:124600K .......... .......... .......... .......... .......... 91% 26.7M 0s 2024-04-20T10:35:48.721 INFO:tasks.workunit.client.0.smithi042.stderr:124650K .......... .......... .......... .......... .......... 91% 18.6M 0s 2024-04-20T10:35:48.723 INFO:tasks.workunit.client.0.smithi042.stderr:124700K .......... .......... .......... .......... .......... 91% 30.3M 0s 2024-04-20T10:35:48.723 INFO:tasks.workunit.client.0.smithi042.stderr:124750K .......... .......... .......... .......... .......... 91% 83.4M 0s 2024-04-20T10:35:48.725 INFO:tasks.workunit.client.0.smithi042.stderr:124800K .......... .......... .......... .......... .......... 91% 27.3M 0s 2024-04-20T10:35:48.727 INFO:tasks.workunit.client.0.smithi042.stderr:124850K .......... .......... .......... .......... .......... 92% 23.1M 0s 2024-04-20T10:35:48.730 INFO:tasks.workunit.client.0.smithi042.stderr:124900K .......... .......... .......... .......... .......... 92% 22.2M 0s 2024-04-20T10:35:48.732 INFO:tasks.workunit.client.0.smithi042.stderr:124950K .......... .......... .......... .......... .......... 92% 25.0M 0s 2024-04-20T10:35:48.732 INFO:tasks.workunit.client.0.smithi042.stderr:125000K .......... .......... .......... .......... .......... 92% 94.0M 0s 2024-04-20T10:35:48.734 INFO:tasks.workunit.client.0.smithi042.stderr:125050K .......... .......... .......... .......... .......... 92% 25.5M 0s 2024-04-20T10:35:48.736 INFO:tasks.workunit.client.0.smithi042.stderr:125100K .......... .......... .......... .......... .......... 92% 23.7M 0s 2024-04-20T10:35:48.738 INFO:tasks.workunit.client.0.smithi042.stderr:125150K .......... .......... .......... .......... .......... 92% 23.8M 0s 2024-04-20T10:35:48.738 INFO:tasks.workunit.client.0.smithi042.stderr:125200K .......... .......... .......... .......... .......... 92% 86.0M 0s 2024-04-20T10:35:48.741 INFO:tasks.workunit.client.0.smithi042.stderr:125250K .......... .......... .......... .......... .......... 92% 25.1M 0s 2024-04-20T10:35:48.743 INFO:tasks.workunit.client.0.smithi042.stderr:125300K .......... .......... .......... .......... .......... 92% 20.0M 0s 2024-04-20T10:35:48.743 INFO:tasks.workunit.client.0.smithi042.stderr:125350K .......... .......... .......... .......... .......... 92% 73.6M 0s 2024-04-20T10:35:48.746 INFO:tasks.workunit.client.0.smithi042.stderr:125400K .......... .......... .......... .......... .......... 92% 24.2M 0s 2024-04-20T10:35:48.747 INFO:tasks.workunit.client.0.smithi042.stderr:125450K .......... .......... .......... .......... .......... 92% 27.9M 0s 2024-04-20T10:35:48.749 INFO:tasks.workunit.client.0.smithi042.stderr:125500K .......... .......... .......... .......... .......... 92% 24.9M 0s 2024-04-20T10:35:48.752 INFO:tasks.workunit.client.0.smithi042.stderr:125550K .......... .......... .......... .......... .......... 92% 24.7M 0s 2024-04-20T10:35:48.752 INFO:tasks.workunit.client.0.smithi042.stderr:125600K .......... .......... .......... .......... .......... 92% 52.0M 0s 2024-04-20T10:35:48.754 INFO:tasks.workunit.client.0.smithi042.stderr:125650K .......... .......... .......... .......... .......... 92% 29.4M 0s 2024-04-20T10:35:48.756 INFO:tasks.workunit.client.0.smithi042.stderr:125700K .......... .......... .......... .......... .......... 92% 28.8M 0s 2024-04-20T10:35:48.758 INFO:tasks.workunit.client.0.smithi042.stderr:125750K .......... .......... .......... .......... .......... 92% 24.8M 0s 2024-04-20T10:35:48.758 INFO:tasks.workunit.client.0.smithi042.stderr:125800K .......... .......... .......... .......... .......... 92% 59.9M 0s 2024-04-20T10:35:48.760 INFO:tasks.workunit.client.0.smithi042.stderr:125850K .......... .......... .......... .......... .......... 92% 23.6M 0s 2024-04-20T10:35:48.762 INFO:tasks.workunit.client.0.smithi042.stderr:125900K .......... .......... .......... .......... .......... 92% 25.6M 0s 2024-04-20T10:35:48.764 INFO:tasks.workunit.client.0.smithi042.stderr:125950K .......... .......... .......... .......... .......... 92% 26.6M 0s 2024-04-20T10:35:48.766 INFO:tasks.workunit.client.0.smithi042.stderr:126000K .......... .......... .......... .......... .......... 92% 19.4M 0s 2024-04-20T10:35:48.768 INFO:tasks.workunit.client.0.smithi042.stderr:126050K .......... .......... .......... .......... .......... 92% 25.4M 0s 2024-04-20T10:35:48.769 INFO:tasks.workunit.client.0.smithi042.stderr:126100K .......... .......... .......... .......... .......... 92% 32.5M 0s 2024-04-20T10:35:48.771 INFO:tasks.workunit.client.0.smithi042.stderr:126150K .......... .......... .......... .......... .......... 92% 23.1M 0s 2024-04-20T10:35:48.775 INFO:tasks.workunit.client.0.smithi042.stderr:126200K .......... .......... .......... .......... .......... 92% 22.2M 0s 2024-04-20T10:35:48.775 INFO:tasks.workunit.client.0.smithi042.stderr:126250K .......... .......... .......... .......... .......... 93% 60.0M 0s 2024-04-20T10:35:48.778 INFO:tasks.workunit.client.0.smithi042.stderr:126300K .......... .......... .......... .......... .......... 93% 16.2M 0s 2024-04-20T10:35:48.778 INFO:tasks.workunit.client.0.smithi042.stderr:126350K .......... .......... .......... .......... .......... 93% 67.7M 0s 2024-04-20T10:35:48.780 INFO:tasks.workunit.client.0.smithi042.stderr:126400K .......... .......... .......... .......... .......... 93% 27.2M 0s 2024-04-20T10:35:48.783 INFO:tasks.workunit.client.0.smithi042.stderr:126450K .......... .......... .......... .......... .......... 93% 23.1M 0s 2024-04-20T10:35:48.785 INFO:tasks.workunit.client.0.smithi042.stderr:126500K .......... .......... .......... .......... .......... 93% 21.8M 0s 2024-04-20T10:35:48.787 INFO:tasks.workunit.client.0.smithi042.stderr:126550K .......... .......... .......... .......... .......... 93% 23.2M 0s 2024-04-20T10:35:48.787 INFO:tasks.workunit.client.0.smithi042.stderr:126600K .......... .......... .......... .......... .......... 93% 70.2M 0s 2024-04-20T10:35:48.789 INFO:tasks.workunit.client.0.smithi042.stderr:126650K .......... .......... .......... .......... .......... 93% 25.3M 0s 2024-04-20T10:35:48.791 INFO:tasks.workunit.client.0.smithi042.stderr:126700K .......... .......... .......... .......... .......... 93% 26.2M 0s 2024-04-20T10:35:48.793 INFO:tasks.workunit.client.0.smithi042.stderr:126750K .......... .......... .......... .......... .......... 93% 26.2M 0s 2024-04-20T10:35:48.795 INFO:tasks.workunit.client.0.smithi042.stderr:126800K .......... .......... .......... .......... .......... 93% 26.0M 0s 2024-04-20T10:35:48.795 INFO:tasks.workunit.client.0.smithi042.stderr:126850K .......... .......... .......... .......... .......... 93% 70.3M 0s 2024-04-20T10:35:48.797 INFO:tasks.workunit.client.0.smithi042.stderr:126900K .......... .......... .......... .......... .......... 93% 24.9M 0s 2024-04-20T10:35:48.801 INFO:tasks.workunit.client.0.smithi042.stderr:126950K .......... .......... .......... .......... .......... 93% 26.6M 0s 2024-04-20T10:35:48.802 INFO:tasks.workunit.client.0.smithi042.stderr:127000K .......... .......... .......... .......... .......... 93% 27.0M 0s 2024-04-20T10:35:48.803 INFO:tasks.workunit.client.0.smithi042.stderr:127050K .......... .......... .......... .......... .......... 93% 25.0M 0s 2024-04-20T10:35:48.806 INFO:tasks.workunit.client.0.smithi042.stderr:127100K .......... .......... .......... .......... .......... 93% 74.2M 0s 2024-04-20T10:35:48.806 INFO:tasks.workunit.client.0.smithi042.stderr:127150K .......... .......... .......... .......... .......... 93% 20.1M 0s 2024-04-20T10:35:48.808 INFO:tasks.workunit.client.0.smithi042.stderr:127200K .......... .......... .......... .......... .......... 93% 26.0M 0s 2024-04-20T10:35:48.810 INFO:tasks.workunit.client.0.smithi042.stderr:127250K .......... .......... .......... .......... .......... 93% 25.4M 0s 2024-04-20T10:35:48.812 INFO:tasks.workunit.client.0.smithi042.stderr:127300K .......... .......... .......... .......... .......... 93% 34.3M 0s 2024-04-20T10:35:48.814 INFO:tasks.workunit.client.0.smithi042.stderr:127350K .......... .......... .......... .......... .......... 93% 44.4M 0s 2024-04-20T10:35:48.814 INFO:tasks.workunit.client.0.smithi042.stderr:127400K .......... .......... .......... .......... .......... 93% 25.3M 0s 2024-04-20T10:35:48.816 INFO:tasks.workunit.client.0.smithi042.stderr:127450K .......... .......... .......... .......... .......... 93% 27.6M 0s 2024-04-20T10:35:48.819 INFO:tasks.workunit.client.0.smithi042.stderr:127500K .......... .......... .......... .......... .......... 93% 21.1M 0s 2024-04-20T10:35:48.821 INFO:tasks.workunit.client.0.smithi042.stderr:127550K .......... .......... .......... .......... .......... 93% 29.3M 0s 2024-04-20T10:35:48.822 INFO:tasks.workunit.client.0.smithi042.stderr:127600K .......... .......... .......... .......... .......... 94% 26.8M 0s 2024-04-20T10:35:48.823 INFO:tasks.workunit.client.0.smithi042.stderr:127650K .......... .......... .......... .......... .......... 94% 55.4M 0s 2024-04-20T10:35:48.825 INFO:tasks.workunit.client.0.smithi042.stderr:127700K .......... .......... .......... .......... .......... 94% 24.6M 0s 2024-04-20T10:35:48.827 INFO:tasks.workunit.client.0.smithi042.stderr:127750K .......... .......... .......... .......... .......... 94% 29.7M 0s 2024-04-20T10:35:48.828 INFO:tasks.workunit.client.0.smithi042.stderr:127800K .......... .......... .......... .......... .......... 94% 25.8M 0s 2024-04-20T10:35:48.831 INFO:tasks.workunit.client.0.smithi042.stderr:127850K .......... .......... .......... .......... .......... 94% 25.7M 0s 2024-04-20T10:35:48.831 INFO:tasks.workunit.client.0.smithi042.stderr:127900K .......... .......... .......... .......... .......... 94% 58.0M 0s 2024-04-20T10:35:48.833 INFO:tasks.workunit.client.0.smithi042.stderr:127950K .......... .......... .......... .......... .......... 94% 24.3M 0s 2024-04-20T10:35:48.835 INFO:tasks.workunit.client.0.smithi042.stderr:128000K .......... .......... .......... .......... .......... 94% 31.5M 0s 2024-04-20T10:35:48.837 INFO:tasks.workunit.client.0.smithi042.stderr:128050K .......... .......... .......... .......... .......... 94% 23.9M 0s 2024-04-20T10:35:48.839 INFO:tasks.workunit.client.0.smithi042.stderr:128100K .......... .......... .......... .......... .......... 94% 25.2M 0s 2024-04-20T10:35:48.840 INFO:tasks.workunit.client.0.smithi042.stderr:128150K .......... .......... .......... .......... .......... 94% 49.3M 0s 2024-04-20T10:35:48.842 INFO:tasks.workunit.client.0.smithi042.stderr:128200K .......... .......... .......... .......... .......... 94% 25.2M 0s 2024-04-20T10:35:48.843 INFO:tasks.workunit.client.0.smithi042.stderr:128250K .......... .......... .......... .......... .......... 94% 31.5M 0s 2024-04-20T10:35:48.846 INFO:tasks.workunit.client.0.smithi042.stderr:128300K .......... .......... .......... .......... .......... 94% 25.0M 0s 2024-04-20T10:35:48.847 INFO:tasks.workunit.client.0.smithi042.stderr:128350K .......... .......... .......... .......... .......... 94% 26.3M 0s 2024-04-20T10:35:48.848 INFO:tasks.workunit.client.0.smithi042.stderr:128400K .......... .......... .......... .......... .......... 94% 42.4M 0s 2024-04-20T10:35:48.850 INFO:tasks.workunit.client.0.smithi042.stderr:128450K .......... .......... .......... .......... .......... 94% 26.9M 0s 2024-04-20T10:35:48.851 INFO:tasks.workunit.client.0.smithi042.stderr:128500K .......... .......... .......... .......... .......... 94% 30.7M 0s 2024-04-20T10:35:48.853 INFO:tasks.workunit.client.0.smithi042.stderr:128550K .......... .......... .......... .......... .......... 94% 27.7M 0s 2024-04-20T10:35:48.855 INFO:tasks.workunit.client.0.smithi042.stderr:128600K .......... .......... .......... .......... .......... 94% 25.0M 0s 2024-04-20T10:35:48.857 INFO:tasks.workunit.client.0.smithi042.stderr:128650K .......... .......... .......... .......... .......... 94% 34.3M 0s 2024-04-20T10:35:48.863 INFO:tasks.workunit.client.0.smithi042.stderr:128700K .......... .......... .......... .......... .......... 94% 28.1M 0s 2024-04-20T10:35:48.863 INFO:tasks.workunit.client.0.smithi042.stderr:128750K .......... .......... .......... .......... .......... 94% 30.3M 0s 2024-04-20T10:35:49.156 INFO:tasks.workunit.client.0.smithi042.stderr:128800K .......... .......... .......... .......... .......... 94% 169K 0s 2024-04-20T10:35:49.157 INFO:tasks.workunit.client.0.smithi042.stderr:128850K .......... .......... .......... .......... .......... 94% 55.8M 0s 2024-04-20T10:35:49.158 INFO:tasks.workunit.client.0.smithi042.stderr:128900K .......... .......... .......... .......... .......... 94% 58.9M 0s 2024-04-20T10:35:49.167 INFO:tasks.workunit.client.0.smithi042.stderr:128950K .......... .......... .......... .......... .......... 95% 13.0M 0s 2024-04-20T10:35:49.167 INFO:tasks.workunit.client.0.smithi042.stderr:129000K .......... .......... .......... .......... .......... 95% 57.3M 0s 2024-04-20T10:35:49.168 INFO:tasks.workunit.client.0.smithi042.stderr:129050K .......... .......... .......... .......... .......... 95% 49.4M 0s 2024-04-20T10:35:49.168 INFO:tasks.workunit.client.0.smithi042.stderr:129100K .......... .......... .......... .......... .......... 95% 58.7M 0s 2024-04-20T10:35:49.168 INFO:tasks.workunit.client.0.smithi042.stderr:129150K .......... .......... .......... .......... .......... 95% 64.5M 0s 2024-04-20T10:35:49.168 INFO:tasks.workunit.client.0.smithi042.stderr:129200K .......... .......... .......... .......... .......... 95% 68.8M 0s 2024-04-20T10:35:49.168 INFO:tasks.workunit.client.0.smithi042.stderr:129250K .......... .......... .......... .......... .......... 95% 61.8M 0s 2024-04-20T10:35:49.203 INFO:tasks.workunit.client.0.smithi042.stderr:129300K .......... .......... .......... .......... .......... 95% 1.34M 0s 2024-04-20T10:35:49.204 INFO:tasks.workunit.client.0.smithi042.stderr:129350K .......... .......... .......... .......... .......... 95% 36.6M 0s 2024-04-20T10:35:49.205 INFO:tasks.workunit.client.0.smithi042.stderr:129400K .......... .......... .......... .......... .......... 95% 35.2M 0s 2024-04-20T10:35:49.207 INFO:tasks.workunit.client.0.smithi042.stderr:129450K .......... .......... .......... .......... .......... 95% 32.4M 0s 2024-04-20T10:35:49.208 INFO:tasks.workunit.client.0.smithi042.stderr:129500K .......... .......... .......... .......... .......... 95% 39.5M 0s 2024-04-20T10:35:49.209 INFO:tasks.workunit.client.0.smithi042.stderr:129550K .......... .......... .......... .......... .......... 95% 39.9M 0s 2024-04-20T10:35:49.211 INFO:tasks.workunit.client.0.smithi042.stderr:129600K .......... .......... .......... .......... .......... 95% 38.1M 0s 2024-04-20T10:35:49.211 INFO:tasks.workunit.client.0.smithi042.stderr:129650K .......... .......... .......... .......... .......... 95% 34.4M 0s 2024-04-20T10:35:49.431 INFO:tasks.workunit.client.0.smithi042.stderr:129700K .......... .......... .......... .......... .......... 95% 228K 0s 2024-04-20T10:35:49.432 INFO:tasks.workunit.client.0.smithi042.stderr:129750K .......... .......... .......... .......... .......... 95% 36.9M 0s 2024-04-20T10:35:49.434 INFO:tasks.workunit.client.0.smithi042.stderr:129800K .......... .......... .......... .......... .......... 95% 37.5M 0s 2024-04-20T10:35:49.435 INFO:tasks.workunit.client.0.smithi042.stderr:129850K .......... .......... .......... .......... .......... 95% 34.6M 0s 2024-04-20T10:35:49.436 INFO:tasks.workunit.client.0.smithi042.stderr:129900K .......... .......... .......... .......... .......... 95% 39.2M 0s 2024-04-20T10:35:49.465 INFO:tasks.workunit.client.0.smithi042.stderr:129950K .......... .......... .......... .......... .......... 95% 1.72M 0s 2024-04-20T10:35:49.466 INFO:tasks.workunit.client.0.smithi042.stderr:130000K .......... .......... .......... .......... .......... 95% 38.5M 0s 2024-04-20T10:35:49.467 INFO:tasks.workunit.client.0.smithi042.stderr:130050K .......... .......... .......... .......... .......... 95% 33.1M 0s 2024-04-20T10:35:49.553 INFO:tasks.workunit.client.0.smithi042.stderr:130100K .......... .......... .......... .......... .......... 95% 585K 0s 2024-04-20T10:35:49.555 INFO:tasks.workunit.client.0.smithi042.stderr:130150K .......... .......... .......... .......... .......... 95% 38.2M 0s 2024-04-20T10:35:49.556 INFO:tasks.workunit.client.0.smithi042.stderr:130200K .......... .......... .......... .......... .......... 95% 36.0M 0s 2024-04-20T10:35:49.557 INFO:tasks.workunit.client.0.smithi042.stderr:130250K .......... .......... .......... .......... .......... 95% 34.7M 0s 2024-04-20T10:35:49.558 INFO:tasks.workunit.client.0.smithi042.stderr:130300K .......... .......... .......... .......... .......... 96% 38.0M 0s 2024-04-20T10:35:49.560 INFO:tasks.workunit.client.0.smithi042.stderr:130350K .......... .......... .......... .......... .......... 96% 39.1M 0s 2024-04-20T10:35:49.561 INFO:tasks.workunit.client.0.smithi042.stderr:130400K .......... .......... .......... .......... .......... 96% 39.7M 0s 2024-04-20T10:35:49.563 INFO:tasks.workunit.client.0.smithi042.stderr:130450K .......... .......... .......... .......... .......... 96% 16.7M 0s 2024-04-20T10:35:49.564 INFO:tasks.workunit.client.0.smithi042.stderr:130500K .......... .......... .......... .......... .......... 96% 36.3M 0s 2024-04-20T10:35:49.566 INFO:tasks.workunit.client.0.smithi042.stderr:130550K .......... .......... .......... .......... .......... 96% 37.0M 0s 2024-04-20T10:35:49.569 INFO:tasks.workunit.client.0.smithi042.stderr:130600K .......... .......... .......... .......... .......... 96% 38.8M 0s 2024-04-20T10:35:49.569 INFO:tasks.workunit.client.0.smithi042.stderr:130650K .......... .......... .......... .......... .......... 96% 32.9M 0s 2024-04-20T10:35:49.571 INFO:tasks.workunit.client.0.smithi042.stderr:130700K .......... .......... .......... .......... .......... 96% 39.0M 0s 2024-04-20T10:35:49.572 INFO:tasks.workunit.client.0.smithi042.stderr:130750K .......... .......... .......... .......... .......... 96% 38.3M 0s 2024-04-20T10:35:49.573 INFO:tasks.workunit.client.0.smithi042.stderr:130800K .......... .......... .......... .......... .......... 96% 37.4M 0s 2024-04-20T10:35:49.574 INFO:tasks.workunit.client.0.smithi042.stderr:130850K .......... .......... .......... .......... .......... 96% 34.2M 0s 2024-04-20T10:35:49.575 INFO:tasks.workunit.client.0.smithi042.stderr:130900K .......... .......... .......... .......... .......... 96% 36.8M 0s 2024-04-20T10:35:49.577 INFO:tasks.workunit.client.0.smithi042.stderr:130950K .......... .......... .......... .......... .......... 96% 39.0M 0s 2024-04-20T10:35:49.579 INFO:tasks.workunit.client.0.smithi042.stderr:131000K .......... .......... .......... .......... .......... 96% 39.8M 0s 2024-04-20T10:35:49.580 INFO:tasks.workunit.client.0.smithi042.stderr:131050K .......... .......... .......... .......... .......... 96% 33.8M 0s 2024-04-20T10:35:49.581 INFO:tasks.workunit.client.0.smithi042.stderr:131100K .......... .......... .......... .......... .......... 96% 36.3M 0s 2024-04-20T10:35:49.583 INFO:tasks.workunit.client.0.smithi042.stderr:131150K .......... .......... .......... .......... .......... 96% 25.9M 0s 2024-04-20T10:35:49.584 INFO:tasks.workunit.client.0.smithi042.stderr:131200K .......... .......... .......... .......... .......... 96% 37.0M 0s 2024-04-20T10:35:49.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:49 smithi162 ceph-mon[21854]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-04-20T10:35:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:49 smithi046 ceph-mon[21574]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-04-20T10:35:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:49 smithi042 ceph-mon[17078]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-04-20T10:35:50.431 INFO:tasks.workunit.client.0.smithi042.stderr:131250K .......... .......... .......... .......... .......... 96% 59.0K 0s 2024-04-20T10:35:50.433 INFO:tasks.workunit.client.0.smithi042.stderr:131300K .......... .......... .......... .......... .......... 96% 34.3M 0s 2024-04-20T10:35:50.434 INFO:tasks.workunit.client.0.smithi042.stderr:131350K .......... .......... .......... .......... .......... 96% 37.9M 0s 2024-04-20T10:35:50.435 INFO:tasks.workunit.client.0.smithi042.stderr:131400K .......... .......... .......... .......... .......... 96% 38.9M 0s 2024-04-20T10:35:50.436 INFO:tasks.workunit.client.0.smithi042.stderr:131450K .......... .......... .......... .......... .......... 96% 33.3M 0s 2024-04-20T10:35:50.616 INFO:tasks.workunit.client.0.smithi042.stderr:131500K .......... .......... .......... .......... .......... 96% 278K 0s 2024-04-20T10:35:50.617 INFO:tasks.workunit.client.0.smithi042.stderr:131550K .......... .......... .......... .......... .......... 96% 40.1M 0s 2024-04-20T10:35:50.618 INFO:tasks.workunit.client.0.smithi042.stderr:131600K .......... .......... .......... .......... .......... 96% 39.6M 0s 2024-04-20T10:35:50.727 INFO:tasks.workunit.client.0.smithi042.stderr:131650K .......... .......... .......... .......... .......... 97% 502K 0s 2024-04-20T10:35:50.728 INFO:tasks.workunit.client.0.smithi042.stderr:131700K .......... .......... .......... .......... .......... 97% 63.5M 0s 2024-04-20T10:35:50.728 INFO:tasks.workunit.client.0.smithi042.stderr:131750K .......... .......... .......... .......... .......... 97% 58.8M 0s 2024-04-20T10:35:50.728 INFO:tasks.workunit.client.0.smithi042.stderr:131800K .......... .......... .......... .......... .......... 97% 62.2M 0s 2024-04-20T10:35:50.728 INFO:tasks.workunit.client.0.smithi042.stderr:131850K .......... .......... .......... .......... .......... 97% 59.4M 0s 2024-04-20T10:35:50.728 INFO:tasks.workunit.client.0.smithi042.stderr:131900K .......... .......... .......... .......... .......... 97% 58.3M 0s 2024-04-20T10:35:50.728 INFO:tasks.workunit.client.0.smithi042.stderr:131950K .......... .......... .......... .......... .......... 97% 68.1M 0s 2024-04-20T10:35:50.728 INFO:tasks.workunit.client.0.smithi042.stderr:132000K .......... .......... .......... .......... .......... 97% 58.9M 0s 2024-04-20T10:35:50.728 INFO:tasks.workunit.client.0.smithi042.stderr:132050K .......... .......... .......... .......... .......... 97% 54.9M 0s 2024-04-20T10:35:50.728 INFO:tasks.workunit.client.0.smithi042.stderr:132100K .......... .......... .......... .......... .......... 97% 56.9M 0s 2024-04-20T10:35:50.728 INFO:tasks.workunit.client.0.smithi042.stderr:132150K .......... .......... .......... .......... .......... 97% 48.8M 0s 2024-04-20T10:35:50.747 INFO:tasks.workunit.client.0.smithi042.stderr:132200K .......... .......... .......... .......... .......... 97% 3.99M 0s 2024-04-20T10:35:50.747 INFO:tasks.workunit.client.0.smithi042.stderr:132250K .......... .......... .......... .......... .......... 97% 67.1M 0s 2024-04-20T10:35:50.747 INFO:tasks.workunit.client.0.smithi042.stderr:132300K .......... .......... .......... .......... .......... 97% 59.2M 0s 2024-04-20T10:35:50.747 INFO:tasks.workunit.client.0.smithi042.stderr:132350K .......... .......... .......... .......... .......... 97% 50.3M 0s 2024-04-20T10:35:50.747 INFO:tasks.workunit.client.0.smithi042.stderr:132400K .......... .......... .......... .......... .......... 97% 68.8M 0s 2024-04-20T10:35:50.747 INFO:tasks.workunit.client.0.smithi042.stderr:132450K .......... .......... .......... .......... .......... 97% 59.3M 0s 2024-04-20T10:35:50.747 INFO:tasks.workunit.client.0.smithi042.stderr:132500K .......... .......... .......... .......... .......... 97% 66.1M 0s 2024-04-20T10:35:50.747 INFO:tasks.workunit.client.0.smithi042.stderr:132550K .......... .......... .......... .......... .......... 97% 52.7M 0s 2024-04-20T10:35:50.747 INFO:tasks.workunit.client.0.smithi042.stderr:132600K .......... .......... .......... .......... .......... 97% 66.6M 0s 2024-04-20T10:35:50.898 INFO:tasks.workunit.client.0.smithi042.stderr:132650K .......... .......... .......... .......... .......... 97% 332K 0s 2024-04-20T10:35:50.898 INFO:tasks.workunit.client.0.smithi042.stderr:132700K .......... .......... .......... .......... .......... 97% 47.2M 0s 2024-04-20T10:35:50.898 INFO:tasks.workunit.client.0.smithi042.stderr:132750K .......... .......... .......... .......... .......... 97% 52.6M 0s 2024-04-20T10:35:50.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:50 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:35:50.909 INFO:tasks.workunit.client.0.smithi042.stderr:132800K .......... .......... .......... .......... .......... 97% 10.6M 0s 2024-04-20T10:35:50.909 INFO:tasks.workunit.client.0.smithi042.stderr:132850K .......... .......... .......... .......... .......... 97% 46.7M 0s 2024-04-20T10:35:50.909 INFO:tasks.workunit.client.0.smithi042.stderr:132900K .......... .......... .......... .......... .......... 97% 61.4M 0s 2024-04-20T10:35:50.909 INFO:tasks.workunit.client.0.smithi042.stderr:132950K .......... .......... .......... .......... .......... 97% 69.5M 0s 2024-04-20T10:35:50.909 INFO:tasks.workunit.client.0.smithi042.stderr:133000K .......... .......... .......... .......... .......... 98% 63.3M 0s 2024-04-20T10:35:50.909 INFO:tasks.workunit.client.0.smithi042.stderr:133050K .......... .......... .......... .......... .......... 98% 53.4M 0s 2024-04-20T10:35:50.909 INFO:tasks.workunit.client.0.smithi042.stderr:133100K .......... .......... .......... .......... .......... 98% 68.0M 0s 2024-04-20T10:35:50.910 INFO:tasks.workunit.client.0.smithi042.stderr:133150K .......... .......... .......... .......... .......... 98% 53.4M 0s 2024-04-20T10:35:50.911 INFO:tasks.workunit.client.0.smithi042.stderr:133200K .......... .......... .......... .......... .......... 98% 21.8M 0s 2024-04-20T10:35:50.911 INFO:tasks.workunit.client.0.smithi042.stderr:133250K .......... .......... .......... .......... .......... 98% 59.4M 0s 2024-04-20T10:35:50.913 INFO:tasks.workunit.client.0.smithi042.stderr:133300K .......... .......... .......... .......... .......... 98% 24.8M 0s 2024-04-20T10:35:50.916 INFO:tasks.workunit.client.0.smithi042.stderr:133350K .......... .......... .......... .......... .......... 98% 25.8M 0s 2024-04-20T10:35:50.918 INFO:tasks.workunit.client.0.smithi042.stderr:133400K .......... .......... .......... .......... .......... 98% 25.5M 0s 2024-04-20T10:35:50.918 INFO:tasks.workunit.client.0.smithi042.stderr:133450K .......... .......... .......... .......... .......... 98% 54.4M 0s 2024-04-20T10:35:50.920 INFO:tasks.workunit.client.0.smithi042.stderr:133500K .......... .......... .......... .......... .......... 98% 25.9M 0s 2024-04-20T10:35:50.922 INFO:tasks.workunit.client.0.smithi042.stderr:133550K .......... .......... .......... .......... .......... 98% 26.8M 0s 2024-04-20T10:35:50.924 INFO:tasks.workunit.client.0.smithi042.stderr:133600K .......... .......... .......... .......... .......... 98% 24.8M 0s 2024-04-20T10:35:50.927 INFO:tasks.workunit.client.0.smithi042.stderr:133650K .......... .......... .......... .......... .......... 98% 24.7M 0s 2024-04-20T10:35:50.927 INFO:tasks.workunit.client.0.smithi042.stderr:133700K .......... .......... .......... .......... .......... 98% 51.9M 0s 2024-04-20T10:35:50.928 INFO:tasks.workunit.client.0.smithi042.stderr:133750K .......... .......... .......... .......... .......... 98% 28.7M 0s 2024-04-20T10:35:50.931 INFO:tasks.workunit.client.0.smithi042.stderr:133800K .......... .......... .......... .......... .......... 98% 25.6M 0s 2024-04-20T10:35:50.933 INFO:tasks.workunit.client.0.smithi042.stderr:133850K .......... .......... .......... .......... .......... 98% 23.9M 0s 2024-04-20T10:35:50.933 INFO:tasks.workunit.client.0.smithi042.stderr:133900K .......... .......... .......... .......... .......... 98% 65.9M 0s 2024-04-20T10:35:50.935 INFO:tasks.workunit.client.0.smithi042.stderr:133950K .......... .......... .......... .......... .......... 98% 25.0M 0s 2024-04-20T10:35:50.937 INFO:tasks.workunit.client.0.smithi042.stderr:134000K .......... .......... .......... .......... .......... 98% 27.1M 0s 2024-04-20T10:35:50.939 INFO:tasks.workunit.client.0.smithi042.stderr:134050K .......... .......... .......... .......... .......... 98% 23.1M 0s 2024-04-20T10:35:50.941 INFO:tasks.workunit.client.0.smithi042.stderr:134100K .......... .......... .......... .......... .......... 98% 27.2M 0s 2024-04-20T10:35:50.941 INFO:tasks.workunit.client.0.smithi042.stderr:134150K .......... .......... .......... .......... .......... 98% 60.2M 0s 2024-04-20T10:35:50.942 INFO:tasks.workunit.client.0.smithi042.stderr:134200K .......... .......... .......... .......... .......... 98% 38.2M 0s 2024-04-20T10:35:50.945 INFO:tasks.workunit.client.0.smithi042.stderr:134250K .......... .......... .......... .......... .......... 98% 23.7M 0s 2024-04-20T10:35:50.947 INFO:tasks.workunit.client.0.smithi042.stderr:134300K .......... .......... .......... .......... .......... 98% 22.2M 0s 2024-04-20T10:35:50.949 INFO:tasks.workunit.client.0.smithi042.stderr:134350K .......... .......... .......... .......... .......... 99% 27.0M 0s 2024-04-20T10:35:50.951 INFO:tasks.workunit.client.0.smithi042.stderr:134400K .......... .......... .......... .......... .......... 99% 24.7M 0s 2024-04-20T10:35:50.951 INFO:tasks.workunit.client.0.smithi042.stderr:134450K .......... .......... .......... .......... .......... 99% 65.9M 0s 2024-04-20T10:35:50.954 INFO:tasks.workunit.client.0.smithi042.stderr:134500K .......... .......... .......... .......... .......... 99% 20.2M 0s 2024-04-20T10:35:50.956 INFO:tasks.workunit.client.0.smithi042.stderr:134550K .......... .......... .......... .......... .......... 99% 20.1M 0s 2024-04-20T10:35:50.959 INFO:tasks.workunit.client.0.smithi042.stderr:134600K .......... .......... .......... .......... .......... 99% 26.1M 0s 2024-04-20T10:35:50.961 INFO:tasks.workunit.client.0.smithi042.stderr:134650K .......... .......... .......... .......... .......... 99% 25.5M 0s 2024-04-20T10:35:50.961 INFO:tasks.workunit.client.0.smithi042.stderr:134700K .......... .......... .......... .......... .......... 99% 67.4M 0s 2024-04-20T10:35:50.963 INFO:tasks.workunit.client.0.smithi042.stderr:134750K .......... .......... .......... .......... .......... 99% 26.5M 0s 2024-04-20T10:35:50.965 INFO:tasks.workunit.client.0.smithi042.stderr:134800K .......... .......... .......... .......... .......... 99% 24.4M 0s 2024-04-20T10:35:50.967 INFO:tasks.workunit.client.0.smithi042.stderr:134850K .......... .......... .......... .......... .......... 99% 21.6M 0s 2024-04-20T10:35:50.969 INFO:tasks.workunit.client.0.smithi042.stderr:134900K .......... .......... .......... .......... .......... 99% 24.4M 0s 2024-04-20T10:35:50.970 INFO:tasks.workunit.client.0.smithi042.stderr:134950K .......... .......... .......... .......... .......... 99% 60.7M 0s 2024-04-20T10:35:50.971 INFO:tasks.workunit.client.0.smithi042.stderr:135000K .......... .......... .......... .......... .......... 99% 31.3M 0s 2024-04-20T10:35:50.974 INFO:tasks.workunit.client.0.smithi042.stderr:135050K .......... .......... .......... .......... .......... 99% 23.5M 0s 2024-04-20T10:35:50.974 INFO:tasks.workunit.client.0.smithi042.stderr:135100K .......... .......... .......... .......... .......... 99% 33.3M 0s 2024-04-20T10:35:50.978 INFO:tasks.workunit.client.0.smithi042.stderr:135150K .......... .......... .......... .......... .......... 99% 19.1M 0s 2024-04-20T10:35:50.978 INFO:tasks.workunit.client.0.smithi042.stderr:135200K .......... .......... .......... .......... .......... 99% 54.7M 0s 2024-04-20T10:35:50.981 INFO:tasks.workunit.client.0.smithi042.stderr:135250K .......... .......... .......... .......... .......... 99% 27.4M 0s 2024-04-20T10:35:50.981 INFO:tasks.workunit.client.0.smithi042.stderr:135300K .......... .......... .......... .......... .......... 99% 52.0M 0s 2024-04-20T10:35:50.984 INFO:tasks.workunit.client.0.smithi042.stderr:135350K .......... .......... .......... .......... .......... 99% 19.3M 0s 2024-04-20T10:35:50.985 INFO:tasks.workunit.client.0.smithi042.stderr:135400K .......... .......... .......... .......... .......... 99% 36.8M 0s 2024-04-20T10:35:50.986 INFO:tasks.workunit.client.0.smithi042.stderr:135450K .......... .......... .......... .......... .......... 99% 41.5M 0s 2024-04-20T10:35:50.988 INFO:tasks.workunit.client.0.smithi042.stderr:135500K .......... .......... .......... .......... .......... 99% 21.1M 0s 2024-04-20T10:35:50.991 INFO:tasks.workunit.client.0.smithi042.stderr:135550K .......... .......... .......... .......... .......... 99% 20.5M 0s 2024-04-20T10:35:50.994 INFO:tasks.workunit.client.0.smithi042.stderr:135600K .......... .......... .......... .......... .......... 99% 25.9M 0s 2024-04-20T10:35:50.994 INFO:tasks.workunit.client.0.smithi042.stderr:135650K .......... .......... .......... .......... .......... 99% 48.7M 0s 2024-04-20T10:35:50.995 INFO:tasks.workunit.client.0.smithi042.stderr:135700K .......... .......... .......... .......... .......... 99% 44.2M 0s 2024-04-20T10:35:50.995 INFO:tasks.workunit.client.0.smithi042.stderr:135750K .. 100% 21.0M=7.6s 2024-04-20T10:35:50.995 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-20T10:35:50.995 INFO:tasks.workunit.client.0.smithi042.stderr:2024-04-20 10:35:50 (17.4 MB/s) - ‘linux.tar.xz’ saved [139010916/139010916] 2024-04-20T10:35:50.995 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-20T10:35:50.998 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir t 2024-04-20T10:35:51.005 INFO:tasks.workunit.client.0.smithi042.stderr:+ cd t 2024-04-20T10:35:51.005 INFO:tasks.workunit.client.0.smithi042.stderr:+ tar xJf ../linux.tar.xz 2024-04-20T10:35:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:50 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:35:51.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:50 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:35:51.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:51 smithi162 ceph-mon[21854]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-04-20T10:35:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:51 smithi046 ceph-mon[21574]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-04-20T10:35:52.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:51 smithi042 ceph-mon[17078]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-04-20T10:35:53.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:53 smithi162 ceph-mon[21854]: pgmap v59: 97 pgs: 97 active+clean; 53 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 MiB/s wr, 5 op/s 2024-04-20T10:35:54.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:53 smithi046 ceph-mon[21574]: pgmap v59: 97 pgs: 97 active+clean; 53 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 MiB/s wr, 5 op/s 2024-04-20T10:35:54.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:53 smithi042 ceph-mon[17078]: pgmap v59: 97 pgs: 97 active+clean; 53 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 MiB/s wr, 5 op/s 2024-04-20T10:35:55.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:55 smithi162 ceph-mon[21854]: pgmap v60: 97 pgs: 97 active+clean; 53 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 MiB/s wr, 4 op/s 2024-04-20T10:35:56.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:55 smithi046 ceph-mon[21574]: pgmap v60: 97 pgs: 97 active+clean; 53 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 MiB/s wr, 4 op/s 2024-04-20T10:35:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:55 smithi042 ceph-mon[17078]: pgmap v60: 97 pgs: 97 active+clean; 53 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 MiB/s wr, 4 op/s 2024-04-20T10:35:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:57 smithi162 ceph-mon[21854]: pgmap v61: 97 pgs: 97 active+clean; 135 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 11 MiB/s wr, 9 op/s 2024-04-20T10:35:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:57 smithi046 ceph-mon[21574]: pgmap v61: 97 pgs: 97 active+clean; 135 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 11 MiB/s wr, 9 op/s 2024-04-20T10:35:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:57 smithi042 ceph-mon[17078]: pgmap v61: 97 pgs: 97 active+clean; 135 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 11 MiB/s wr, 9 op/s 2024-04-20T10:36:00.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:35:59 smithi046 ceph-mon[21574]: pgmap v62: 97 pgs: 97 active+clean; 135 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 11 MiB/s wr, 7 op/s 2024-04-20T10:36:00.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:35:59 smithi042 ceph-mon[17078]: pgmap v62: 97 pgs: 97 active+clean; 135 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 11 MiB/s wr, 7 op/s 2024-04-20T10:36:00.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:35:59 smithi162 ceph-mon[21854]: pgmap v62: 97 pgs: 97 active+clean; 135 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 11 MiB/s wr, 7 op/s 2024-04-20T10:36:02.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:01 smithi046 ceph-mon[21574]: pgmap v63: 97 pgs: 97 active+clean; 135 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 11 MiB/s wr, 7 op/s 2024-04-20T10:36:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:01 smithi042 ceph-mon[17078]: pgmap v63: 97 pgs: 97 active+clean; 135 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 11 MiB/s wr, 7 op/s 2024-04-20T10:36:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:01 smithi162 ceph-mon[21854]: pgmap v63: 97 pgs: 97 active+clean; 135 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 11 MiB/s wr, 7 op/s 2024-04-20T10:36:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:03 smithi046 ceph-mon[21574]: pgmap v64: 97 pgs: 97 active+clean; 141 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 12 MiB/s wr, 29 op/s 2024-04-20T10:36:04.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:03 smithi042 ceph-mon[17078]: pgmap v64: 97 pgs: 97 active+clean; 141 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 12 MiB/s wr, 29 op/s 2024-04-20T10:36:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:03 smithi162 ceph-mon[21854]: pgmap v64: 97 pgs: 97 active+clean; 141 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 12 MiB/s wr, 29 op/s 2024-04-20T10:36:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:05 smithi046 ceph-mon[21574]: pgmap v65: 97 pgs: 97 active+clean; 141 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 7.3 MiB/s wr, 26 op/s 2024-04-20T10:36:06.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:05 smithi042 ceph-mon[17078]: pgmap v65: 97 pgs: 97 active+clean; 141 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 7.3 MiB/s wr, 26 op/s 2024-04-20T10:36:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:05 smithi162 ceph-mon[21854]: pgmap v65: 97 pgs: 97 active+clean; 141 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 7.3 MiB/s wr, 26 op/s 2024-04-20T10:36:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:07 smithi046 ceph-mon[21574]: pgmap v66: 97 pgs: 97 active+clean; 144 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 7.7 MiB/s wr, 37 op/s 2024-04-20T10:36:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:07 smithi042 ceph-mon[17078]: pgmap v66: 97 pgs: 97 active+clean; 144 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 7.7 MiB/s wr, 37 op/s 2024-04-20T10:36:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:07 smithi162 ceph-mon[21854]: pgmap v66: 97 pgs: 97 active+clean; 144 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 7.7 MiB/s wr, 37 op/s 2024-04-20T10:36:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:09 smithi046 ceph-mon[21574]: pgmap v67: 97 pgs: 97 active+clean; 144 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 894 KiB/s wr, 32 op/s 2024-04-20T10:36:10.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:09 smithi042 ceph-mon[17078]: pgmap v67: 97 pgs: 97 active+clean; 144 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 894 KiB/s wr, 32 op/s 2024-04-20T10:36:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:09 smithi162 ceph-mon[21854]: pgmap v67: 97 pgs: 97 active+clean; 144 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 894 KiB/s wr, 32 op/s 2024-04-20T10:36:12.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:11 smithi046 ceph-mon[21574]: pgmap v68: 97 pgs: 97 active+clean; 144 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 894 KiB/s wr, 32 op/s 2024-04-20T10:36:12.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:11 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:36:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:11 smithi042 ceph-mon[17078]: pgmap v68: 97 pgs: 97 active+clean; 144 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 894 KiB/s wr, 32 op/s 2024-04-20T10:36:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:11 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:36:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:11 smithi162 ceph-mon[21854]: pgmap v68: 97 pgs: 97 active+clean; 144 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 894 KiB/s wr, 32 op/s 2024-04-20T10:36:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:11 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:36:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:13 smithi046 ceph-mon[21574]: pgmap v69: 97 pgs: 97 active+clean; 148 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-04-20T10:36:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:13 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:36:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:13 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:36:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:13 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:36:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:13 smithi042 ceph-mon[17078]: pgmap v69: 97 pgs: 97 active+clean; 148 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-04-20T10:36:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:13 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:36:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:13 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:36:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:13 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:36:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:13 smithi162 ceph-mon[21854]: pgmap v69: 97 pgs: 97 active+clean; 148 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-04-20T10:36:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:13 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:36:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:13 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:36:14.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:13 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:36:16.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:15 smithi046 ceph-mon[21574]: pgmap v70: 97 pgs: 97 active+clean; 148 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 729 KiB/s wr, 25 op/s 2024-04-20T10:36:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:15 smithi042 ceph-mon[17078]: pgmap v70: 97 pgs: 97 active+clean; 148 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 729 KiB/s wr, 25 op/s 2024-04-20T10:36:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:15 smithi162 ceph-mon[21854]: pgmap v70: 97 pgs: 97 active+clean; 148 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 729 KiB/s wr, 25 op/s 2024-04-20T10:36:16.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:36:16.433 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:36:16.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.661+0000 7fc5d5800640 1 -- 172.21.15.42:0/1335641452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d0100dc0 msgr2=0x7fc5d01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:16.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.661+0000 7fc5d5800640 1 --2- 172.21.15.42:0/1335641452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d0100dc0 0x7fc5d01011a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009ca0 tx=0x7fc5c002c190 comp rx=0 tx=0).stop 2024-04-20T10:36:16.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.662+0000 7fc5d5800640 1 -- 172.21.15.42:0/1335641452 shutdown_connections 2024-04-20T10:36:16.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.662+0000 7fc5d5800640 1 --2- 172.21.15.42:0/1335641452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5d01061c0 0x7fc5d0106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.662+0000 7fc5d5800640 1 --2- 172.21.15.42:0/1335641452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d01016e0 0x7fc5d0105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.662+0000 7fc5d5800640 1 --2- 172.21.15.42:0/1335641452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d0100dc0 0x7fc5d01011a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.662+0000 7fc5d5800640 1 -- 172.21.15.42:0/1335641452 >> 172.21.15.42:0/1335641452 conn(0x7fc5d00fc7b0 msgr2=0x7fc5d00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:16.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.662+0000 7fc5d5800640 1 -- 172.21.15.42:0/1335641452 shutdown_connections 2024-04-20T10:36:16.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.662+0000 7fc5d5800640 1 -- 172.21.15.42:0/1335641452 wait complete. 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.663+0000 7fc5d5800640 1 Processor -- start 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.663+0000 7fc5d5800640 1 -- start start 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.663+0000 7fc5d5800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5d0100dc0 0x7fc5d01a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.663+0000 7fc5d5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d01016e0 0x7fc5d01a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.663+0000 7fc5d5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d01061c0 0x7fc5d019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.663+0000 7fc5d5800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0118780 con 0x7fc5d01061c0 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.663+0000 7fc5d5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0118600 con 0x7fc5d01016e0 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.663+0000 7fc5d5800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0118900 con 0x7fc5d0100dc0 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.663+0000 7fc5c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d01016e0 0x7fc5d01a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.663+0000 7fc5c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d01016e0 0x7fc5d01a4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:45826/0 (socket says 172.21.15.42:45826) 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5c7400640 1 -- 172.21.15.42:0/4061712428 learned_addr learned my addr 172.21.15.42:0/4061712428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5d4e00640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d01061c0 0x7fc5d019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5cfe00640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5d0100dc0 0x7fc5d01a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5c7400640 1 -- 172.21.15.42:0/4061712428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5d0100dc0 msgr2=0x7fc5d01a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5c7400640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5d0100dc0 0x7fc5d01a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5c7400640 1 -- 172.21.15.42:0/4061712428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d01061c0 msgr2=0x7fc5d019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5c7400640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d01061c0 0x7fc5d019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5c7400640 1 -- 172.21.15.42:0/4061712428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d019f280 con 0x7fc5d01016e0 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5c7400640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d01016e0 0x7fc5d01a4e90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800e9f0 tx=0x7fc5b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5cd600640 1 -- 172.21.15.42:0/4061712428 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b800ccc0 con 0x7fc5d01016e0 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5cd600640 1 -- 172.21.15.42:0/4061712428 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b8010910 con 0x7fc5d01016e0 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5d5800640 1 -- 172.21.15.42:0/4061712428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d019f570 con 0x7fc5d01016e0 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.664+0000 7fc5cd600640 1 -- 172.21.15.42:0/4061712428 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8018920 con 0x7fc5d01016e0 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.665+0000 7fc5d5800640 1 -- 172.21.15.42:0/4061712428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d01ab810 con 0x7fc5d01016e0 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.668+0000 7fc5cd600640 1 -- 172.21.15.42:0/4061712428 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc5b8010430 con 0x7fc5d01016e0 2024-04-20T10:36:16.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.668+0000 7fc5d9bb4740 1 Processor -- start 2024-04-20T10:36:16.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.668+0000 7fc5d9bb4740 1 -- start start 2024-04-20T10:36:16.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.669+0000 7fc5d9bb4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32c380 0x560b8d32c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.669+0000 7fc5d9bb4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d32cca0 0x560b8d335340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.669+0000 7fc5d9bb4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d335880 0x560b8d337c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.669+0000 7fc5d9bb4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560b8d292150 con 0x560b8d32cca0 2024-04-20T10:36:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.669+0000 7fc5d9bb4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b8d238370 con 0x560b8d335880 2024-04-20T10:36:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.669+0000 7fc5d9bb4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x560b8d28e110 con 0x560b8d32c380 2024-04-20T10:36:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.669+0000 7fc5c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d335880 0x560b8d337c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.669+0000 7fc5cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d32cca0 0x560b8d335340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.669+0000 7fc5c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d335880 0x560b8d337c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:45836/0 (socket says 172.21.15.42:45836) 2024-04-20T10:36:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.669+0000 7fc5c7400640 1 -- 172.21.15.42:0/3287633355 learned_addr learned my addr 172.21.15.42:0/3287633355 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.669+0000 7fc5d4e00640 1 --2- 172.21.15.42:0/3287633355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32c380 0x560b8d32c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.669+0000 7fc5cd600640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc5b40a3da0 0x7fc5b40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.670+0000 7fc5cd600640 1 -- 172.21.15.42:0/4061712428 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc5b80d09f0 con 0x7fc5d01016e0 2024-04-20T10:36:16.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.670+0000 7fc5c7400640 1 -- 172.21.15.42:0/3287633355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32c380 msgr2=0x560b8d32c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.670+0000 7fc5c7400640 1 --2- 172.21.15.42:0/3287633355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32c380 0x560b8d32c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.670+0000 7fc5c7400640 1 -- 172.21.15.42:0/3287633355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d32cca0 msgr2=0x560b8d335340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.670+0000 7fc5c7400640 1 --2- 172.21.15.42:0/3287633355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d32cca0 0x560b8d335340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.670+0000 7fc5c7400640 1 -- 172.21.15.42:0/3287633355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b8d06e7d0 con 0x560b8d335880 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.670+0000 7fc5d4e00640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc5b40a3da0 0x7fc5b40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.670+0000 7fc5c7400640 1 --2- 172.21.15.42:0/3287633355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d335880 0x560b8d337c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80ce050 tx=0x7fc5b8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92fe9f88c96967a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.670+0000 7fc5c5600640 1 -- 172.21.15.42:0/3287633355 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b80a0ce0 con 0x560b8d335880 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.670+0000 7fc5c5600640 1 -- 172.21.15.42:0/3287633355 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b80a55a0 con 0x560b8d335880 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.670+0000 7fc5c5600640 1 -- 172.21.15.42:0/3287633355 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b80870d0 con 0x560b8d335880 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.671+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/3287633355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d335880 msgr2=0x560b8d337c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.671+0000 7fc5d4e00640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc5b40a3da0 0x7fc5b40a6260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80058c0 tx=0x7fc5c80046c0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.671+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/3287633355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d335880 0x560b8d337c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80ce050 tx=0x7fc5b8009f90 comp rx=0 tx=0).stop 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.671+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/3287633355 shutdown_connections 2024-04-20T10:36:16.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.671+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/3287633355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d335880 0x560b8d337c70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.671+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/3287633355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d32cca0 0x560b8d335340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.671+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/3287633355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32c380 0x560b8d32c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.671+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/3287633355 >> 172.21.15.42:0/3287633355 conn(0x560b8d326ab0 msgr2=0x560b8d326e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.671+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/3287633355 shutdown_connections 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.671+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/3287633355 wait complete. 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.672+0000 7fc5d9bb4740 1 -- start start 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.672+0000 7fc5d9bb4740 1 Processor -- start 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.673+0000 7fc5d9bb4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32c380 0x560b8d332910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.673+0000 7fc5d9bb4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32cca0 0x560b8d332e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.673+0000 7fc5d9bb4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d335880 0x560b8d333390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.673+0000 7fc5d9bb4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560b8d338610 con 0x560b8d335880 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.673+0000 7fc5d9bb4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560b8d24b740 con 0x560b8d32c380 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.673+0000 7fc5d9bb4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x560b8d338790 con 0x560b8d32cca0 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.673+0000 7fc5d9bb4740 10 client.0 _ll_register_callbacks cb 0x560b8d246b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:16.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.673+0000 7fc5cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32c380 0x560b8d332910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.673+0000 7fc5cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32c380 0x560b8d332910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:45838/0 (socket says 172.21.15.42:45838) 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.673+0000 7fc5cfe00640 1 -- 172.21.15.42:0/899946996 learned_addr learned my addr 172.21.15.42:0/899946996 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.673+0000 7fc5c7400640 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32cca0 0x560b8d332e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.673+0000 7fc5d4e00640 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d335880 0x560b8d333390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.674+0000 7fc5cfe00640 1 -- 172.21.15.42:0/899946996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32cca0 msgr2=0x560b8d332e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.674+0000 7fc5cfe00640 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32cca0 0x560b8d332e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.674+0000 7fc5cfe00640 1 -- 172.21.15.42:0/899946996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d335880 msgr2=0x560b8d333390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.674+0000 7fc5cfe00640 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d335880 0x560b8d333390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.674+0000 7fc5cfe00640 1 -- 172.21.15.42:0/899946996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b8cfa43f0 con 0x560b8d32c380 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.674+0000 7fc5d4e00640 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d335880 0x560b8d333390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.674+0000 7fc5cfe00640 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32c380 0x560b8d332910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0037040 tx=0x7fc5c0003f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb6eb68e5577b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.674+0000 7fc5be000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:36:16.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.674+0000 7fc5be000640 1 -- 172.21.15.42:0/899946996 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0044070 con 0x560b8d32c380 2024-04-20T10:36:16.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.674+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/899946996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5b80a55a0 con 0x560b8d32c380 2024-04-20T10:36:16.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.674+0000 7fc5be000640 1 -- 172.21.15.42:0/899946996 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc5c0004040 con 0x560b8d32c380 2024-04-20T10:36:16.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.675+0000 7fc5be000640 1 -- 172.21.15.42:0/899946996 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc5c0039040 con 0x560b8d32c380 2024-04-20T10:36:16.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.675+0000 7fc5d9bb4740 10 client.25604 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:16.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.675+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/899946996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560b8cf640f0 con 0x560b8d32c380 2024-04-20T10:36:16.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.675+0000 7fc5be000640 1 -- 172.21.15.42:0/899946996 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b80a55a0 con 0x560b8d32c380 2024-04-20T10:36:16.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.675+0000 7fc5be000640 1 -- 172.21.15.42:0/899946996 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc5c0032ab0 con 0x560b8d32c380 2024-04-20T10:36:16.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.675+0000 7fc5d9bb4740 10 client.25604 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:16.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.675+0000 7fc5d9bb4740 10 client.25604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:16.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.675+0000 7fc5d9bb4740 20 client.25604 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:16.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.675+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560b8d345ba0 0x560b8d347ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.675+0000 7fc5d9bb4740 4 client.25604 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:36:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.675+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/899946996 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b8d29fb70 con 0x560b8d345ba0 2024-04-20T10:36:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.676+0000 7fc5c7400640 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560b8d345ba0 0x560b8d347ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.676+0000 7fc5c7400640 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560b8d345ba0 0x560b8d347ff0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d273837bf1baa37 server_cookie=87537fe558977d8c in_seq=0 out_seq=0 2024-04-20T10:36:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.676+0000 7fc5be000640 10 client.25604 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.678+0000 7fc5be000640 1 -- 172.21.15.42:0/899946996 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b8d29fb70 con 0x560b8d345ba0 2024-04-20T10:36:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.678+0000 7fc5be000640 10 client.25604 handle_command_reply: tid=0 2024-04-20T10:36:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.678+0000 7fc5d9bb4740 1 client.25604 shutdown 2024-04-20T10:36:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.678+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/899946996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32c380 msgr2=0x560b8d332910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.678+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32c380 0x560b8d332910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0037040 tx=0x7fc5c0003f00 comp rx=0 tx=0).stop 2024-04-20T10:36:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.678+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/899946996 shutdown_connections 2024-04-20T10:36:16.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.678+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560b8d345ba0 0x560b8d347ff0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.678+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d335880 0x560b8d333390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.678+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32cca0 0x560b8d332e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.678+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/899946996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32c380 0x560b8d332910 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.678+0000 7fc5be000640 0 client.25604 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:16.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.678+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/899946996 >> 172.21.15.42:0/899946996 conn(0x560b8d326ab0 msgr2=0x560b8d328800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:16.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.679+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/899946996 shutdown_connections 2024-04-20T10:36:16.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.679+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/899946996 wait complete. 2024-04-20T10:36:16.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.679+0000 7fc5d9bb4740 20 client.25604 trim_cache size 0 max 16384 2024-04-20T10:36:16.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.701+0000 7fc5d9bb4740 1 Processor -- start 2024-04-20T10:36:16.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.701+0000 7fc5d9bb4740 1 -- start start 2024-04-20T10:36:16.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.701+0000 7fc5d9bb4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32c380 0x560b8d335040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.701+0000 7fc5d9bb4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 0x560b8d3414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.701+0000 7fc5d9bb4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d335880 0x560b8d331a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.701+0000 7fc5d9bb4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01b2210 con 0x560b8d335880 2024-04-20T10:36:16.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.701+0000 7fc5d9bb4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01b4a90 con 0x560b8d32cca0 2024-04-20T10:36:16.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.701+0000 7fc5d9bb4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01b2880 con 0x560b8d32c380 2024-04-20T10:36:16.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.702+0000 7fc5c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 0x560b8d3414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.702+0000 7fc5c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 0x560b8d3414d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:45852/0 (socket says 172.21.15.42:45852) 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.702+0000 7fc5c7400640 1 -- 172.21.15.42:0/3445028312 learned_addr learned my addr 172.21.15.42:0/3445028312 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.702+0000 7fc5cfe00640 1 --2- 172.21.15.42:0/3445028312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32c380 0x560b8d335040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.702+0000 7fc5d4e00640 1 --2- 172.21.15.42:0/3445028312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d335880 0x560b8d331a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.702+0000 7fc5c7400640 1 -- 172.21.15.42:0/3445028312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32c380 msgr2=0x560b8d335040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.702+0000 7fc5c7400640 1 --2- 172.21.15.42:0/3445028312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32c380 0x560b8d335040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.702+0000 7fc5c7400640 1 -- 172.21.15.42:0/3445028312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d335880 msgr2=0x560b8d331a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.702+0000 7fc5c7400640 1 --2- 172.21.15.42:0/3445028312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d335880 0x560b8d331a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.702+0000 7fc5c7400640 1 -- 172.21.15.42:0/3445028312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b8d32b6a0 con 0x560b8d32cca0 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.702+0000 7fc5cfe00640 1 --2- 172.21.15.42:0/3445028312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32c380 0x560b8d335040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.702+0000 7fc5c7400640 1 --2- 172.21.15.42:0/3445028312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 0x560b8d3414d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800e9c0 tx=0x7fc5b80bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=873e512cfb8fd447 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.703+0000 7fc5c4c00640 1 -- 172.21.15.42:0/3445028312 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8010720 con 0x560b8d32cca0 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.703+0000 7fc5c4c00640 1 -- 172.21.15.42:0/3445028312 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b80a55a0 con 0x560b8d32cca0 2024-04-20T10:36:16.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.703+0000 7fc5c4c00640 1 -- 172.21.15.42:0/3445028312 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b80991c0 con 0x560b8d32cca0 2024-04-20T10:36:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.703+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/3445028312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 msgr2=0x560b8d3414d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.703+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/3445028312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 0x560b8d3414d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800e9c0 tx=0x7fc5b80bd040 comp rx=0 tx=0).stop 2024-04-20T10:36:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.704+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/3445028312 shutdown_connections 2024-04-20T10:36:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.704+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/3445028312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d335880 0x560b8d331a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.704+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/3445028312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 0x560b8d3414d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.704+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/3445028312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d32c380 0x560b8d335040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.704+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/3445028312 >> 172.21.15.42:0/3445028312 conn(0x560b8d326ab0 msgr2=0x560b8d327ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.704+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/3445028312 shutdown_connections 2024-04-20T10:36:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.704+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/3445028312 wait complete. 2024-04-20T10:36:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.704+0000 7fc5d9bb4740 1 -- start start 2024-04-20T10:36:16.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.704+0000 7fc5d9bb4740 1 Processor -- start 2024-04-20T10:36:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5d9bb4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d32c380 0x560b8d33dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5d9bb4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 0x560b8d339020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5d9bb4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d335880 0x560b8d339560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5d9bb4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01b2f50 con 0x560b8d32c380 2024-04-20T10:36:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5d9bb4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01b43c0 con 0x560b8d32cca0 2024-04-20T10:36:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5d9bb4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01b3620 con 0x560b8d335880 2024-04-20T10:36:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5d9bb4740 10 client.0 _ll_register_callbacks cb 0x7fc5c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d32c380 0x560b8d33dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 0x560b8d339020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 0x560b8d339020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:45866/0 (socket says 172.21.15.42:45866) 2024-04-20T10:36:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5c7400640 1 -- 172.21.15.42:0/2846901083 learned_addr learned my addr 172.21.15.42:0/2846901083 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:16.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d32c380 0x560b8d33dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36462/0 (socket says 172.21.15.42:36462) 2024-04-20T10:36:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.705+0000 7fc5d4e00640 1 --2- 172.21.15.42:0/2846901083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d335880 0x560b8d339560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5c7400640 1 -- 172.21.15.42:0/2846901083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d335880 msgr2=0x560b8d339560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5c7400640 1 --2- 172.21.15.42:0/2846901083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d335880 0x560b8d339560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5c7400640 1 -- 172.21.15.42:0/2846901083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d32c380 msgr2=0x560b8d33dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5c7400640 1 --2- 172.21.15.42:0/2846901083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d32c380 0x560b8d33dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5c7400640 1 -- 172.21.15.42:0/2846901083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b8d339c50 con 0x560b8d32cca0 2024-04-20T10:36:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5c7400640 1 --2- 172.21.15.42:0/2846901083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 0x560b8d339020 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80993b0 tx=0x7fc5b80a0d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc929ab5a78efb4b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5be000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:36:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5be000640 1 -- 172.21.15.42:0/2846901083 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8010ac0 con 0x560b8d32cca0 2024-04-20T10:36:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5be000640 1 -- 172.21.15.42:0/2846901083 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc5b8099050 con 0x560b8d32cca0 2024-04-20T10:36:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/2846901083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5b80a55a0 con 0x560b8d32cca0 2024-04-20T10:36:16.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5be000640 1 -- 172.21.15.42:0/2846901083 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc5b80ce360 con 0x560b8d32cca0 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5d9bb4740 10 client.25610 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.706+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/2846901083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560b8d3ac040 con 0x560b8d32cca0 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.707+0000 7fc5be000640 1 -- 172.21.15.42:0/2846901083 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b80a55a0 con 0x560b8d32cca0 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.707+0000 7fc5be000640 1 -- 172.21.15.42:0/2846901083 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc5b809be30 con 0x560b8d32cca0 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.707+0000 7fc5d9bb4740 10 client.25610 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.707+0000 7fc5d9bb4740 10 client.25610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.707+0000 7fc5d9bb4740 20 client.25610 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.707+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/2846901083 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560b8d3ac2d0 0x560b8d3ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.707+0000 7fc5d9bb4740 4 client.25610 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.707+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/2846901083 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b8d2999e0 con 0x560b8d3ac2d0 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.707+0000 7fc5cfe00640 1 --2- 172.21.15.42:0/2846901083 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560b8d3ac2d0 0x560b8d3ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.708+0000 7fc5cfe00640 1 --2- 172.21.15.42:0/2846901083 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560b8d3ac2d0 0x560b8d3ae6c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724ec06cd73d2401 server_cookie=552bf21f9b9b7991 in_seq=0 out_seq=0 2024-04-20T10:36:16.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.708+0000 7fc5be000640 10 client.25610 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:16.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.710+0000 7fc5be000640 1 -- 172.21.15.42:0/2846901083 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560b8d2999e0 con 0x560b8d3ac2d0 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.710+0000 7fc5be000640 10 client.25610 handle_command_reply: tid=0 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.710+0000 7fc5d9bb4740 1 client.25610 shutdown 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.710+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/2846901083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 msgr2=0x560b8d339020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.710+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/2846901083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 0x560b8d339020 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80993b0 tx=0x7fc5b80a0d80 comp rx=0 tx=0).stop 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.711+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/2846901083 shutdown_connections 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.711+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/2846901083 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560b8d3ac2d0 0x560b8d3ae6c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.711+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/2846901083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560b8d335880 0x560b8d339560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.711+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/2846901083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8d32cca0 0x560b8d339020 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.711+0000 7fc5d9bb4740 1 --2- 172.21.15.42:0/2846901083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b8d32c380 0x560b8d33dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.711+0000 7fc5be000640 0 client.25610 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.711+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/2846901083 >> 172.21.15.42:0/2846901083 conn(0x560b8d326ab0 msgr2=0x560b8d327ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.711+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/2846901083 shutdown_connections 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.711+0000 7fc5d9bb4740 1 -- 172.21.15.42:0/2846901083 wait complete. 2024-04-20T10:36:16.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.711+0000 7fc5d9bb4740 20 client.25610 trim_cache size 0 max 16384 2024-04-20T10:36:16.714 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:36:16.714 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:36:16.714 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:36:16.714 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:36:16.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.712+0000 7fc5d5800640 1 -- 172.21.15.42:0/4061712428 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc5b40a3da0 msgr2=0x7fc5b40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:16.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.712+0000 7fc5d5800640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc5b40a3da0 0x7fc5b40a6260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80058c0 tx=0x7fc5c80046c0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.712+0000 7fc5d5800640 1 -- 172.21.15.42:0/4061712428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d01016e0 msgr2=0x7fc5d01a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:16.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.712+0000 7fc5d5800640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d01016e0 0x7fc5d01a4e90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800e9f0 tx=0x7fc5b800eeb0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.712+0000 7fc5d5800640 1 -- 172.21.15.42:0/4061712428 shutdown_connections 2024-04-20T10:36:16.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.712+0000 7fc5d5800640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc5b40a3da0 0x7fc5b40a6260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.712+0000 7fc5d5800640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5d01061c0 0x7fc5d019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.712+0000 7fc5d5800640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d01016e0 0x7fc5d01a4e90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.712+0000 7fc5d5800640 1 --2- 172.21.15.42:0/4061712428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5d0100dc0 0x7fc5d01a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.712+0000 7fc5d5800640 1 -- 172.21.15.42:0/4061712428 >> 172.21.15.42:0/4061712428 conn(0x7fc5d00fc7b0 msgr2=0x7fc5d00fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:16.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.712+0000 7fc5d5800640 1 -- 172.21.15.42:0/4061712428 shutdown_connections 2024-04-20T10:36:16.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.712+0000 7fc5d5800640 1 -- 172.21.15.42:0/4061712428 wait complete. 2024-04-20T10:36:16.731 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:36:16.731 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:36:16.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.950+0000 7f68b0e00640 1 -- 172.21.15.42:0/3011824441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ac102b50 msgr2=0x7f68ac106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:16.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.950+0000 7f68b0e00640 1 --2- 172.21.15.42:0/3011824441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ac102b50 0x7f68ac106b90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6894009900 tx=0x7f689402bc30 comp rx=0 tx=0).stop 2024-04-20T10:36:16.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.950+0000 7f68b0e00640 1 -- 172.21.15.42:0/3011824441 shutdown_connections 2024-04-20T10:36:16.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.950+0000 7f68b0e00640 1 --2- 172.21.15.42:0/3011824441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68ac107270 0x7f68ac10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.950+0000 7f68b0e00640 1 --2- 172.21.15.42:0/3011824441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ac102b50 0x7f68ac106b90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.950+0000 7f68b0e00640 1 --2- 172.21.15.42:0/3011824441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68ac1021a0 0x7f68ac102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.950+0000 7f68b0e00640 1 -- 172.21.15.42:0/3011824441 >> 172.21.15.42:0/3011824441 conn(0x7f68ac079320 msgr2=0x7f68ac079730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:16.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.950+0000 7f68b0e00640 1 -- 172.21.15.42:0/3011824441 shutdown_connections 2024-04-20T10:36:16.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.950+0000 7f68b0e00640 1 -- 172.21.15.42:0/3011824441 wait complete. 2024-04-20T10:36:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.951+0000 7f68b0e00640 1 Processor -- start 2024-04-20T10:36:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.951+0000 7f68b0e00640 1 -- start start 2024-04-20T10:36:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.952+0000 7f68b0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ac1021a0 0x7f68ac19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.952+0000 7f68b0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68ac107270 0x7f68ac19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.952+0000 7f68b0e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68ac198670 0x7f68ac198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.952+0000 7f68b0e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac112350 con 0x7f68ac107270 2024-04-20T10:36:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.952+0000 7f68b0e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac1121d0 con 0x7f68ac1021a0 2024-04-20T10:36:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.952+0000 7f68b0e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac1124d0 con 0x7f68ac198670 2024-04-20T10:36:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.952+0000 7f68a9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ac1021a0 0x7f68ac19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.952+0000 7f68a9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ac1021a0 0x7f68ac19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:45886/0 (socket says 172.21.15.42:45886) 2024-04-20T10:36:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.952+0000 7f68a9e00640 1 -- 172.21.15.42:0/3346222627 learned_addr learned my addr 172.21.15.42:0/3346222627 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:16.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.952+0000 7f68a3e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68ac107270 0x7f68ac19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.952+0000 7f68aa800640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68ac198670 0x7f68ac198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.953+0000 7f68a9e00640 1 -- 172.21.15.42:0/3346222627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68ac198670 msgr2=0x7f68ac198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:16.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.953+0000 7f68a9e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68ac198670 0x7f68ac198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.953+0000 7f68a9e00640 1 -- 172.21.15.42:0/3346222627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68ac107270 msgr2=0x7f68ac19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:16.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.953+0000 7f68a9e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68ac107270 0x7f68ac19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:16.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.953+0000 7f68a9e00640 1 -- 172.21.15.42:0/3346222627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ac199270 con 0x7f68ac1021a0 2024-04-20T10:36:16.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.953+0000 7f68a3e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68ac107270 0x7f68ac19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.953+0000 7f68a9e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ac1021a0 0x7f68ac19e5b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f689c00e9f0 tx=0x7f689c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.953+0000 7f68a2a00640 1 -- 172.21.15.42:0/3346222627 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c00cd80 con 0x7f68ac1021a0 2024-04-20T10:36:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.953+0000 7f68b0e00640 1 -- 172.21.15.42:0/3346222627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ac1a5350 con 0x7f68ac1021a0 2024-04-20T10:36:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.953+0000 7f68a2a00640 1 -- 172.21.15.42:0/3346222627 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f689c010910 con 0x7f68ac1021a0 2024-04-20T10:36:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.953+0000 7f68a2a00640 1 -- 172.21.15.42:0/3346222627 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c018990 con 0x7f68ac1021a0 2024-04-20T10:36:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.953+0000 7f68b0e00640 1 -- 172.21.15.42:0/3346222627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ac1a5700 con 0x7f68ac1021a0 2024-04-20T10:36:16.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.955+0000 7f68b0e00640 1 -- 172.21.15.42:0/3346222627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ac103670 con 0x7f68ac1021a0 2024-04-20T10:36:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.956+0000 7f68a2a00640 1 -- 172.21.15.42:0/3346222627 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f689c010ac0 con 0x7f68ac1021a0 2024-04-20T10:36:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.958+0000 7f68a2a00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f68900a3d20 0x7f68900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.958+0000 7f68a2a00640 1 -- 172.21.15.42:0/3346222627 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f689c0d0b80 con 0x7f68ac1021a0 2024-04-20T10:36:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.958+0000 7f68a3e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f68900a3d20 0x7f68900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:16.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.959+0000 7f68a3e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f68900a3d20 0x7f68900a61e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f689402c240 tx=0x7f689402df90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:16.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:16.962+0000 7f68a2a00640 1 -- 172.21.15.42:0/3346222627 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f689c09cb70 con 0x7f68ac1021a0 2024-04-20T10:36:17.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.223+0000 7f68b0e00640 1 -- 172.21.15.42:0/3346222627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68ac10a500 con 0x7f68ac1021a0 2024-04-20T10:36:17.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.225+0000 7f68a2a00640 1 -- 172.21.15.42:0/3346222627 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f689c0a0e70 con 0x7f68ac1021a0 2024-04-20T10:36:17.226 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:36:17.226 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:36:17.227 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:36:17.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.228+0000 7f68b0e00640 1 -- 172.21.15.42:0/3346222627 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f68900a3d20 msgr2=0x7f68900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:17.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.228+0000 7f68b0e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f68900a3d20 0x7f68900a61e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f689402c240 tx=0x7f689402df90 comp rx=0 tx=0).stop 2024-04-20T10:36:17.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.228+0000 7f68b0e00640 1 -- 172.21.15.42:0/3346222627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ac1021a0 msgr2=0x7f68ac19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:17.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.228+0000 7f68b0e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ac1021a0 0x7f68ac19e5b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f689c00e9f0 tx=0x7f689c00eeb0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.228+0000 7f68b0e00640 1 -- 172.21.15.42:0/3346222627 shutdown_connections 2024-04-20T10:36:17.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.228+0000 7f68b0e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f68900a3d20 0x7f68900a61e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.228+0000 7f68b0e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68ac198670 0x7f68ac198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.228+0000 7f68b0e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68ac107270 0x7f68ac19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.228+0000 7f68b0e00640 1 --2- 172.21.15.42:0/3346222627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ac1021a0 0x7f68ac19e5b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.228+0000 7f68b0e00640 1 -- 172.21.15.42:0/3346222627 >> 172.21.15.42:0/3346222627 conn(0x7f68ac079320 msgr2=0x7f68ac1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:17.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.228+0000 7f68b0e00640 1 -- 172.21.15.42:0/3346222627 shutdown_connections 2024-04-20T10:36:17.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.229+0000 7f68b0e00640 1 -- 172.21.15.42:0/3346222627 wait complete. 2024-04-20T10:36:17.250 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:36:17.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.471+0000 7ff4c2400640 1 -- 172.21.15.42:0/692048761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc10d3d0 msgr2=0x7ff4bc10d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:17.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.471+0000 7ff4c2400640 1 --2- 172.21.15.42:0/692048761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc10d3d0 0x7ff4bc10d7b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009860 tx=0x7ff4b002bc10 comp rx=0 tx=0).stop 2024-04-20T10:36:17.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.471+0000 7ff4c2400640 1 -- 172.21.15.42:0/692048761 shutdown_connections 2024-04-20T10:36:17.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.471+0000 7ff4c2400640 1 --2- 172.21.15.42:0/692048761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4bc108ea0 0x7ff4bc1152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.471+0000 7ff4c2400640 1 --2- 172.21.15.42:0/692048761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4bc108500 0x7ff4bc108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.471+0000 7ff4c2400640 1 --2- 172.21.15.42:0/692048761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc10d3d0 0x7ff4bc10d7b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.471+0000 7ff4c2400640 1 -- 172.21.15.42:0/692048761 >> 172.21.15.42:0/692048761 conn(0x7ff4bc077d80 msgr2=0x7ff4bc106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:17.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.471+0000 7ff4c2400640 1 -- 172.21.15.42:0/692048761 shutdown_connections 2024-04-20T10:36:17.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.471+0000 7ff4c2400640 1 -- 172.21.15.42:0/692048761 wait complete. 2024-04-20T10:36:17.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.472+0000 7ff4c2400640 1 Processor -- start 2024-04-20T10:36:17.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.473+0000 7ff4c2400640 1 -- start start 2024-04-20T10:36:17.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.473+0000 7ff4c2400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc108500 0x7ff4bc1a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.473+0000 7ff4c2400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4bc108ea0 0x7ff4bc1a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.473+0000 7ff4c2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4bc10d3d0 0x7ff4bc19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.473+0000 7ff4c2400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1187f0 con 0x7ff4bc108ea0 2024-04-20T10:36:17.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.473+0000 7ff4c2400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc118670 con 0x7ff4bc10d3d0 2024-04-20T10:36:17.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.473+0000 7ff4c2400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc118970 con 0x7ff4bc108500 2024-04-20T10:36:17.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.473+0000 7ff4bbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4bc108ea0 0x7ff4bc1a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.473+0000 7ff4c1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4bc10d3d0 0x7ff4bc19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.473+0000 7ff4c1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4bc10d3d0 0x7ff4bc19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:45916/0 (socket says 172.21.15.42:45916) 2024-04-20T10:36:17.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.473+0000 7ff4c1a00640 1 -- 172.21.15.42:0/1494645863 learned_addr learned my addr 172.21.15.42:0/1494645863 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:17.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.473+0000 7ff4c1000640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc108500 0x7ff4bc1a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.474+0000 7ff4c1a00640 1 -- 172.21.15.42:0/1494645863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc108500 msgr2=0x7ff4bc1a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:17.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.474+0000 7ff4c1a00640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc108500 0x7ff4bc1a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.474+0000 7ff4c1a00640 1 -- 172.21.15.42:0/1494645863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4bc108ea0 msgr2=0x7ff4bc1a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:17.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.474+0000 7ff4c1a00640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4bc108ea0 0x7ff4bc1a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.474+0000 7ff4c1a00640 1 -- 172.21.15.42:0/1494645863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc19f2d0 con 0x7ff4bc10d3d0 2024-04-20T10:36:17.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.474+0000 7ff4c1a00640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4bc10d3d0 0x7ff4bc19eb50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400e960 tx=0x7ff4b400ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:17.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.474+0000 7ff4b9600640 1 -- 172.21.15.42:0/1494645863 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4004d20 con 0x7ff4bc10d3d0 2024-04-20T10:36:17.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.474+0000 7ff4b9600640 1 -- 172.21.15.42:0/1494645863 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b4005020 con 0x7ff4bc10d3d0 2024-04-20T10:36:17.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.474+0000 7ff4c2400640 1 -- 172.21.15.42:0/1494645863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4bc19f5c0 con 0x7ff4bc10d3d0 2024-04-20T10:36:17.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.475+0000 7ff4b9600640 1 -- 172.21.15.42:0/1494645863 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40116d0 con 0x7ff4bc10d3d0 2024-04-20T10:36:17.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.475+0000 7ff4c2400640 1 -- 172.21.15.42:0/1494645863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4bc1ab810 con 0x7ff4bc10d3d0 2024-04-20T10:36:17.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.478+0000 7ff4c6797740 1 Processor -- start 2024-04-20T10:36:17.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.478+0000 7ff4b9600640 1 -- 172.21.15.42:0/1494645863 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff4b4005560 con 0x7ff4bc10d3d0 2024-04-20T10:36:17.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.478+0000 7ff4c6797740 1 -- start start 2024-04-20T10:36:17.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.478+0000 7ff4c6797740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc6e170 0x55c64dc6e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.479+0000 7ff4c6797740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc77130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.479+0000 7ff4c6797740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc77670 0x55c64dc79a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.479+0000 7ff4c6797740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c64dbd3050 con 0x55c64dc6e170 2024-04-20T10:36:17.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.479+0000 7ff4c6797740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c64db79490 con 0x55c64dc77670 2024-04-20T10:36:17.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.479+0000 7ff4c6797740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c64dbcf010 con 0x55c64dc6ea90 2024-04-20T10:36:17.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.479+0000 7ff4bbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc6e170 0x55c64dc6e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.479+0000 7ff4c1000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc77130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.479+0000 7ff4c1000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc77130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37002/0 (socket says 172.21.15.42:37002) 2024-04-20T10:36:17.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.479+0000 7ff4c1000640 1 -- 172.21.15.42:0/482036107 learned_addr learned my addr 172.21.15.42:0/482036107 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:17.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.479+0000 7ff4bbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc6e170 0x55c64dc6e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36516/0 (socket says 172.21.15.42:36516) 2024-04-20T10:36:17.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.479+0000 7ff4bbe00640 1 --2- 172.21.15.42:0/482036107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc77670 0x55c64dc79a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4bbe00640 1 -- 172.21.15.42:0/482036107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 msgr2=0x55c64dc77130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:17.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4bbe00640 1 --2- 172.21.15.42:0/482036107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc77130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4bbe00640 1 -- 172.21.15.42:0/482036107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc77670 msgr2=0x55c64dc79a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:17.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4bbe00640 1 --2- 172.21.15.42:0/482036107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc77670 0x55c64dc79a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4bbe00640 1 -- 172.21.15.42:0/482036107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c64db81070 con 0x55c64dc6e170 2024-04-20T10:36:17.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4c1000640 1 --2- 172.21.15.42:0/482036107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc77130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:17.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4b9600640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff4a40a3da0 0x7ff4a40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4b9600640 1 -- 172.21.15.42:0/1494645863 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff4b40d0830 con 0x7ff4bc10d3d0 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4bbe00640 1 --2- 172.21.15.42:0/482036107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc6e170 0x55c64dc6e550 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc10e870 tx=0x7ff4a8009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295e8ff258ff1faf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4c1000640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff4a40a3da0 0x7ff4a40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4ae000640 1 -- 172.21.15.42:0/482036107 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8013cd0 con 0x55c64dc6e170 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4ae000640 1 -- 172.21.15.42:0/482036107 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a8005020 con 0x55c64dc6e170 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.480+0000 7ff4ae000640 1 -- 172.21.15.42:0/482036107 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80049c0 con 0x55c64dc6e170 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.481+0000 7ff4c6797740 1 -- 172.21.15.42:0/482036107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc6e170 msgr2=0x55c64dc6e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.481+0000 7ff4c1000640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff4a40a3da0 0x7ff4a40a6260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0000c00 tx=0x7ff4b00056d0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.481+0000 7ff4c6797740 1 --2- 172.21.15.42:0/482036107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc6e170 0x55c64dc6e550 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc10e870 tx=0x7ff4a8009640 comp rx=0 tx=0).stop 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.481+0000 7ff4c6797740 1 -- 172.21.15.42:0/482036107 shutdown_connections 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.481+0000 7ff4c6797740 1 --2- 172.21.15.42:0/482036107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc77670 0x55c64dc79a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.481+0000 7ff4c6797740 1 --2- 172.21.15.42:0/482036107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc77130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.481+0000 7ff4c6797740 1 --2- 172.21.15.42:0/482036107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc6e170 0x55c64dc6e550 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.481+0000 7ff4c6797740 1 -- 172.21.15.42:0/482036107 >> 172.21.15.42:0/482036107 conn(0x55c64dc688a0 msgr2=0x55c64dc68c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.481+0000 7ff4c6797740 1 -- 172.21.15.42:0/482036107 shutdown_connections 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.481+0000 7ff4c6797740 1 -- 172.21.15.42:0/482036107 wait complete. 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.482+0000 7ff4c6797740 1 -- start start 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.482+0000 7ff4c6797740 1 Processor -- start 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.482+0000 7ff4c6797740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c6797740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc74b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c6797740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc77670 0x55c64dc75040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c6797740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c64dc7a450 con 0x55c64dc77670 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c6797740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c64db8c860 con 0x55c64dc6e170 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c6797740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c64dc7a5d0 con 0x55c64dc6ea90 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c6797740 10 client.0 _ll_register_callbacks cb 0x55c64db5afe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc745c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:45938/0 (socket says 172.21.15.42:45938) 2024-04-20T10:36:17.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c1000640 1 -- 172.21.15.42:0/2493975636 learned_addr learned my addr 172.21.15.42:0/2493975636 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:17.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c1a00640 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc77670 0x55c64dc75040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4bbe00640 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc74b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c1000640 1 -- 172.21.15.42:0/2493975636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 msgr2=0x55c64dc74b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:17.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c1000640 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc74b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c1000640 1 -- 172.21.15.42:0/2493975636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc77670 msgr2=0x55c64dc75040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:17.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c1000640 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc77670 0x55c64dc75040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.483+0000 7ff4c1000640 1 -- 172.21.15.42:0/2493975636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c64db90950 con 0x55c64dc6e170 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.484+0000 7ff4c1000640 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc745c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff4b002c220 tx=0x7ff4b0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d5bbdd42ea95f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.484+0000 7ff4a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.484+0000 7ff4a2a00640 1 -- 172.21.15.42:0/2493975636 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0039040 con 0x55c64dc6e170 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.484+0000 7ff4a2a00640 1 -- 172.21.15.42:0/2493975636 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff4b002c7b0 con 0x55c64dc6e170 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.484+0000 7ff4c6797740 1 -- 172.21.15.42:0/2493975636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a8005020 con 0x55c64dc6e170 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.484+0000 7ff4a2a00640 1 -- 172.21.15.42:0/2493975636 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff4b0059050 con 0x55c64dc6e170 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.484+0000 7ff4c6797740 10 client.25628 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.484+0000 7ff4c6797740 1 -- 172.21.15.42:0/2493975636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c64db5e820 con 0x55c64dc6e170 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.485+0000 7ff4a2a00640 1 -- 172.21.15.42:0/2493975636 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a8005020 con 0x55c64dc6e170 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.485+0000 7ff4a2a00640 1 -- 172.21.15.42:0/2493975636 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7ff4b004b460 con 0x55c64dc6e170 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.485+0000 7ff4c6797740 10 client.25628 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.485+0000 7ff4c6797740 10 client.25628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.485+0000 7ff4c6797740 20 client.25628 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.485+0000 7ff4c6797740 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c64dc87620 0x55c64dc89aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.485+0000 7ff4c6797740 4 client.25628 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.485+0000 7ff4c6797740 1 -- 172.21.15.42:0/2493975636 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c64dbe0a70 con 0x55c64dc87620 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.485+0000 7ff4bbe00640 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c64dc87620 0x55c64dc89aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.486+0000 7ff4bbe00640 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c64dc87620 0x55c64dc89aa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd691f8c0bc758a server_cookie=df3c6dcff3aca399 in_seq=0 out_seq=0 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.486+0000 7ff4a2a00640 10 client.25628 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4a2a00640 1 -- 172.21.15.42:0/2493975636 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c64dbe0a70 con 0x55c64dc87620 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4a2a00640 10 client.25628 handle_command_reply: tid=0 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4c6797740 1 client.25628 shutdown 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4c6797740 1 -- 172.21.15.42:0/2493975636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 msgr2=0x55c64dc745c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4c6797740 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc745c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff4b002c220 tx=0x7ff4b0005f90 comp rx=0 tx=0).stop 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4c6797740 1 -- 172.21.15.42:0/2493975636 shutdown_connections 2024-04-20T10:36:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4c6797740 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c64dc87620 0x55c64dc89aa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4c6797740 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc77670 0x55c64dc75040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4c6797740 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc74b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4c6797740 1 --2- 172.21.15.42:0/2493975636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc745c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4c6797740 1 -- 172.21.15.42:0/2493975636 >> 172.21.15.42:0/2493975636 conn(0x55c64dc688a0 msgr2=0x55c64dc6a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:17.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4a2a00640 0 client.25628 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:17.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.487+0000 7ff4c6797740 1 -- 172.21.15.42:0/2493975636 shutdown_connections 2024-04-20T10:36:17.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.488+0000 7ff4c6797740 1 -- 172.21.15.42:0/2493975636 wait complete. 2024-04-20T10:36:17.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.488+0000 7ff4c6797740 20 client.25628 trim_cache size 0 max 16384 2024-04-20T10:36:17.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4c6797740 1 Processor -- start 2024-04-20T10:36:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4c6797740 1 -- start start 2024-04-20T10:36:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4c6797740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc76cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4c6797740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc6ea90 0x55c64dc83370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4c6797740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc77670 0x55c64dc74c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4c6797740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1b2100 con 0x55c64dc6ea90 2024-04-20T10:36:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4c6797740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1b4980 con 0x55c64dc6e170 2024-04-20T10:36:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4c6797740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1b2770 con 0x55c64dc77670 2024-04-20T10:36:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc76cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc76cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:45944/0 (socket says 172.21.15.42:45944) 2024-04-20T10:36:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4c1000640 1 -- 172.21.15.42:0/3596357295 learned_addr learned my addr 172.21.15.42:0/3596357295 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4bbe00640 1 --2- 172.21.15.42:0/3596357295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc6ea90 0x55c64dc83370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.503+0000 7ff4c1a00640 1 --2- 172.21.15.42:0/3596357295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc77670 0x55c64dc74c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.504+0000 7ff4c1000640 1 -- 172.21.15.42:0/3596357295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc77670 msgr2=0x55c64dc74c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:17.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.504+0000 7ff4c1000640 1 --2- 172.21.15.42:0/3596357295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc77670 0x55c64dc74c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.504+0000 7ff4c1000640 1 -- 172.21.15.42:0/3596357295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc6ea90 msgr2=0x55c64dc83370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:17.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.504+0000 7ff4c1000640 1 --2- 172.21.15.42:0/3596357295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc6ea90 0x55c64dc83370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.504+0000 7ff4c1000640 1 -- 172.21.15.42:0/3596357295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c64d8e53f0 con 0x55c64dc6e170 2024-04-20T10:36:17.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.504+0000 7ff4c1a00640 1 --2- 172.21.15.42:0/3596357295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc77670 0x55c64dc74c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:17.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.504+0000 7ff4c1000640 1 --2- 172.21.15.42:0/3596357295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc76cf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009780 tx=0x7ff4b004cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9c478789c743e41 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:17.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.505+0000 7ff4ad600640 1 -- 172.21.15.42:0/3596357295 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0061020 con 0x55c64dc6e170 2024-04-20T10:36:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.505+0000 7ff4ad600640 1 -- 172.21.15.42:0/3596357295 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b0050070 con 0x55c64dc6e170 2024-04-20T10:36:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.505+0000 7ff4ad600640 1 -- 172.21.15.42:0/3596357295 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0039040 con 0x55c64dc6e170 2024-04-20T10:36:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.505+0000 7ff4c6797740 1 -- 172.21.15.42:0/3596357295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 msgr2=0x55c64dc76cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.505+0000 7ff4c6797740 1 --2- 172.21.15.42:0/3596357295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc76cf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009780 tx=0x7ff4b004cfa0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.505+0000 7ff4c6797740 1 -- 172.21.15.42:0/3596357295 shutdown_connections 2024-04-20T10:36:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.505+0000 7ff4c6797740 1 --2- 172.21.15.42:0/3596357295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc77670 0x55c64dc74c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.505+0000 7ff4c6797740 1 --2- 172.21.15.42:0/3596357295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc6ea90 0x55c64dc83370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.505+0000 7ff4c6797740 1 --2- 172.21.15.42:0/3596357295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc76cf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.505+0000 7ff4c6797740 1 -- 172.21.15.42:0/3596357295 >> 172.21.15.42:0/3596357295 conn(0x55c64dc688a0 msgr2=0x55c64dc74580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.506+0000 7ff4c6797740 1 -- 172.21.15.42:0/3596357295 shutdown_connections 2024-04-20T10:36:17.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.506+0000 7ff4c6797740 1 -- 172.21.15.42:0/3596357295 wait complete. 2024-04-20T10:36:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.506+0000 7ff4c6797740 1 -- start start 2024-04-20T10:36:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.506+0000 7ff4c6797740 1 Processor -- start 2024-04-20T10:36:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c6797740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc7fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c6797740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc71020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c6797740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc77670 0x55c64dc715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c6797740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1b2e40 con 0x55c64dc77670 2024-04-20T10:36:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c6797740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1b42b0 con 0x55c64dc6e170 2024-04-20T10:36:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c6797740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc1b3510 con 0x55c64dc6ea90 2024-04-20T10:36:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c6797740 10 client.0 _ll_register_callbacks cb 0x7ff4b0004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c1a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc77670 0x55c64dc715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc7fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc7fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:45956/0 (socket says 172.21.15.42:45956) 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c1000640 1 -- 172.21.15.42:0/1228555175 learned_addr learned my addr 172.21.15.42:0/1228555175 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4bbe00640 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc71020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c1000640 1 -- 172.21.15.42:0/1228555175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 msgr2=0x55c64dc71020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.507+0000 7ff4c1000640 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc71020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.508+0000 7ff4c1000640 1 -- 172.21.15.42:0/1228555175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc77670 msgr2=0x55c64dc715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.508+0000 7ff4c1000640 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc77670 0x55c64dc715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.508+0000 7ff4c1000640 1 -- 172.21.15.42:0/1228555175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c64d9af7d0 con 0x55c64dc6e170 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.508+0000 7ff4c1a00640 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc77670 0x55c64dc715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.508+0000 7ff4c1000640 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc7fd60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff4b004bb30 tx=0x7ff4b004b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe3ced89ab2e1ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.508+0000 7ff4a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.508+0000 7ff4a2a00640 1 -- 172.21.15.42:0/1228555175 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b006a020 con 0x55c64dc6e170 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.508+0000 7ff4a2a00640 1 -- 172.21.15.42:0/1228555175 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff4b002c9c0 con 0x55c64dc6e170 2024-04-20T10:36:17.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.508+0000 7ff4c6797740 1 -- 172.21.15.42:0/1228555175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4b0050070 con 0x55c64dc6e170 2024-04-20T10:36:17.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.509+0000 7ff4a2a00640 1 -- 172.21.15.42:0/1228555175 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff4b0061070 con 0x55c64dc6e170 2024-04-20T10:36:17.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.509+0000 7ff4c6797740 10 client.25634 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:17.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.509+0000 7ff4c6797740 1 -- 172.21.15.42:0/1228555175 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c64dceda70 con 0x55c64dc6e170 2024-04-20T10:36:17.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.509+0000 7ff4a2a00640 1 -- 172.21.15.42:0/1228555175 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b0050070 con 0x55c64dc6e170 2024-04-20T10:36:17.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.509+0000 7ff4a2a00640 1 -- 172.21.15.42:0/1228555175 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7ff4b00664e0 con 0x55c64dc6e170 2024-04-20T10:36:17.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.509+0000 7ff4c6797740 10 client.25634 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:17.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.509+0000 7ff4c6797740 10 client.25634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:17.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.509+0000 7ff4c6797740 20 client.25634 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:17.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.509+0000 7ff4c6797740 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c64dcedf40 0x55c64dcf0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:17.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.509+0000 7ff4c6797740 4 client.25634 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:36:17.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.509+0000 7ff4c6797740 1 -- 172.21.15.42:0/1228555175 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c64dbda8e0 con 0x55c64dcedf40 2024-04-20T10:36:17.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.509+0000 7ff4bbe00640 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c64dcedf40 0x55c64dcf0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:17.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.510+0000 7ff4bbe00640 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c64dcedf40 0x55c64dcf0330 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db53a8fcb5abc199 server_cookie=5ff4ddc38a44cfaf in_seq=0 out_seq=0 2024-04-20T10:36:17.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.510+0000 7ff4a2a00640 10 client.25634 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:17.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.510+0000 7ff4a2a00640 1 -- 172.21.15.42:0/1228555175 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c64dbda8e0 con 0x55c64dcedf40 2024-04-20T10:36:17.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.510+0000 7ff4a2a00640 10 client.25634 handle_command_reply: tid=0 2024-04-20T10:36:17.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.510+0000 7ff4c6797740 1 client.25634 shutdown 2024-04-20T10:36:17.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.510+0000 7ff4c6797740 1 -- 172.21.15.42:0/1228555175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 msgr2=0x55c64dc7fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:17.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.510+0000 7ff4c6797740 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc7fd60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff4b004bb30 tx=0x7ff4b004b710 comp rx=0 tx=0).stop 2024-04-20T10:36:17.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.511+0000 7ff4c6797740 1 -- 172.21.15.42:0/1228555175 shutdown_connections 2024-04-20T10:36:17.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.511+0000 7ff4c6797740 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c64dcedf40 0x55c64dcf0330 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.511+0000 7ff4c6797740 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c64dc77670 0x55c64dc715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.511+0000 7ff4c6797740 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c64dc6ea90 0x55c64dc71020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.511+0000 7ff4c6797740 1 --2- 172.21.15.42:0/1228555175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c64dc6e170 0x55c64dc7fd60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.511+0000 7ff4c6797740 1 -- 172.21.15.42:0/1228555175 >> 172.21.15.42:0/1228555175 conn(0x55c64dc688a0 msgr2=0x55c64dc67960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:17.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.511+0000 7ff4a2a00640 0 client.25634 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:17.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.511+0000 7ff4c6797740 1 -- 172.21.15.42:0/1228555175 shutdown_connections 2024-04-20T10:36:17.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.511+0000 7ff4c6797740 1 -- 172.21.15.42:0/1228555175 wait complete. 2024-04-20T10:36:17.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.511+0000 7ff4c6797740 20 client.25634 trim_cache size 0 max 16384 2024-04-20T10:36:17.515 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:36:17.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.512+0000 7ff4c2400640 1 -- 172.21.15.42:0/1494645863 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff4a40a3da0 msgr2=0x7ff4a40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:17.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.512+0000 7ff4c2400640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff4a40a3da0 0x7ff4a40a6260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0000c00 tx=0x7ff4b00056d0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.512+0000 7ff4c2400640 1 -- 172.21.15.42:0/1494645863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4bc10d3d0 msgr2=0x7ff4bc19eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:17.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.512+0000 7ff4c2400640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4bc10d3d0 0x7ff4bc19eb50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400e960 tx=0x7ff4b400ee20 comp rx=0 tx=0).stop 2024-04-20T10:36:17.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.512+0000 7ff4c2400640 1 -- 172.21.15.42:0/1494645863 shutdown_connections 2024-04-20T10:36:17.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.512+0000 7ff4c2400640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff4a40a3da0 0x7ff4a40a6260 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.512+0000 7ff4c2400640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4bc10d3d0 0x7ff4bc19eb50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.512+0000 7ff4c2400640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4bc108ea0 0x7ff4bc1a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.512+0000 7ff4c2400640 1 --2- 172.21.15.42:0/1494645863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4bc108500 0x7ff4bc1a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:17.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.512+0000 7ff4c2400640 1 -- 172.21.15.42:0/1494645863 >> 172.21.15.42:0/1494645863 conn(0x7ff4bc077d80 msgr2=0x7ff4bc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:17.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.512+0000 7ff4c2400640 1 -- 172.21.15.42:0/1494645863 shutdown_connections 2024-04-20T10:36:17.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:17.512+0000 7ff4c2400640 1 -- 172.21.15.42:0/1494645863 wait complete. 2024-04-20T10:36:17.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:17 smithi042 ceph-mon[17078]: pgmap v71: 97 pgs: 97 active+clean; 153 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 35 op/s 2024-04-20T10:36:17.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:17 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3346222627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:36:18.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:17 smithi046 ceph-mon[21574]: pgmap v71: 97 pgs: 97 active+clean; 153 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 35 op/s 2024-04-20T10:36:18.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:17 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3346222627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:36:18.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:17 smithi162 ceph-mon[21854]: pgmap v71: 97 pgs: 97 active+clean; 153 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 35 op/s 2024-04-20T10:36:18.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:17 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3346222627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:36:18.531 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:36:18.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:36:18.531 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:36:18.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.762+0000 7f82fc600640 1 -- 172.21.15.42:0/3159478468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4104dc0 msgr2=0x7f82f4108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:18.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.762+0000 7f82fc600640 1 --2- 172.21.15.42:0/3159478468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4104dc0 0x7f82f4108e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009900 tx=0x7f82e002bc30 comp rx=0 tx=0).stop 2024-04-20T10:36:18.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.762+0000 7f82fc600640 1 -- 172.21.15.42:0/3159478468 shutdown_connections 2024-04-20T10:36:18.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.762+0000 7f82fc600640 1 --2- 172.21.15.42:0/3159478468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82f41094e0 0x7f82f41110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.762+0000 7f82fc600640 1 --2- 172.21.15.42:0/3159478468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4104dc0 0x7f82f4108e00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.762+0000 7f82fc600640 1 --2- 172.21.15.42:0/3159478468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82f4104410 0x7f82f41047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.762+0000 7f82fc600640 1 -- 172.21.15.42:0/3159478468 >> 172.21.15.42:0/3159478468 conn(0x7f82f40ffcf0 msgr2=0x7f82f4102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:18.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.762+0000 7f82fc600640 1 -- 172.21.15.42:0/3159478468 shutdown_connections 2024-04-20T10:36:18.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.762+0000 7f82fc600640 1 -- 172.21.15.42:0/3159478468 wait complete. 2024-04-20T10:36:18.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.763+0000 7f82fc600640 1 Processor -- start 2024-04-20T10:36:18.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.763+0000 7f82fc600640 1 -- start start 2024-04-20T10:36:18.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.764+0000 7f82fc600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4104410 0x7f82f41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.764+0000 7f82fc600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82f41094e0 0x7f82f41a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.764+0000 7f82fc600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82f419a910 0x7f82f419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.764+0000 7f82fc600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4114570 con 0x7f82f419a910 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.764+0000 7f82fc600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41143f0 con 0x7f82f4104410 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.764+0000 7f82fc600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41146f0 con 0x7f82f41094e0 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.764+0000 7f82fa600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82f419a910 0x7f82f419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.764+0000 7f82fa600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82f419a910 0x7f82f419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36580/0 (socket says 172.21.15.42:36580) 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.764+0000 7f82fa600640 1 -- 172.21.15.42:0/3681100755 learned_addr learned my addr 172.21.15.42:0/3681100755 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.764+0000 7f82f9c00640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4104410 0x7f82f41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.764+0000 7f82f9200640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82f41094e0 0x7f82f41a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.765+0000 7f82fa600640 1 -- 172.21.15.42:0/3681100755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82f41094e0 msgr2=0x7f82f41a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.765+0000 7f82fa600640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82f41094e0 0x7f82f41a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.765+0000 7f82fa600640 1 -- 172.21.15.42:0/3681100755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4104410 msgr2=0x7f82f41a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.765+0000 7f82fa600640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4104410 0x7f82f41a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.765+0000 7f82fa600640 1 -- 172.21.15.42:0/3681100755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f419b430 con 0x7f82f419a910 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.765+0000 7f82f9c00640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4104410 0x7f82f41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:18.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.765+0000 7f82fa600640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82f419a910 0x7f82f419adc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00eb10 tx=0x7f82ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:18.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.765+0000 7f82f2000640 1 -- 172.21.15.42:0/3681100755 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec004b20 con 0x7f82f419a910 2024-04-20T10:36:18.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.765+0000 7f82f2000640 1 -- 172.21.15.42:0/3681100755 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ec004cc0 con 0x7f82f419a910 2024-04-20T10:36:18.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.765+0000 7f82fc600640 1 -- 172.21.15.42:0/3681100755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f419b720 con 0x7f82f419a910 2024-04-20T10:36:18.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.765+0000 7f82f2000640 1 -- 172.21.15.42:0/3681100755 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec010760 con 0x7f82f419a910 2024-04-20T10:36:18.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.765+0000 7f82fc600640 1 -- 172.21.15.42:0/3681100755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f41a7920 con 0x7f82f419a910 2024-04-20T10:36:18.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.769+0000 7f82ff284740 1 Processor -- start 2024-04-20T10:36:18.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.769+0000 7f82f2000640 1 -- 172.21.15.42:0/3681100755 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f82ec005020 con 0x7f82f419a910 2024-04-20T10:36:18.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.769+0000 7f82ff284740 1 -- start start 2024-04-20T10:36:18.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.769+0000 7f82ff284740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f62162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.769+0000 7f82ff284740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6216830 0x55f0f621eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.769+0000 7f82ff284740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f621f410 0x55f0f6221800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.769+0000 7f82ff284740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f0f617ad20 con 0x55f0f621f410 2024-04-20T10:36:18.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.769+0000 7f82ff284740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f0f61213e0 con 0x55f0f6215f10 2024-04-20T10:36:18.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.769+0000 7f82ff284740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f0f6176ce0 con 0x55f0f6216830 2024-04-20T10:36:18.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f621f410 0x55f0f6221800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f621f410 0x55f0f6221800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36596/0 (socket says 172.21.15.42:36596) 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f9200640 1 -- 172.21.15.42:0/1590080922 learned_addr learned my addr 172.21.15.42:0/1590080922 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f9200640 1 --2- 172.21.15.42:0/1590080922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f62162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f9c00640 1 --2- 172.21.15.42:0/1590080922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6216830 0x55f0f621eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f2000640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f82dc0a3d20 0x7f82dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f9200640 1 -- 172.21.15.42:0/1590080922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6216830 msgr2=0x55f0f621eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f9200640 1 --2- 172.21.15.42:0/1590080922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6216830 0x55f0f621eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f9200640 1 -- 172.21.15.42:0/1590080922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 msgr2=0x55f0f62162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f9200640 1 --2- 172.21.15.42:0/1590080922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f62162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f9200640 1 -- 172.21.15.42:0/1590080922 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0f6140980 con 0x55f0f621f410 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f9c00640 1 --2- 172.21.15.42:0/1590080922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6216830 0x55f0f621eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.770+0000 7f82f2000640 1 -- 172.21.15.42:0/3681100755 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f82ec0d0d00 con 0x7f82f419a910 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.771+0000 7f82fa600640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f82dc0a3d20 0x7f82dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.771+0000 7f82f9200640 1 --2- 172.21.15.42:0/1590080922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f621f410 0x55f0f6221800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f82e0005ec0 tx=0x7f82e0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0995729a3f0c48b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:18.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.771+0000 7f82e6a00640 1 -- 172.21.15.42:0/1590080922 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0004030 con 0x55f0f621f410 2024-04-20T10:36:18.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.771+0000 7f82e6a00640 1 -- 172.21.15.42:0/1590080922 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e0004ba0 con 0x55f0f621f410 2024-04-20T10:36:18.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.771+0000 7f82e6a00640 1 -- 172.21.15.42:0/1590080922 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0005250 con 0x55f0f621f410 2024-04-20T10:36:18.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.772+0000 7f82ff284740 1 -- 172.21.15.42:0/1590080922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f621f410 msgr2=0x55f0f6221800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:18.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.772+0000 7f82ff284740 1 --2- 172.21.15.42:0/1590080922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f621f410 0x55f0f6221800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f82e0005ec0 tx=0x7f82e0005dd0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.772+0000 7f82fa600640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f82dc0a3d20 0x7f82dc0a61e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0ce500 tx=0x7f82ec005b40 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:18.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.772+0000 7f82ff284740 1 -- 172.21.15.42:0/1590080922 shutdown_connections 2024-04-20T10:36:18.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.772+0000 7f82ff284740 1 --2- 172.21.15.42:0/1590080922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f621f410 0x55f0f6221800 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.772+0000 7f82ff284740 1 --2- 172.21.15.42:0/1590080922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6216830 0x55f0f621eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.772+0000 7f82ff284740 1 --2- 172.21.15.42:0/1590080922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f62162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.772+0000 7f82ff284740 1 -- 172.21.15.42:0/1590080922 >> 172.21.15.42:0/1590080922 conn(0x55f0f6210640 msgr2=0x55f0f6210a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.772+0000 7f82ff284740 1 -- 172.21.15.42:0/1590080922 shutdown_connections 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.772+0000 7f82ff284740 1 -- 172.21.15.42:0/1590080922 wait complete. 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.772+0000 7f82ff284740 1 -- start start 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.773+0000 7f82ff284740 1 Processor -- start 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.773+0000 7f82ff284740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f621c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.773+0000 7f82ff284740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f621c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.773+0000 7f82ff284740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f621f410 0x55f0f621cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.773+0000 7f82ff284740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f0f62221a0 con 0x55f0f6216830 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.773+0000 7f82ff284740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f0f61347a0 con 0x55f0f6215f10 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.773+0000 7f82ff284740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f0f6222320 con 0x55f0f621f410 2024-04-20T10:36:18.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.773+0000 7f82ff284740 10 client.0 _ll_register_callbacks cb 0x55f0f6136b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.773+0000 7f82f9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f621c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82f9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f621c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82f9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f621c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36598/0 (socket says 172.21.15.42:36598) 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82f9c00640 1 -- 172.21.15.42:0/1729821678 learned_addr learned my addr 172.21.15.42:0/1729821678 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82f9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f621c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:45998/0 (socket says 172.21.15.42:45998) 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82f9200640 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f621f410 0x55f0f621cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82f9c00640 1 -- 172.21.15.42:0/1729821678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f621f410 msgr2=0x55f0f621cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82f9c00640 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f621f410 0x55f0f621cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82f9c00640 1 -- 172.21.15.42:0/1729821678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 msgr2=0x55f0f621c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82f9c00640 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f621c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82f9c00640 1 -- 172.21.15.42:0/1729821678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0f5f557e0 con 0x55f0f6216830 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82f9200640 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f621c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82f9c00640 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f621c9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f82e8013d30 tx=0x7f82e80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b573d8323b211d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:18.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.774+0000 7f82db400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.775+0000 7f82db400640 1 -- 172.21.15.42:0/1729821678 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e801b630 con 0x55f0f6216830 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.775+0000 7f82db400640 1 -- 172.21.15.42:0/1729821678 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f82e801b7d0 con 0x55f0f6216830 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.775+0000 7f82ff284740 1 -- 172.21.15.42:0/1729821678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e0004ba0 con 0x55f0f6216830 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.775+0000 7f82db400640 1 -- 172.21.15.42:0/1729821678 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f82e801b9d0 con 0x55f0f6216830 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.775+0000 7f82ff284740 10 client.15534 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.775+0000 7f82ff284740 1 -- 172.21.15.42:0/1729821678 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f0f5fb5f70 con 0x55f0f6216830 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.775+0000 7f82db400640 1 -- 172.21.15.42:0/1729821678 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e0004ba0 con 0x55f0f6216830 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.775+0000 7f82db400640 1 -- 172.21.15.42:0/1729821678 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f82e800e3b0 con 0x55f0f6216830 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.776+0000 7f82ff284740 10 client.15534 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.776+0000 7f82ff284740 10 client.15534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.776+0000 7f82ff284740 20 client.15534 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.776+0000 7f82ff284740 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f0f622f750 0x55f0f6231b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.776+0000 7f82ff284740 4 client.15534 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:36:18.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.776+0000 7f82ff284740 1 -- 172.21.15.42:0/1729821678 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0f6188740 con 0x55f0f622f750 2024-04-20T10:36:18.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.776+0000 7f82f9200640 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f0f622f750 0x55f0f6231b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.777+0000 7f82f9200640 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f0f622f750 0x55f0f6231b80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21bcb7335276c20 server_cookie=ac9c5cce68b33d70 in_seq=0 out_seq=0 2024-04-20T10:36:18.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.777+0000 7f82db400640 10 client.15534 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:18.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.777+0000 7f82db400640 1 -- 172.21.15.42:0/1729821678 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0f6188740 con 0x55f0f622f750 2024-04-20T10:36:18.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.777+0000 7f82db400640 10 client.15534 handle_command_reply: tid=0 2024-04-20T10:36:18.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82ff284740 1 client.15534 shutdown 2024-04-20T10:36:18.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82ff284740 1 -- 172.21.15.42:0/1729821678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 msgr2=0x55f0f621c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:18.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82ff284740 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f621c9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f82e8013d30 tx=0x7f82e80117b0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82ff284740 1 -- 172.21.15.42:0/1729821678 shutdown_connections 2024-04-20T10:36:18.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82ff284740 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f0f622f750 0x55f0f6231b80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82ff284740 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f621f410 0x55f0f621cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82ff284740 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f621c9c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82db400640 0 client.15534 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:18.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82ff284740 1 --2- 172.21.15.42:0/1729821678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f621c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82ff284740 1 -- 172.21.15.42:0/1729821678 >> 172.21.15.42:0/1729821678 conn(0x55f0f6210640 msgr2=0x55f0f6212390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:18.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82ff284740 1 -- 172.21.15.42:0/1729821678 shutdown_connections 2024-04-20T10:36:18.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82ff284740 1 -- 172.21.15.42:0/1729821678 wait complete. 2024-04-20T10:36:18.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.778+0000 7f82ff284740 20 client.15534 trim_cache size 0 max 16384 2024-04-20T10:36:18.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.798+0000 7f82ff284740 1 Processor -- start 2024-04-20T10:36:18.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.799+0000 7f82ff284740 1 -- start start 2024-04-20T10:36:18.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.799+0000 7f82ff284740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6215f10 0x55f0f621ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.799+0000 7f82ff284740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f622b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.799+0000 7f82ff284740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f621f410 0x55f0f62121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.799+0000 7f82ff284740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41adf50 con 0x55f0f6216830 2024-04-20T10:36:18.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.799+0000 7f82ff284740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41b07d0 con 0x55f0f621f410 2024-04-20T10:36:18.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.799+0000 7f82ff284740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41ae5c0 con 0x55f0f6215f10 2024-04-20T10:36:18.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.799+0000 7f82f9200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6215f10 0x55f0f621ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.799+0000 7f82f9200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6215f10 0x55f0f621ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37060/0 (socket says 172.21.15.42:37060) 2024-04-20T10:36:18.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.799+0000 7f82f9200640 1 -- 172.21.15.42:0/3890190443 learned_addr learned my addr 172.21.15.42:0/3890190443 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:18.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.799+0000 7f82f9c00640 1 --2- 172.21.15.42:0/3890190443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f622b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.799+0000 7f82f9200640 1 --2- 172.21.15.42:0/3890190443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f621f410 0x55f0f62121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.800+0000 7f82f9200640 1 -- 172.21.15.42:0/3890190443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f621f410 msgr2=0x55f0f62121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:18.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.800+0000 7f82f9200640 1 --2- 172.21.15.42:0/3890190443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f621f410 0x55f0f62121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.800+0000 7f82f9200640 1 -- 172.21.15.42:0/3890190443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 msgr2=0x55f0f622b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:18.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.800+0000 7f82f9200640 1 --2- 172.21.15.42:0/3890190443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f622b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.800+0000 7f82f9200640 1 -- 172.21.15.42:0/3890190443 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0f5e4d170 con 0x55f0f6215f10 2024-04-20T10:36:18.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.800+0000 7f82f9200640 1 --2- 172.21.15.42:0/3890190443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6215f10 0x55f0f621ebb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009900 tx=0x7f82e0004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11401f8cccae991b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.800+0000 7f82e6000640 1 -- 172.21.15.42:0/3890190443 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e00047e0 con 0x55f0f6215f10 2024-04-20T10:36:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.800+0000 7f82e6000640 1 -- 172.21.15.42:0/3890190443 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e0059820 con 0x55f0f6215f10 2024-04-20T10:36:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.800+0000 7f82e6000640 1 -- 172.21.15.42:0/3890190443 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0005460 con 0x55f0f6215f10 2024-04-20T10:36:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.801+0000 7f82ff284740 1 -- 172.21.15.42:0/3890190443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6215f10 msgr2=0x55f0f621ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.801+0000 7f82ff284740 1 --2- 172.21.15.42:0/3890190443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6215f10 0x55f0f621ebb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009900 tx=0x7f82e0004610 comp rx=0 tx=0).stop 2024-04-20T10:36:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.801+0000 7f82ff284740 1 -- 172.21.15.42:0/3890190443 shutdown_connections 2024-04-20T10:36:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.801+0000 7f82ff284740 1 --2- 172.21.15.42:0/3890190443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f621f410 0x55f0f62121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.801+0000 7f82ff284740 1 --2- 172.21.15.42:0/3890190443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f622b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.801+0000 7f82ff284740 1 --2- 172.21.15.42:0/3890190443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f6215f10 0x55f0f621ebb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.801+0000 7f82ff284740 1 -- 172.21.15.42:0/3890190443 >> 172.21.15.42:0/3890190443 conn(0x55f0f6210640 msgr2=0x55f0f6211a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.801+0000 7f82ff284740 1 -- 172.21.15.42:0/3890190443 shutdown_connections 2024-04-20T10:36:18.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.801+0000 7f82ff284740 1 -- 172.21.15.42:0/3890190443 wait complete. 2024-04-20T10:36:18.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.802+0000 7f82ff284740 1 -- start start 2024-04-20T10:36:18.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.802+0000 7f82ff284740 1 Processor -- start 2024-04-20T10:36:18.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.802+0000 7f82ff284740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f6223150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.802+0000 7f82ff284740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f6223690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.802+0000 7f82ff284740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f621f410 0x55f0f6223bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.802+0000 7f82ff284740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41aec90 con 0x55f0f6216830 2024-04-20T10:36:18.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.802+0000 7f82ff284740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41b0100 con 0x55f0f6215f10 2024-04-20T10:36:18.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.802+0000 7f82ff284740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f82f41af360 con 0x55f0f621f410 2024-04-20T10:36:18.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.802+0000 7f82ff284740 10 client.0 _ll_register_callbacks cb 0x7f82e8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:18.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82f9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f6223150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82f9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f6223150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46024/0 (socket says 172.21.15.42:46024) 2024-04-20T10:36:18.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82f9200640 1 -- 172.21.15.42:0/1704708572 learned_addr learned my addr 172.21.15.42:0/1704708572 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:18.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82f9c00640 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f6223690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82f9200640 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f621f410 0x55f0f6223bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82f9200640 1 -- 172.21.15.42:0/1704708572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f621f410 msgr2=0x55f0f6223bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:18.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82f9200640 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f621f410 0x55f0f6223bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82f9200640 1 -- 172.21.15.42:0/1704708572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 msgr2=0x55f0f6223690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:18.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82f9200640 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f6223690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82f9200640 1 -- 172.21.15.42:0/1704708572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0f6294e30 con 0x55f0f6215f10 2024-04-20T10:36:18.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82f9200640 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f6223150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f82e002c4e0 tx=0x7f82e0005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e0d6b9e4df500dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:18.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82db400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:36:18.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.803+0000 7f82db400640 1 -- 172.21.15.42:0/1704708572 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e00049e0 con 0x55f0f6215f10 2024-04-20T10:36:18.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.804+0000 7f82ff284740 1 -- 172.21.15.42:0/1704708572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e0059820 con 0x55f0f6215f10 2024-04-20T10:36:18.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.804+0000 7f82db400640 1 -- 172.21.15.42:0/1704708572 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f82e00050e0 con 0x55f0f6215f10 2024-04-20T10:36:18.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.804+0000 7f82db400640 1 -- 172.21.15.42:0/1704708572 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f82e0004b80 con 0x55f0f6215f10 2024-04-20T10:36:18.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.804+0000 7f82ff284740 10 client.25652 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:18.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.804+0000 7f82ff284740 1 -- 172.21.15.42:0/1704708572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f0f6296150 con 0x55f0f6215f10 2024-04-20T10:36:18.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.804+0000 7f82db400640 1 -- 172.21.15.42:0/1704708572 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e0059820 con 0x55f0f6215f10 2024-04-20T10:36:18.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.804+0000 7f82db400640 1 -- 172.21.15.42:0/1704708572 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f82e0034b30 con 0x55f0f6215f10 2024-04-20T10:36:18.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.804+0000 7f82ff284740 10 client.25652 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:18.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.804+0000 7f82ff284740 10 client.25652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:18.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.805+0000 7f82ff284740 20 client.25652 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:18.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.805+0000 7f82ff284740 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f0f6296380 0x55f0f6298770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:18.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.805+0000 7f82ff284740 4 client.25652 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:36:18.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.805+0000 7f82ff284740 1 -- 172.21.15.42:0/1704708572 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f0f61825b0 con 0x55f0f6296380 2024-04-20T10:36:18.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.805+0000 7f82f9c00640 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f0f6296380 0x55f0f6298770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:18.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.805+0000 7f82f9c00640 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f0f6296380 0x55f0f6298770 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a1626c6a076689 server_cookie=2bce92af5ec6e577 in_seq=0 out_seq=0 2024-04-20T10:36:18.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.805+0000 7f82db400640 10 client.25652 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:18.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82db400640 1 -- 172.21.15.42:0/1704708572 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f0f61825b0 con 0x55f0f6296380 2024-04-20T10:36:18.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82db400640 10 client.25652 handle_command_reply: tid=0 2024-04-20T10:36:18.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82ff284740 1 client.25652 shutdown 2024-04-20T10:36:18.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82ff284740 1 -- 172.21.15.42:0/1704708572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 msgr2=0x55f0f6223150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:18.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82ff284740 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f6223150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f82e002c4e0 tx=0x7f82e0005040 comp rx=0 tx=0).stop 2024-04-20T10:36:18.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82ff284740 1 -- 172.21.15.42:0/1704708572 shutdown_connections 2024-04-20T10:36:18.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82ff284740 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f0f6296380 0x55f0f6298770 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82ff284740 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0f621f410 0x55f0f6223bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82ff284740 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0f6216830 0x55f0f6223690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82ff284740 1 --2- 172.21.15.42:0/1704708572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f0f6215f10 0x55f0f6223150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82db400640 0 client.25652 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:18.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82ff284740 1 -- 172.21.15.42:0/1704708572 >> 172.21.15.42:0/1704708572 conn(0x55f0f6210640 msgr2=0x55f0f620f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:18.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82ff284740 1 -- 172.21.15.42:0/1704708572 shutdown_connections 2024-04-20T10:36:18.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82ff284740 1 -- 172.21.15.42:0/1704708572 wait complete. 2024-04-20T10:36:18.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.807+0000 7f82ff284740 20 client.25652 trim_cache size 0 max 16384 2024-04-20T10:36:18.812 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:36:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:36:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "fd2c6ec7-3de8-4564-8578-c0186dbc5ffd", 2024-04-20T10:36:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:36:18.813 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:36:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.808+0000 7f82fc600640 1 -- 172.21.15.42:0/3681100755 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f82dc0a3d20 msgr2=0x7f82dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.808+0000 7f82fc600640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f82dc0a3d20 0x7f82dc0a61e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0ce500 tx=0x7f82ec005b40 comp rx=0 tx=0).stop 2024-04-20T10:36:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.808+0000 7f82fc600640 1 -- 172.21.15.42:0/3681100755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82f419a910 msgr2=0x7f82f419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.808+0000 7f82fc600640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82f419a910 0x7f82f419adc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00eb10 tx=0x7f82ec00c2d0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.808+0000 7f82fc600640 1 -- 172.21.15.42:0/3681100755 shutdown_connections 2024-04-20T10:36:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.808+0000 7f82fc600640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f82dc0a3d20 0x7f82dc0a61e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.808+0000 7f82fc600640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82f419a910 0x7f82f419adc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.808+0000 7f82fc600640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82f41094e0 0x7f82f41a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.808+0000 7f82fc600640 1 --2- 172.21.15.42:0/3681100755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f4104410 0x7f82f41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.808+0000 7f82fc600640 1 -- 172.21.15.42:0/3681100755 >> 172.21.15.42:0/3681100755 conn(0x7f82f40ffcf0 msgr2=0x7f82f4101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.808+0000 7f82fc600640 1 -- 172.21.15.42:0/3681100755 shutdown_connections 2024-04-20T10:36:18.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:18.808+0000 7f82fc600640 1 -- 172.21.15.42:0/3681100755 wait complete. 2024-04-20T10:36:18.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:36:18.829 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:36:19.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.040+0000 7fdae5000640 1 -- 172.21.15.42:0/3489551261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae0104c20 msgr2=0x7fdae0108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:19.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.040+0000 7fdae5000640 1 --2- 172.21.15.42:0/3489551261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae0104c20 0x7fdae0108c60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009900 tx=0x7fdac802bc30 comp rx=0 tx=0).stop 2024-04-20T10:36:19.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.040+0000 7fdae5000640 1 -- 172.21.15.42:0/3489551261 shutdown_connections 2024-04-20T10:36:19.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.040+0000 7fdae5000640 1 --2- 172.21.15.42:0/3489551261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdae0109340 0x7fdae0110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.040+0000 7fdae5000640 1 --2- 172.21.15.42:0/3489551261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae0104c20 0x7fdae0108c60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.040+0000 7fdae5000640 1 --2- 172.21.15.42:0/3489551261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdae0104270 0x7fdae0104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.040+0000 7fdae5000640 1 -- 172.21.15.42:0/3489551261 >> 172.21.15.42:0/3489551261 conn(0x7fdae00ffa70 msgr2=0x7fdae0101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:19.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.041+0000 7fdae5000640 1 -- 172.21.15.42:0/3489551261 shutdown_connections 2024-04-20T10:36:19.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.041+0000 7fdae5000640 1 -- 172.21.15.42:0/3489551261 wait complete. 2024-04-20T10:36:19.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdae5000640 1 Processor -- start 2024-04-20T10:36:19.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdae5000640 1 -- start start 2024-04-20T10:36:19.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdae5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae0104270 0x7fdae01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdae5000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdae0109340 0x7fdae01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdae5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdae019a720 0x7fdae019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdae5000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdae0114420 con 0x7fdae019a720 2024-04-20T10:36:19.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdae5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01142a0 con 0x7fdae0104270 2024-04-20T10:36:19.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdae5000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01145a0 con 0x7fdae0109340 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdadf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae0104270 0x7fdae01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdadf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae0104270 0x7fdae01a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:58556/0 (socket says 172.21.15.42:58556) 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdadf400640 1 -- 172.21.15.42:0/4154345906 learned_addr learned my addr 172.21.15.42:0/4154345906 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdadfe00640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdae019a720 0x7fdae019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.042+0000 7fdad6a00640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdae0109340 0x7fdae01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.043+0000 7fdadf400640 1 -- 172.21.15.42:0/4154345906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdae0109340 msgr2=0x7fdae01a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.043+0000 7fdadf400640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdae0109340 0x7fdae01a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.043+0000 7fdadf400640 1 -- 172.21.15.42:0/4154345906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdae019a720 msgr2=0x7fdae019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.043+0000 7fdadf400640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdae019a720 0x7fdae019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.043+0000 7fdadf400640 1 -- 172.21.15.42:0/4154345906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae019b320 con 0x7fdae0104270 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.043+0000 7fdadf400640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae0104270 0x7fdae01a0610 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdad000d950 tx=0x7fdad000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.043+0000 7fdadcc00640 1 -- 172.21.15.42:0/4154345906 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0017070 con 0x7fdae0104270 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.043+0000 7fdadcc00640 1 -- 172.21.15.42:0/4154345906 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad0009c30 con 0x7fdae0104270 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.043+0000 7fdae5000640 1 -- 172.21.15.42:0/4154345906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae01a73b0 con 0x7fdae0104270 2024-04-20T10:36:19.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.043+0000 7fdadcc00640 1 -- 172.21.15.42:0/4154345906 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad000f6b0 con 0x7fdae0104270 2024-04-20T10:36:19.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.044+0000 7fdae5000640 1 -- 172.21.15.42:0/4154345906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae01a7760 con 0x7fdae0104270 2024-04-20T10:36:19.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.047+0000 7fdae925f740 1 Processor -- start 2024-04-20T10:36:19.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.048+0000 7fdae925f740 1 -- start start 2024-04-20T10:36:19.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.048+0000 7fdadcc00640 1 -- 172.21.15.42:0/4154345906 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fdad00109d0 con 0x7fdae0104270 2024-04-20T10:36:19.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.048+0000 7fdae925f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 0x5653f9b35760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.048+0000 7fdae925f740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.048+0000 7fdae925f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b3e880 0x5653f9b40c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.048+0000 7fdae925f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5653f9a9b150 con 0x5653f9b35380 2024-04-20T10:36:19.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.048+0000 7fdae925f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5653f9a41370 con 0x5653f9b3e880 2024-04-20T10:36:19.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.048+0000 7fdae925f740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5653f9a97110 con 0x5653f9b35ca0 2024-04-20T10:36:19.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.048+0000 7fdadfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 0x5653f9b35760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.048+0000 7fdadfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 0x5653f9b35760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41810/0 (socket says 172.21.15.42:41810) 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.048+0000 7fdadfe00640 1 -- 172.21.15.42:0/4046601914 learned_addr learned my addr 172.21.15.42:0/4046601914 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.049+0000 7fdad6a00640 1 --2- 172.21.15.42:0/4046601914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b3e880 0x5653f9b40c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.049+0000 7fdadf400640 1 --2- 172.21.15.42:0/4046601914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.049+0000 7fdadfe00640 1 -- 172.21.15.42:0/4046601914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 msgr2=0x5653f9b3e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.049+0000 7fdadfe00640 1 --2- 172.21.15.42:0/4046601914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.049+0000 7fdadfe00640 1 -- 172.21.15.42:0/4046601914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b3e880 msgr2=0x5653f9b40c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.049+0000 7fdadfe00640 1 --2- 172.21.15.42:0/4046601914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b3e880 0x5653f9b40c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.049+0000 7fdadfe00640 1 -- 172.21.15.42:0/4046601914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f98777d0 con 0x5653f9b35380 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.049+0000 7fdad6a00640 1 --2- 172.21.15.42:0/4046601914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b3e880 0x5653f9b40c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.049+0000 7fdadcc00640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fdac00a3d20 0x7fdac00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.049+0000 7fdadfe00640 1 --2- 172.21.15.42:0/4046601914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 0x5653f9b35760 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009f70 tx=0x7fdad8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2497e150cc33fb9a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.050+0000 7fdadcc00640 1 -- 172.21.15.42:0/4154345906 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fdad00d06e0 con 0x7fdae0104270 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.050+0000 7fdadf400640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fdac00a3d20 0x7fdac00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.050+0000 7fdad4c00640 1 -- 172.21.15.42:0/4046601914 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad8016070 con 0x5653f9b35380 2024-04-20T10:36:19.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.050+0000 7fdad4c00640 1 -- 172.21.15.42:0/4046601914 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad8004d20 con 0x5653f9b35380 2024-04-20T10:36:19.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.050+0000 7fdad4c00640 1 -- 172.21.15.42:0/4046601914 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad80128f0 con 0x5653f9b35380 2024-04-20T10:36:19.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.051+0000 7fdadf400640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fdac00a3d20 0x7fdac00a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdad00cdee0 tx=0x7fdad00d6550 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:19.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.051+0000 7fdae925f740 1 -- 172.21.15.42:0/4046601914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 msgr2=0x5653f9b35760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:19.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.051+0000 7fdae925f740 1 --2- 172.21.15.42:0/4046601914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 0x5653f9b35760 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009f70 tx=0x7fdad8010800 comp rx=0 tx=0).stop 2024-04-20T10:36:19.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.051+0000 7fdae925f740 1 -- 172.21.15.42:0/4046601914 shutdown_connections 2024-04-20T10:36:19.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.051+0000 7fdae925f740 1 --2- 172.21.15.42:0/4046601914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b3e880 0x5653f9b40c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.051+0000 7fdae925f740 1 --2- 172.21.15.42:0/4046601914 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.051+0000 7fdae925f740 1 --2- 172.21.15.42:0/4046601914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 0x5653f9b35760 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.051+0000 7fdae925f740 1 -- 172.21.15.42:0/4046601914 >> 172.21.15.42:0/4046601914 conn(0x5653f9b2fab0 msgr2=0x5653f9b2fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:19.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.051+0000 7fdae925f740 1 -- 172.21.15.42:0/4046601914 shutdown_connections 2024-04-20T10:36:19.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.051+0000 7fdae925f740 1 -- 172.21.15.42:0/4046601914 wait complete. 2024-04-20T10:36:19.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.051+0000 7fdae925f740 1 -- start start 2024-04-20T10:36:19.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.052+0000 7fdae925f740 1 Processor -- start 2024-04-20T10:36:19.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.052+0000 7fdae925f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b35380 0x5653f9b3b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.052+0000 7fdae925f740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.052+0000 7fdae925f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b3e880 0x5653f9b3c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.052+0000 7fdae925f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5653f9b41670 con 0x5653f9b3e880 2024-04-20T10:36:19.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.052+0000 7fdae925f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5653f9a54740 con 0x5653f9b35380 2024-04-20T10:36:19.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.052+0000 7fdae925f740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5653f9b417f0 con 0x5653f9b35ca0 2024-04-20T10:36:19.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.052+0000 7fdae925f740 10 client.0 _ll_register_callbacks cb 0x5653f9a4fb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:19.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdadfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b35380 0x5653f9b3b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdadfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b35380 0x5653f9b3b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:58560/0 (socket says 172.21.15.42:58560) 2024-04-20T10:36:19.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdad6a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdadfe00640 1 -- 172.21.15.42:0/2100594548 learned_addr learned my addr 172.21.15.42:0/2100594548 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:19.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdadfe00640 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b3e880 0x5653f9b3c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdad6a00640 1 -- 172.21.15.42:0/2100594548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b35380 msgr2=0x5653f9b3b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdad6a00640 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b35380 0x5653f9b3b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdad6a00640 1 -- 172.21.15.42:0/2100594548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b3e880 msgr2=0x5653f9b3c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdad6a00640 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b3e880 0x5653f9b3c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdad6a00640 1 -- 172.21.15.42:0/2100594548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f97ad3f0 con 0x5653f9b35ca0 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdadfe00640 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b3e880 0x5653f9b3c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdad6a00640 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3bec0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdac8036040 tx=0x7fdac8009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6779a058263713d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.053+0000 7fdacd600640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.054+0000 7fdacd600640 1 -- 172.21.15.42:0/2100594548 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac802ccd0 con 0x5653f9b35ca0 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.054+0000 7fdacd600640 1 -- 172.21.15.42:0/2100594548 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fdac80046c0 con 0x5653f9b35ca0 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.054+0000 7fdae925f740 1 -- 172.21.15.42:0/2100594548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad8004d20 con 0x5653f9b35ca0 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.054+0000 7fdacd600640 1 -- 172.21.15.42:0/2100594548 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdac80048c0 con 0x5653f9b35ca0 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.054+0000 7fdae925f740 10 client.25684 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.054+0000 7fdae925f740 1 -- 172.21.15.42:0/2100594548 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5653f976d0f0 con 0x5653f9b35ca0 2024-04-20T10:36:19.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.054+0000 7fdacd600640 1 -- 172.21.15.42:0/2100594548 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad8004d20 con 0x5653f9b35ca0 2024-04-20T10:36:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.055+0000 7fdacd600640 1 -- 172.21.15.42:0/2100594548 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fdac8003710 con 0x5653f9b35ca0 2024-04-20T10:36:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.055+0000 7fdae925f740 10 client.25684 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.055+0000 7fdae925f740 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.055+0000 7fdae925f740 20 client.25684 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.055+0000 7fdae925f740 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5653f9b4ec80 0x5653f9b510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.055+0000 7fdae925f740 4 client.25684 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:36:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.055+0000 7fdae925f740 1 -- 172.21.15.42:0/2100594548 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653f9aa8b70 con 0x5653f9b4ec80 2024-04-20T10:36:19.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.055+0000 7fdadfe00640 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5653f9b4ec80 0x5653f9b510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.056+0000 7fdadfe00640 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5653f9b4ec80 0x5653f9b510c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26739d9f471497b3 server_cookie=f6e1eae97887335f in_seq=0 out_seq=0 2024-04-20T10:36:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.056+0000 7fdacd600640 10 client.25684 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdacd600640 1 -- 172.21.15.42:0/2100594548 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653f9aa8b70 con 0x5653f9b4ec80 2024-04-20T10:36:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdacd600640 10 client.25684 handle_command_reply: tid=0 2024-04-20T10:36:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdae925f740 1 client.25684 shutdown 2024-04-20T10:36:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdae925f740 1 -- 172.21.15.42:0/2100594548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 msgr2=0x5653f9b3bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdae925f740 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3bec0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdac8036040 tx=0x7fdac8009610 comp rx=0 tx=0).stop 2024-04-20T10:36:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdae925f740 1 -- 172.21.15.42:0/2100594548 shutdown_connections 2024-04-20T10:36:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdae925f740 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5653f9b4ec80 0x5653f9b510c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdae925f740 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b3e880 0x5653f9b3c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdae925f740 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3bec0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdae925f740 1 --2- 172.21.15.42:0/2100594548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b35380 0x5653f9b3b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdae925f740 1 -- 172.21.15.42:0/2100594548 >> 172.21.15.42:0/2100594548 conn(0x5653f9b2fab0 msgr2=0x5653f9b31800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:19.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdacd600640 0 client.25684 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:19.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdae925f740 1 -- 172.21.15.42:0/2100594548 shutdown_connections 2024-04-20T10:36:19.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.058+0000 7fdae925f740 1 -- 172.21.15.42:0/2100594548 wait complete. 2024-04-20T10:36:19.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.059+0000 7fdae925f740 20 client.25684 trim_cache size 0 max 16384 2024-04-20T10:36:19.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.078+0000 7fdae925f740 1 Processor -- start 2024-04-20T10:36:19.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.078+0000 7fdae925f740 1 -- start start 2024-04-20T10:36:19.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.078+0000 7fdae925f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 0x5653f9b4a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.078+0000 7fdae925f740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.078+0000 7fdae925f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b3e880 0x5653f9b3a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.078+0000 7fdae925f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01adcd0 con 0x5653f9b35380 2024-04-20T10:36:19.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.078+0000 7fdae925f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01b0530 con 0x5653f9b3e880 2024-04-20T10:36:19.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.078+0000 7fdae925f740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01ae320 con 0x5653f9b35ca0 2024-04-20T10:36:19.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.079+0000 7fdadfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 0x5653f9b4a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.079+0000 7fdadfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 0x5653f9b4a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41824/0 (socket says 172.21.15.42:41824) 2024-04-20T10:36:19.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.079+0000 7fdadfe00640 1 -- 172.21.15.42:0/3718281070 learned_addr learned my addr 172.21.15.42:0/3718281070 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:19.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.079+0000 7fdad6a00640 1 --2- 172.21.15.42:0/3718281070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.079+0000 7fdadfe00640 1 --2- 172.21.15.42:0/3718281070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b3e880 0x5653f9b3a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.080+0000 7fdad6a00640 1 -- 172.21.15.42:0/3718281070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b3e880 msgr2=0x5653f9b3a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.080+0000 7fdad6a00640 1 --2- 172.21.15.42:0/3718281070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b3e880 0x5653f9b3a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.080+0000 7fdad6a00640 1 -- 172.21.15.42:0/3718281070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 msgr2=0x5653f9b4a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.080+0000 7fdad6a00640 1 --2- 172.21.15.42:0/3718281070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 0x5653f9b4a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.080+0000 7fdad6a00640 1 -- 172.21.15.42:0/3718281070 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f9b40fe0 con 0x5653f9b35ca0 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.080+0000 7fdad6a00640 1 --2- 172.21.15.42:0/3718281070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3e040 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009850 tx=0x7fdac8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32484f9b89a44c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.080+0000 7fdacfe00640 1 -- 172.21.15.42:0/3718281070 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8058070 con 0x5653f9b35ca0 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.080+0000 7fdacfe00640 1 -- 172.21.15.42:0/3718281070 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac802ccd0 con 0x5653f9b35ca0 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.080+0000 7fdacfe00640 1 -- 172.21.15.42:0/3718281070 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8004cc0 con 0x5653f9b35ca0 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.081+0000 7fdae925f740 1 -- 172.21.15.42:0/3718281070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 msgr2=0x5653f9b3e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.081+0000 7fdae925f740 1 --2- 172.21.15.42:0/3718281070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3e040 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009850 tx=0x7fdac8005c00 comp rx=0 tx=0).stop 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.081+0000 7fdae925f740 1 -- 172.21.15.42:0/3718281070 shutdown_connections 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.081+0000 7fdae925f740 1 --2- 172.21.15.42:0/3718281070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b3e880 0x5653f9b3a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.081+0000 7fdae925f740 1 --2- 172.21.15.42:0/3718281070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b3e040 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.081+0000 7fdae925f740 1 --2- 172.21.15.42:0/3718281070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b35380 0x5653f9b4a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.081+0000 7fdae925f740 1 -- 172.21.15.42:0/3718281070 >> 172.21.15.42:0/3718281070 conn(0x5653f9b2fab0 msgr2=0x5653f9b316c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:19.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.081+0000 7fdae925f740 1 -- 172.21.15.42:0/3718281070 shutdown_connections 2024-04-20T10:36:19.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.081+0000 7fdae925f740 1 -- 172.21.15.42:0/3718281070 wait complete. 2024-04-20T10:36:19.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.081+0000 7fdae925f740 1 -- start start 2024-04-20T10:36:19.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.082+0000 7fdae925f740 1 Processor -- start 2024-04-20T10:36:19.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.082+0000 7fdae925f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b35380 0x5653f9b38790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.082+0000 7fdae925f740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b38cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.082+0000 7fdae925f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b3e880 0x5653f9b39210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.082+0000 7fdae925f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01ae9f0 con 0x5653f9b3e880 2024-04-20T10:36:19.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.082+0000 7fdae925f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01afe60 con 0x5653f9b35380 2024-04-20T10:36:19.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.082+0000 7fdae925f740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fdae01af0c0 con 0x5653f9b35ca0 2024-04-20T10:36:19.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.082+0000 7fdae925f740 10 client.0 _ll_register_callbacks cb 0x7fdac8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:19.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.083+0000 7fdad6a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b38cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.083+0000 7fdad6a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b38cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:47040/0 (socket says 172.21.15.42:47040) 2024-04-20T10:36:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.083+0000 7fdad6a00640 1 -- 172.21.15.42:0/1282873135 learned_addr learned my addr 172.21.15.42:0/1282873135 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.083+0000 7fdadfe00640 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b35380 0x5653f9b38790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.083+0000 7fdadfe00640 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b3e880 0x5653f9b39210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.083+0000 7fdad6a00640 1 -- 172.21.15.42:0/1282873135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b35380 msgr2=0x5653f9b38790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.083+0000 7fdad6a00640 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b35380 0x5653f9b38790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.083+0000 7fdad6a00640 1 -- 172.21.15.42:0/1282873135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b3e880 msgr2=0x5653f9b39210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.083+0000 7fdad6a00640 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b3e880 0x5653f9b39210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.083+0000 7fdad6a00640 1 -- 172.21.15.42:0/1282873135 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f9bbcac0 con 0x5653f9b35ca0 2024-04-20T10:36:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.084+0000 7fdad6a00640 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b38cd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009290 tx=0x7fdac8004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2f6f9fe7d54336b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.084+0000 7fdacd600640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:36:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.084+0000 7fdacd600640 1 -- 172.21.15.42:0/1282873135 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8061060 con 0x5653f9b35ca0 2024-04-20T10:36:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.084+0000 7fdacd600640 1 -- 172.21.15.42:0/1282873135 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fdac8003800 con 0x5653f9b35ca0 2024-04-20T10:36:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.084+0000 7fdae925f740 1 -- 172.21.15.42:0/1282873135 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac802ccd0 con 0x5653f9b35ca0 2024-04-20T10:36:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.084+0000 7fdacd600640 1 -- 172.21.15.42:0/1282873135 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdac8058070 con 0x5653f9b35ca0 2024-04-20T10:36:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.084+0000 7fdae925f740 10 client.25690 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.084+0000 7fdae925f740 1 -- 172.21.15.42:0/1282873135 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5653f9bbddb0 con 0x5653f9b35ca0 2024-04-20T10:36:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.084+0000 7fdacd600640 1 -- 172.21.15.42:0/1282873135 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac802ccd0 con 0x5653f9b35ca0 2024-04-20T10:36:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.085+0000 7fdacd600640 1 -- 172.21.15.42:0/1282873135 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fdac80534c0 con 0x5653f9b35ca0 2024-04-20T10:36:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.085+0000 7fdae925f740 10 client.25690 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.085+0000 7fdae925f740 10 client.25690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.085+0000 7fdae925f740 20 client.25690 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.085+0000 7fdae925f740 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5653f9bbe410 0x5653f9bc0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.085+0000 7fdae925f740 4 client.25690 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:36:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.085+0000 7fdae925f740 1 -- 172.21.15.42:0/1282873135 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5653f9aa29e0 con 0x5653f9bbe410 2024-04-20T10:36:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.085+0000 7fdadfe00640 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5653f9bbe410 0x5653f9bc0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.085+0000 7fdadfe00640 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5653f9bbe410 0x5653f9bc0830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c54a52ba0fd472a0 server_cookie=5abd6d28a99eec87 in_seq=0 out_seq=0 2024-04-20T10:36:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.085+0000 7fdacd600640 10 client.25690 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.087+0000 7fdacd600640 1 -- 172.21.15.42:0/1282873135 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5653f9aa29e0 con 0x5653f9bbe410 2024-04-20T10:36:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.087+0000 7fdacd600640 10 client.25690 handle_command_reply: tid=0 2024-04-20T10:36:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.087+0000 7fdae925f740 1 client.25690 shutdown 2024-04-20T10:36:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.087+0000 7fdae925f740 1 -- 172.21.15.42:0/1282873135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 msgr2=0x5653f9b38cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.087+0000 7fdae925f740 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b38cd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009290 tx=0x7fdac8004d10 comp rx=0 tx=0).stop 2024-04-20T10:36:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.087+0000 7fdae925f740 1 -- 172.21.15.42:0/1282873135 shutdown_connections 2024-04-20T10:36:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.087+0000 7fdae925f740 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5653f9bbe410 0x5653f9bc0830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.087+0000 7fdae925f740 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5653f9b3e880 0x5653f9b39210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.087+0000 7fdae925f740 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5653f9b35ca0 0x5653f9b38cd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.087+0000 7fdacd600640 0 client.25690 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.087+0000 7fdae925f740 1 --2- 172.21.15.42:0/1282873135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5653f9b35380 0x5653f9b38790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.087+0000 7fdae925f740 1 -- 172.21.15.42:0/1282873135 >> 172.21.15.42:0/1282873135 conn(0x5653f9b2fab0 msgr2=0x5653f9b2e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae925f740 1 -- 172.21.15.42:0/1282873135 shutdown_connections 2024-04-20T10:36:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae925f740 1 -- 172.21.15.42:0/1282873135 wait complete. 2024-04-20T10:36:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae925f740 20 client.25690 trim_cache size 0 max 16384 2024-04-20T10:36:19.091 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:36:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (260 inodes in the stack)", 2024-04-20T10:36:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:36:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "fd2c6ec7-3de8-4564-8578-c0186dbc5ffd": { 2024-04-20T10:36:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:36:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "fd2c6ec7-3de8-4564-8578-c0186dbc5ffd", 2024-04-20T10:36:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:36:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:36:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:36:19.092 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:36:19.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae5000640 1 -- 172.21.15.42:0/4154345906 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fdac00a3d20 msgr2=0x7fdac00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:19.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae5000640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fdac00a3d20 0x7fdac00a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdad00cdee0 tx=0x7fdad00d6550 comp rx=0 tx=0).stop 2024-04-20T10:36:19.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae5000640 1 -- 172.21.15.42:0/4154345906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae0104270 msgr2=0x7fdae01a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:19.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae5000640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae0104270 0x7fdae01a0610 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdad000d950 tx=0x7fdad000de10 comp rx=0 tx=0).stop 2024-04-20T10:36:19.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae5000640 1 -- 172.21.15.42:0/4154345906 shutdown_connections 2024-04-20T10:36:19.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae5000640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fdac00a3d20 0x7fdac00a61e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae5000640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdae019a720 0x7fdae019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae5000640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdae0109340 0x7fdae01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae5000640 1 --2- 172.21.15.42:0/4154345906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae0104270 0x7fdae01a0610 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:19.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.088+0000 7fdae5000640 1 -- 172.21.15.42:0/4154345906 >> 172.21.15.42:0/4154345906 conn(0x7fdae00ffa70 msgr2=0x7fdae0101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:19.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.089+0000 7fdae5000640 1 -- 172.21.15.42:0/4154345906 shutdown_connections 2024-04-20T10:36:19.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:19.089+0000 7fdae5000640 1 -- 172.21.15.42:0/4154345906 wait complete. 2024-04-20T10:36:19.107 INFO:tasks.cephfs.filesystem:scrub status for tag:fd2c6ec7-3de8-4564-8578-c0186dbc5ffd - {'path': '/', 'tag': 'fd2c6ec7-3de8-4564-8578-c0186dbc5ffd', 'options': 'recursive,force'} 2024-04-20T10:36:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:19 smithi046 ceph-mon[21574]: pgmap v72: 97 pgs: 97 active+clean; 153 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 845 KiB/s wr, 24 op/s 2024-04-20T10:36:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:19 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:36:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:19 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:36:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:19 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:36:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:19 smithi042 ceph-mon[17078]: pgmap v72: 97 pgs: 97 active+clean; 153 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 845 KiB/s wr, 24 op/s 2024-04-20T10:36:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:19 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:36:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:19 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:36:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:19 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:36:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:19 smithi162 ceph-mon[21854]: pgmap v72: 97 pgs: 97 active+clean; 153 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 845 KiB/s wr, 24 op/s 2024-04-20T10:36:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:19 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:36:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:19 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:36:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:19 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:36:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:21 smithi046 ceph-mon[21574]: pgmap v73: 97 pgs: 97 active+clean; 153 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 845 KiB/s wr, 24 op/s 2024-04-20T10:36:22.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:21 smithi042 ceph-mon[17078]: pgmap v73: 97 pgs: 97 active+clean; 153 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 845 KiB/s wr, 24 op/s 2024-04-20T10:36:22.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:21 smithi162 ceph-mon[21854]: pgmap v73: 97 pgs: 97 active+clean; 153 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 845 KiB/s wr, 24 op/s 2024-04-20T10:36:24.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:23 smithi046 ceph-mon[21574]: pgmap v74: 97 pgs: 97 active+clean; 157 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 37 op/s 2024-04-20T10:36:24.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:23 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:36:24.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:23 smithi042 ceph-mon[17078]: pgmap v74: 97 pgs: 97 active+clean; 157 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 37 op/s 2024-04-20T10:36:24.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:23 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:36:24.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:23 smithi162 ceph-mon[21854]: pgmap v74: 97 pgs: 97 active+clean; 157 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 37 op/s 2024-04-20T10:36:24.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:23 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:36:26.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:25 smithi046 ceph-mon[21574]: pgmap v75: 97 pgs: 97 active+clean; 157 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 930 KiB/s wr, 22 op/s 2024-04-20T10:36:26.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:25 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:36:26.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:25 smithi042 ceph-mon[17078]: pgmap v75: 97 pgs: 97 active+clean; 157 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 930 KiB/s wr, 22 op/s 2024-04-20T10:36:26.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:25 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:36:26.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:25 smithi162 ceph-mon[21854]: pgmap v75: 97 pgs: 97 active+clean; 157 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 930 KiB/s wr, 22 op/s 2024-04-20T10:36:26.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:25 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:36:28.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:27 smithi046 ceph-mon[21574]: pgmap v76: 97 pgs: 97 active+clean; 173 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.3 MiB/s wr, 210 op/s 2024-04-20T10:36:28.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:27 smithi042 ceph-mon[17078]: pgmap v76: 97 pgs: 97 active+clean; 173 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.3 MiB/s wr, 210 op/s 2024-04-20T10:36:28.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:27 smithi162 ceph-mon[21854]: pgmap v76: 97 pgs: 97 active+clean; 173 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.3 MiB/s wr, 210 op/s 2024-04-20T10:36:30.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:29 smithi046 ceph-mon[21574]: pgmap v77: 97 pgs: 97 active+clean; 173 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-20T10:36:30.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:29 smithi042 ceph-mon[17078]: pgmap v77: 97 pgs: 97 active+clean; 173 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-20T10:36:30.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:29 smithi162 ceph-mon[21854]: pgmap v77: 97 pgs: 97 active+clean; 173 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-20T10:36:32.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:31 smithi046 ceph-mon[21574]: pgmap v78: 97 pgs: 97 active+clean; 173 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-20T10:36:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:31 smithi042 ceph-mon[17078]: pgmap v78: 97 pgs: 97 active+clean; 173 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-20T10:36:32.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:31 smithi162 ceph-mon[21854]: pgmap v78: 97 pgs: 97 active+clean; 173 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-20T10:36:34.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:33 smithi046 ceph-mon[21574]: pgmap v79: 97 pgs: 97 active+clean; 195 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 545 op/s 2024-04-20T10:36:34.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:33 smithi042 ceph-mon[17078]: pgmap v79: 97 pgs: 97 active+clean; 195 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 545 op/s 2024-04-20T10:36:34.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:33 smithi162 ceph-mon[21854]: pgmap v79: 97 pgs: 97 active+clean; 195 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 545 op/s 2024-04-20T10:36:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:35 smithi046 ceph-mon[21574]: pgmap v80: 97 pgs: 97 active+clean; 195 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 532 op/s 2024-04-20T10:36:36.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:35 smithi042 ceph-mon[17078]: pgmap v80: 97 pgs: 97 active+clean; 195 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 532 op/s 2024-04-20T10:36:36.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:35 smithi162 ceph-mon[21854]: pgmap v80: 97 pgs: 97 active+clean; 195 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 532 op/s 2024-04-20T10:36:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:37 smithi046 ceph-mon[21574]: pgmap v81: 97 pgs: 97 active+clean; 204 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 586 op/s 2024-04-20T10:36:38.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:37 smithi042 ceph-mon[17078]: pgmap v81: 97 pgs: 97 active+clean; 204 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 586 op/s 2024-04-20T10:36:38.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:37 smithi162 ceph-mon[21854]: pgmap v81: 97 pgs: 97 active+clean; 204 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 586 op/s 2024-04-20T10:36:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:39 smithi046 ceph-mon[21574]: pgmap v82: 97 pgs: 97 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 749 op/s 2024-04-20T10:36:40.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:39 smithi042 ceph-mon[17078]: pgmap v82: 97 pgs: 97 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 749 op/s 2024-04-20T10:36:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:39 smithi162 ceph-mon[21854]: pgmap v82: 97 pgs: 97 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 749 op/s 2024-04-20T10:36:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:41 smithi046 ceph-mon[21574]: pgmap v83: 97 pgs: 97 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 749 op/s 2024-04-20T10:36:42.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:41 smithi042 ceph-mon[17078]: pgmap v83: 97 pgs: 97 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 749 op/s 2024-04-20T10:36:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:41 smithi162 ceph-mon[21854]: pgmap v83: 97 pgs: 97 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 749 op/s 2024-04-20T10:36:44.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:43 smithi046 ceph-mon[21574]: pgmap v84: 97 pgs: 97 active+clean; 245 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 770 op/s 2024-04-20T10:36:44.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:43 smithi042 ceph-mon[17078]: pgmap v84: 97 pgs: 97 active+clean; 245 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 770 op/s 2024-04-20T10:36:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:43 smithi162 ceph-mon[21854]: pgmap v84: 97 pgs: 97 active+clean; 245 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 770 op/s 2024-04-20T10:36:46.055 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:45 smithi046 ceph-mon[21574]: pgmap v85: 97 pgs: 97 active+clean; 245 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 426 op/s 2024-04-20T10:36:46.305 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:45 smithi162 ceph-mon[21854]: pgmap v85: 97 pgs: 97 active+clean; 245 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 426 op/s 2024-04-20T10:36:46.305 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:45 smithi042 ceph-mon[17078]: pgmap v85: 97 pgs: 97 active+clean; 245 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 426 op/s 2024-04-20T10:36:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:47 smithi046 ceph-mon[21574]: pgmap v86: 97 pgs: 97 active+clean; 248 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 431 op/s 2024-04-20T10:36:48.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:47 smithi042 ceph-mon[17078]: pgmap v86: 97 pgs: 97 active+clean; 248 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 431 op/s 2024-04-20T10:36:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:47 smithi162 ceph-mon[21854]: pgmap v86: 97 pgs: 97 active+clean; 248 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 431 op/s 2024-04-20T10:36:49.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:36:49.108 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:36:49.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.336+0000 7f962be00640 1 -- 172.21.15.42:0/34094763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9624104410 msgr2=0x7f96241047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.336+0000 7f962be00640 1 --2- 172.21.15.42:0/34094763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9624104410 0x7f96241047f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9614009860 tx=0x7f961402bb70 comp rx=0 tx=0).stop 2024-04-20T10:36:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.336+0000 7f962be00640 1 -- 172.21.15.42:0/34094763 shutdown_connections 2024-04-20T10:36:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.336+0000 7f962be00640 1 --2- 172.21.15.42:0/34094763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96241094e0 0x7f96241110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.336+0000 7f962be00640 1 --2- 172.21.15.42:0/34094763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9624104dc0 0x7f9624108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.336+0000 7f962be00640 1 --2- 172.21.15.42:0/34094763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9624104410 0x7f96241047f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.336+0000 7f962be00640 1 -- 172.21.15.42:0/34094763 >> 172.21.15.42:0/34094763 conn(0x7f96240ffcf0 msgr2=0x7f9624102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.336+0000 7f962be00640 1 -- 172.21.15.42:0/34094763 shutdown_connections 2024-04-20T10:36:49.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.336+0000 7f962be00640 1 -- 172.21.15.42:0/34094763 wait complete. 2024-04-20T10:36:49.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.337+0000 7f962be00640 1 Processor -- start 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.337+0000 7f962be00640 1 -- start start 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f962be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9624104410 0x7f96241a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f962be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96241094e0 0x7f96241a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f962be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962419a940 0x7f962419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f962be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96241145a0 con 0x7f962419a940 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f962be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9624114420 con 0x7f96241094e0 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f962be00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9624114720 con 0x7f9624104410 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f9629400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9624104410 0x7f96241a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f9629400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9624104410 0x7f96241a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37044/0 (socket says 172.21.15.42:37044) 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f9629400640 1 -- 172.21.15.42:0/3347731821 learned_addr learned my addr 172.21.15.42:0/3347731821 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f9628a00640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96241094e0 0x7f96241a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f9629e00640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962419a940 0x7f962419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f9629400640 1 -- 172.21.15.42:0/3347731821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96241094e0 msgr2=0x7f96241a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f9629400640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96241094e0 0x7f96241a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f9629400640 1 -- 172.21.15.42:0/3347731821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962419a940 msgr2=0x7f962419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f9629400640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962419a940 0x7f962419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.338+0000 7f9629400640 1 -- 172.21.15.42:0/3347731821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962419b460 con 0x7f9624104410 2024-04-20T10:36:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.339+0000 7f9629e00640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962419a940 0x7f962419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:36:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.339+0000 7f9629400640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9624104410 0x7f96241a0740 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9614005e00 tx=0x7f96140045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.339+0000 7f9622000640 1 -- 172.21.15.42:0/3347731821 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614041020 con 0x7f9624104410 2024-04-20T10:36:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.339+0000 7f962be00640 1 -- 172.21.15.42:0/3347731821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962419b6f0 con 0x7f9624104410 2024-04-20T10:36:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.339+0000 7f9622000640 1 -- 172.21.15.42:0/3347731821 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961402c9f0 con 0x7f9624104410 2024-04-20T10:36:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.339+0000 7f9622000640 1 -- 172.21.15.42:0/3347731821 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614039040 con 0x7f9624104410 2024-04-20T10:36:49.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.339+0000 7f962be00640 1 -- 172.21.15.42:0/3347731821 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96241a78f0 con 0x7f9624104410 2024-04-20T10:36:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.342+0000 7f962eb69740 1 Processor -- start 2024-04-20T10:36:49.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f962eb69740 1 -- start start 2024-04-20T10:36:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f9622000640 1 -- 172.21.15.42:0/3347731821 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9614044020 con 0x7f9624104410 2024-04-20T10:36:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f962eb69740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0bd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f962eb69740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f962eb69740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f962eb69740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c7bd023150 con 0x55c7bd0c66a0 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f962eb69740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c7bcfc9370 con 0x55c7bd0bdac0 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f962eb69740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c7bd01f110 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f9629e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f9629e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0bd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37050/0 (socket says 172.21.15.42:37050) 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f9629e00640 1 -- 172.21.15.42:0/1417531377 learned_addr learned my addr 172.21.15.42:0/1417531377 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f9629400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.343+0000 7f9628a00640 1 --2- 172.21.15.42:0/1417531377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.344+0000 7f9629e00640 1 -- 172.21.15.42:0/1417531377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bdac0 msgr2=0x55c7bd0c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.344+0000 7f9629e00640 1 --2- 172.21.15.42:0/1417531377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.344+0000 7f9629e00640 1 -- 172.21.15.42:0/1417531377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0c66a0 msgr2=0x55c7bd0c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.344+0000 7f9629e00640 1 --2- 172.21.15.42:0/1417531377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.344+0000 7f9629e00640 1 -- 172.21.15.42:0/1417531377 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7bcdff7d0 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.344+0000 7f9629e00640 1 --2- 172.21.15.42:0/1417531377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0bd580 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f961c009fd0 tx=0x7f961c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e64fd904ef2608d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:49.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.344+0000 7f961a600640 1 -- 172.21.15.42:0/1417531377 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c011700 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.344+0000 7f961a600640 1 -- 172.21.15.42:0/1417531377 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961c011d80 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.344+0000 7f961a600640 1 -- 172.21.15.42:0/1417531377 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c00e3c0 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.345+0000 7f9622000640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f96080a3d20 0x7f96080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.345+0000 7f9622000640 1 -- 172.21.15.42:0/3347731821 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f96140f1f50 con 0x7f9624104410 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.345+0000 7f9629400640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f96080a3d20 0x7f96080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.345+0000 7f962eb69740 1 -- 172.21.15.42:0/1417531377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bd1a0 msgr2=0x55c7bd0bd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.345+0000 7f962eb69740 1 --2- 172.21.15.42:0/1417531377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0bd580 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f961c009fd0 tx=0x7f961c010860 comp rx=0 tx=0).stop 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.345+0000 7f962eb69740 1 -- 172.21.15.42:0/1417531377 shutdown_connections 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.345+0000 7f962eb69740 1 --2- 172.21.15.42:0/1417531377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.345+0000 7f962eb69740 1 --2- 172.21.15.42:0/1417531377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.345+0000 7f962eb69740 1 --2- 172.21.15.42:0/1417531377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0bd580 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.345+0000 7f962eb69740 1 -- 172.21.15.42:0/1417531377 >> 172.21.15.42:0/1417531377 conn(0x55c7bd0b78d0 msgr2=0x55c7bd0b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.346+0000 7f962eb69740 1 -- 172.21.15.42:0/1417531377 shutdown_connections 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.346+0000 7f9629400640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f96080a3d20 0x7f96080a61e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55c7bd0bcc70 tx=0x7f96140f5a00 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.346+0000 7f962eb69740 1 -- 172.21.15.42:0/1417531377 wait complete. 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.346+0000 7f962eb69740 1 -- start start 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.346+0000 7f962eb69740 1 Processor -- start 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.347+0000 7f962eb69740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.347+0000 7f962eb69740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.347+0000 7f962eb69740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0c4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.347+0000 7f962eb69740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c7bd0c9490 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.347+0000 7f962eb69740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c7bcfdc740 con 0x55c7bd0c66a0 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.347+0000 7f962eb69740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c7bd0c9610 con 0x55c7bd0bdac0 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.347+0000 7f962eb69740 10 client.0 _ll_register_callbacks cb 0x55c7bcfd7b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.347+0000 7f9629e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.347+0000 7f9629e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55196/0 (socket says 172.21.15.42:55196) 2024-04-20T10:36:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.347+0000 7f9629e00640 1 -- 172.21.15.42:0/2804661289 learned_addr learned my addr 172.21.15.42:0/2804661289 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.347+0000 7f9628a00640 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.347+0000 7f9629e00640 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.348+0000 7f9628a00640 1 -- 172.21.15.42:0/2804661289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0c66a0 msgr2=0x55c7bd0c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.348+0000 7f9628a00640 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.348+0000 7f9628a00640 1 -- 172.21.15.42:0/2804661289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 msgr2=0x55c7bd0c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.348+0000 7f9628a00640 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.348+0000 7f9628a00640 1 -- 172.21.15.42:0/2804661289 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7bcd353f0 con 0x55c7bd0bdac0 2024-04-20T10:36:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.348+0000 7f9628a00640 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c3c50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f962419c0e0 tx=0x7f9610011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7240904c10825768 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.348+0000 7f960ea00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:36:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.348+0000 7f960ea00640 1 -- 172.21.15.42:0/2804661289 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961001e020 con 0x55c7bd0bdac0 2024-04-20T10:36:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.348+0000 7f960ea00640 1 -- 172.21.15.42:0/2804661289 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f961000fbc0 con 0x55c7bd0bdac0 2024-04-20T10:36:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.348+0000 7f962eb69740 1 -- 172.21.15.42:0/2804661289 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f961c011d80 con 0x55c7bd0bdac0 2024-04-20T10:36:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.349+0000 7f960ea00640 1 -- 172.21.15.42:0/2804661289 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f961000fdc0 con 0x55c7bd0bdac0 2024-04-20T10:36:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.349+0000 7f962eb69740 10 client.25702 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.349+0000 7f962eb69740 1 -- 172.21.15.42:0/2804661289 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c7bccf50f0 con 0x55c7bd0bdac0 2024-04-20T10:36:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.349+0000 7f960ea00640 1 -- 172.21.15.42:0/2804661289 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961c011d80 con 0x55c7bd0bdac0 2024-04-20T10:36:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.349+0000 7f960ea00640 1 -- 172.21.15.42:0/2804661289 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f9610019070 con 0x55c7bd0bdac0 2024-04-20T10:36:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.349+0000 7f962eb69740 10 client.25702 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.349+0000 7f962eb69740 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.349+0000 7f962eb69740 20 client.25702 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.349+0000 7f962eb69740 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c7bd0d6a80 0x55c7bd0d8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.349+0000 7f962eb69740 4 client.25702 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:36:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.349+0000 7f962eb69740 1 -- 172.21.15.42:0/2804661289 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7bd030b70 con 0x55c7bd0d6a80 2024-04-20T10:36:49.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.350+0000 7f9629e00640 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c7bd0d6a80 0x55c7bd0d8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.350+0000 7f9629e00640 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c7bd0d6a80 0x55c7bd0d8eb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=982971aeb129d136 server_cookie=5e37b4ab3d8f6351 in_seq=0 out_seq=0 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.350+0000 7f960ea00640 10 client.25702 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.351+0000 7f960ea00640 1 -- 172.21.15.42:0/2804661289 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7bd030b70 con 0x55c7bd0d6a80 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.351+0000 7f960ea00640 10 client.25702 handle_command_reply: tid=0 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.351+0000 7f962eb69740 1 client.25702 shutdown 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.351+0000 7f962eb69740 1 -- 172.21.15.42:0/2804661289 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bdac0 msgr2=0x55c7bd0c3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.351+0000 7f962eb69740 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c3c50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f962419c0e0 tx=0x7f9610011b50 comp rx=0 tx=0).stop 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.351+0000 7f962eb69740 1 -- 172.21.15.42:0/2804661289 shutdown_connections 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.351+0000 7f962eb69740 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c7bd0d6a80 0x55c7bd0d8eb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.351+0000 7f962eb69740 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.351+0000 7f962eb69740 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c3c50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.351+0000 7f962eb69740 1 --2- 172.21.15.42:0/2804661289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.351+0000 7f962eb69740 1 -- 172.21.15.42:0/2804661289 >> 172.21.15.42:0/2804661289 conn(0x55c7bd0b78d0 msgr2=0x55c7bd0b9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.351+0000 7f960ea00640 0 client.25702 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.352+0000 7f962eb69740 1 -- 172.21.15.42:0/2804661289 shutdown_connections 2024-04-20T10:36:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.352+0000 7f962eb69740 1 -- 172.21.15.42:0/2804661289 wait complete. 2024-04-20T10:36:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.352+0000 7f962eb69740 20 client.25702 trim_cache size 0 max 16384 2024-04-20T10:36:49.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.374+0000 7f962eb69740 1 Processor -- start 2024-04-20T10:36:49.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.374+0000 7f962eb69740 1 -- start start 2024-04-20T10:36:49.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.374+0000 7f962eb69740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.374+0000 7f962eb69740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.374+0000 7f962eb69740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0bea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.374+0000 7f962eb69740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96241adef0 con 0x55c7bd0bdac0 2024-04-20T10:36:49.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.374+0000 7f962eb69740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f96241b0770 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.374+0000 7f962eb69740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f96241ae560 con 0x55c7bd0c66a0 2024-04-20T10:36:49.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.375+0000 7f9628a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.375+0000 7f9628a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55206/0 (socket says 172.21.15.42:55206) 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.375+0000 7f9628a00640 1 -- 172.21.15.42:0/3411712362 learned_addr learned my addr 172.21.15.42:0/3411712362 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.375+0000 7f9629e00640 1 --2- 172.21.15.42:0/3411712362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.375+0000 7f9629e00640 1 --2- 172.21.15.42:0/3411712362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0bea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.375+0000 7f9628a00640 1 -- 172.21.15.42:0/3411712362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0c66a0 msgr2=0x55c7bd0bea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.375+0000 7f9628a00640 1 --2- 172.21.15.42:0/3411712362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0bea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.375+0000 7f9628a00640 1 -- 172.21.15.42:0/3411712362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bd1a0 msgr2=0x55c7bd0c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.375+0000 7f9628a00640 1 --2- 172.21.15.42:0/3411712362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.375+0000 7f9628a00640 1 -- 172.21.15.42:0/3411712362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7bd0b1d10 con 0x55c7bd0bdac0 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.375+0000 7f9629e00640 1 --2- 172.21.15.42:0/3411712362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.376+0000 7f9628a00640 1 --2- 172.21.15.42:0/3411712362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c2650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9610000c00 tx=0x7f961000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2ed43010204343 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.376+0000 7f9619c00640 1 -- 172.21.15.42:0/3411712362 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961002e020 con 0x55c7bd0bdac0 2024-04-20T10:36:49.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.376+0000 7f9619c00640 1 -- 172.21.15.42:0/3411712362 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9610019370 con 0x55c7bd0bdac0 2024-04-20T10:36:49.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.376+0000 7f9619c00640 1 -- 172.21.15.42:0/3411712362 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610000e60 con 0x55c7bd0bdac0 2024-04-20T10:36:49.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.377+0000 7f962eb69740 1 -- 172.21.15.42:0/3411712362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bdac0 msgr2=0x55c7bd0c2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.377+0000 7f962eb69740 1 --2- 172.21.15.42:0/3411712362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c2650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9610000c00 tx=0x7f961000f790 comp rx=0 tx=0).stop 2024-04-20T10:36:49.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.377+0000 7f962eb69740 1 -- 172.21.15.42:0/3411712362 shutdown_connections 2024-04-20T10:36:49.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.377+0000 7f962eb69740 1 --2- 172.21.15.42:0/3411712362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0bea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.377+0000 7f962eb69740 1 --2- 172.21.15.42:0/3411712362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0c2650 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.377+0000 7f962eb69740 1 --2- 172.21.15.42:0/3411712362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.377+0000 7f962eb69740 1 -- 172.21.15.42:0/3411712362 >> 172.21.15.42:0/3411712362 conn(0x55c7bd0b78d0 msgr2=0x55c7bd0c3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.377+0000 7f962eb69740 1 -- 172.21.15.42:0/3411712362 shutdown_connections 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.377+0000 7f962eb69740 1 -- 172.21.15.42:0/3411712362 wait complete. 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.377+0000 7f962eb69740 1 -- start start 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.378+0000 7f962eb69740 1 Processor -- start 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.378+0000 7f962eb69740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.378+0000 7f962eb69740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.378+0000 7f962eb69740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.378+0000 7f962eb69740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96241aec30 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.378+0000 7f962eb69740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f96241b00a0 con 0x55c7bd0bdac0 2024-04-20T10:36:49.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.378+0000 7f962eb69740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f96241af300 con 0x55c7bd0c66a0 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.378+0000 7f962eb69740 10 client.0 _ll_register_callbacks cb 0x7f9610009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.378+0000 7f9629e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.378+0000 7f9629e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55214/0 (socket says 172.21.15.42:55214) 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.378+0000 7f9629e00640 1 -- 172.21.15.42:0/3180305729 learned_addr learned my addr 172.21.15.42:0/3180305729 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.379+0000 7f9628a00640 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.379+0000 7f9629e00640 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.379+0000 7f9629e00640 1 -- 172.21.15.42:0/3180305729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0c66a0 msgr2=0x55c7bd0caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.379+0000 7f9629e00640 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.379+0000 7f9629e00640 1 -- 172.21.15.42:0/3180305729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bdac0 msgr2=0x55c7bd0ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.379+0000 7f9629e00640 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.379+0000 7f9629e00640 1 -- 172.21.15.42:0/3180305729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7bd13c120 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.379+0000 7f9628a00640 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.379+0000 7f9629e00640 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0ca460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f961c01bac0 tx=0x7f961c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb30b532a8fcf63 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.379+0000 7f960ea00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.380+0000 7f960ea00640 1 -- 172.21.15.42:0/3180305729 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c01b700 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.380+0000 7f962eb69740 1 -- 172.21.15.42:0/3180305729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9610019370 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.380+0000 7f960ea00640 1 -- 172.21.15.42:0/3180305729 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f961c023040 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.380+0000 7f960ea00640 1 -- 172.21.15.42:0/3180305729 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f961c040050 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.380+0000 7f962eb69740 10 client.15576 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:49.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.380+0000 7f962eb69740 1 -- 172.21.15.42:0/3180305729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c7bd13d410 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.380+0000 7f960ea00640 1 -- 172.21.15.42:0/3180305729 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9610019370 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.380+0000 7f960ea00640 1 -- 172.21.15.42:0/3180305729 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f961c039060 con 0x55c7bd0bd1a0 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.380+0000 7f962eb69740 10 client.15576 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.380+0000 7f962eb69740 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.381+0000 7f962eb69740 20 client.15576 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.381+0000 7f962eb69740 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c7bd13d9f0 0x55c7bd13fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.381+0000 7f962eb69740 4 client.15576 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.381+0000 7f962eb69740 1 -- 172.21.15.42:0/3180305729 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c7bd02a9e0 con 0x55c7bd13d9f0 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.381+0000 7f9628a00640 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c7bd13d9f0 0x55c7bd13fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f9628a00640 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c7bd13d9f0 0x55c7bd13fde0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa07edde550fa53 server_cookie=7d976fd8cc0677b in_seq=0 out_seq=0 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f960ea00640 10 client.15576 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f960ea00640 1 -- 172.21.15.42:0/3180305729 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c7bd02a9e0 con 0x55c7bd13d9f0 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f960ea00640 10 client.15576 handle_command_reply: tid=0 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f962eb69740 1 client.15576 shutdown 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f962eb69740 1 -- 172.21.15.42:0/3180305729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 msgr2=0x55c7bd0ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f962eb69740 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0ca460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f961c01bac0 tx=0x7f961c010860 comp rx=0 tx=0).stop 2024-04-20T10:36:49.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f962eb69740 1 -- 172.21.15.42:0/3180305729 shutdown_connections 2024-04-20T10:36:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f962eb69740 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c7bd13d9f0 0x55c7bd13fde0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f962eb69740 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c7bd0c66a0 0x55c7bd0caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f962eb69740 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7bd0bdac0 0x55c7bd0ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f962eb69740 1 --2- 172.21.15.42:0/3180305729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c7bd0bd1a0 0x55c7bd0ca460 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f960ea00640 0 client.15576 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f962eb69740 1 -- 172.21.15.42:0/3180305729 >> 172.21.15.42:0/3180305729 conn(0x55c7bd0b78d0 msgr2=0x55c7bd0b8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f962eb69740 1 -- 172.21.15.42:0/3180305729 shutdown_connections 2024-04-20T10:36:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f962eb69740 1 -- 172.21.15.42:0/3180305729 wait complete. 2024-04-20T10:36:49.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.382+0000 7f962eb69740 20 client.15576 trim_cache size 0 max 16384 2024-04-20T10:36:49.385 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:36:49.385 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:36:49.385 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:36:49.386 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:36:49.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.383+0000 7f962be00640 1 -- 172.21.15.42:0/3347731821 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f96080a3d20 msgr2=0x7f96080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.383+0000 7f962be00640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f96080a3d20 0x7f96080a61e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55c7bd0bcc70 tx=0x7f96140f5a00 comp rx=0 tx=0).stop 2024-04-20T10:36:49.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.383+0000 7f962be00640 1 -- 172.21.15.42:0/3347731821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9624104410 msgr2=0x7f96241a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.383+0000 7f962be00640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9624104410 0x7f96241a0740 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9614005e00 tx=0x7f96140045d0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.383+0000 7f962be00640 1 -- 172.21.15.42:0/3347731821 shutdown_connections 2024-04-20T10:36:49.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.383+0000 7f962be00640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f96080a3d20 0x7f96080a61e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.383+0000 7f962be00640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962419a940 0x7f962419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.383+0000 7f962be00640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96241094e0 0x7f96241a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.383+0000 7f962be00640 1 --2- 172.21.15.42:0/3347731821 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9624104410 0x7f96241a0740 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.383+0000 7f962be00640 1 -- 172.21.15.42:0/3347731821 >> 172.21.15.42:0/3347731821 conn(0x7f96240ffcf0 msgr2=0x7f9624101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:49.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.383+0000 7f962be00640 1 -- 172.21.15.42:0/3347731821 shutdown_connections 2024-04-20T10:36:49.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.383+0000 7f962be00640 1 -- 172.21.15.42:0/3347731821 wait complete. 2024-04-20T10:36:49.401 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:36:49.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:36:49.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.610+0000 7ff091e00640 1 -- 172.21.15.42:0/3196772181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08c069360 msgr2=0x7ff08c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.610+0000 7ff091e00640 1 --2- 172.21.15.42:0/3196772181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08c069360 0x7ff08c069740 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009900 tx=0x7ff07c02bc40 comp rx=0 tx=0).stop 2024-04-20T10:36:49.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.610+0000 7ff091e00640 1 -- 172.21.15.42:0/3196772181 shutdown_connections 2024-04-20T10:36:49.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.610+0000 7ff091e00640 1 --2- 172.21.15.42:0/3196772181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff08c107100 0x7ff08c10eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.610+0000 7ff091e00640 1 --2- 172.21.15.42:0/3196772181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff08c069d10 0x7ff08c1069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.610+0000 7ff091e00640 1 --2- 172.21.15.42:0/3196772181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08c069360 0x7ff08c069740 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.610+0000 7ff091e00640 1 -- 172.21.15.42:0/3196772181 >> 172.21.15.42:0/3196772181 conn(0x7ff08c0767a0 msgr2=0x7ff08c076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:49.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.610+0000 7ff091e00640 1 -- 172.21.15.42:0/3196772181 shutdown_connections 2024-04-20T10:36:49.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.610+0000 7ff091e00640 1 -- 172.21.15.42:0/3196772181 wait complete. 2024-04-20T10:36:49.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.611+0000 7ff091e00640 1 Processor -- start 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.611+0000 7ff091e00640 1 -- start start 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff08c069360 0x7ff08c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff08c107100 0x7ff08c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08c19a710 0x7ff08c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c1121d0 con 0x7ff08c19a710 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c112050 con 0x7ff08c069360 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c112350 con 0x7ff08c107100 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08c19a710 0x7ff08c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08c19a710 0x7ff08c19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55238/0 (socket says 172.21.15.42:55238) 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091400640 1 -- 172.21.15.42:0/494242441 learned_addr learned my addr 172.21.15.42:0/494242441 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff083e00640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff08c107100 0x7ff08c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff090a00640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff08c069360 0x7ff08c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091400640 1 -- 172.21.15.42:0/494242441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff08c107100 msgr2=0x7ff08c1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091400640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff08c107100 0x7ff08c1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091400640 1 -- 172.21.15.42:0/494242441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff08c069360 msgr2=0x7ff08c1a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091400640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff08c069360 0x7ff08c1a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.612+0000 7ff091400640 1 -- 172.21.15.42:0/494242441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c19b310 con 0x7ff08c19a710 2024-04-20T10:36:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.613+0000 7ff091400640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08c19a710 0x7ff08c19ab90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff08400eb10 tx=0x7ff08400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.613+0000 7ff08a000640 1 -- 172.21.15.42:0/494242441 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084004b20 con 0x7ff08c19a710 2024-04-20T10:36:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.613+0000 7ff08a000640 1 -- 172.21.15.42:0/494242441 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff084004cc0 con 0x7ff08c19a710 2024-04-20T10:36:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.613+0000 7ff08a000640 1 -- 172.21.15.42:0/494242441 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084010760 con 0x7ff08c19a710 2024-04-20T10:36:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.613+0000 7ff091e00640 1 -- 172.21.15.42:0/494242441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c1a73a0 con 0x7ff08c19a710 2024-04-20T10:36:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.613+0000 7ff091e00640 1 -- 172.21.15.42:0/494242441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c1a77a0 con 0x7ff08c19a710 2024-04-20T10:36:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.614+0000 7ff091e00640 1 -- 172.21.15.42:0/494242441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c1034b0 con 0x7ff08c19a710 2024-04-20T10:36:49.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.617+0000 7ff08a000640 1 -- 172.21.15.42:0/494242441 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff084005020 con 0x7ff08c19a710 2024-04-20T10:36:49.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.618+0000 7ff08a000640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff0700a3da0 0x7ff0700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:49.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.618+0000 7ff08a000640 1 -- 172.21.15.42:0/494242441 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ff0840d0d10 con 0x7ff08c19a710 2024-04-20T10:36:49.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.618+0000 7ff090a00640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff0700a3da0 0x7ff0700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:49.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.619+0000 7ff090a00640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff0700a3da0 0x7ff0700a6260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff07c02efd0 tx=0x7ff07c02cf90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.623+0000 7ff08a000640 1 -- 172.21.15.42:0/494242441 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff0840d6020 con 0x7ff08c19a710 2024-04-20T10:36:49.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.873+0000 7ff091e00640 1 -- 172.21.15.42:0/494242441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff08c1004d0 con 0x7ff08c19a710 2024-04-20T10:36:49.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.875+0000 7ff08a000640 1 -- 172.21.15.42:0/494242441 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7ff08409d350 con 0x7ff08c19a710 2024-04-20T10:36:49.875 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:36:49.875 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:36:49.875 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:36:49.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.878+0000 7ff091e00640 1 -- 172.21.15.42:0/494242441 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff0700a3da0 msgr2=0x7ff0700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.878+0000 7ff091e00640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff0700a3da0 0x7ff0700a6260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff07c02efd0 tx=0x7ff07c02cf90 comp rx=0 tx=0).stop 2024-04-20T10:36:49.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.878+0000 7ff091e00640 1 -- 172.21.15.42:0/494242441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08c19a710 msgr2=0x7ff08c19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:49.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.878+0000 7ff091e00640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08c19a710 0x7ff08c19ab90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff08400eb10 tx=0x7ff08400c2d0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.878+0000 7ff091e00640 1 -- 172.21.15.42:0/494242441 shutdown_connections 2024-04-20T10:36:49.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.878+0000 7ff091e00640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff0700a3da0 0x7ff0700a6260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.878+0000 7ff091e00640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff08c19a710 0x7ff08c19ab90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.878+0000 7ff091e00640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff08c107100 0x7ff08c1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.878+0000 7ff091e00640 1 --2- 172.21.15.42:0/494242441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff08c069360 0x7ff08c1a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:49.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.878+0000 7ff091e00640 1 -- 172.21.15.42:0/494242441 >> 172.21.15.42:0/494242441 conn(0x7ff08c0767a0 msgr2=0x7ff08c10d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.878+0000 7ff091e00640 1 -- 172.21.15.42:0/494242441 shutdown_connections 2024-04-20T10:36:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:49.878+0000 7ff091e00640 1 -- 172.21.15.42:0/494242441 wait complete. 2024-04-20T10:36:49.898 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:36:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:49 smithi046 ceph-mon[21574]: pgmap v87: 97 pgs: 97 active+clean; 250 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 390 op/s 2024-04-20T10:36:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:49 smithi042 ceph-mon[17078]: pgmap v87: 97 pgs: 97 active+clean; 250 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 390 op/s 2024-04-20T10:36:50.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.118+0000 7fad35200640 1 -- 172.21.15.42:0/1619652045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad30104c30 msgr2=0x7fad30108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:50.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.118+0000 7fad35200640 1 --2- 172.21.15.42:0/1619652045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad30104c30 0x7fad30108c70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fad20009ca0 tx=0x7fad2002c100 comp rx=0 tx=0).stop 2024-04-20T10:36:50.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.119+0000 7fad35200640 1 -- 172.21.15.42:0/1619652045 shutdown_connections 2024-04-20T10:36:50.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.119+0000 7fad35200640 1 --2- 172.21.15.42:0/1619652045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad30109350 0x7fad30110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.119+0000 7fad35200640 1 --2- 172.21.15.42:0/1619652045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad30104c30 0x7fad30108c70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.119+0000 7fad35200640 1 --2- 172.21.15.42:0/1619652045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad30104280 0x7fad30104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.119+0000 7fad35200640 1 -- 172.21.15.42:0/1619652045 >> 172.21.15.42:0/1619652045 conn(0x7fad300ffaa0 msgr2=0x7fad30101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:50.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.119+0000 7fad35200640 1 -- 172.21.15.42:0/1619652045 shutdown_connections 2024-04-20T10:36:50.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.119+0000 7fad35200640 1 -- 172.21.15.42:0/1619652045 wait complete. 2024-04-20T10:36:50.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.120+0000 7fad35200640 1 Processor -- start 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.120+0000 7fad35200640 1 -- start start 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.120+0000 7fad35200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad30104280 0x7fad301a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.120+0000 7fad35200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad30109350 0x7fad301a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.120+0000 7fad35200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3019a760 0x7fad3019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.120+0000 7fad35200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fad301143c0 con 0x7fad3019a760 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.120+0000 7fad35200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad30114240 con 0x7fad30109350 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.120+0000 7fad35200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fad30114540 con 0x7fad30104280 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3019a760 0x7fad3019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2f400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad30104280 0x7fad301a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2f400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad30104280 0x7fad301a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37082/0 (socket says 172.21.15.42:37082) 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2f400640 1 -- 172.21.15.42:0/36146876 learned_addr learned my addr 172.21.15.42:0/36146876 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2ea00640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad30109350 0x7fad301a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2ea00640 1 -- 172.21.15.42:0/36146876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad30104280 msgr2=0x7fad301a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:50.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2ea00640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad30104280 0x7fad301a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2ea00640 1 -- 172.21.15.42:0/36146876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3019a760 msgr2=0x7fad3019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:50.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2ea00640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3019a760 0x7fad3019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2ea00640 1 -- 172.21.15.42:0/36146876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3019b360 con 0x7fad30109350 2024-04-20T10:36:50.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2f400640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad30104280 0x7fad301a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:50.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2fe00640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3019a760 0x7fad3019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:50.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.121+0000 7fad2ea00640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad30109350 0x7fad301a0b90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fad20037040 tx=0x7fad20035ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:50.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.122+0000 7fad27e00640 1 -- 172.21.15.42:0/36146876 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20039040 con 0x7fad30109350 2024-04-20T10:36:50.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.122+0000 7fad35200640 1 -- 172.21.15.42:0/36146876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3019b5f0 con 0x7fad30109350 2024-04-20T10:36:50.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.122+0000 7fad27e00640 1 -- 172.21.15.42:0/36146876 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad20035cd0 con 0x7fad30109350 2024-04-20T10:36:50.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.122+0000 7fad27e00640 1 -- 172.21.15.42:0/36146876 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad200042a0 con 0x7fad30109350 2024-04-20T10:36:50.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.122+0000 7fad35200640 1 -- 172.21.15.42:0/36146876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad301a76e0 con 0x7fad30109350 2024-04-20T10:36:50.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.125+0000 7fad39595740 1 Processor -- start 2024-04-20T10:36:50.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.125+0000 7fad27e00640 1 -- 172.21.15.42:0/36146876 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fad20046030 con 0x7fad30109350 2024-04-20T10:36:50.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.125+0000 7fad39595740 1 -- start start 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.126+0000 7fad39595740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415170 0x5614b6415550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.126+0000 7fad39595740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415a90 0x5614b641e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.126+0000 7fad39595740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b6420a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.126+0000 7fad39595740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5614b637a050 con 0x5614b641e670 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.126+0000 7fad39595740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5614b6320490 con 0x5614b6415170 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.126+0000 7fad39595740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5614b6376010 con 0x5614b6415a90 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.126+0000 7fad2ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b6420a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.126+0000 7fad2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415170 0x5614b6415550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.126+0000 7fad2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415170 0x5614b6415550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49954/0 (socket says 172.21.15.42:49954) 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.126+0000 7fad2fe00640 1 -- 172.21.15.42:0/1171697648 learned_addr learned my addr 172.21.15.42:0/1171697648 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.126+0000 7fad2f400640 1 --2- 172.21.15.42:0/1171697648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415a90 0x5614b641e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.126+0000 7fad2fe00640 1 -- 172.21.15.42:0/1171697648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415a90 msgr2=0x5614b641e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:50.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.127+0000 7fad2fe00640 1 --2- 172.21.15.42:0/1171697648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415a90 0x5614b641e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.127+0000 7fad2fe00640 1 -- 172.21.15.42:0/1171697648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 msgr2=0x5614b6420a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:50.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.127+0000 7fad2fe00640 1 --2- 172.21.15.42:0/1171697648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b6420a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.127+0000 7fad2fe00640 1 -- 172.21.15.42:0/1171697648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b6328070 con 0x5614b6415170 2024-04-20T10:36:50.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.127+0000 7fad27e00640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fad140a3d20 0x7fad140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.127+0000 7fad2fe00640 1 --2- 172.21.15.42:0/1171697648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415170 0x5614b6415550 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad28010400 tx=0x7fad280108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2a091432cc300f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:50.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.127+0000 7fad27e00640 1 -- 172.21.15.42:0/36146876 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fad200f2270 con 0x7fad30109350 2024-04-20T10:36:50.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.127+0000 7fad2f400640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fad140a3d20 0x7fad140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.127+0000 7fad24c00640 1 -- 172.21.15.42:0/1171697648 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28011760 con 0x5614b6415170 2024-04-20T10:36:50.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.127+0000 7fad24c00640 1 -- 172.21.15.42:0/1171697648 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad28011de0 con 0x5614b6415170 2024-04-20T10:36:50.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.127+0000 7fad24c00640 1 -- 172.21.15.42:0/1171697648 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28018380 con 0x5614b6415170 2024-04-20T10:36:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.128+0000 7fad39595740 1 -- 172.21.15.42:0/1171697648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415170 msgr2=0x5614b6415550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.128+0000 7fad39595740 1 --2- 172.21.15.42:0/1171697648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415170 0x5614b6415550 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad28010400 tx=0x7fad280108c0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.128+0000 7fad2f400640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fad140a3d20 0x7fad140a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fad18009620 tx=0x7fad18005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.128+0000 7fad39595740 1 -- 172.21.15.42:0/1171697648 shutdown_connections 2024-04-20T10:36:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.128+0000 7fad39595740 1 --2- 172.21.15.42:0/1171697648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b6420a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.128+0000 7fad39595740 1 --2- 172.21.15.42:0/1171697648 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415a90 0x5614b641e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.128+0000 7fad39595740 1 --2- 172.21.15.42:0/1171697648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415170 0x5614b6415550 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.128+0000 7fad39595740 1 -- 172.21.15.42:0/1171697648 >> 172.21.15.42:0/1171697648 conn(0x5614b640f8a0 msgr2=0x5614b640fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.128+0000 7fad39595740 1 -- 172.21.15.42:0/1171697648 shutdown_connections 2024-04-20T10:36:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.128+0000 7fad39595740 1 -- 172.21.15.42:0/1171697648 wait complete. 2024-04-20T10:36:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.129+0000 7fad39595740 1 -- start start 2024-04-20T10:36:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.129+0000 7fad39595740 1 Processor -- start 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.129+0000 7fad39595740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415170 0x5614b641b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.129+0000 7fad39595740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415a90 0x5614b641bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.129+0000 7fad39595740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b641c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.129+0000 7fad39595740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5614b6421460 con 0x5614b641e670 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.129+0000 7fad39595740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5614b6333860 con 0x5614b6415a90 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.129+0000 7fad39595740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5614b64215e0 con 0x5614b6415170 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.130+0000 7fad39595740 10 client.0 _ll_register_callbacks cb 0x5614b6301fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.130+0000 7fad2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b641c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.130+0000 7fad2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b641c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55274/0 (socket says 172.21.15.42:55274) 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.130+0000 7fad2fe00640 1 -- 172.21.15.42:0/2263825764 learned_addr learned my addr 172.21.15.42:0/2263825764 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.130+0000 7fad2f400640 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415170 0x5614b641b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.130+0000 7fad2ea00640 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415a90 0x5614b641bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.130+0000 7fad2fe00640 1 -- 172.21.15.42:0/2263825764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415170 msgr2=0x5614b641b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.130+0000 7fad2fe00640 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415170 0x5614b641b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.130+0000 7fad2fe00640 1 -- 172.21.15.42:0/2263825764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415a90 msgr2=0x5614b641bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.130+0000 7fad2fe00640 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415a90 0x5614b641bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.130+0000 7fad2fe00640 1 -- 172.21.15.42:0/2263825764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b6337950 con 0x5614b641e670 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.131+0000 7fad2fe00640 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b641c1e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fad28000c00 tx=0x7fad28011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07b8047f65b5327 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.131+0000 7fad1d600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.131+0000 7fad1d600640 1 -- 172.21.15.42:0/2263825764 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28027020 con 0x5614b641e670 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.131+0000 7fad1d600640 1 -- 172.21.15.42:0/2263825764 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fad28019cd0 con 0x5614b641e670 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.131+0000 7fad39595740 1 -- 172.21.15.42:0/2263825764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad28011de0 con 0x5614b641e670 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.131+0000 7fad1d600640 1 -- 172.21.15.42:0/2263825764 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fad28032050 con 0x5614b641e670 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.131+0000 7fad39595740 10 client.15594 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.131+0000 7fad39595740 1 -- 172.21.15.42:0/2263825764 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5614b6305820 con 0x5614b641e670 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.132+0000 7fad1d600640 1 -- 172.21.15.42:0/2263825764 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad28011de0 con 0x5614b641e670 2024-04-20T10:36:50.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.132+0000 7fad1d600640 1 -- 172.21.15.42:0/2263825764 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fad28022030 con 0x5614b641e670 2024-04-20T10:36:50.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.132+0000 7fad39595740 10 client.15594 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:50.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.132+0000 7fad39595740 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:50.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.132+0000 7fad39595740 20 client.15594 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:50.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.132+0000 7fad39595740 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5614b642e6c0 0x5614b6430ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.132+0000 7fad39595740 4 client.15594 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:36:50.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.132+0000 7fad39595740 1 -- 172.21.15.42:0/2263825764 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614b6387a70 con 0x5614b642e6c0 2024-04-20T10:36:50.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.132+0000 7fad2f400640 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5614b642e6c0 0x5614b6430ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.133+0000 7fad2f400640 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5614b642e6c0 0x5614b6430ab0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa340f3cf6e56b2 server_cookie=18c8e20d75be9b7a in_seq=0 out_seq=0 2024-04-20T10:36:50.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.133+0000 7fad1d600640 10 client.15594 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:50.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.134+0000 7fad1d600640 1 -- 172.21.15.42:0/2263825764 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614b6387a70 con 0x5614b642e6c0 2024-04-20T10:36:50.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.134+0000 7fad1d600640 10 client.15594 handle_command_reply: tid=0 2024-04-20T10:36:50.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.134+0000 7fad39595740 1 client.15594 shutdown 2024-04-20T10:36:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.134+0000 7fad39595740 1 -- 172.21.15.42:0/2263825764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 msgr2=0x5614b641c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.134+0000 7fad39595740 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b641c1e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fad28000c00 tx=0x7fad28011f90 comp rx=0 tx=0).stop 2024-04-20T10:36:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.134+0000 7fad39595740 1 -- 172.21.15.42:0/2263825764 shutdown_connections 2024-04-20T10:36:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.135+0000 7fad39595740 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5614b642e6c0 0x5614b6430ab0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.135+0000 7fad39595740 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b641c1e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.135+0000 7fad39595740 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415a90 0x5614b641bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.135+0000 7fad39595740 1 --2- 172.21.15.42:0/2263825764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415170 0x5614b641b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.135+0000 7fad1d600640 0 client.15594 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.135+0000 7fad39595740 1 -- 172.21.15.42:0/2263825764 >> 172.21.15.42:0/2263825764 conn(0x5614b640f8a0 msgr2=0x5614b64115f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.135+0000 7fad39595740 1 -- 172.21.15.42:0/2263825764 shutdown_connections 2024-04-20T10:36:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.135+0000 7fad39595740 1 -- 172.21.15.42:0/2263825764 wait complete. 2024-04-20T10:36:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.135+0000 7fad39595740 20 client.15594 trim_cache size 0 max 16384 2024-04-20T10:36:50.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.150+0000 7fad39595740 1 Processor -- start 2024-04-20T10:36:50.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.150+0000 7fad39595740 1 -- start start 2024-04-20T10:36:50.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.150+0000 7fad39595740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415170 0x5614b641de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.150+0000 7fad39595740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415a90 0x5614b642a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.150+0000 7fad39595740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b640e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.150+0000 7fad39595740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fad301adc80 con 0x5614b641e670 2024-04-20T10:36:50.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.150+0000 7fad39595740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad301b04e0 con 0x5614b6415a90 2024-04-20T10:36:50.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.150+0000 7fad39595740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fad301ae2d0 con 0x5614b6415170 2024-04-20T10:36:50.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:49 smithi162 ceph-mon[21854]: pgmap v87: 97 pgs: 97 active+clean; 250 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 390 op/s 2024-04-20T10:36:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.150+0000 7fad2f400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415170 0x5614b641de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.150+0000 7fad2f400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415170 0x5614b641de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37100/0 (socket says 172.21.15.42:37100) 2024-04-20T10:36:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.150+0000 7fad2f400640 1 -- 172.21.15.42:0/3293856144 learned_addr learned my addr 172.21.15.42:0/3293856144 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.150+0000 7fad2fe00640 1 --2- 172.21.15.42:0/3293856144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b640e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.151+0000 7fad2ea00640 1 --2- 172.21.15.42:0/3293856144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415a90 0x5614b642a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.151+0000 7fad2f400640 1 -- 172.21.15.42:0/3293856144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415a90 msgr2=0x5614b642a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.151+0000 7fad2f400640 1 --2- 172.21.15.42:0/3293856144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415a90 0x5614b642a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.151+0000 7fad2f400640 1 -- 172.21.15.42:0/3293856144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 msgr2=0x5614b640e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.151+0000 7fad2f400640 1 --2- 172.21.15.42:0/3293856144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b640e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.151+0000 7fad2f400640 1 -- 172.21.15.42:0/3293856144 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b608c3f0 con 0x5614b6415170 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.151+0000 7fad2ea00640 1 --2- 172.21.15.42:0/3293856144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415a90 0x5614b642a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.151+0000 7fad2f400640 1 --2- 172.21.15.42:0/3293856144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415170 0x5614b641de90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x5614b6414c40 tx=0x7fad18017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d992ed9514126f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.152+0000 7fad1fe00640 1 -- 172.21.15.42:0/3293856144 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad180210d0 con 0x5614b6415170 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.152+0000 7fad1fe00640 1 -- 172.21.15.42:0/3293856144 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad18021270 con 0x5614b6415170 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.152+0000 7fad1fe00640 1 -- 172.21.15.42:0/3293856144 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18021b90 con 0x5614b6415170 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.152+0000 7fad39595740 1 -- 172.21.15.42:0/3293856144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415170 msgr2=0x5614b641de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.152+0000 7fad39595740 1 --2- 172.21.15.42:0/3293856144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415170 0x5614b641de90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x5614b6414c40 tx=0x7fad18017000 comp rx=0 tx=0).stop 2024-04-20T10:36:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.152+0000 7fad39595740 1 -- 172.21.15.42:0/3293856144 shutdown_connections 2024-04-20T10:36:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.152+0000 7fad39595740 1 --2- 172.21.15.42:0/3293856144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b641e670 0x5614b640e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.152+0000 7fad39595740 1 --2- 172.21.15.42:0/3293856144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b6415a90 0x5614b642a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.152+0000 7fad39595740 1 --2- 172.21.15.42:0/3293856144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415170 0x5614b641de90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.152+0000 7fad39595740 1 -- 172.21.15.42:0/3293856144 >> 172.21.15.42:0/3293856144 conn(0x5614b640f8a0 msgr2=0x5614b640e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.153+0000 7fad39595740 1 -- 172.21.15.42:0/3293856144 shutdown_connections 2024-04-20T10:36:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.153+0000 7fad39595740 1 -- 172.21.15.42:0/3293856144 wait complete. 2024-04-20T10:36:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.153+0000 7fad39595740 1 -- start start 2024-04-20T10:36:50.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.153+0000 7fad39595740 1 Processor -- start 2024-04-20T10:36:50.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.153+0000 7fad39595740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415a90 0x5614b6418590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad39595740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b641e670 0x5614b6418ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad39595740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b6419010 0x5614b6493950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad39595740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fad301ae9a0 con 0x5614b6419010 2024-04-20T10:36:50.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad39595740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad301afe10 con 0x5614b641e670 2024-04-20T10:36:50.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad39595740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fad301af070 con 0x5614b6415a90 2024-04-20T10:36:50.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad39595740 10 client.0 _ll_register_callbacks cb 0x7fad2800be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:50.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b6419010 0x5614b6493950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b6419010 0x5614b6493950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55280/0 (socket says 172.21.15.42:55280) 2024-04-20T10:36:50.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad2fe00640 1 -- 172.21.15.42:0/1325276441 learned_addr learned my addr 172.21.15.42:0/1325276441 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:50.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad2f400640 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415a90 0x5614b6418590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b641e670 0x5614b6418ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad2fe00640 1 -- 172.21.15.42:0/1325276441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415a90 msgr2=0x5614b6418590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad2fe00640 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415a90 0x5614b6418590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad2fe00640 1 -- 172.21.15.42:0/1325276441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b641e670 msgr2=0x5614b6418ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad2fe00640 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b641e670 0x5614b6418ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad2fe00640 1 -- 172.21.15.42:0/1325276441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b61567d0 con 0x5614b6419010 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.154+0000 7fad2ea00640 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b641e670 0x5614b6418ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.155+0000 7fad2fe00640 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b6419010 0x5614b6493950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fad2800bf10 tx=0x7fad28011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b196c8c67e56603b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.155+0000 7fad1d600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.155+0000 7fad1d600640 1 -- 172.21.15.42:0/1325276441 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28027040 con 0x5614b6419010 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.155+0000 7fad39595740 1 -- 172.21.15.42:0/1325276441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad18021270 con 0x5614b6419010 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.155+0000 7fad1d600640 1 -- 172.21.15.42:0/1325276441 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fad28019d30 con 0x5614b6419010 2024-04-20T10:36:50.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.155+0000 7fad1d600640 1 -- 172.21.15.42:0/1325276441 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fad2801f040 con 0x5614b6419010 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.155+0000 7fad39595740 10 client.15600 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.155+0000 7fad39595740 1 -- 172.21.15.42:0/1325276441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5614b6495050 con 0x5614b6419010 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.156+0000 7fad1d600640 1 -- 172.21.15.42:0/1325276441 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad18021270 con 0x5614b6419010 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.156+0000 7fad1d600640 1 -- 172.21.15.42:0/1325276441 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fad28022070 con 0x5614b6419010 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.156+0000 7fad39595740 10 client.15600 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.156+0000 7fad39595740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.156+0000 7fad39595740 20 client.15600 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.156+0000 7fad39595740 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5614b64956e0 0x5614b6497b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.156+0000 7fad39595740 4 client.15600 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.156+0000 7fad39595740 1 -- 172.21.15.42:0/1325276441 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614b63818e0 con 0x5614b64956e0 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.156+0000 7fad2f400640 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5614b64956e0 0x5614b6497b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad2f400640 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5614b64956e0 0x5614b6497b00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45a87af9d9a286ab server_cookie=397d25a0a86f01cf in_seq=0 out_seq=0 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad1d600640 10 client.15600 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:50.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad1d600640 1 -- 172.21.15.42:0/1325276441 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614b63818e0 con 0x5614b64956e0 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad1d600640 10 client.15600 handle_command_reply: tid=0 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad39595740 1 client.15600 shutdown 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad39595740 1 -- 172.21.15.42:0/1325276441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b6419010 msgr2=0x5614b6493950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad39595740 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b6419010 0x5614b6493950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fad2800bf10 tx=0x7fad28011dc0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad39595740 1 -- 172.21.15.42:0/1325276441 shutdown_connections 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad39595740 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5614b64956e0 0x5614b6497b00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad39595740 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5614b6419010 0x5614b6493950 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad39595740 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614b641e670 0x5614b6418ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad39595740 1 --2- 172.21.15.42:0/1325276441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5614b6415a90 0x5614b6418590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad1d600640 0 client.15600 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.157+0000 7fad39595740 1 -- 172.21.15.42:0/1325276441 >> 172.21.15.42:0/1325276441 conn(0x5614b640f8a0 msgr2=0x5614b641a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad39595740 1 -- 172.21.15.42:0/1325276441 shutdown_connections 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad39595740 1 -- 172.21.15.42:0/1325276441 wait complete. 2024-04-20T10:36:50.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad39595740 20 client.15600 trim_cache size 0 max 16384 2024-04-20T10:36:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:36:50.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad35200640 1 -- 172.21.15.42:0/36146876 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fad140a3d20 msgr2=0x7fad140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:50.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad35200640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fad140a3d20 0x7fad140a61e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fad18009620 tx=0x7fad18005f90 comp rx=0 tx=0).stop 2024-04-20T10:36:50.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad35200640 1 -- 172.21.15.42:0/36146876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad30109350 msgr2=0x7fad301a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:50.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad35200640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad30109350 0x7fad301a0b90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fad20037040 tx=0x7fad20035ae0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad35200640 1 -- 172.21.15.42:0/36146876 shutdown_connections 2024-04-20T10:36:50.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad35200640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fad140a3d20 0x7fad140a61e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad35200640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad3019a760 0x7fad3019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad35200640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad30109350 0x7fad301a0b90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad35200640 1 --2- 172.21.15.42:0/36146876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad30104280 0x7fad301a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:50.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.158+0000 7fad35200640 1 -- 172.21.15.42:0/36146876 >> 172.21.15.42:0/36146876 conn(0x7fad300ffaa0 msgr2=0x7fad30100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:50.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.159+0000 7fad35200640 1 -- 172.21.15.42:0/36146876 shutdown_connections 2024-04-20T10:36:50.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:50.159+0000 7fad35200640 1 -- 172.21.15.42:0/36146876 wait complete. 2024-04-20T10:36:51.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:50 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/494242441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:36:51.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:50 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/494242441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:36:51.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:50 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/494242441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:36:51.177 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:36:51.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:36:51.178 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:36:51.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.415+0000 7f7f26400640 1 -- 172.21.15.42:0/4101604075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f20104c30 msgr2=0x7f7f20108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.415+0000 7f7f26400640 1 --2- 172.21.15.42:0/4101604075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f20104c30 0x7f7f20108c70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009900 tx=0x7f7f0802bc40 comp rx=0 tx=0).stop 2024-04-20T10:36:51.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.416+0000 7f7f26400640 1 -- 172.21.15.42:0/4101604075 shutdown_connections 2024-04-20T10:36:51.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.416+0000 7f7f26400640 1 --2- 172.21.15.42:0/4101604075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f20109350 0x7f7f20110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.416+0000 7f7f26400640 1 --2- 172.21.15.42:0/4101604075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f20104c30 0x7f7f20108c70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.416+0000 7f7f26400640 1 --2- 172.21.15.42:0/4101604075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20104280 0x7f7f20104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.416+0000 7f7f26400640 1 -- 172.21.15.42:0/4101604075 >> 172.21.15.42:0/4101604075 conn(0x7f7f200ffaa0 msgr2=0x7f7f20101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:51.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.416+0000 7f7f26400640 1 -- 172.21.15.42:0/4101604075 shutdown_connections 2024-04-20T10:36:51.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.416+0000 7f7f26400640 1 -- 172.21.15.42:0/4101604075 wait complete. 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.417+0000 7f7f26400640 1 Processor -- start 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.417+0000 7f7f26400640 1 -- start start 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f26400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20104280 0x7f7f201a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f26400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f20109350 0x7f7f201a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f26400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f2019a7b0 0x7f7f2019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f26400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20114430 con 0x7f7f2019a7b0 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f26400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201142b0 con 0x7f7f20104280 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f26400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201145b0 con 0x7f7f20109350 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f25000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20104280 0x7f7f201a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f25000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20104280 0x7f7f201a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49988/0 (socket says 172.21.15.42:49988) 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f25000640 1 -- 172.21.15.42:0/2829522260 learned_addr learned my addr 172.21.15.42:0/2829522260 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f25a00640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f2019a7b0 0x7f7f2019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f17e00640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f20109350 0x7f7f201a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f25000640 1 -- 172.21.15.42:0/2829522260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f20109350 msgr2=0x7f7f201a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f25000640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f20109350 0x7f7f201a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f25000640 1 -- 172.21.15.42:0/2829522260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f2019a7b0 msgr2=0x7f7f2019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f25000640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f2019a7b0 0x7f7f2019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.418+0000 7f7f25000640 1 -- 172.21.15.42:0/2829522260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2019b3b0 con 0x7f7f20104280 2024-04-20T10:36:51.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.419+0000 7f7f25000640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20104280 0x7f7f201a06a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000e9f0 tx=0x7f7f1000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:51.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.419+0000 7f7f1e000640 1 -- 172.21.15.42:0/2829522260 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1000cd80 con 0x7f7f20104280 2024-04-20T10:36:51.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.419+0000 7f7f26400640 1 -- 172.21.15.42:0/2829522260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f201a7440 con 0x7f7f20104280 2024-04-20T10:36:51.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.419+0000 7f7f1e000640 1 -- 172.21.15.42:0/2829522260 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f10005500 con 0x7f7f20104280 2024-04-20T10:36:51.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.419+0000 7f7f1e000640 1 -- 172.21.15.42:0/2829522260 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10010690 con 0x7f7f20104280 2024-04-20T10:36:51.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.419+0000 7f7f26400640 1 -- 172.21.15.42:0/2829522260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f201a77f0 con 0x7f7f20104280 2024-04-20T10:36:51.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.423+0000 7f7f2a633740 1 Processor -- start 2024-04-20T10:36:51.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.423+0000 7f7f2a633740 1 -- start start 2024-04-20T10:36:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.424+0000 7f7f1e000640 1 -- 172.21.15.42:0/2829522260 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7f100046c0 con 0x7f7f20104280 2024-04-20T10:36:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.424+0000 7f7f2a633740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a51684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.424+0000 7f7f2a633740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a51710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.424+0000 7f7f2a633740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a51739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.424+0000 7f7f2a633740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5630a50ccd20 con 0x5630a51715f0 2024-04-20T10:36:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.424+0000 7f7f2a633740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630a50733e0 con 0x5630a5168a10 2024-04-20T10:36:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.424+0000 7f7f2a633740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5630a50c8ce0 con 0x5630a51680f0 2024-04-20T10:36:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.424+0000 7f7f25000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a51710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.424+0000 7f7f25000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a51710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49996/0 (socket says 172.21.15.42:49996) 2024-04-20T10:36:51.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.424+0000 7f7f25000640 1 -- 172.21.15.42:0/3830572318 learned_addr learned my addr 172.21.15.42:0/3830572318 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:51.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.424+0000 7f7f17e00640 1 --2- 172.21.15.42:0/3830572318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a51739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.424+0000 7f7f25a00640 1 --2- 172.21.15.42:0/3830572318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a51684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.425+0000 7f7f25000640 1 -- 172.21.15.42:0/3830572318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 msgr2=0x5630a51684d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.425+0000 7f7f25000640 1 --2- 172.21.15.42:0/3830572318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a51684d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.425+0000 7f7f25000640 1 -- 172.21.15.42:0/3830572318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 msgr2=0x5630a51739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.425+0000 7f7f25000640 1 --2- 172.21.15.42:0/3830572318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a51739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.425+0000 7f7f25000640 1 -- 172.21.15.42:0/3830572318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630a5092980 con 0x5630a5168a10 2024-04-20T10:36:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.425+0000 7f7f25000640 1 --2- 172.21.15.42:0/3830572318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a51710b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f100ce250 tx=0x7f7f100d4cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da4f800ec98621c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.425+0000 7f7f16000640 1 -- 172.21.15.42:0/3830572318 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f100e0070 con 0x5630a5168a10 2024-04-20T10:36:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.425+0000 7f7f16000640 1 -- 172.21.15.42:0/3830572318 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f100d9cc0 con 0x5630a5168a10 2024-04-20T10:36:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.425+0000 7f7f16000640 1 -- 172.21.15.42:0/3830572318 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f100db660 con 0x5630a5168a10 2024-04-20T10:36:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.426+0000 7f7f1e000640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7f040a3da0 0x7f7f040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.426+0000 7f7f2a633740 1 -- 172.21.15.42:0/3830572318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 msgr2=0x5630a51710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.426+0000 7f7f2a633740 1 --2- 172.21.15.42:0/3830572318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a51710b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f100ce250 tx=0x7f7f100d4cd0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.426+0000 7f7f2a633740 1 -- 172.21.15.42:0/3830572318 shutdown_connections 2024-04-20T10:36:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.426+0000 7f7f2a633740 1 --2- 172.21.15.42:0/3830572318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a51739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.426+0000 7f7f2a633740 1 --2- 172.21.15.42:0/3830572318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a51710b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.426+0000 7f7f2a633740 1 --2- 172.21.15.42:0/3830572318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a51684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.426+0000 7f7f2a633740 1 -- 172.21.15.42:0/3830572318 >> 172.21.15.42:0/3830572318 conn(0x5630a5162820 msgr2=0x5630a5162c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.426+0000 7f7f1e000640 1 -- 172.21.15.42:0/2829522260 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f7f100d0b90 con 0x7f7f20104280 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.426+0000 7f7f25a00640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7f040a3da0 0x7f7f040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.426+0000 7f7f2a633740 1 -- 172.21.15.42:0/3830572318 shutdown_connections 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.426+0000 7f7f2a633740 1 -- 172.21.15.42:0/3830572318 wait complete. 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.427+0000 7f7f2a633740 1 -- start start 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.427+0000 7f7f25a00640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7f040a3da0 0x7f7f040a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f18005fd0 tx=0x7f7f180096a0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.427+0000 7f7f2a633740 1 Processor -- start 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.428+0000 7f7f2a633740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a516e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.428+0000 7f7f2a633740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a516eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.428+0000 7f7f2a633740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a516f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.428+0000 7f7f2a633740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5630a51743e0 con 0x5630a51715f0 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.428+0000 7f7f2a633740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630a50867a0 con 0x5630a5168a10 2024-04-20T10:36:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.428+0000 7f7f2a633740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5630a5174560 con 0x5630a51680f0 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.428+0000 7f7f2a633740 10 client.0 _ll_register_callbacks cb 0x5630a5088b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.428+0000 7f7f17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a516eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.428+0000 7f7f17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a516eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:50002/0 (socket says 172.21.15.42:50002) 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.428+0000 7f7f25a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a516f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.428+0000 7f7f17e00640 1 -- 172.21.15.42:0/1049953503 learned_addr learned my addr 172.21.15.42:0/1049953503 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.428+0000 7f7f25000640 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a516e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.429+0000 7f7f17e00640 1 -- 172.21.15.42:0/1049953503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 msgr2=0x5630a516e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.429+0000 7f7f17e00640 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a516e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.429+0000 7f7f17e00640 1 -- 172.21.15.42:0/1049953503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 msgr2=0x5630a516f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.429+0000 7f7f17e00640 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a516f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.429+0000 7f7f17e00640 1 -- 172.21.15.42:0/1049953503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630a4ea77e0 con 0x5630a5168a10 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.429+0000 7f7f25a00640 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a516f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.429+0000 7f7f17e00640 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a516eb40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009900 tx=0x7f7f08004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e7d713881340b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.429+0000 7f7f0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.429+0000 7f7f0ea00640 1 -- 172.21.15.42:0/1049953503 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080046c0 con 0x5630a5168a10 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.429+0000 7f7f0ea00640 1 -- 172.21.15.42:0/1049953503 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f7f08004860 con 0x5630a5168a10 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.429+0000 7f7f2a633740 1 -- 172.21.15.42:0/1049953503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f100d9cc0 con 0x5630a5168a10 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.430+0000 7f7f0ea00640 1 -- 172.21.15.42:0/1049953503 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7f08004a60 con 0x5630a5168a10 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.430+0000 7f7f2a633740 10 client.25724 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.430+0000 7f7f2a633740 1 -- 172.21.15.42:0/1049953503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5630a4f07f70 con 0x5630a5168a10 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.430+0000 7f7f0ea00640 1 -- 172.21.15.42:0/1049953503 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f100d9cc0 con 0x5630a5168a10 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.430+0000 7f7f0ea00640 1 -- 172.21.15.42:0/1049953503 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f7f08046070 con 0x5630a5168a10 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.430+0000 7f7f2a633740 10 client.25724 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.430+0000 7f7f2a633740 10 client.25724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.430+0000 7f7f2a633740 20 client.25724 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.430+0000 7f7f2a633740 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5630a51818a0 0x5630a5183d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.430+0000 7f7f2a633740 4 client.25724 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:36:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.430+0000 7f7f2a633740 1 -- 172.21.15.42:0/1049953503 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630a50da740 con 0x5630a51818a0 2024-04-20T10:36:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.431+0000 7f7f25000640 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5630a51818a0 0x5630a5183d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.431+0000 7f7f25000640 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5630a51818a0 0x5630a5183d20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4479d52a234d68fa server_cookie=629c191661f72fc0 in_seq=0 out_seq=0 2024-04-20T10:36:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.431+0000 7f7f0ea00640 10 client.25724 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f0ea00640 1 -- 172.21.15.42:0/1049953503 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630a50da740 con 0x5630a51818a0 2024-04-20T10:36:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f0ea00640 10 client.25724 handle_command_reply: tid=0 2024-04-20T10:36:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f2a633740 1 client.25724 shutdown 2024-04-20T10:36:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f2a633740 1 -- 172.21.15.42:0/1049953503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 msgr2=0x5630a516eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f2a633740 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a516eb40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009900 tx=0x7f7f08004610 comp rx=0 tx=0).stop 2024-04-20T10:36:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f2a633740 1 -- 172.21.15.42:0/1049953503 shutdown_connections 2024-04-20T10:36:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f2a633740 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5630a51818a0 0x5630a5183d20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f0ea00640 0 client.25724 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:51.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f2a633740 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a516f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f2a633740 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a516eb40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f2a633740 1 --2- 172.21.15.42:0/1049953503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a516e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f2a633740 1 -- 172.21.15.42:0/1049953503 >> 172.21.15.42:0/1049953503 conn(0x5630a5162820 msgr2=0x5630a5164570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:51.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f2a633740 1 -- 172.21.15.42:0/1049953503 shutdown_connections 2024-04-20T10:36:51.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.433+0000 7f7f2a633740 1 -- 172.21.15.42:0/1049953503 wait complete. 2024-04-20T10:36:51.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.434+0000 7f7f2a633740 20 client.25724 trim_cache size 0 max 16384 2024-04-20T10:36:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.455+0000 7f7f2a633740 1 Processor -- start 2024-04-20T10:36:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.455+0000 7f7f2a633740 1 -- start start 2024-04-20T10:36:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.455+0000 7f7f2a633740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a5170d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.455+0000 7f7f2a633740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a517d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.455+0000 7f7f2a633740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a51895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.455+0000 7f7f2a633740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201add70 con 0x5630a51715f0 2024-04-20T10:36:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.455+0000 7f7f2a633740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201b05d0 con 0x5630a5168a10 2024-04-20T10:36:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.455+0000 7f7f2a633740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201ae3c0 con 0x5630a51680f0 2024-04-20T10:36:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f25000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a5170d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f25a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a51895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a517d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f25000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a5170d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37156/0 (socket says 172.21.15.42:37156) 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f25a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a51895f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55342/0 (socket says 172.21.15.42:55342) 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f25000640 1 -- 172.21.15.42:0/2509363565 learned_addr learned my addr 172.21.15.42:0/2509363565 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a517d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:50012/0 (socket says 172.21.15.42:50012) 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f17e00640 1 -- 172.21.15.42:0/2509363565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 msgr2=0x5630a5170d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f17e00640 1 --2- 172.21.15.42:0/2509363565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a5170d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f17e00640 1 -- 172.21.15.42:0/2509363565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 msgr2=0x5630a51895f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f17e00640 1 --2- 172.21.15.42:0/2509363565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a51895f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f17e00640 1 -- 172.21.15.42:0/2509363565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630a5168840 con 0x5630a5168a10 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f25000640 1 --2- 172.21.15.42:0/2509363565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a5170d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:51.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.456+0000 7f7f25a00640 1 --2- 172.21.15.42:0/2509363565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a51895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:36:51.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.457+0000 7f7f17e00640 1 --2- 172.21.15.42:0/2509363565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a517d2a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7f0802c250 tx=0x7f7f08003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9eb4264e0fe5037 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:51.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.457+0000 7f7f15600640 1 -- 172.21.15.42:0/2509363565 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08005d50 con 0x5630a5168a10 2024-04-20T10:36:51.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.457+0000 7f7f15600640 1 -- 172.21.15.42:0/2509363565 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f080057f0 con 0x5630a5168a10 2024-04-20T10:36:51.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.457+0000 7f7f15600640 1 -- 172.21.15.42:0/2509363565 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0802cb60 con 0x5630a5168a10 2024-04-20T10:36:51.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.458+0000 7f7f2a633740 1 -- 172.21.15.42:0/2509363565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 msgr2=0x5630a517d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.458+0000 7f7f2a633740 1 --2- 172.21.15.42:0/2509363565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a517d2a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7f0802c250 tx=0x7f7f08003c40 comp rx=0 tx=0).stop 2024-04-20T10:36:51.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.458+0000 7f7f2a633740 1 -- 172.21.15.42:0/2509363565 shutdown_connections 2024-04-20T10:36:51.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.458+0000 7f7f2a633740 1 --2- 172.21.15.42:0/2509363565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51715f0 0x5630a51895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.458+0000 7f7f2a633740 1 --2- 172.21.15.42:0/2509363565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a5168a10 0x5630a517d2a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.458+0000 7f7f2a633740 1 --2- 172.21.15.42:0/2509363565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a51680f0 0x5630a5170d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.458+0000 7f7f2a633740 1 -- 172.21.15.42:0/2509363565 >> 172.21.15.42:0/2509363565 conn(0x5630a5162820 msgr2=0x5630a5161a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:51.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.460+0000 7f7f2a633740 1 -- 172.21.15.42:0/2509363565 shutdown_connections 2024-04-20T10:36:51.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.460+0000 7f7f2a633740 1 -- 172.21.15.42:0/2509363565 wait complete. 2024-04-20T10:36:51.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.460+0000 7f7f2a633740 1 -- start start 2024-04-20T10:36:51.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.460+0000 7f7f2a633740 1 Processor -- start 2024-04-20T10:36:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.461+0000 7f7f2a633740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51680f0 0x5630a5179a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.461+0000 7f7f2a633740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a5168a10 0x5630a5175e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.461+0000 7f7f2a633740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a51715f0 0x5630a51763a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.461+0000 7f7f2a633740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201aea90 con 0x5630a51680f0 2024-04-20T10:36:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.461+0000 7f7f2a633740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201aff00 con 0x5630a51715f0 2024-04-20T10:36:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.461+0000 7f7f2a633740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201af160 con 0x5630a5168a10 2024-04-20T10:36:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.461+0000 7f7f2a633740 10 client.0 _ll_register_callbacks cb 0x7f7f08009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.461+0000 7f7f17e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a5168a10 0x5630a5175e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.461+0000 7f7f25a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a51715f0 0x5630a51763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.462+0000 7f7f25a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a51715f0 0x5630a51763a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:50014/0 (socket says 172.21.15.42:50014) 2024-04-20T10:36:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.462+0000 7f7f25000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51680f0 0x5630a5179a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.462+0000 7f7f25a00640 1 -- 172.21.15.42:0/3178810497 learned_addr learned my addr 172.21.15.42:0/3178810497 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.462+0000 7f7f17e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a5168a10 0x5630a5175e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37170/0 (socket says 172.21.15.42:37170) 2024-04-20T10:36:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.462+0000 7f7f25a00640 1 -- 172.21.15.42:0/3178810497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a5168a10 msgr2=0x5630a5175e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.462+0000 7f7f25a00640 1 --2- 172.21.15.42:0/3178810497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a5168a10 0x5630a5175e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.462+0000 7f7f25a00640 1 -- 172.21.15.42:0/3178810497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51680f0 msgr2=0x5630a5179a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.462+0000 7f7f25a00640 1 --2- 172.21.15.42:0/3178810497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51680f0 0x5630a5179a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.462+0000 7f7f25a00640 1 -- 172.21.15.42:0/3178810497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630a4d9f170 con 0x5630a51715f0 2024-04-20T10:36:51.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.462+0000 7f7f25a00640 1 --2- 172.21.15.42:0/3178810497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a51715f0 0x5630a51763a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7f18010a20 tx=0x7f7f18004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25920e18196f04c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:51.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.462+0000 7f7f0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:36:51.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.463+0000 7f7f0ea00640 1 -- 172.21.15.42:0/3178810497 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1802e060 con 0x5630a51715f0 2024-04-20T10:36:51.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.463+0000 7f7f0ea00640 1 -- 172.21.15.42:0/3178810497 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f7f18003b80 con 0x5630a51715f0 2024-04-20T10:36:51.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.463+0000 7f7f2a633740 1 -- 172.21.15.42:0/3178810497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f080057f0 con 0x5630a51715f0 2024-04-20T10:36:51.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.463+0000 7f7f0ea00640 1 -- 172.21.15.42:0/3178810497 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7f18003d80 con 0x5630a51715f0 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.463+0000 7f7f2a633740 10 client.25730 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.463+0000 7f7f2a633740 1 -- 172.21.15.42:0/3178810497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5630a51e8c70 con 0x5630a51715f0 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.463+0000 7f7f0ea00640 1 -- 172.21.15.42:0/3178810497 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f080057f0 con 0x5630a51715f0 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.463+0000 7f7f0ea00640 1 -- 172.21.15.42:0/3178810497 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f7f18028030 con 0x5630a51715f0 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.464+0000 7f7f2a633740 10 client.25730 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.464+0000 7f7f2a633740 10 client.25730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.464+0000 7f7f2a633740 20 client.25730 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.464+0000 7f7f2a633740 1 --2- 172.21.15.42:0/3178810497 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5630a51818a0 0x5630a51eb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.464+0000 7f7f2a633740 4 client.25730 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.464+0000 7f7f2a633740 1 -- 172.21.15.42:0/3178810497 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5630a50d45b0 con 0x5630a51818a0 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.464+0000 7f7f25000640 1 --2- 172.21.15.42:0/3178810497 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5630a51818a0 0x5630a51eb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.464+0000 7f7f25000640 1 --2- 172.21.15.42:0/3178810497 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5630a51818a0 0x5630a51eb460 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16f36c4cb2c90e74 server_cookie=2233ceeb65dcb74f in_seq=0 out_seq=0 2024-04-20T10:36:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.464+0000 7f7f0ea00640 10 client.25730 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:51.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.538+0000 7f7f0ea00640 1 -- 172.21.15.42:0/3178810497 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5630a50d45b0 con 0x5630a51818a0 2024-04-20T10:36:51.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.538+0000 7f7f0ea00640 10 client.25730 handle_command_reply: tid=0 2024-04-20T10:36:51.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.538+0000 7f7f2a633740 1 client.25730 shutdown 2024-04-20T10:36:51.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.538+0000 7f7f2a633740 1 -- 172.21.15.42:0/3178810497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a51715f0 msgr2=0x5630a51763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.538+0000 7f7f2a633740 1 --2- 172.21.15.42:0/3178810497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a51715f0 0x5630a51763a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7f18010a20 tx=0x7f7f18004ab0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.539+0000 7f7f2a633740 1 -- 172.21.15.42:0/3178810497 shutdown_connections 2024-04-20T10:36:51.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.539+0000 7f7f2a633740 1 --2- 172.21.15.42:0/3178810497 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5630a51818a0 0x5630a51eb460 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.539+0000 7f7f2a633740 1 --2- 172.21.15.42:0/3178810497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630a51715f0 0x5630a51763a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.539+0000 7f7f2a633740 1 --2- 172.21.15.42:0/3178810497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5630a5168a10 0x5630a5175e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.539+0000 7f7f2a633740 1 --2- 172.21.15.42:0/3178810497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5630a51680f0 0x5630a5179a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.539+0000 7f7f2a633740 1 -- 172.21.15.42:0/3178810497 >> 172.21.15.42:0/3178810497 conn(0x5630a5162820 msgr2=0x5630a516e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:51.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.539+0000 7f7f0ea00640 0 client.25730 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:51.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.539+0000 7f7f2a633740 1 -- 172.21.15.42:0/3178810497 shutdown_connections 2024-04-20T10:36:51.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.540+0000 7f7f2a633740 1 -- 172.21.15.42:0/3178810497 wait complete. 2024-04-20T10:36:51.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.540+0000 7f7f2a633740 20 client.25730 trim_cache size 0 max 16384 2024-04-20T10:36:51.540 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:36:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:36:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "f1675da7-9a74-4618-8841-7372371c5ed9", 2024-04-20T10:36:51.541 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:36:51.541 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:36:51.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.541+0000 7f7f26400640 1 -- 172.21.15.42:0/2829522260 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7f040a3da0 msgr2=0x7f7f040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.541+0000 7f7f26400640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7f040a3da0 0x7f7f040a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f18005fd0 tx=0x7f7f180096a0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.541+0000 7f7f26400640 1 -- 172.21.15.42:0/2829522260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20104280 msgr2=0x7f7f201a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.541+0000 7f7f26400640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20104280 0x7f7f201a06a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000e9f0 tx=0x7f7f1000edf0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.541+0000 7f7f26400640 1 -- 172.21.15.42:0/2829522260 shutdown_connections 2024-04-20T10:36:51.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.541+0000 7f7f26400640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7f040a3da0 0x7f7f040a6260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.541+0000 7f7f26400640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f2019a7b0 0x7f7f2019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.541+0000 7f7f26400640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f20109350 0x7f7f201a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.541+0000 7f7f26400640 1 --2- 172.21.15.42:0/2829522260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f20104280 0x7f7f201a06a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.541+0000 7f7f26400640 1 -- 172.21.15.42:0/2829522260 >> 172.21.15.42:0/2829522260 conn(0x7f7f200ffaa0 msgr2=0x7f7f20100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:51.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.541+0000 7f7f26400640 1 -- 172.21.15.42:0/2829522260 shutdown_connections 2024-04-20T10:36:51.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.541+0000 7f7f26400640 1 -- 172.21.15.42:0/2829522260 wait complete. 2024-04-20T10:36:51.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:36:51.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:36:51.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.787+0000 7f1fc7200640 1 -- 172.21.15.42:0/1451575602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc8102fc0 msgr2=0x7f1fc81033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.787+0000 7f1fc7200640 1 --2- 172.21.15.42:0/1451575602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc8102fc0 0x7f1fc81033a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009900 tx=0x7f1fb802bcd0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.788+0000 7f1fc7200640 1 -- 172.21.15.42:0/1451575602 shutdown_connections 2024-04-20T10:36:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.788+0000 7f1fc7200640 1 --2- 172.21.15.42:0/1451575602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fc810c3e0 0x7f1fc810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.788+0000 7f1fc7200640 1 --2- 172.21.15.42:0/1451575602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc81038e0 0x7f1fc810bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.788+0000 7f1fc7200640 1 --2- 172.21.15.42:0/1451575602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc8102fc0 0x7f1fc81033a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.788+0000 7f1fc7200640 1 -- 172.21.15.42:0/1451575602 >> 172.21.15.42:0/1451575602 conn(0x7f1fc80fc7e0 msgr2=0x7f1fc80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.788+0000 7f1fc7200640 1 -- 172.21.15.42:0/1451575602 shutdown_connections 2024-04-20T10:36:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.788+0000 7f1fc7200640 1 -- 172.21.15.42:0/1451575602 wait complete. 2024-04-20T10:36:51.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.789+0000 7f1fc7200640 1 Processor -- start 2024-04-20T10:36:51.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.789+0000 7f1fc7200640 1 -- start start 2024-04-20T10:36:51.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.789+0000 7f1fc7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fc8102fc0 0x7f1fc81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.789+0000 7f1fc7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc81038e0 0x7f1fc81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.789+0000 7f1fc7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc810c3e0 0x7f1fc819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.789+0000 7f1fc7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81121a0 con 0x7f1fc81038e0 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.789+0000 7f1fc7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8112020 con 0x7f1fc810c3e0 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.789+0000 7f1fc7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8112320 con 0x7f1fc8102fc0 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.789+0000 7f1fc5e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fc8102fc0 0x7f1fc81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.790+0000 7f1fc5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc81038e0 0x7f1fc81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.790+0000 7f1fc5e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fc8102fc0 0x7f1fc81a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37190/0 (socket says 172.21.15.42:37190) 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.790+0000 7f1fc5e00640 1 -- 172.21.15.42:0/2374213872 learned_addr learned my addr 172.21.15.42:0/2374213872 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.790+0000 7f1fc6800640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc810c3e0 0x7f1fc819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.790+0000 7f1fc5e00640 1 -- 172.21.15.42:0/2374213872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc810c3e0 msgr2=0x7f1fc819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.790+0000 7f1fc5e00640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc810c3e0 0x7f1fc819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.790+0000 7f1fc5e00640 1 -- 172.21.15.42:0/2374213872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc81038e0 msgr2=0x7f1fc81a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.790+0000 7f1fc5e00640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc81038e0 0x7f1fc81a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.790+0000 7f1fc5e00640 1 -- 172.21.15.42:0/2374213872 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc819afa0 con 0x7f1fc8102fc0 2024-04-20T10:36:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.790+0000 7f1fc5400640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc81038e0 0x7f1fc81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:36:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.790+0000 7f1fc5e00640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fc8102fc0 0x7f1fc81a0620 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8005940 tx=0x7f1fb802cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.791+0000 7f1fbea00640 1 -- 172.21.15.42:0/2374213872 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8041070 con 0x7f1fc8102fc0 2024-04-20T10:36:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.791+0000 7f1fc7200640 1 -- 172.21.15.42:0/2374213872 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc819b230 con 0x7f1fc8102fc0 2024-04-20T10:36:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.791+0000 7f1fbea00640 1 -- 172.21.15.42:0/2374213872 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb8004450 con 0x7f1fc8102fc0 2024-04-20T10:36:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.791+0000 7f1fbea00640 1 -- 172.21.15.42:0/2374213872 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8003d80 con 0x7f1fc8102fc0 2024-04-20T10:36:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.791+0000 7f1fc7200640 1 -- 172.21.15.42:0/2374213872 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc81a75a0 con 0x7f1fc8102fc0 2024-04-20T10:36:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.795+0000 7f1fcfbb1740 1 Processor -- start 2024-04-20T10:36:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.795+0000 7f1fcfbb1740 1 -- start start 2024-04-20T10:36:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.795+0000 7f1fbea00640 1 -- 172.21.15.42:0/2374213872 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1fb8044070 con 0x7f1fc8102fc0 2024-04-20T10:36:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.795+0000 7f1fcfbb1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 0x5610d8150760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.795+0000 7f1fcfbb1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150ca0 0x5610d8159340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.795+0000 7f1fcfbb1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8159880 0x5610d815bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.795+0000 7f1fcfbb1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5610d80b6150 con 0x5610d8150ca0 2024-04-20T10:36:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.795+0000 7f1fcfbb1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5610d805c370 con 0x5610d8159880 2024-04-20T10:36:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.795+0000 7f1fcfbb1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5610d80b2110 con 0x5610d8150380 2024-04-20T10:36:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.796+0000 7f1fc6800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 0x5610d8150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.796+0000 7f1fc5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8159880 0x5610d815bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.796+0000 7f1fc6800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 0x5610d8150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37192/0 (socket says 172.21.15.42:37192) 2024-04-20T10:36:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.796+0000 7f1fc6800640 1 -- 172.21.15.42:0/3938042977 learned_addr learned my addr 172.21.15.42:0/3938042977 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:51.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.796+0000 7f1fc5e00640 1 --2- 172.21.15.42:0/3938042977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150ca0 0x5610d8159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.796+0000 7f1fc6800640 1 -- 172.21.15.42:0/3938042977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8159880 msgr2=0x5610d815bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.796+0000 7f1fc6800640 1 --2- 172.21.15.42:0/3938042977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8159880 0x5610d815bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.796+0000 7f1fc6800640 1 -- 172.21.15.42:0/3938042977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150ca0 msgr2=0x5610d8159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.796+0000 7f1fc6800640 1 --2- 172.21.15.42:0/3938042977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150ca0 0x5610d8159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.796+0000 7f1fc6800640 1 -- 172.21.15.42:0/3938042977 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610d7e927d0 con 0x5610d8150380 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.796+0000 7f1fc5e00640 1 --2- 172.21.15.42:0/3938042977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150ca0 0x5610d8159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.797+0000 7f1fc6800640 1 --2- 172.21.15.42:0/3938042977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 0x5610d8150760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009f50 tx=0x7f1fc00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c347ad2b2058683a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.797+0000 7f1fbea00640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f1fac0a3da0 0x7f1fac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.797+0000 7f1fb3400640 1 -- 172.21.15.42:0/3938042977 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc001e070 con 0x5610d8150380 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.797+0000 7f1fb3400640 1 -- 172.21.15.42:0/3938042977 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc0004030 con 0x5610d8150380 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.797+0000 7f1fb3400640 1 -- 172.21.15.42:0/3938042977 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc000f720 con 0x5610d8150380 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.797+0000 7f1fbea00640 1 -- 172.21.15.42:0/2374213872 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f1fb80f21d0 con 0x7f1fc8102fc0 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.797+0000 7f1fc5e00640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f1fac0a3da0 0x7f1fac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.798+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/3938042977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 msgr2=0x5610d8150760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.798+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/3938042977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 0x5610d8150760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009f50 tx=0x7f1fc00118d0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.798+0000 7f1fc5e00640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f1fac0a3da0 0x7f1fac0a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80efae0 tx=0x7f1fb802bcd0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.798+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/3938042977 shutdown_connections 2024-04-20T10:36:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.798+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/3938042977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8159880 0x5610d815bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.798+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/3938042977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150ca0 0x5610d8159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.798+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/3938042977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 0x5610d8150760 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.798+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/3938042977 >> 172.21.15.42:0/3938042977 conn(0x5610d814aab0 msgr2=0x5610d814ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.798+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/3938042977 shutdown_connections 2024-04-20T10:36:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.798+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/3938042977 wait complete. 2024-04-20T10:36:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.799+0000 7f1fcfbb1740 1 -- start start 2024-04-20T10:36:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.799+0000 7f1fcfbb1740 1 Processor -- start 2024-04-20T10:36:51.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fcfbb1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150380 0x5610d8156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fcfbb1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 0x5610d8156e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fcfbb1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8159880 0x5610d81573c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fcfbb1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5610d815c670 con 0x5610d8150380 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fcfbb1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5610d806f740 con 0x5610d8150ca0 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fcfbb1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5610d815c7f0 con 0x5610d8159880 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fcfbb1740 10 client.0 _ll_register_callbacks cb 0x5610d806ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fc5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 0x5610d8156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fc5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 0x5610d8156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:50046/0 (socket says 172.21.15.42:50046) 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fc5400640 1 -- 172.21.15.42:0/1880118318 learned_addr learned my addr 172.21.15.42:0/1880118318 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fc6800640 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150380 0x5610d8156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fc6800640 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8159880 0x5610d81573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fc5400640 1 -- 172.21.15.42:0/1880118318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8159880 msgr2=0x5610d81573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fc5400640 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8159880 0x5610d81573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fc5400640 1 -- 172.21.15.42:0/1880118318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150380 msgr2=0x5610d8156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fc5400640 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150380 0x5610d8156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.800+0000 7f1fc5400640 1 -- 172.21.15.42:0/1880118318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610d7dc83f0 con 0x5610d8150ca0 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.801+0000 7f1fc6800640 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150380 0x5610d8156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.801+0000 7f1fc5400640 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 0x5610d8156e80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4012700 tx=0x7f1fb4012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf2a7be09fc1cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.801+0000 7f1fa7e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.801+0000 7f1fa7e00640 1 -- 172.21.15.42:0/1880118318 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4004320 con 0x5610d8150ca0 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.801+0000 7f1fa7e00640 1 -- 172.21.15.42:0/1880118318 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f1fb4011940 con 0x5610d8150ca0 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.801+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1880118318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fc0004030 con 0x5610d8150ca0 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.801+0000 7f1fa7e00640 1 -- 172.21.15.42:0/1880118318 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1fb40044c0 con 0x5610d8150ca0 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.802+0000 7f1fcfbb1740 10 client.25742 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.802+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1880118318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5610d7d880f0 con 0x5610d8150ca0 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.802+0000 7f1fa7e00640 1 -- 172.21.15.42:0/1880118318 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc0004030 con 0x5610d8150ca0 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.802+0000 7f1fa7e00640 1 -- 172.21.15.42:0/1880118318 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f1fb4025480 con 0x5610d8150ca0 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.802+0000 7f1fcfbb1740 10 client.25742 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.802+0000 7f1fcfbb1740 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.802+0000 7f1fcfbb1740 20 client.25742 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:51.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.802+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5610d8169c40 0x5610d816c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.802+0000 7f1fcfbb1740 4 client.25742 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:36:51.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.802+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1880118318 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610d80c3b70 con 0x5610d8169c40 2024-04-20T10:36:51.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.803+0000 7f1fc6800640 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5610d8169c40 0x5610d816c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.803+0000 7f1fc6800640 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5610d8169c40 0x5610d816c050 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b61b74c0b22e5d server_cookie=d17ce577f78a240f in_seq=0 out_seq=0 2024-04-20T10:36:51.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.803+0000 7f1fa7e00640 10 client.25742 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:51.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fa7e00640 1 -- 172.21.15.42:0/1880118318 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610d80c3b70 con 0x5610d8169c40 2024-04-20T10:36:51.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fa7e00640 10 client.25742 handle_command_reply: tid=0 2024-04-20T10:36:51.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fcfbb1740 1 client.25742 shutdown 2024-04-20T10:36:51.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1880118318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 msgr2=0x5610d8156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 0x5610d8156e80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4012700 tx=0x7f1fb4012bc0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1880118318 shutdown_connections 2024-04-20T10:36:51.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5610d8169c40 0x5610d816c050 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8159880 0x5610d81573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 0x5610d8156e80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fa7e00640 0 client.25742 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:51.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/1880118318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150380 0x5610d8156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1880118318 >> 172.21.15.42:0/1880118318 conn(0x5610d814aab0 msgr2=0x5610d814c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:51.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1880118318 shutdown_connections 2024-04-20T10:36:51.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1880118318 wait complete. 2024-04-20T10:36:51.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.807+0000 7f1fcfbb1740 20 client.25742 trim_cache size 0 max 16384 2024-04-20T10:36:51.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.830+0000 7f1fcfbb1740 1 Processor -- start 2024-04-20T10:36:51.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.830+0000 7f1fcfbb1740 1 -- start start 2024-04-20T10:36:51.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.831+0000 7f1fcfbb1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150380 0x5610d8159070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.831+0000 7f1fcfbb1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150ca0 0x5610d8165530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.831+0000 7f1fcfbb1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8159880 0x5610d8157010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.831+0000 7f1fcfbb1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81add70 con 0x5610d8150ca0 2024-04-20T10:36:51.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.831+0000 7f1fcfbb1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81b05f0 con 0x5610d8150380 2024-04-20T10:36:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.831+0000 7f1fcfbb1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81ae3e0 con 0x5610d8159880 2024-04-20T10:36:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.831+0000 7f1fc6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150380 0x5610d8159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.831+0000 7f1fc6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150380 0x5610d8159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:50050/0 (socket says 172.21.15.42:50050) 2024-04-20T10:36:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.831+0000 7f1fc6800640 1 -- 172.21.15.42:0/4087825337 learned_addr learned my addr 172.21.15.42:0/4087825337 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.831+0000 7f1fc6800640 1 --2- 172.21.15.42:0/4087825337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8159880 0x5610d8157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.831+0000 7f1fc5400640 1 --2- 172.21.15.42:0/4087825337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150ca0 0x5610d8165530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.832+0000 7f1fc6800640 1 -- 172.21.15.42:0/4087825337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8159880 msgr2=0x5610d8157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.832+0000 7f1fc6800640 1 --2- 172.21.15.42:0/4087825337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8159880 0x5610d8157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.832+0000 7f1fc6800640 1 -- 172.21.15.42:0/4087825337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150ca0 msgr2=0x5610d8165530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.832+0000 7f1fc6800640 1 --2- 172.21.15.42:0/4087825337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150ca0 0x5610d8165530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.832+0000 7f1fc6800640 1 -- 172.21.15.42:0/4087825337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610d815bfa0 con 0x5610d8150380 2024-04-20T10:36:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.832+0000 7f1fc6800640 1 --2- 172.21.15.42:0/4087825337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150380 0x5610d8159070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0014040 tx=0x7f1fc0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0e9fbd0bf82349c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:51.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.832+0000 7f1fb2a00640 1 -- 172.21.15.42:0/4087825337 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc001e040 con 0x5610d8150380 2024-04-20T10:36:51.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.832+0000 7f1fb2a00640 1 -- 172.21.15.42:0/4087825337 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc0004510 con 0x5610d8150380 2024-04-20T10:36:51.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.832+0000 7f1fb2a00640 1 -- 172.21.15.42:0/4087825337 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc002b6a0 con 0x5610d8150380 2024-04-20T10:36:51.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.833+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/4087825337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150380 msgr2=0x5610d8159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.833+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/4087825337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150380 0x5610d8159070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0014040 tx=0x7f1fc0005fd0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.833+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/4087825337 shutdown_connections 2024-04-20T10:36:51.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.833+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/4087825337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8159880 0x5610d8157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.833+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/4087825337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8150ca0 0x5610d8165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.833+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/4087825337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150380 0x5610d8159070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.833+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/4087825337 >> 172.21.15.42:0/4087825337 conn(0x5610d814aab0 msgr2=0x5610d8156900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:51.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.833+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/4087825337 shutdown_connections 2024-04-20T10:36:51.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.833+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/4087825337 wait complete. 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.834+0000 7f1fcfbb1740 1 -- start start 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.834+0000 7f1fcfbb1740 1 Processor -- start 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.834+0000 7f1fcfbb1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 0x5610d8161f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.834+0000 7f1fcfbb1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 0x5610d8153440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.834+0000 7f1fcfbb1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8159880 0x5610d8153b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.834+0000 7f1fcfbb1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81aeab0 con 0x5610d8159880 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.834+0000 7f1fcfbb1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81aff20 con 0x5610d8150ca0 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.834+0000 7f1fcfbb1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81af180 con 0x5610d8150380 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.835+0000 7f1fcfbb1740 10 client.0 _ll_register_callbacks cb 0x7f1fb4009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.835+0000 7f1fc5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 0x5610d8153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.835+0000 7f1fc5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 0x5610d8153440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:50062/0 (socket says 172.21.15.42:50062) 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.835+0000 7f1fc5400640 1 -- 172.21.15.42:0/1709531653 learned_addr learned my addr 172.21.15.42:0/1709531653 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.835+0000 7f1fc6800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 0x5610d8161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.835+0000 7f1fc6800640 1 --2- 172.21.15.42:0/1709531653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8159880 0x5610d8153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.835+0000 7f1fc6800640 1 -- 172.21.15.42:0/1709531653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 msgr2=0x5610d8153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.835+0000 7f1fc6800640 1 --2- 172.21.15.42:0/1709531653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 0x5610d8153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.835+0000 7f1fc6800640 1 -- 172.21.15.42:0/1709531653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8159880 msgr2=0x5610d8153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:36:51.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.835+0000 7f1fc6800640 1 --2- 172.21.15.42:0/1709531653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8159880 0x5610d8153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.835+0000 7f1fc6800640 1 -- 172.21.15.42:0/1709531653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610d81cf1d0 con 0x5610d8150380 2024-04-20T10:36:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.836+0000 7f1fc6800640 1 --2- 172.21.15.42:0/1709531653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 0x5610d8161f40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009d60 tx=0x7f1fc002b280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e919ddc99dfbf9fc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:36:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.836+0000 7f1fa7e00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:36:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.836+0000 7f1fa7e00640 1 -- 172.21.15.42:0/1709531653 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc001e070 con 0x5610d8150380 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.836+0000 7f1fa7e00640 1 -- 172.21.15.42:0/1709531653 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f1fc0004200 con 0x5610d8150380 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.836+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1709531653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fc0004510 con 0x5610d8150380 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.836+0000 7f1fa7e00640 1 -- 172.21.15.42:0/1709531653 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1fc0004ac0 con 0x5610d8150380 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.836+0000 7f1fcfbb1740 10 client.25768 fetch_fsmap learned FSMap version 19 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.836+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1709531653 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5610d81d02f0 con 0x5610d8150380 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.836+0000 7f1fa7e00640 1 -- 172.21.15.42:0/1709531653 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc0004510 con 0x5610d8150380 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.837+0000 7f1fa7e00640 1 -- 172.21.15.42:0/1709531653 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f1fc000fa80 con 0x5610d8150380 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.837+0000 7f1fcfbb1740 10 client.25768 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.837+0000 7f1fcfbb1740 10 client.25768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.837+0000 7f1fcfbb1740 20 client.25768 populate_metadata read hostname 'smithi042' 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.837+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/1709531653 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5610d81d06c0 0x5610d81d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.837+0000 7f1fcfbb1740 4 client.25768 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:36:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.837+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1709531653 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610d8146170 con 0x5610d81d06c0 2024-04-20T10:36:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.837+0000 7f1fc5400640 1 --2- 172.21.15.42:0/1709531653 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5610d81d06c0 0x5610d81d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:36:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.838+0000 7f1fc5400640 1 --2- 172.21.15.42:0/1709531653 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5610d81d06c0 0x5610d81d2ab0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d2e6b89308755c server_cookie=7b3ca86ba9ba7ee in_seq=0 out_seq=0 2024-04-20T10:36:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.838+0000 7f1fa7e00640 10 client.25768 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fa7e00640 1 -- 172.21.15.42:0/1709531653 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5610d8146170 con 0x5610d81d06c0 2024-04-20T10:36:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fa7e00640 10 client.25768 handle_command_reply: tid=0 2024-04-20T10:36:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fcfbb1740 1 client.25768 shutdown 2024-04-20T10:36:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1709531653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 msgr2=0x5610d8161f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/1709531653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 0x5610d8161f40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009d60 tx=0x7f1fc002b280 comp rx=0 tx=0).stop 2024-04-20T10:36:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1709531653 shutdown_connections 2024-04-20T10:36:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/1709531653 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5610d81d06c0 0x5610d81d2ab0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/1709531653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5610d8159880 0x5610d8153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/1709531653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610d8150ca0 0x5610d8153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fa7e00640 0 client.25768 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:36:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fcfbb1740 1 --2- 172.21.15.42:0/1709531653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5610d8150380 0x5610d8161f40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1709531653 >> 172.21.15.42:0/1709531653 conn(0x5610d814aab0 msgr2=0x5610d8149b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1709531653 shutdown_connections 2024-04-20T10:36:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fcfbb1740 1 -- 172.21.15.42:0/1709531653 wait complete. 2024-04-20T10:36:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.839+0000 7f1fcfbb1740 20 client.25768 trim_cache size 0 max 16384 2024-04-20T10:36:51.842 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:36:51.842 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (3272 inodes in the stack)", 2024-04-20T10:36:51.842 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:36:51.843 INFO:teuthology.orchestra.run.smithi042.stdout: "f1675da7-9a74-4618-8841-7372371c5ed9": { 2024-04-20T10:36:51.843 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:36:51.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "f1675da7-9a74-4618-8841-7372371c5ed9", 2024-04-20T10:36:51.843 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:36:51.843 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:36:51.843 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:36:51.843 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:36:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.840+0000 7f1fc7200640 1 -- 172.21.15.42:0/2374213872 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f1fac0a3da0 msgr2=0x7f1fac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.840+0000 7f1fc7200640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f1fac0a3da0 0x7f1fac0a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80efae0 tx=0x7f1fb802bcd0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.840+0000 7f1fc7200640 1 -- 172.21.15.42:0/2374213872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fc8102fc0 msgr2=0x7f1fc81a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:36:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.840+0000 7f1fc7200640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fc8102fc0 0x7f1fc81a0620 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8005940 tx=0x7f1fb802cbd0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.840+0000 7f1fc7200640 1 -- 172.21.15.42:0/2374213872 shutdown_connections 2024-04-20T10:36:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.840+0000 7f1fc7200640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f1fac0a3da0 0x7f1fac0a6260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.840+0000 7f1fc7200640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc810c3e0 0x7f1fc819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.840+0000 7f1fc7200640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fc81038e0 0x7f1fc81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.840+0000 7f1fc7200640 1 --2- 172.21.15.42:0/2374213872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fc8102fc0 0x7f1fc81a0620 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:36:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.840+0000 7f1fc7200640 1 -- 172.21.15.42:0/2374213872 >> 172.21.15.42:0/2374213872 conn(0x7f1fc80fc7e0 msgr2=0x7f1fc8100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:36:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.840+0000 7f1fc7200640 1 -- 172.21.15.42:0/2374213872 shutdown_connections 2024-04-20T10:36:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:36:51.840+0000 7f1fc7200640 1 -- 172.21.15.42:0/2374213872 wait complete. 2024-04-20T10:36:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:51 smithi042 ceph-mon[17078]: pgmap v88: 97 pgs: 97 active+clean; 250 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 39 op/s 2024-04-20T10:36:51.862 INFO:tasks.cephfs.filesystem:scrub status for tag:f1675da7-9a74-4618-8841-7372371c5ed9 - {'path': '/', 'tag': 'f1675da7-9a74-4618-8841-7372371c5ed9', 'options': 'recursive,force'} 2024-04-20T10:36:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:51 smithi046 ceph-mon[21574]: pgmap v88: 97 pgs: 97 active+clean; 250 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 39 op/s 2024-04-20T10:36:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:51 smithi162 ceph-mon[21854]: pgmap v88: 97 pgs: 97 active+clean; 250 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 39 op/s 2024-04-20T10:36:53.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:52 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:36:53.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:52 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:36:53.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:52 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:36:53.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:52 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:36:53.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:52 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:36:53.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:52 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:36:53.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:52 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:36:53.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:52 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:36:53.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:52 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:36:54.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:53 smithi042 ceph-mon[17078]: pgmap v89: 97 pgs: 97 active+clean; 252 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 47 op/s 2024-04-20T10:36:54.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:53 smithi162 ceph-mon[21854]: pgmap v89: 97 pgs: 97 active+clean; 252 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 47 op/s 2024-04-20T10:36:54.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:53 smithi046 ceph-mon[21574]: pgmap v89: 97 pgs: 97 active+clean; 252 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 47 op/s 2024-04-20T10:36:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:55 smithi042 ceph-mon[17078]: pgmap v90: 97 pgs: 97 active+clean; 252 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 702 KiB/s wr, 26 op/s 2024-04-20T10:36:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:55 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:36:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:55 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:36:56.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:55 smithi162 ceph-mon[21854]: pgmap v90: 97 pgs: 97 active+clean; 252 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 702 KiB/s wr, 26 op/s 2024-04-20T10:36:56.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:55 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:36:56.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:55 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:36:56.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:55 smithi046 ceph-mon[21574]: pgmap v90: 97 pgs: 97 active+clean; 252 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 702 KiB/s wr, 26 op/s 2024-04-20T10:36:56.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:55 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:36:56.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:55 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:36:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:57 smithi042 ceph-mon[17078]: pgmap v91: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 811 KiB/s wr, 32 op/s 2024-04-20T10:36:58.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:57 smithi162 ceph-mon[21854]: pgmap v91: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 811 KiB/s wr, 32 op/s 2024-04-20T10:36:58.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:57 smithi046 ceph-mon[21574]: pgmap v91: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 811 KiB/s wr, 32 op/s 2024-04-20T10:37:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:36:59 smithi042 ceph-mon[17078]: pgmap v92: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 611 KiB/s wr, 38 op/s 2024-04-20T10:37:00.105 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:36:59 smithi046 ceph-mon[21574]: pgmap v92: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 611 KiB/s wr, 38 op/s 2024-04-20T10:37:00.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:36:59 smithi162 ceph-mon[21854]: pgmap v92: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 611 KiB/s wr, 38 op/s 2024-04-20T10:37:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:01 smithi042 ceph-mon[17078]: pgmap v93: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 358 KiB/s wr, 25 op/s 2024-04-20T10:37:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:01 smithi162 ceph-mon[21854]: pgmap v93: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 358 KiB/s wr, 25 op/s 2024-04-20T10:37:02.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:01 smithi046 ceph-mon[21574]: pgmap v93: 97 pgs: 97 active+clean; 253 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 358 KiB/s wr, 25 op/s 2024-04-20T10:37:04.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:03 smithi042 ceph-mon[17078]: pgmap v94: 97 pgs: 97 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-04-20T10:37:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:03 smithi162 ceph-mon[21854]: pgmap v94: 97 pgs: 97 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-04-20T10:37:04.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:03 smithi046 ceph-mon[21574]: pgmap v94: 97 pgs: 97 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.7 MiB/s wr, 133 op/s 2024-04-20T10:37:06.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:05 smithi042 ceph-mon[17078]: pgmap v95: 97 pgs: 97 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-04-20T10:37:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:05 smithi162 ceph-mon[21854]: pgmap v95: 97 pgs: 97 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-04-20T10:37:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:05 smithi046 ceph-mon[21574]: pgmap v95: 97 pgs: 97 active+clean; 268 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-04-20T10:37:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:07 smithi042 ceph-mon[17078]: pgmap v96: 97 pgs: 97 active+clean; 279 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-04-20T10:37:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:07 smithi162 ceph-mon[21854]: pgmap v96: 97 pgs: 97 active+clean; 279 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-04-20T10:37:08.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:07 smithi046 ceph-mon[21574]: pgmap v96: 97 pgs: 97 active+clean; 279 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-04-20T10:37:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:09 smithi042 ceph-mon[17078]: pgmap v97: 97 pgs: 97 active+clean; 317 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 501 op/s 2024-04-20T10:37:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:09 smithi162 ceph-mon[21854]: pgmap v97: 97 pgs: 97 active+clean; 317 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 501 op/s 2024-04-20T10:37:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:09 smithi046 ceph-mon[21574]: pgmap v97: 97 pgs: 97 active+clean; 317 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 501 op/s 2024-04-20T10:37:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:11 smithi162 ceph-mon[21854]: pgmap v98: 97 pgs: 97 active+clean; 317 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 490 op/s 2024-04-20T10:37:12.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:11 smithi046 ceph-mon[21574]: pgmap v98: 97 pgs: 97 active+clean; 317 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 490 op/s 2024-04-20T10:37:12.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:11 smithi042 ceph-mon[17078]: pgmap v98: 97 pgs: 97 active+clean; 317 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 490 op/s 2024-04-20T10:37:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:13 smithi162 ceph-mon[21854]: pgmap v99: 97 pgs: 97 active+clean; 327 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.6 MiB/s wr, 520 op/s 2024-04-20T10:37:14.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:13 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:37:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:13 smithi046 ceph-mon[21574]: pgmap v99: 97 pgs: 97 active+clean; 327 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.6 MiB/s wr, 520 op/s 2024-04-20T10:37:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:13 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:37:14.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:13 smithi042 ceph-mon[17078]: pgmap v99: 97 pgs: 97 active+clean; 327 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.6 MiB/s wr, 520 op/s 2024-04-20T10:37:14.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:13 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:37:15.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:14 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:37:15.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:14 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:37:15.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:14 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:37:15.306 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:14 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:37:15.306 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:14 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:37:15.306 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:14 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:37:15.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:14 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:37:15.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:14 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:37:15.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:14 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:37:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:15 smithi162 ceph-mon[21854]: pgmap v100: 97 pgs: 97 active+clean; 327 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 412 op/s 2024-04-20T10:37:16.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:15 smithi046 ceph-mon[21574]: pgmap v100: 97 pgs: 97 active+clean; 327 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 412 op/s 2024-04-20T10:37:16.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:15 smithi042 ceph-mon[17078]: pgmap v100: 97 pgs: 97 active+clean; 327 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 412 op/s 2024-04-20T10:37:18.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:17 smithi162 ceph-mon[21854]: pgmap v101: 97 pgs: 97 active+clean; 327 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 417 op/s 2024-04-20T10:37:18.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:17 smithi046 ceph-mon[21574]: pgmap v101: 97 pgs: 97 active+clean; 327 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 417 op/s 2024-04-20T10:37:18.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:17 smithi042 ceph-mon[17078]: pgmap v101: 97 pgs: 97 active+clean; 327 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 417 op/s 2024-04-20T10:37:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:19 smithi162 ceph-mon[21854]: pgmap v102: 97 pgs: 97 active+clean; 332 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 333 op/s 2024-04-20T10:37:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:19 smithi046 ceph-mon[21574]: pgmap v102: 97 pgs: 97 active+clean; 332 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 333 op/s 2024-04-20T10:37:20.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:19 smithi042 ceph-mon[17078]: pgmap v102: 97 pgs: 97 active+clean; 332 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 333 op/s 2024-04-20T10:37:21.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:20 smithi162 ceph-mon[21854]: pgmap v103: 97 pgs: 97 active+clean; 332 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 50 op/s 2024-04-20T10:37:21.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:20 smithi046 ceph-mon[21574]: pgmap v103: 97 pgs: 97 active+clean; 332 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 50 op/s 2024-04-20T10:37:21.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:20 smithi042 ceph-mon[17078]: pgmap v103: 97 pgs: 97 active+clean; 332 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 50 op/s 2024-04-20T10:37:21.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:37:21.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:37:22.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.098+0000 7f4c6fc00640 1 -- 172.21.15.42:0/807383417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c681021f0 msgr2=0x7f4c681025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.098+0000 7f4c6fc00640 1 --2- 172.21.15.42:0/807383417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c681021f0 0x7f4c681025d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009900 tx=0x7f4c5802bc40 comp rx=0 tx=0).stop 2024-04-20T10:37:22.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.099+0000 7f4c6fc00640 1 -- 172.21.15.42:0/807383417 shutdown_connections 2024-04-20T10:37:22.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.099+0000 7f4c6fc00640 1 --2- 172.21.15.42:0/807383417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c681072c0 0x7f4c6810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.099+0000 7f4c6fc00640 1 --2- 172.21.15.42:0/807383417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c68102ba0 0x7f4c68106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.099+0000 7f4c6fc00640 1 --2- 172.21.15.42:0/807383417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c681021f0 0x7f4c681025d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.099+0000 7f4c6fc00640 1 -- 172.21.15.42:0/807383417 >> 172.21.15.42:0/807383417 conn(0x7f4c680fda70 msgr2=0x7f4c680ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.099+0000 7f4c6fc00640 1 -- 172.21.15.42:0/807383417 shutdown_connections 2024-04-20T10:37:22.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.099+0000 7f4c6fc00640 1 -- 172.21.15.42:0/807383417 wait complete. 2024-04-20T10:37:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.100+0000 7f4c6fc00640 1 Processor -- start 2024-04-20T10:37:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.100+0000 7f4c6fc00640 1 -- start start 2024-04-20T10:37:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.100+0000 7f4c6fc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c681021f0 0x7f4c68074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.100+0000 7f4c6fc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c681072c0 0x7f4c680747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.100+0000 7f4c6fc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c68074ce0 0x7f4c681a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.100+0000 7f4c6fc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681123d0 con 0x7f4c68074ce0 2024-04-20T10:37:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c6fc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68112250 con 0x7f4c681021f0 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c6fc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68112550 con 0x7f4c681072c0 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c6d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c681021f0 0x7f4c68074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c67e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c681072c0 0x7f4c680747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c6d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c681021f0 0x7f4c68074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49548/0 (socket says 172.21.15.42:49548) 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c6d200640 1 -- 172.21.15.42:0/2748381890 learned_addr learned my addr 172.21.15.42:0/2748381890 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c6dc00640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c68074ce0 0x7f4c681a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c6d200640 1 -- 172.21.15.42:0/2748381890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c681072c0 msgr2=0x7f4c680747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c6d200640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c681072c0 0x7f4c680747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c6d200640 1 -- 172.21.15.42:0/2748381890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c68074ce0 msgr2=0x7f4c681a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c6d200640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c68074ce0 0x7f4c681a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c6d200640 1 -- 172.21.15.42:0/2748381890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c681a71c0 con 0x7f4c681021f0 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.101+0000 7f4c6dc00640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c68074ce0 0x7f4c681a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:37:22.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.102+0000 7f4c6d200640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c681021f0 0x7f4c68074260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c58005ab0 tx=0x7f4c58004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.102+0000 7f4c65600640 1 -- 172.21.15.42:0/2748381890 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58041070 con 0x7f4c681021f0 2024-04-20T10:37:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.102+0000 7f4c65600640 1 -- 172.21.15.42:0/2748381890 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5802cca0 con 0x7f4c681021f0 2024-04-20T10:37:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.102+0000 7f4c65600640 1 -- 172.21.15.42:0/2748381890 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58003e50 con 0x7f4c681021f0 2024-04-20T10:37:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.102+0000 7f4c6fc00640 1 -- 172.21.15.42:0/2748381890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c681a7450 con 0x7f4c681021f0 2024-04-20T10:37:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.102+0000 7f4c6fc00640 1 -- 172.21.15.42:0/2748381890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c681a78f0 con 0x7f4c681021f0 2024-04-20T10:37:22.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.106+0000 7f4c65600640 1 -- 172.21.15.42:0/2748381890 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4c58044070 con 0x7f4c681021f0 2024-04-20T10:37:22.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.107+0000 7f4c72863740 1 Processor -- start 2024-04-20T10:37:22.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.107+0000 7f4c72863740 1 -- start start 2024-04-20T10:37:22.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.107+0000 7f4c72863740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67f41a0 0x5556b67f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.107+0000 7f4c72863740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b67fd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.107+0000 7f4c72863740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 0x5556b67ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.107+0000 7f4c72863740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5556b675a150 con 0x5556b67f4ac0 2024-04-20T10:37:22.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.107+0000 7f4c72863740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556b6700370 con 0x5556b67fd6a0 2024-04-20T10:37:22.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.107+0000 7f4c72863740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5556b6756110 con 0x5556b67f41a0 2024-04-20T10:37:22.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 0x5556b67ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c6dc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67f41a0 0x5556b67f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c6d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b67fd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c65600640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c4c0a3f40 0x7f4c4c0a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 0x5556b67ffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49554/0 (socket says 172.21.15.42:49554) 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c67e00640 1 -- 172.21.15.42:0/1123343657 learned_addr learned my addr 172.21.15.42:0/1123343657 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c65600640 1 -- 172.21.15.42:0/2748381890 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4c580f2160 con 0x7f4c681021f0 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c67e00640 1 -- 172.21.15.42:0/1123343657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67f41a0 msgr2=0x5556b67f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c67e00640 1 --2- 172.21.15.42:0/1123343657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67f41a0 0x5556b67f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c67e00640 1 -- 172.21.15.42:0/1123343657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 msgr2=0x5556b67fd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c67e00640 1 --2- 172.21.15.42:0/1123343657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b67fd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c67e00640 1 -- 172.21.15.42:0/1123343657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556b65367d0 con 0x5556b67fd6a0 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c6d200640 1 --2- 172.21.15.42:0/1123343657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b67fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c6dc00640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c4c0a3f40 0x7f4c4c0a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.108+0000 7f4c67e00640 1 --2- 172.21.15.42:0/1123343657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 0x5556b67ffa90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4c500039f0 tx=0x7f4c50009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82622492e917db21 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.109+0000 7f4c5e000640 1 -- 172.21.15.42:0/1123343657 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50004b20 con 0x5556b67fd6a0 2024-04-20T10:37:22.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.109+0000 7f4c5e000640 1 -- 172.21.15.42:0/1123343657 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c50004cc0 con 0x5556b67fd6a0 2024-04-20T10:37:22.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.109+0000 7f4c5e000640 1 -- 172.21.15.42:0/1123343657 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c500127b0 con 0x5556b67fd6a0 2024-04-20T10:37:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.110+0000 7f4c72863740 1 -- 172.21.15.42:0/1123343657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 msgr2=0x5556b67ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.110+0000 7f4c72863740 1 --2- 172.21.15.42:0/1123343657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 0x5556b67ffa90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4c500039f0 tx=0x7f4c50009f90 comp rx=0 tx=0).stop 2024-04-20T10:37:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.110+0000 7f4c6dc00640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c4c0a3f40 0x7f4c4c0a6400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009c20 tx=0x7f4c600046c0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.110+0000 7f4c72863740 1 -- 172.21.15.42:0/1123343657 shutdown_connections 2024-04-20T10:37:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.110+0000 7f4c72863740 1 --2- 172.21.15.42:0/1123343657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 0x5556b67ffa90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.110+0000 7f4c72863740 1 --2- 172.21.15.42:0/1123343657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b67fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.110+0000 7f4c72863740 1 --2- 172.21.15.42:0/1123343657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67f41a0 0x5556b67f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.110+0000 7f4c72863740 1 -- 172.21.15.42:0/1123343657 >> 172.21.15.42:0/1123343657 conn(0x5556b67ee8d0 msgr2=0x5556b67eecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.110+0000 7f4c72863740 1 -- 172.21.15.42:0/1123343657 shutdown_connections 2024-04-20T10:37:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.110+0000 7f4c72863740 1 -- 172.21.15.42:0/1123343657 wait complete. 2024-04-20T10:37:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.110+0000 7f4c72863740 1 -- start start 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.111+0000 7f4c72863740 1 Processor -- start 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.111+0000 7f4c72863740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f41a0 0x5556b67fa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.111+0000 7f4c72863740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b67fac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.111+0000 7f4c72863740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67fd6a0 0x5556b67fb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.111+0000 7f4c72863740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5556b6800490 con 0x5556b67f4ac0 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.111+0000 7f4c72863740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5556b6713740 con 0x5556b67f41a0 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.111+0000 7f4c72863740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5556b6800610 con 0x5556b67fd6a0 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.111+0000 7f4c72863740 10 client.0 _ll_register_callbacks cb 0x5556b670eb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c6d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f41a0 0x5556b67fa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c6d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f41a0 0x5556b67fa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49564/0 (socket says 172.21.15.42:49564) 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c6d200640 1 -- 172.21.15.42:0/3423077427 learned_addr learned my addr 172.21.15.42:0/3423077427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c67e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b67fac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c6dc00640 1 --2- 172.21.15.42:0/3423077427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67fd6a0 0x5556b67fb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c6d200640 1 -- 172.21.15.42:0/3423077427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67fd6a0 msgr2=0x5556b67fb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c6d200640 1 --2- 172.21.15.42:0/3423077427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67fd6a0 0x5556b67fb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c6d200640 1 -- 172.21.15.42:0/3423077427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 msgr2=0x5556b67fac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c6d200640 1 --2- 172.21.15.42:0/3423077427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b67fac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c6d200640 1 -- 172.21.15.42:0/3423077427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556b646c3f0 con 0x5556b67f41a0 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c6d200640 1 --2- 172.21.15.42:0/3423077427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f41a0 0x5556b67fa750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4c580bdeb0 tx=0x7f4c5802bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de5fd73fd2dd6d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c56a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c56a00640 1 -- 172.21.15.42:0/3423077427 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5804b040 con 0x5556b67f41a0 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.112+0000 7f4c56a00640 1 -- 172.21.15.42:0/3423077427 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4c580c6990 con 0x5556b67f41a0 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.113+0000 7f4c72863740 1 -- 172.21.15.42:0/3423077427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c50004cc0 con 0x5556b67f41a0 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.113+0000 7f4c56a00640 1 -- 172.21.15.42:0/3423077427 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4c580efcd0 con 0x5556b67f41a0 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.113+0000 7f4c72863740 10 client.25760 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.113+0000 7f4c72863740 1 -- 172.21.15.42:0/3423077427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5556b642c0f0 con 0x5556b67f41a0 2024-04-20T10:37:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.113+0000 7f4c56a00640 1 -- 172.21.15.42:0/3423077427 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c50004cc0 con 0x5556b67f41a0 2024-04-20T10:37:22.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.113+0000 7f4c56a00640 1 -- 172.21.15.42:0/3423077427 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f4c580c2140 con 0x5556b67f41a0 2024-04-20T10:37:22.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.113+0000 7f4c72863740 10 client.25760 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:22.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.113+0000 7f4c72863740 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:22.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.113+0000 7f4c72863740 20 client.25760 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:22.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.113+0000 7f4c72863740 1 --2- 172.21.15.42:0/3423077427 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5556b680d940 0x5556b680fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.113+0000 7f4c72863740 4 client.25760 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:37:22.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.113+0000 7f4c72863740 1 -- 172.21.15.42:0/3423077427 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556b6767b70 con 0x5556b680d940 2024-04-20T10:37:22.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.114+0000 7f4c67e00640 1 --2- 172.21.15.42:0/3423077427 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5556b680d940 0x5556b680fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.114+0000 7f4c67e00640 1 --2- 172.21.15.42:0/3423077427 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5556b680d940 0x5556b680fdb0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a40da93de99019be server_cookie=990986ff9cd86e86 in_seq=0 out_seq=0 2024-04-20T10:37:22.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.114+0000 7f4c56a00640 10 client.25760 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:22.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c56a00640 1 -- 172.21.15.42:0/3423077427 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556b6767b70 con 0x5556b680d940 2024-04-20T10:37:22.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c56a00640 10 client.25760 handle_command_reply: tid=0 2024-04-20T10:37:22.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c72863740 1 client.25760 shutdown 2024-04-20T10:37:22.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c72863740 1 -- 172.21.15.42:0/3423077427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f41a0 msgr2=0x5556b67fa750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c72863740 1 --2- 172.21.15.42:0/3423077427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f41a0 0x5556b67fa750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4c580bdeb0 tx=0x7f4c5802bc40 comp rx=0 tx=0).stop 2024-04-20T10:37:22.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c72863740 1 -- 172.21.15.42:0/3423077427 shutdown_connections 2024-04-20T10:37:22.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c72863740 1 --2- 172.21.15.42:0/3423077427 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5556b680d940 0x5556b680fdb0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c72863740 1 --2- 172.21.15.42:0/3423077427 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67fd6a0 0x5556b67fb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c72863740 1 --2- 172.21.15.42:0/3423077427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b67fac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c72863740 1 --2- 172.21.15.42:0/3423077427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f41a0 0x5556b67fa750 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c56a00640 0 client.25760 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:22.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c72863740 1 -- 172.21.15.42:0/3423077427 >> 172.21.15.42:0/3423077427 conn(0x5556b67ee8d0 msgr2=0x5556b67f0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.115+0000 7f4c72863740 1 -- 172.21.15.42:0/3423077427 shutdown_connections 2024-04-20T10:37:22.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.116+0000 7f4c72863740 1 -- 172.21.15.42:0/3423077427 wait complete. 2024-04-20T10:37:22.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.116+0000 7f4c72863740 20 client.25760 trim_cache size 0 max 16384 2024-04-20T10:37:22.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.136+0000 7f4c72863740 1 Processor -- start 2024-04-20T10:37:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.136+0000 7f4c72863740 1 -- start start 2024-04-20T10:37:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.136+0000 7f4c72863740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f41a0 0x5556b67fce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.136+0000 7f4c72863740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f4ac0 0x5556b6809350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.136+0000 7f4c72863740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67fd6a0 0x5556b67ed740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.136+0000 7f4c72863740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681adec0 con 0x5556b67f41a0 2024-04-20T10:37:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.136+0000 7f4c72863740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681b0740 con 0x5556b67f4ac0 2024-04-20T10:37:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.136+0000 7f4c72863740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681ae530 con 0x5556b67fd6a0 2024-04-20T10:37:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.136+0000 7f4c67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f4ac0 0x5556b6809350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.136+0000 7f4c67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f4ac0 0x5556b6809350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49578/0 (socket says 172.21.15.42:49578) 2024-04-20T10:37:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.136+0000 7f4c6d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f41a0 0x5556b67fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.136+0000 7f4c67e00640 1 -- 172.21.15.42:0/1225841897 learned_addr learned my addr 172.21.15.42:0/1225841897 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.137+0000 7f4c6dc00640 1 --2- 172.21.15.42:0/1225841897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67fd6a0 0x5556b67ed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.137+0000 7f4c67e00640 1 -- 172.21.15.42:0/1225841897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67fd6a0 msgr2=0x5556b67ed740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.137+0000 7f4c67e00640 1 --2- 172.21.15.42:0/1225841897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67fd6a0 0x5556b67ed740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.137+0000 7f4c67e00640 1 -- 172.21.15.42:0/1225841897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f41a0 msgr2=0x5556b67fce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.137+0000 7f4c67e00640 1 --2- 172.21.15.42:0/1225841897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f41a0 0x5556b67fce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.137+0000 7f4c67e00640 1 -- 172.21.15.42:0/1225841897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556b67f48f0 con 0x5556b67f4ac0 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.137+0000 7f4c67e00640 1 --2- 172.21.15.42:0/1225841897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f4ac0 0x5556b6809350 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009f60 tx=0x7f4c50012450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9877777034a97ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.137+0000 7f4c5d600640 1 -- 172.21.15.42:0/1225841897 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c500046e0 con 0x5556b67f4ac0 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.137+0000 7f4c5d600640 1 -- 172.21.15.42:0/1225841897 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c50004880 con 0x5556b67f4ac0 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.137+0000 7f4c5d600640 1 -- 172.21.15.42:0/1225841897 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50012aa0 con 0x5556b67f4ac0 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.138+0000 7f4c72863740 1 -- 172.21.15.42:0/1225841897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f4ac0 msgr2=0x5556b6809350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.138+0000 7f4c72863740 1 --2- 172.21.15.42:0/1225841897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f4ac0 0x5556b6809350 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009f60 tx=0x7f4c50012450 comp rx=0 tx=0).stop 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.138+0000 7f4c72863740 1 -- 172.21.15.42:0/1225841897 shutdown_connections 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.138+0000 7f4c72863740 1 --2- 172.21.15.42:0/1225841897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67fd6a0 0x5556b67ed740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.138+0000 7f4c72863740 1 --2- 172.21.15.42:0/1225841897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67f4ac0 0x5556b6809350 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.138+0000 7f4c72863740 1 --2- 172.21.15.42:0/1225841897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f41a0 0x5556b67fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.138+0000 7f4c72863740 1 -- 172.21.15.42:0/1225841897 >> 172.21.15.42:0/1225841897 conn(0x5556b67ee8d0 msgr2=0x5556b67ed030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.138+0000 7f4c72863740 1 -- 172.21.15.42:0/1225841897 shutdown_connections 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.138+0000 7f4c72863740 1 -- 172.21.15.42:0/1225841897 wait complete. 2024-04-20T10:37:22.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.139+0000 7f4c72863740 1 -- start start 2024-04-20T10:37:22.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.139+0000 7f4c72863740 1 Processor -- start 2024-04-20T10:37:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.139+0000 7f4c72863740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67f41a0 0x5556b6801240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.139+0000 7f4c72863740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b6801780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.139+0000 7f4c72863740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 0x5556b6801cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.139+0000 7f4c72863740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681aec00 con 0x5556b67f4ac0 2024-04-20T10:37:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.139+0000 7f4c72863740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681b0070 con 0x5556b67fd6a0 2024-04-20T10:37:22.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.139+0000 7f4c72863740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681af2d0 con 0x5556b67f41a0 2024-04-20T10:37:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.140+0000 7f4c72863740 10 client.0 _ll_register_callbacks cb 0x7f4c58005700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.140+0000 7f4c67e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b6801780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.140+0000 7f4c6dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 0x5556b6801cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.140+0000 7f4c67e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b6801780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55696/0 (socket says 172.21.15.42:55696) 2024-04-20T10:37:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.140+0000 7f4c67e00640 1 -- 172.21.15.42:0/2070992920 learned_addr learned my addr 172.21.15.42:0/2070992920 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.140+0000 7f4c6d200640 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67f41a0 0x5556b6801240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.140+0000 7f4c67e00640 1 -- 172.21.15.42:0/2070992920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67f41a0 msgr2=0x5556b6801240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.140+0000 7f4c67e00640 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67f41a0 0x5556b6801240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.140+0000 7f4c67e00640 1 -- 172.21.15.42:0/2070992920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 msgr2=0x5556b6801cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.140+0000 7f4c67e00640 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 0x5556b6801cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.140+0000 7f4c67e00640 1 -- 172.21.15.42:0/2070992920 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556b6872ef0 con 0x5556b67f4ac0 2024-04-20T10:37:22.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.140+0000 7f4c6dc00640 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 0x5556b6801cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.141+0000 7f4c67e00640 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b6801780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4c50000f80 tx=0x7f4c500124c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772866f19e5d18c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.141+0000 7f4c56a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.141+0000 7f4c56a00640 1 -- 172.21.15.42:0/2070992920 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50039060 con 0x5556b67f4ac0 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.141+0000 7f4c56a00640 1 -- 172.21.15.42:0/2070992920 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4c50012af0 con 0x5556b67f4ac0 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.141+0000 7f4c72863740 1 -- 172.21.15.42:0/2070992920 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5556b67619e0 con 0x5556b67f4ac0 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.141+0000 7f4c56a00640 1 -- 172.21.15.42:0/2070992920 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4c500046e0 con 0x5556b67f4ac0 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.141+0000 7f4c72863740 10 client.15654 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.141+0000 7f4c72863740 1 -- 172.21.15.42:0/2070992920 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5556b6874160 con 0x5556b67f4ac0 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.142+0000 7f4c56a00640 1 -- 172.21.15.42:0/2070992920 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5556b67619e0 con 0x5556b67f4ac0 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.142+0000 7f4c56a00640 1 -- 172.21.15.42:0/2070992920 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f4c5000f040 con 0x5556b67f4ac0 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.142+0000 7f4c72863740 10 client.15654 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.142+0000 7f4c72863740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.142+0000 7f4c72863740 20 client.15654 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.142+0000 7f4c72863740 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5556b6874770 0x5556b6876b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.142+0000 7f4c72863740 4 client.15654 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:37:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.142+0000 7f4c72863740 1 -- 172.21.15.42:0/2070992920 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5556b66dd450 con 0x5556b6874770 2024-04-20T10:37:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.142+0000 7f4c6d200640 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5556b6874770 0x5556b6876b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.143+0000 7f4c6d200640 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5556b6874770 0x5556b6876b90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6728e2bdbdda3556 server_cookie=a6ed0b7d0d69018c in_seq=0 out_seq=0 2024-04-20T10:37:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.143+0000 7f4c56a00640 10 client.15654 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c56a00640 1 -- 172.21.15.42:0/2070992920 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5556b66dd450 con 0x5556b6874770 2024-04-20T10:37:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c56a00640 10 client.15654 handle_command_reply: tid=0 2024-04-20T10:37:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c72863740 1 client.15654 shutdown 2024-04-20T10:37:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c72863740 1 -- 172.21.15.42:0/2070992920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 msgr2=0x5556b6801780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c72863740 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b6801780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4c50000f80 tx=0x7f4c500124c0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c72863740 1 -- 172.21.15.42:0/2070992920 shutdown_connections 2024-04-20T10:37:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c72863740 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5556b6874770 0x5556b6876b90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c72863740 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5556b67fd6a0 0x5556b6801cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c72863740 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5556b67f4ac0 0x5556b6801780 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c72863740 1 --2- 172.21.15.42:0/2070992920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556b67f41a0 0x5556b6801240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c56a00640 0 client.15654 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c72863740 1 -- 172.21.15.42:0/2070992920 >> 172.21.15.42:0/2070992920 conn(0x5556b67ee8d0 msgr2=0x5556b67fafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c72863740 1 -- 172.21.15.42:0/2070992920 shutdown_connections 2024-04-20T10:37:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c72863740 1 -- 172.21.15.42:0/2070992920 wait complete. 2024-04-20T10:37:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.144+0000 7f4c72863740 20 client.15654 trim_cache size 0 max 16384 2024-04-20T10:37:22.148 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:37:22.148 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:37:22.148 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:37:22.148 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:37:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.145+0000 7f4c6fc00640 1 -- 172.21.15.42:0/2748381890 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c4c0a3f40 msgr2=0x7f4c4c0a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.145+0000 7f4c6fc00640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c4c0a3f40 0x7f4c4c0a6400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009c20 tx=0x7f4c600046c0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.145+0000 7f4c6fc00640 1 -- 172.21.15.42:0/2748381890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c681021f0 msgr2=0x7f4c68074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.145+0000 7f4c6fc00640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c681021f0 0x7f4c68074260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c58005ab0 tx=0x7f4c58004360 comp rx=0 tx=0).stop 2024-04-20T10:37:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.145+0000 7f4c6fc00640 1 -- 172.21.15.42:0/2748381890 shutdown_connections 2024-04-20T10:37:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.145+0000 7f4c6fc00640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c4c0a3f40 0x7f4c4c0a6400 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.145+0000 7f4c6fc00640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c68074ce0 0x7f4c681a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.145+0000 7f4c6fc00640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c681072c0 0x7f4c680747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.145+0000 7f4c6fc00640 1 --2- 172.21.15.42:0/2748381890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c681021f0 0x7f4c68074260 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.145+0000 7f4c6fc00640 1 -- 172.21.15.42:0/2748381890 >> 172.21.15.42:0/2748381890 conn(0x7f4c680fda70 msgr2=0x7f4c680ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.145+0000 7f4c6fc00640 1 -- 172.21.15.42:0/2748381890 shutdown_connections 2024-04-20T10:37:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.145+0000 7f4c6fc00640 1 -- 172.21.15.42:0/2748381890 wait complete. 2024-04-20T10:37:22.163 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:37:22.163 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:37:22.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.355+0000 7f6857200640 1 -- 172.21.15.42:0/1565162491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6858102980 msgr2=0x7f68581069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.355+0000 7f6857200640 1 --2- 172.21.15.42:0/1565162491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6858102980 0x7f68581069c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6840009900 tx=0x7f684002bc30 comp rx=0 tx=0).stop 2024-04-20T10:37:22.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.355+0000 7f6857200640 1 -- 172.21.15.42:0/1565162491 shutdown_connections 2024-04-20T10:37:22.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.355+0000 7f6857200640 1 --2- 172.21.15.42:0/1565162491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68581070a0 0x7f685810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.355+0000 7f6857200640 1 --2- 172.21.15.42:0/1565162491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6858102980 0x7f68581069c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.355+0000 7f6857200640 1 --2- 172.21.15.42:0/1565162491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6858101fd0 0x7f68581023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.355+0000 7f6857200640 1 -- 172.21.15.42:0/1565162491 >> 172.21.15.42:0/1565162491 conn(0x7f68580fd890 msgr2=0x7f68580ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.356+0000 7f6857200640 1 -- 172.21.15.42:0/1565162491 shutdown_connections 2024-04-20T10:37:22.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.356+0000 7f6857200640 1 -- 172.21.15.42:0/1565162491 wait complete. 2024-04-20T10:37:22.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.357+0000 7f6857200640 1 Processor -- start 2024-04-20T10:37:22.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.357+0000 7f6857200640 1 -- start start 2024-04-20T10:37:22.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.358+0000 7f6857200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6858101fd0 0x7f68581a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.358+0000 7f6857200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68581070a0 0x7f68581a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.358+0000 7f6857200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685819a780 0x7f685819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.358+0000 7f6857200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6858112180 con 0x7f6858101fd0 2024-04-20T10:37:22.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.358+0000 7f6857200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6858112000 con 0x7f685819a780 2024-04-20T10:37:22.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.358+0000 7f6857200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6858112300 con 0x7f68581070a0 2024-04-20T10:37:22.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.358+0000 7f6855e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6858101fd0 0x7f68581a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.358+0000 7f6856800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685819a780 0x7f685819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.358+0000 7f6855e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6858101fd0 0x7f68581a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55712/0 (socket says 172.21.15.42:55712) 2024-04-20T10:37:22.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.358+0000 7f6855e00640 1 -- 172.21.15.42:0/1693130126 learned_addr learned my addr 172.21.15.42:0/1693130126 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:22.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.358+0000 7f684fe00640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68581070a0 0x7f68581a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.359+0000 7f6856800640 1 -- 172.21.15.42:0/1693130126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68581070a0 msgr2=0x7f68581a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.359+0000 7f6856800640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68581070a0 0x7f68581a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.359+0000 7f6856800640 1 -- 172.21.15.42:0/1693130126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6858101fd0 msgr2=0x7f68581a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.359+0000 7f6856800640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6858101fd0 0x7f68581a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.359+0000 7f6856800640 1 -- 172.21.15.42:0/1693130126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685819b310 con 0x7f685819a780 2024-04-20T10:37:22.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.359+0000 7f6856800640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685819a780 0x7f685819ac30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6850009fd0 tx=0x7f685000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.359+0000 7f684ea00640 1 -- 172.21.15.42:0/1693130126 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685000ce20 con 0x7f685819a780 2024-04-20T10:37:22.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.359+0000 7f684ea00640 1 -- 172.21.15.42:0/1693130126 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6850005020 con 0x7f685819a780 2024-04-20T10:37:22.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.359+0000 7f6857200640 1 -- 172.21.15.42:0/1693130126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685819b600 con 0x7f685819a780 2024-04-20T10:37:22.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.359+0000 7f684ea00640 1 -- 172.21.15.42:0/1693130126 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850010720 con 0x7f685819a780 2024-04-20T10:37:22.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.359+0000 7f6857200640 1 -- 172.21.15.42:0/1693130126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68581a7760 con 0x7f685819a780 2024-04-20T10:37:22.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.361+0000 7f6857200640 1 -- 172.21.15.42:0/1693130126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68581034a0 con 0x7f685819a780 2024-04-20T10:37:22.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.362+0000 7f684ea00640 1 -- 172.21.15.42:0/1693130126 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f6850005540 con 0x7f685819a780 2024-04-20T10:37:22.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.364+0000 7f684ea00640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f683c0a3da0 0x7f683c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.364+0000 7f684ea00640 1 -- 172.21.15.42:0/1693130126 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f68500d0d70 con 0x7f685819a780 2024-04-20T10:37:22.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.364+0000 7f6855e00640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f683c0a3da0 0x7f683c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.365+0000 7f6855e00640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f683c0a3da0 0x7f683c0a6260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6848009c90 tx=0x7f6848005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.368+0000 7f684ea00640 1 -- 172.21.15.42:0/1693130126 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f685819b600 con 0x7f685819a780 2024-04-20T10:37:22.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.627+0000 7f6857200640 1 -- 172.21.15.42:0/1693130126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6858068ca0 con 0x7f685819a780 2024-04-20T10:37:22.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.628+0000 7f684ea00640 1 -- 172.21.15.42:0/1693130126 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f6858068ca0 con 0x7f685819a780 2024-04-20T10:37:22.629 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:37:22.629 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:37:22.629 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:37:22.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.631+0000 7f6857200640 1 -- 172.21.15.42:0/1693130126 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f683c0a3da0 msgr2=0x7f683c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.631+0000 7f6857200640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f683c0a3da0 0x7f683c0a6260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6848009c90 tx=0x7f6848005f90 comp rx=0 tx=0).stop 2024-04-20T10:37:22.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.631+0000 7f6857200640 1 -- 172.21.15.42:0/1693130126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685819a780 msgr2=0x7f685819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.631+0000 7f6857200640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685819a780 0x7f685819ac30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6850009fd0 tx=0x7f685000ef10 comp rx=0 tx=0).stop 2024-04-20T10:37:22.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.631+0000 7f6857200640 1 -- 172.21.15.42:0/1693130126 shutdown_connections 2024-04-20T10:37:22.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.631+0000 7f6857200640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f683c0a3da0 0x7f683c0a6260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.631+0000 7f6857200640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685819a780 0x7f685819ac30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.631+0000 7f6857200640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68581070a0 0x7f68581a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.631+0000 7f6857200640 1 --2- 172.21.15.42:0/1693130126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6858101fd0 0x7f68581a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.631+0000 7f6857200640 1 -- 172.21.15.42:0/1693130126 >> 172.21.15.42:0/1693130126 conn(0x7f68580fd890 msgr2=0x7f68580ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.631+0000 7f6857200640 1 -- 172.21.15.42:0/1693130126 shutdown_connections 2024-04-20T10:37:22.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.631+0000 7f6857200640 1 -- 172.21.15.42:0/1693130126 wait complete. 2024-04-20T10:37:22.654 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:37:22.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.875+0000 7fbcaf200640 1 -- 172.21.15.42:0/3080679557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcb0102a30 msgr2=0x7fbcb0106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.875+0000 7fbcaf200640 1 --2- 172.21.15.42:0/3080679557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcb0102a30 0x7fbcb0106a70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009900 tx=0x7fbc9c02bc40 comp rx=0 tx=0).stop 2024-04-20T10:37:22.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.875+0000 7fbcaf200640 1 -- 172.21.15.42:0/3080679557 shutdown_connections 2024-04-20T10:37:22.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.875+0000 7fbcaf200640 1 --2- 172.21.15.42:0/3080679557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcb0107150 0x7fbcb010ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.875+0000 7fbcaf200640 1 --2- 172.21.15.42:0/3080679557 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcb0102a30 0x7fbcb0106a70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.875+0000 7fbcaf200640 1 --2- 172.21.15.42:0/3080679557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb0102080 0x7fbcb0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.875+0000 7fbcaf200640 1 -- 172.21.15.42:0/3080679557 >> 172.21.15.42:0/3080679557 conn(0x7fbcb00fd900 msgr2=0x7fbcb00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.876+0000 7fbcaf200640 1 -- 172.21.15.42:0/3080679557 shutdown_connections 2024-04-20T10:37:22.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.876+0000 7fbcaf200640 1 -- 172.21.15.42:0/3080679557 wait complete. 2024-04-20T10:37:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.877+0000 7fbcaf200640 1 Processor -- start 2024-04-20T10:37:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.877+0000 7fbcaf200640 1 -- start start 2024-04-20T10:37:22.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.877+0000 7fbcaf200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcb0102080 0x7fbcb01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.877+0000 7fbcaf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb0107150 0x7fbcb01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.877+0000 7fbcaf200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcb019a780 0x7fbcb019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.877+0000 7fbcaf200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb0112210 con 0x7fbcb0102080 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.877+0000 7fbcaf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb0112090 con 0x7fbcb0107150 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.877+0000 7fbcaf200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb0112390 con 0x7fbcb019a780 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.878+0000 7fbcade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcb0102080 0x7fbcb01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.878+0000 7fbcad400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb0107150 0x7fbcb01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.878+0000 7fbcade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcb0102080 0x7fbcb01a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55724/0 (socket says 172.21.15.42:55724) 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.878+0000 7fbcade00640 1 -- 172.21.15.42:0/2056000791 learned_addr learned my addr 172.21.15.42:0/2056000791 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.878+0000 7fbcae800640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcb019a780 0x7fbcb019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.878+0000 7fbcade00640 1 -- 172.21.15.42:0/2056000791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcb019a780 msgr2=0x7fbcb019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.878+0000 7fbcade00640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcb019a780 0x7fbcb019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.878+0000 7fbcade00640 1 -- 172.21.15.42:0/2056000791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb0107150 msgr2=0x7fbcb01a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.878+0000 7fbcade00640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb0107150 0x7fbcb01a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.878+0000 7fbcade00640 1 -- 172.21.15.42:0/2056000791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb019b310 con 0x7fbcb0102080 2024-04-20T10:37:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.878+0000 7fbcae800640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcb019a780 0x7fbcb019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:37:22.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.878+0000 7fbcade00640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcb0102080 0x7fbcb01a0610 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbca000e9f0 tx=0x7fbca000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.879+0000 7fbca6a00640 1 -- 172.21.15.42:0/2056000791 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca000cd80 con 0x7fbcb0102080 2024-04-20T10:37:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.879+0000 7fbca6a00640 1 -- 172.21.15.42:0/2056000791 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca0010910 con 0x7fbcb0102080 2024-04-20T10:37:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.879+0000 7fbcaf200640 1 -- 172.21.15.42:0/2056000791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb019b600 con 0x7fbcb0102080 2024-04-20T10:37:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.879+0000 7fbca6a00640 1 -- 172.21.15.42:0/2056000791 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0018990 con 0x7fbcb0102080 2024-04-20T10:37:22.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.879+0000 7fbcaf200640 1 -- 172.21.15.42:0/2056000791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb01a7800 con 0x7fbcb0102080 2024-04-20T10:37:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.883+0000 7fbcb7b94740 1 Processor -- start 2024-04-20T10:37:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.883+0000 7fbcb7b94740 1 -- start start 2024-04-20T10:37:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcb7b94740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b1e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcb7b94740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1ea90 0x559ce8b27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcb7b94740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 0x559ce8b29a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcb7b94740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559ce8a83050 con 0x559ce8b1ea90 2024-04-20T10:37:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcb7b94740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559ce8a29490 con 0x559ce8b27670 2024-04-20T10:37:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcb7b94740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559ce8a7f010 con 0x559ce8b1e170 2024-04-20T10:37:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcad400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 0x559ce8b29a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcae800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b1e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcad400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 0x559ce8b29a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49626/0 (socket says 172.21.15.42:49626) 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcad400640 1 -- 172.21.15.42:0/436696524 learned_addr learned my addr 172.21.15.42:0/436696524 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcad400640 1 -- 172.21.15.42:0/436696524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 msgr2=0x559ce8b1e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcad400640 1 --2- 172.21.15.42:0/436696524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b1e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcad400640 1 -- 172.21.15.42:0/436696524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1ea90 msgr2=0x559ce8b27130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcad400640 1 --2- 172.21.15.42:0/436696524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1ea90 0x559ce8b27130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.884+0000 7fbcad400640 1 -- 172.21.15.42:0/436696524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce8a31070 con 0x559ce8b27670 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.885+0000 7fbca6a00640 1 -- 172.21.15.42:0/2056000791 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbca0010ac0 con 0x7fbcb0102080 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.885+0000 7fbcad400640 1 --2- 172.21.15.42:0/436696524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 0x559ce8b29a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbcb006b540 tx=0x7fbc9c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cffc296d281a2656 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.885+0000 7fbc9b400640 1 -- 172.21.15.42:0/436696524 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c042070 con 0x559ce8b27670 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.885+0000 7fbc9b400640 1 -- 172.21.15.42:0/436696524 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9c02c890 con 0x559ce8b27670 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.885+0000 7fbc9b400640 1 -- 172.21.15.42:0/436696524 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c004c00 con 0x559ce8b27670 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.886+0000 7fbcb7b94740 1 -- 172.21.15.42:0/436696524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 msgr2=0x559ce8b29a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.886+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/436696524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 0x559ce8b29a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbcb006b540 tx=0x7fbc9c000bc0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.886+0000 7fbca6a00640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbc940a3da0 0x7fbc940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.886+0000 7fbca6a00640 1 -- 172.21.15.42:0/2056000791 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbca00d0b90 con 0x7fbcb0102080 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.886+0000 7fbcb7b94740 1 -- 172.21.15.42:0/436696524 shutdown_connections 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.886+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/436696524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 0x559ce8b29a60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.886+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/436696524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1ea90 0x559ce8b27130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.886+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/436696524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b1e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.886+0000 7fbcb7b94740 1 -- 172.21.15.42:0/436696524 >> 172.21.15.42:0/436696524 conn(0x559ce8b188a0 msgr2=0x559ce8b18c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.886+0000 7fbcae800640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbc940a3da0 0x7fbc940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.887+0000 7fbcb7b94740 1 -- 172.21.15.42:0/436696524 shutdown_connections 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.887+0000 7fbcb7b94740 1 -- 172.21.15.42:0/436696524 wait complete. 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.887+0000 7fbcb7b94740 1 -- start start 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.887+0000 7fbcae800640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbc940a3da0 0x7fbc940a6260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbca8005fd0 tx=0x7fbca8009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.887+0000 7fbcb7b94740 1 Processor -- start 2024-04-20T10:37:22.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcb7b94740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1e170 0x559ce8b24760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcb7b94740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 0x559ce8b24ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcb7b94740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b27670 0x559ce8b251e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcb7b94740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559ce8b2a400 con 0x559ce8b1e170 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcb7b94740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559ce8a3c860 con 0x559ce8b1ea90 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcb7b94740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559ce8b2a580 con 0x559ce8b27670 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcb7b94740 10 client.0 _ll_register_callbacks cb 0x559ce8a0afe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcad400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 0x559ce8b24ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1e170 0x559ce8b24760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcad400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 0x559ce8b24ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49640/0 (socket says 172.21.15.42:49640) 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcae800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b27670 0x559ce8b251e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcad400640 1 -- 172.21.15.42:0/1971439736 learned_addr learned my addr 172.21.15.42:0/1971439736 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcad400640 1 -- 172.21.15.42:0/1971439736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b27670 msgr2=0x559ce8b251e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcad400640 1 --2- 172.21.15.42:0/1971439736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b27670 0x559ce8b251e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcad400640 1 -- 172.21.15.42:0/1971439736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1e170 msgr2=0x559ce8b24760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcad400640 1 --2- 172.21.15.42:0/1971439736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1e170 0x559ce8b24760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.888+0000 7fbcad400640 1 -- 172.21.15.42:0/1971439736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce8a40950 con 0x559ce8b1ea90 2024-04-20T10:37:22.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.889+0000 7fbcad400640 1 --2- 172.21.15.42:0/1971439736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 0x559ce8b24ca0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c036040 tx=0x7fbc9c004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1681c170e77f1667 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.889+0000 7fbc8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:37:22.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.889+0000 7fbc8fe00640 1 -- 172.21.15.42:0/1971439736 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c042040 con 0x559ce8b1ea90 2024-04-20T10:37:22.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.889+0000 7fbc8fe00640 1 -- 172.21.15.42:0/1971439736 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbc9c005150 con 0x559ce8b1ea90 2024-04-20T10:37:22.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.889+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1971439736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc9c02c890 con 0x559ce8b1ea90 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.889+0000 7fbc8fe00640 1 -- 172.21.15.42:0/1971439736 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbc9c005350 con 0x559ce8b1ea90 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.889+0000 7fbcb7b94740 10 client.25784 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.889+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1971439736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559ce8a0e820 con 0x559ce8b1ea90 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.890+0000 7fbc8fe00640 1 -- 172.21.15.42:0/1971439736 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9c02c890 con 0x559ce8b1ea90 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.890+0000 7fbc8fe00640 1 -- 172.21.15.42:0/1971439736 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fbc9c034440 con 0x559ce8b1ea90 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.890+0000 7fbcb7b94740 10 client.25784 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.890+0000 7fbcb7b94740 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.890+0000 7fbcb7b94740 20 client.25784 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.890+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/1971439736 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559ce8b37550 0x559ce8b39940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.890+0000 7fbcb7b94740 4 client.25784 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.890+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1971439736 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ce8a90a70 con 0x559ce8b37550 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.890+0000 7fbcade00640 1 --2- 172.21.15.42:0/1971439736 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559ce8b37550 0x559ce8b39940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.891+0000 7fbcade00640 1 --2- 172.21.15.42:0/1971439736 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559ce8b37550 0x559ce8b39940 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9a6a9bf530d9003 server_cookie=9a01bd45e85df1c0 in_seq=0 out_seq=0 2024-04-20T10:37:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.891+0000 7fbc8fe00640 10 client.25784 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbc8fe00640 1 -- 172.21.15.42:0/1971439736 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ce8a90a70 con 0x559ce8b37550 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbc8fe00640 10 client.25784 handle_command_reply: tid=0 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbcb7b94740 1 client.25784 shutdown 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1971439736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 msgr2=0x559ce8b24ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/1971439736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 0x559ce8b24ca0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c036040 tx=0x7fbc9c004dc0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1971439736 shutdown_connections 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/1971439736 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559ce8b37550 0x559ce8b39940 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/1971439736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b27670 0x559ce8b251e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/1971439736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 0x559ce8b24ca0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbc8fe00640 0 client.25784 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/1971439736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1e170 0x559ce8b24760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1971439736 >> 172.21.15.42:0/1971439736 conn(0x559ce8b188a0 msgr2=0x559ce8b1a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1971439736 shutdown_connections 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1971439736 wait complete. 2024-04-20T10:37:22.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.893+0000 7fbcb7b94740 20 client.25784 trim_cache size 0 max 16384 2024-04-20T10:37:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.908+0000 7fbcb7b94740 1 Processor -- start 2024-04-20T10:37:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.908+0000 7fbcb7b94740 1 -- start start 2024-04-20T10:37:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcb7b94740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b26e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcb7b94740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1ea90 0x559ce8b332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcb7b94740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 0x559ce8b3eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcb7b94740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01adc20 con 0x559ce8b1ea90 2024-04-20T10:37:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcb7b94740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01b04a0 con 0x559ce8b27670 2024-04-20T10:37:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcb7b94740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01ae290 con 0x559ce8b1e170 2024-04-20T10:37:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcad400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1ea90 0x559ce8b332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcad400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1ea90 0x559ce8b332c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55740/0 (socket says 172.21.15.42:55740) 2024-04-20T10:37:22.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcad400640 1 -- 172.21.15.42:0/1281430775 learned_addr learned my addr 172.21.15.42:0/1281430775 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:22.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcae800640 1 --2- 172.21.15.42:0/1281430775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 0x559ce8b3eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcade00640 1 --2- 172.21.15.42:0/1281430775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b26e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcad400640 1 -- 172.21.15.42:0/1281430775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 msgr2=0x559ce8b26e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcad400640 1 --2- 172.21.15.42:0/1281430775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b26e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcad400640 1 -- 172.21.15.42:0/1281430775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 msgr2=0x559ce8b3eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcad400640 1 --2- 172.21.15.42:0/1281430775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 0x559ce8b3eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcad400640 1 -- 172.21.15.42:0/1281430775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce87953f0 con 0x559ce8b1ea90 2024-04-20T10:37:22.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcade00640 1 --2- 172.21.15.42:0/1281430775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b26e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:22.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.909+0000 7fbcae800640 1 --2- 172.21.15.42:0/1281430775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 0x559ce8b3eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.910+0000 7fbcad400640 1 --2- 172.21.15.42:0/1281430775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1ea90 0x559ce8b332c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c02c180 tx=0x7fbc9c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb05e83a15bfdef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.910+0000 7fbc9aa00640 1 -- 172.21.15.42:0/1281430775 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c05d020 con 0x559ce8b1ea90 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.910+0000 7fbc9aa00640 1 -- 172.21.15.42:0/1281430775 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9c02c890 con 0x559ce8b1ea90 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.910+0000 7fbc9aa00640 1 -- 172.21.15.42:0/1281430775 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c042040 con 0x559ce8b1ea90 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.911+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1281430775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1ea90 msgr2=0x559ce8b332c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.911+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/1281430775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1ea90 0x559ce8b332c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c02c180 tx=0x7fbc9c000bc0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.911+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1281430775 shutdown_connections 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.911+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/1281430775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b27670 0x559ce8b3eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.911+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/1281430775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b1ea90 0x559ce8b332c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.911+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/1281430775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b26e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.911+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1281430775 >> 172.21.15.42:0/1281430775 conn(0x559ce8b188a0 msgr2=0x559ce8b17960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.911+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1281430775 shutdown_connections 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.911+0000 7fbcb7b94740 1 -- 172.21.15.42:0/1281430775 wait complete. 2024-04-20T10:37:22.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.911+0000 7fbcb7b94740 1 -- start start 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcb7b94740 1 Processor -- start 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcb7b94740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcb7b94740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 0x559ce8b21c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcb7b94740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b27670 0x559ce8b22150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcb7b94740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01ae960 con 0x559ce8b27670 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcb7b94740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01afdd0 con 0x559ce8b1ea90 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcb7b94740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01af030 con 0x559ce8b1e170 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcb7b94740 10 client.0 _ll_register_callbacks cb 0x7fbc9c004cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcad400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 0x559ce8b21c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcad400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 0x559ce8b21c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49666/0 (socket says 172.21.15.42:49666) 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcad400640 1 -- 172.21.15.42:0/173676027 learned_addr learned my addr 172.21.15.42:0/173676027 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcade00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.912+0000 7fbcae800640 1 --2- 172.21.15.42:0/173676027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b27670 0x559ce8b22150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbcad400640 1 -- 172.21.15.42:0/173676027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 msgr2=0x559ce8b216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbcad400640 1 --2- 172.21.15.42:0/173676027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b216d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbcad400640 1 -- 172.21.15.42:0/173676027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b27670 msgr2=0x559ce8b22150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbcad400640 1 --2- 172.21.15.42:0/173676027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b27670 0x559ce8b22150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbcad400640 1 -- 172.21.15.42:0/173676027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce885f7d0 con 0x559ce8b1ea90 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbcad400640 1 --2- 172.21.15.42:0/173676027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 0x559ce8b21c10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009900 tx=0x7fbc9c0048e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=241694bbfca3daa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbc8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbc8fe00640 1 -- 172.21.15.42:0/173676027 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c05d070 con 0x559ce8b1ea90 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbc8fe00640 1 -- 172.21.15.42:0/173676027 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbc9c005150 con 0x559ce8b1ea90 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbcb7b94740 1 -- 172.21.15.42:0/173676027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc9c02c890 con 0x559ce8b1ea90 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbc8fe00640 1 -- 172.21.15.42:0/173676027 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbc9c069050 con 0x559ce8b1ea90 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbcb7b94740 10 client.25790 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.913+0000 7fbcb7b94740 1 -- 172.21.15.42:0/173676027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559ce8b9db00 con 0x559ce8b1ea90 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.914+0000 7fbc8fe00640 1 -- 172.21.15.42:0/173676027 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9c02c890 con 0x559ce8b1ea90 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.914+0000 7fbc8fe00640 1 -- 172.21.15.42:0/173676027 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fbc9c047cc0 con 0x559ce8b1ea90 2024-04-20T10:37:22.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.914+0000 7fbcb7b94740 10 client.25790 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.914+0000 7fbcb7b94740 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.914+0000 7fbcb7b94740 20 client.25790 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.914+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/173676027 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559ce8b9e0e0 0x559ce8ba0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.914+0000 7fbcb7b94740 4 client.25790 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:37:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.914+0000 7fbcb7b94740 1 -- 172.21.15.42:0/173676027 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ce8a8a8e0 con 0x559ce8b9e0e0 2024-04-20T10:37:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.914+0000 7fbcade00640 1 --2- 172.21.15.42:0/173676027 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559ce8b9e0e0 0x559ce8ba0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:22.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.915+0000 7fbcade00640 1 --2- 172.21.15.42:0/173676027 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559ce8b9e0e0 0x559ce8ba0500 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cbf84d69058335b server_cookie=a8fa1b90d46b6f6 in_seq=0 out_seq=0 2024-04-20T10:37:22.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.915+0000 7fbc8fe00640 10 client.25790 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:22.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.916+0000 7fbc8fe00640 1 -- 172.21.15.42:0/173676027 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ce8a8a8e0 con 0x559ce8b9e0e0 2024-04-20T10:37:22.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.916+0000 7fbc8fe00640 10 client.25790 handle_command_reply: tid=0 2024-04-20T10:37:22.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.916+0000 7fbcb7b94740 1 client.25790 shutdown 2024-04-20T10:37:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.916+0000 7fbcb7b94740 1 -- 172.21.15.42:0/173676027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 msgr2=0x559ce8b21c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.916+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/173676027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 0x559ce8b21c10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009900 tx=0x7fbc9c0048e0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.917+0000 7fbcb7b94740 1 -- 172.21.15.42:0/173676027 shutdown_connections 2024-04-20T10:37:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.917+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/173676027 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559ce8b9e0e0 0x559ce8ba0500 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.917+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/173676027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559ce8b27670 0x559ce8b22150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.917+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/173676027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ce8b1ea90 0x559ce8b21c10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.917+0000 7fbcb7b94740 1 --2- 172.21.15.42:0/173676027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559ce8b1e170 0x559ce8b216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.917+0000 7fbcb7b94740 1 -- 172.21.15.42:0/173676027 >> 172.21.15.42:0/173676027 conn(0x559ce8b188a0 msgr2=0x559ce8b170d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.917+0000 7fbc8fe00640 0 client.25790 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.917+0000 7fbcb7b94740 1 -- 172.21.15.42:0/173676027 shutdown_connections 2024-04-20T10:37:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.917+0000 7fbcb7b94740 1 -- 172.21.15.42:0/173676027 wait complete. 2024-04-20T10:37:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.917+0000 7fbcb7b94740 20 client.25790 trim_cache size 0 max 16384 2024-04-20T10:37:22.921 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:37:22.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.918+0000 7fbcaf200640 1 -- 172.21.15.42:0/2056000791 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbc940a3da0 msgr2=0x7fbc940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.918+0000 7fbcaf200640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbc940a3da0 0x7fbc940a6260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbca8005fd0 tx=0x7fbca8009500 comp rx=0 tx=0).stop 2024-04-20T10:37:22.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.918+0000 7fbcaf200640 1 -- 172.21.15.42:0/2056000791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcb0102080 msgr2=0x7fbcb01a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:22.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.918+0000 7fbcaf200640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcb0102080 0x7fbcb01a0610 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbca000e9f0 tx=0x7fbca000eeb0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.918+0000 7fbcaf200640 1 -- 172.21.15.42:0/2056000791 shutdown_connections 2024-04-20T10:37:22.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.918+0000 7fbcaf200640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbc940a3da0 0x7fbc940a6260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.918+0000 7fbcaf200640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcb019a780 0x7fbcb019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.918+0000 7fbcaf200640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcb0107150 0x7fbcb01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.918+0000 7fbcaf200640 1 --2- 172.21.15.42:0/2056000791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcb0102080 0x7fbcb01a0610 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.918+0000 7fbcaf200640 1 -- 172.21.15.42:0/2056000791 >> 172.21.15.42:0/2056000791 conn(0x7fbcb00fd900 msgr2=0x7fbcb00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.918+0000 7fbcaf200640 1 -- 172.21.15.42:0/2056000791 shutdown_connections 2024-04-20T10:37:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:22.918+0000 7fbcaf200640 1 -- 172.21.15.42:0/2056000791 wait complete. 2024-04-20T10:37:23.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:23 smithi046 ceph-mon[21574]: pgmap v104: 97 pgs: 97 active+clean; 333 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 67 op/s 2024-04-20T10:37:23.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:23 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1693130126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:37:23.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:23 smithi042 ceph-mon[17078]: pgmap v104: 97 pgs: 97 active+clean; 333 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 67 op/s 2024-04-20T10:37:23.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:23 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1693130126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:37:23.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:23 smithi162 ceph-mon[21854]: pgmap v104: 97 pgs: 97 active+clean; 333 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 67 op/s 2024-04-20T10:37:23.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:23 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1693130126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:37:23.939 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:37:23.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:37:23.939 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:37:24.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.172+0000 7fc97e400640 1 -- 172.21.15.42:0/4034854918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc978108500 msgr2=0x7fc978108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.172+0000 7fc97e400640 1 --2- 172.21.15.42:0/4034854918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc978108500 0x7fc978108960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc964009900 tx=0x7fc96402bcd0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.172+0000 7fc97e400640 1 -- 172.21.15.42:0/4034854918 shutdown_connections 2024-04-20T10:37:24.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.172+0000 7fc97e400640 1 --2- 172.21.15.42:0/4034854918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc978108ea0 0x7fc9781152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.172+0000 7fc97e400640 1 --2- 172.21.15.42:0/4034854918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc978108500 0x7fc978108960 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.172+0000 7fc97e400640 1 --2- 172.21.15.42:0/4034854918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc97810d3d0 0x7fc97810d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.172+0000 7fc97e400640 1 -- 172.21.15.42:0/4034854918 >> 172.21.15.42:0/4034854918 conn(0x7fc978077d80 msgr2=0x7fc978106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:24.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.172+0000 7fc97e400640 1 -- 172.21.15.42:0/4034854918 shutdown_connections 2024-04-20T10:37:24.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.172+0000 7fc97e400640 1 -- 172.21.15.42:0/4034854918 wait complete. 2024-04-20T10:37:24.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.173+0000 7fc97e400640 1 Processor -- start 2024-04-20T10:37:24.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.174+0000 7fc97e400640 1 -- start start 2024-04-20T10:37:24.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.174+0000 7fc97e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc978108500 0x7fc9781a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.174+0000 7fc97e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc978108ea0 0x7fc9781a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.174+0000 7fc97e400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc97810d3d0 0x7fc97819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.174+0000 7fc97e400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781187f0 con 0x7fc978108ea0 2024-04-20T10:37:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.174+0000 7fc97e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc978118670 con 0x7fc978108500 2024-04-20T10:37:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.174+0000 7fc97e400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc978118970 con 0x7fc97810d3d0 2024-04-20T10:37:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.174+0000 7fc97d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc978108500 0x7fc9781a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.174+0000 7fc977e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc978108ea0 0x7fc9781a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.174+0000 7fc97da00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc97810d3d0 0x7fc97819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.174+0000 7fc977e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc978108ea0 0x7fc9781a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55766/0 (socket says 172.21.15.42:55766) 2024-04-20T10:37:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.174+0000 7fc977e00640 1 -- 172.21.15.42:0/1700747866 learned_addr learned my addr 172.21.15.42:0/1700747866 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.175+0000 7fc97d000640 1 -- 172.21.15.42:0/1700747866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc97810d3d0 msgr2=0x7fc97819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.175+0000 7fc97d000640 1 --2- 172.21.15.42:0/1700747866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc97810d3d0 0x7fc97819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.175+0000 7fc97d000640 1 -- 172.21.15.42:0/1700747866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc978108ea0 msgr2=0x7fc9781a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.175+0000 7fc97d000640 1 --2- 172.21.15.42:0/1700747866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc978108ea0 0x7fc9781a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.175+0000 7fc97d000640 1 -- 172.21.15.42:0/1700747866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97819f430 con 0x7fc978108500 2024-04-20T10:37:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.175+0000 7fc97d000640 1 --2- 172.21.15.42:0/1700747866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc978108500 0x7fc9781a4a60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00e9f0 tx=0x7fc96c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.175+0000 7fc975600640 1 -- 172.21.15.42:0/1700747866 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00ccc0 con 0x7fc978108500 2024-04-20T10:37:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.176+0000 7fc975600640 1 -- 172.21.15.42:0/1700747866 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96c010910 con 0x7fc978108500 2024-04-20T10:37:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.176+0000 7fc97e400640 1 -- 172.21.15.42:0/1700747866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97819f720 con 0x7fc978108500 2024-04-20T10:37:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.176+0000 7fc975600640 1 -- 172.21.15.42:0/1700747866 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c018960 con 0x7fc978108500 2024-04-20T10:37:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.176+0000 7fc97e400640 1 -- 172.21.15.42:0/1700747866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9781ab9e0 con 0x7fc978108500 2024-04-20T10:37:24.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.180+0000 7fc9827a7740 1 Processor -- start 2024-04-20T10:37:24.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.180+0000 7fc9827a7740 1 -- start start 2024-04-20T10:37:24.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.181+0000 7fc975600640 1 -- 172.21.15.42:0/1700747866 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc96c010ac0 con 0x7fc978108500 2024-04-20T10:37:24.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.181+0000 7fc9827a7740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d00f0 0x55c9373d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.181+0000 7fc9827a7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d0a10 0x55c9373d90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.181+0000 7fc9827a7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d95f0 0x55c9373db9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.181+0000 7fc9827a7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c937334d20 con 0x55c9373d0a10 2024-04-20T10:37:24.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.181+0000 7fc9827a7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c9372db3e0 con 0x55c9373d95f0 2024-04-20T10:37:24.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.181+0000 7fc9827a7740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c937330ce0 con 0x55c9373d00f0 2024-04-20T10:37:24.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.181+0000 7fc977e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d95f0 0x55c9373db9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.181+0000 7fc977e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d95f0 0x55c9373db9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49694/0 (socket says 172.21.15.42:49694) 2024-04-20T10:37:24.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.181+0000 7fc977e00640 1 -- 172.21.15.42:0/490467628 learned_addr learned my addr 172.21.15.42:0/490467628 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:24.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.181+0000 7fc97da00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d00f0 0x55c9373d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.181+0000 7fc97d000640 1 --2- 172.21.15.42:0/490467628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d0a10 0x55c9373d90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.182+0000 7fc977e00640 1 -- 172.21.15.42:0/490467628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d00f0 msgr2=0x55c9373d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.182+0000 7fc977e00640 1 --2- 172.21.15.42:0/490467628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d00f0 0x55c9373d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.182+0000 7fc977e00640 1 -- 172.21.15.42:0/490467628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d0a10 msgr2=0x55c9373d90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.182+0000 7fc977e00640 1 --2- 172.21.15.42:0/490467628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d0a10 0x55c9373d90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.182+0000 7fc977e00640 1 -- 172.21.15.42:0/490467628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9372fa980 con 0x55c9373d95f0 2024-04-20T10:37:24.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.182+0000 7fc977e00640 1 --2- 172.21.15.42:0/490467628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d95f0 0x55c9373db9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc964036040 tx=0x7fc9640058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3c2bbbfbc609d25 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:24.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.182+0000 7fc96a000640 1 -- 172.21.15.42:0/490467628 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964042070 con 0x55c9373d95f0 2024-04-20T10:37:24.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.182+0000 7fc96a000640 1 -- 172.21.15.42:0/490467628 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96402cb20 con 0x55c9373d95f0 2024-04-20T10:37:24.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.182+0000 7fc96a000640 1 -- 172.21.15.42:0/490467628 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964004f80 con 0x55c9373d95f0 2024-04-20T10:37:24.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.183+0000 7fc975600640 1 --2- 172.21.15.42:0/1700747866 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc9600a3d20 0x7fc9600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.183+0000 7fc975600640 1 -- 172.21.15.42:0/1700747866 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc96c0d0b10 con 0x7fc978108500 2024-04-20T10:37:24.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.183+0000 7fc97da00640 1 --2- 172.21.15.42:0/1700747866 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc9600a3d20 0x7fc9600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.183+0000 7fc9827a7740 1 -- 172.21.15.42:0/490467628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d95f0 msgr2=0x55c9373db9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.183+0000 7fc9827a7740 1 --2- 172.21.15.42:0/490467628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d95f0 0x55c9373db9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc964036040 tx=0x7fc9640058d0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.184+0000 7fc9827a7740 1 -- 172.21.15.42:0/490467628 shutdown_connections 2024-04-20T10:37:24.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.184+0000 7fc9827a7740 1 --2- 172.21.15.42:0/490467628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d95f0 0x55c9373db9e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.184+0000 7fc9827a7740 1 --2- 172.21.15.42:0/490467628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d0a10 0x55c9373d90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.184+0000 7fc9827a7740 1 --2- 172.21.15.42:0/490467628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d00f0 0x55c9373d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.184+0000 7fc9827a7740 1 -- 172.21.15.42:0/490467628 >> 172.21.15.42:0/490467628 conn(0x55c9373ca820 msgr2=0x55c9373cac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.184+0000 7fc9827a7740 1 -- 172.21.15.42:0/490467628 shutdown_connections 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.184+0000 7fc97da00640 1 --2- 172.21.15.42:0/1700747866 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc9600a3d20 0x7fc9600a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc970005fd0 tx=0x7fc970009550 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.184+0000 7fc9827a7740 1 -- 172.21.15.42:0/490467628 wait complete. 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.184+0000 7fc9827a7740 1 -- start start 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.184+0000 7fc9827a7740 1 Processor -- start 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.185+0000 7fc9827a7740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d00f0 0x55c9373d6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.185+0000 7fc9827a7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 0x55c9373d6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.185+0000 7fc9827a7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d95f0 0x55c9373d70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.185+0000 7fc9827a7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c9373dc380 con 0x55c9373d95f0 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.185+0000 7fc9827a7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c9372ee7a0 con 0x55c9373d0a10 2024-04-20T10:37:24.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.185+0000 7fc9827a7740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c9373dc500 con 0x55c9373d00f0 2024-04-20T10:37:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.185+0000 7fc9827a7740 10 client.0 _ll_register_callbacks cb 0x55c9372f0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.185+0000 7fc977e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 0x55c9373d6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.185+0000 7fc977e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 0x55c9373d6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49702/0 (socket says 172.21.15.42:49702) 2024-04-20T10:37:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.185+0000 7fc977e00640 1 -- 172.21.15.42:0/3371469993 learned_addr learned my addr 172.21.15.42:0/3371469993 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.185+0000 7fc97d000640 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d00f0 0x55c9373d6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.185+0000 7fc97da00640 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d95f0 0x55c9373d70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.186+0000 7fc977e00640 1 -- 172.21.15.42:0/3371469993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d00f0 msgr2=0x55c9373d6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.186+0000 7fc977e00640 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d00f0 0x55c9373d6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.186+0000 7fc977e00640 1 -- 172.21.15.42:0/3371469993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d95f0 msgr2=0x55c9373d70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.186+0000 7fc977e00640 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d95f0 0x55c9373d70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.186+0000 7fc977e00640 1 -- 172.21.15.42:0/3371469993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c93710f7e0 con 0x55c9373d0a10 2024-04-20T10:37:24.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.186+0000 7fc97da00640 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d95f0 0x55c9373d70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:37:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.186+0000 7fc977e00640 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 0x55c9373d6b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc964009430 tx=0x7fc964004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3801b4b3102ae1d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.186+0000 7fc95ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:37:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.186+0000 7fc95ea00640 1 -- 172.21.15.42:0/3371469993 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964042070 con 0x55c9373d0a10 2024-04-20T10:37:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.186+0000 7fc95ea00640 1 -- 172.21.15.42:0/3371469993 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc964004bb0 con 0x55c9373d0a10 2024-04-20T10:37:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.186+0000 7fc9827a7740 1 -- 172.21.15.42:0/3371469993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc96402cb20 con 0x55c9373d0a10 2024-04-20T10:37:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.186+0000 7fc95ea00640 1 -- 172.21.15.42:0/3371469993 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc964004f80 con 0x55c9373d0a10 2024-04-20T10:37:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.187+0000 7fc9827a7740 10 client.25802 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.187+0000 7fc9827a7740 1 -- 172.21.15.42:0/3371469993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c93716ff70 con 0x55c9373d0a10 2024-04-20T10:37:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.187+0000 7fc95ea00640 1 -- 172.21.15.42:0/3371469993 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96402cb20 con 0x55c9373d0a10 2024-04-20T10:37:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.187+0000 7fc95ea00640 1 -- 172.21.15.42:0/3371469993 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc964003a80 con 0x55c9373d0a10 2024-04-20T10:37:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.187+0000 7fc9827a7740 10 client.25802 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:24.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.187+0000 7fc9827a7740 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:24.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.187+0000 7fc9827a7740 20 client.25802 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:24.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.187+0000 7fc9827a7740 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c9373e97d0 0x55c9373ebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.187+0000 7fc9827a7740 4 client.25802 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:37:24.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.187+0000 7fc9827a7740 1 -- 172.21.15.42:0/3371469993 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c937342740 con 0x55c9373e97d0 2024-04-20T10:37:24.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.188+0000 7fc97d000640 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c9373e97d0 0x55c9373ebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.188+0000 7fc97d000640 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c9373e97d0 0x55c9373ebbc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4abf7bcd17b50387 server_cookie=27fe192396511b15 in_seq=0 out_seq=0 2024-04-20T10:37:24.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.188+0000 7fc95ea00640 10 client.25802 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:24.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc95ea00640 1 -- 172.21.15.42:0/3371469993 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c937342740 con 0x55c9373e97d0 2024-04-20T10:37:24.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc95ea00640 10 client.25802 handle_command_reply: tid=0 2024-04-20T10:37:24.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc9827a7740 1 client.25802 shutdown 2024-04-20T10:37:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc9827a7740 1 -- 172.21.15.42:0/3371469993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 msgr2=0x55c9373d6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc9827a7740 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 0x55c9373d6b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc964009430 tx=0x7fc964004ec0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc9827a7740 1 -- 172.21.15.42:0/3371469993 shutdown_connections 2024-04-20T10:37:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc9827a7740 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c9373e97d0 0x55c9373ebbc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc9827a7740 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d95f0 0x55c9373d70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc9827a7740 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 0x55c9373d6b90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc9827a7740 1 --2- 172.21.15.42:0/3371469993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d00f0 0x55c9373d6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc9827a7740 1 -- 172.21.15.42:0/3371469993 >> 172.21.15.42:0/3371469993 conn(0x55c9373ca820 msgr2=0x55c9373cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc95ea00640 0 client.25802 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc9827a7740 1 -- 172.21.15.42:0/3371469993 shutdown_connections 2024-04-20T10:37:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.189+0000 7fc9827a7740 1 -- 172.21.15.42:0/3371469993 wait complete. 2024-04-20T10:37:24.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.190+0000 7fc9827a7740 20 client.25802 trim_cache size 0 max 16384 2024-04-20T10:37:24.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.212+0000 7fc9827a7740 1 Processor -- start 2024-04-20T10:37:24.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.212+0000 7fc9827a7740 1 -- start start 2024-04-20T10:37:24.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.212+0000 7fc9827a7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d00f0 0x55c9373d8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.212+0000 7fc9827a7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 0x55c9373e5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.212+0000 7fc9827a7740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d95f0 0x55c9373f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.212+0000 7fc9827a7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781b2090 con 0x55c9373d00f0 2024-04-20T10:37:24.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.212+0000 7fc9827a7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781b4910 con 0x55c9373d0a10 2024-04-20T10:37:24.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.212+0000 7fc9827a7740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781b2700 con 0x55c9373d95f0 2024-04-20T10:37:24.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.213+0000 7fc97d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d00f0 0x55c9373d8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.212+0000 7fc977e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 0x55c9373e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.213+0000 7fc97d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d00f0 0x55c9373d8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55788/0 (socket says 172.21.15.42:55788) 2024-04-20T10:37:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.213+0000 7fc97d000640 1 -- 172.21.15.42:0/2811550297 learned_addr learned my addr 172.21.15.42:0/2811550297 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.213+0000 7fc97da00640 1 --2- 172.21.15.42:0/2811550297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d95f0 0x55c9373f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.213+0000 7fc97d000640 1 -- 172.21.15.42:0/2811550297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d95f0 msgr2=0x55c9373f1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.213+0000 7fc97d000640 1 --2- 172.21.15.42:0/2811550297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d95f0 0x55c9373f1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.213+0000 7fc97d000640 1 -- 172.21.15.42:0/2811550297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 msgr2=0x55c9373e5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.213+0000 7fc97d000640 1 --2- 172.21.15.42:0/2811550297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 0x55c9373e5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.213+0000 7fc97d000640 1 -- 172.21.15.42:0/2811550297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c937007170 con 0x55c9373d00f0 2024-04-20T10:37:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.213+0000 7fc977e00640 1 --2- 172.21.15.42:0/2811550297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 0x55c9373e5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.213+0000 7fc97d000640 1 --2- 172.21.15.42:0/2811550297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d00f0 0x55c9373d8d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0d4400 tx=0x7fc96c0d3020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624085f7eb360d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:24.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.214+0000 7fc969600640 1 -- 172.21.15.42:0/2811550297 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0d5040 con 0x55c9373d00f0 2024-04-20T10:37:24.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.214+0000 7fc969600640 1 -- 172.21.15.42:0/2811550297 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96c09cb60 con 0x55c9373d00f0 2024-04-20T10:37:24.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.214+0000 7fc969600640 1 -- 172.21.15.42:0/2811550297 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0ce4e0 con 0x55c9373d00f0 2024-04-20T10:37:24.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.214+0000 7fc9827a7740 1 -- 172.21.15.42:0/2811550297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d00f0 msgr2=0x55c9373d8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.214+0000 7fc9827a7740 1 --2- 172.21.15.42:0/2811550297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d00f0 0x55c9373d8d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0d4400 tx=0x7fc96c0d3020 comp rx=0 tx=0).stop 2024-04-20T10:37:24.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.215+0000 7fc9827a7740 1 -- 172.21.15.42:0/2811550297 shutdown_connections 2024-04-20T10:37:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.215+0000 7fc9827a7740 1 --2- 172.21.15.42:0/2811550297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d95f0 0x55c9373f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.215+0000 7fc9827a7740 1 --2- 172.21.15.42:0/2811550297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d0a10 0x55c9373e5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.215+0000 7fc9827a7740 1 --2- 172.21.15.42:0/2811550297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d00f0 0x55c9373d8d80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.215+0000 7fc9827a7740 1 -- 172.21.15.42:0/2811550297 >> 172.21.15.42:0/2811550297 conn(0x55c9373ca820 msgr2=0x55c9373c9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.215+0000 7fc9827a7740 1 -- 172.21.15.42:0/2811550297 shutdown_connections 2024-04-20T10:37:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.215+0000 7fc9827a7740 1 -- 172.21.15.42:0/2811550297 wait complete. 2024-04-20T10:37:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.215+0000 7fc9827a7740 1 -- start start 2024-04-20T10:37:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.216+0000 7fc9827a7740 1 Processor -- start 2024-04-20T10:37:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.216+0000 7fc9827a7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d00f0 0x55c937447d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.216+0000 7fc9827a7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d0a10 0x55c937448240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.216+0000 7fc9827a7740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d95f0 0x55c9373ddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.216+0000 7fc9827a7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781b2dd0 con 0x55c9373d0a10 2024-04-20T10:37:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.216+0000 7fc9827a7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781b4240 con 0x55c9373d00f0 2024-04-20T10:37:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.216+0000 7fc9827a7740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781b34a0 con 0x55c9373d95f0 2024-04-20T10:37:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.216+0000 7fc9827a7740 10 client.0 _ll_register_callbacks cb 0x7fc96c0d4530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.216+0000 7fc97d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d00f0 0x55c937447d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.216+0000 7fc97d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d00f0 0x55c937447d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49726/0 (socket says 172.21.15.42:49726) 2024-04-20T10:37:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.216+0000 7fc97d000640 1 -- 172.21.15.42:0/1118872739 learned_addr learned my addr 172.21.15.42:0/1118872739 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.216+0000 7fc97da00640 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d95f0 0x55c9373ddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc977e00640 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d0a10 0x55c937448240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc97d000640 1 -- 172.21.15.42:0/1118872739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d95f0 msgr2=0x55c9373ddaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc97d000640 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d95f0 0x55c9373ddaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc97d000640 1 -- 172.21.15.42:0/1118872739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d0a10 msgr2=0x55c937448240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc97d000640 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d0a10 0x55c937448240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc97d000640 1 -- 172.21.15.42:0/1118872739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9373de190 con 0x55c9373d00f0 2024-04-20T10:37:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc977e00640 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d0a10 0x55c937448240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:37:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc97d000640 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d00f0 0x55c937447d00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0a51f0 tx=0x7fc96c0d44b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5deef94470a4474 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc95ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:37:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc95ea00640 1 -- 172.21.15.42:0/1118872739 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0b5070 con 0x55c9373d00f0 2024-04-20T10:37:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc95ea00640 1 -- 172.21.15.42:0/1118872739 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc96c09ae60 con 0x55c9373d00f0 2024-04-20T10:37:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc9827a7740 1 -- 172.21.15.42:0/1118872739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc96c09cb60 con 0x55c9373d00f0 2024-04-20T10:37:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc95ea00640 1 -- 172.21.15.42:0/1118872739 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc96c09c6a0 con 0x55c9373d00f0 2024-04-20T10:37:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc9827a7740 10 client.25808 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:24.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.217+0000 7fc9827a7740 1 -- 172.21.15.42:0/1118872739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c937450610 con 0x55c9373d00f0 2024-04-20T10:37:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.218+0000 7fc95ea00640 1 -- 172.21.15.42:0/1118872739 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96c09cb60 con 0x55c9373d00f0 2024-04-20T10:37:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.218+0000 7fc95ea00640 1 -- 172.21.15.42:0/1118872739 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc96c09c140 con 0x55c9373d00f0 2024-04-20T10:37:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.218+0000 7fc9827a7740 10 client.25808 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.218+0000 7fc9827a7740 10 client.25808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.218+0000 7fc9827a7740 20 client.25808 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.218+0000 7fc9827a7740 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c937450cd0 0x55c9374530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.218+0000 7fc9827a7740 4 client.25808 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:37:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.218+0000 7fc9827a7740 1 -- 172.21.15.42:0/1118872739 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c93733c5b0 con 0x55c937450cd0 2024-04-20T10:37:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.218+0000 7fc977e00640 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c937450cd0 0x55c9374530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.218+0000 7fc977e00640 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c937450cd0 0x55c9374530f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a586cff6d565f7bb server_cookie=26974c1413aae4aa in_seq=0 out_seq=0 2024-04-20T10:37:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.218+0000 7fc95ea00640 10 client.25808 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc95ea00640 1 -- 172.21.15.42:0/1118872739 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c93733c5b0 con 0x55c937450cd0 2024-04-20T10:37:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc95ea00640 10 client.25808 handle_command_reply: tid=0 2024-04-20T10:37:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc9827a7740 1 client.25808 shutdown 2024-04-20T10:37:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc9827a7740 1 -- 172.21.15.42:0/1118872739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d00f0 msgr2=0x55c937447d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc9827a7740 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d00f0 0x55c937447d00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0a51f0 tx=0x7fc96c0d44b0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc9827a7740 1 -- 172.21.15.42:0/1118872739 shutdown_connections 2024-04-20T10:37:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc9827a7740 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c937450cd0 0x55c9374530f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc9827a7740 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c9373d95f0 0x55c9373ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc9827a7740 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9373d0a10 0x55c937448240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc9827a7740 1 --2- 172.21.15.42:0/1118872739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9373d00f0 0x55c937447d00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc95ea00640 0 client.25808 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc9827a7740 1 -- 172.21.15.42:0/1118872739 >> 172.21.15.42:0/1118872739 conn(0x55c9373ca820 msgr2=0x55c9373cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:24.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc9827a7740 1 -- 172.21.15.42:0/1118872739 shutdown_connections 2024-04-20T10:37:24.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc9827a7740 1 -- 172.21.15.42:0/1118872739 wait complete. 2024-04-20T10:37:24.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.221+0000 7fc9827a7740 20 client.25808 trim_cache size 0 max 16384 2024-04-20T10:37:24.226 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:37:24.226 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:37:24.226 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "8c604d9f-b430-4968-ba98-d4bffb506417", 2024-04-20T10:37:24.226 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:37:24.227 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:37:24.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.222+0000 7fc97e400640 1 -- 172.21.15.42:0/1700747866 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc9600a3d20 msgr2=0x7fc9600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.222+0000 7fc97e400640 1 --2- 172.21.15.42:0/1700747866 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc9600a3d20 0x7fc9600a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc970005fd0 tx=0x7fc970009550 comp rx=0 tx=0).stop 2024-04-20T10:37:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.222+0000 7fc97e400640 1 -- 172.21.15.42:0/1700747866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc978108500 msgr2=0x7fc9781a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.222+0000 7fc97e400640 1 --2- 172.21.15.42:0/1700747866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc978108500 0x7fc9781a4a60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00e9f0 tx=0x7fc96c00eeb0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.222+0000 7fc97e400640 1 -- 172.21.15.42:0/1700747866 shutdown_connections 2024-04-20T10:37:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.222+0000 7fc97e400640 1 --2- 172.21.15.42:0/1700747866 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc9600a3d20 0x7fc9600a61e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.222+0000 7fc97e400640 1 --2- 172.21.15.42:0/1700747866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc97810d3d0 0x7fc97819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.222+0000 7fc97e400640 1 --2- 172.21.15.42:0/1700747866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc978108ea0 0x7fc9781a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.222+0000 7fc97e400640 1 --2- 172.21.15.42:0/1700747866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc978108500 0x7fc9781a4a60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.222+0000 7fc97e400640 1 -- 172.21.15.42:0/1700747866 >> 172.21.15.42:0/1700747866 conn(0x7fc978077d80 msgr2=0x7fc9780783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.222+0000 7fc97e400640 1 -- 172.21.15.42:0/1700747866 shutdown_connections 2024-04-20T10:37:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.222+0000 7fc97e400640 1 -- 172.21.15.42:0/1700747866 wait complete. 2024-04-20T10:37:24.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:37:24.243 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:37:24.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.452+0000 7fc290000640 1 -- 172.21.15.42:0/252867781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc28806a540 msgr2=0x7fc28806a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.452+0000 7fc290000640 1 --2- 172.21.15.42:0/252867781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc28806a540 0x7fc28806a9a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc274009930 tx=0x7fc27402bc20 comp rx=0 tx=0).stop 2024-04-20T10:37:24.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.452+0000 7fc290000640 1 -- 172.21.15.42:0/252867781 shutdown_connections 2024-04-20T10:37:24.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.452+0000 7fc290000640 1 --2- 172.21.15.42:0/252867781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806aee0 0x7fc28810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.452+0000 7fc290000640 1 --2- 172.21.15.42:0/252867781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc28806a540 0x7fc28806a9a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.452+0000 7fc290000640 1 --2- 172.21.15.42:0/252867781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc288107fd0 0x7fc2881083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.452+0000 7fc290000640 1 -- 172.21.15.42:0/252867781 >> 172.21.15.42:0/252867781 conn(0x7fc288076f70 msgr2=0x7fc288077380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:24.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.452+0000 7fc290000640 1 -- 172.21.15.42:0/252867781 shutdown_connections 2024-04-20T10:37:24.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.452+0000 7fc290000640 1 -- 172.21.15.42:0/252867781 wait complete. 2024-04-20T10:37:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.453+0000 7fc290000640 1 Processor -- start 2024-04-20T10:37:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.453+0000 7fc290000640 1 -- start start 2024-04-20T10:37:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc290000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc28806a540 0x7fc28819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc290000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806aee0 0x7fc28819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc290000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc288107fd0 0x7fc288198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc290000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc2881122d0 con 0x7fc288107fd0 2024-04-20T10:37:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc290000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc288112150 con 0x7fc28806aee0 2024-04-20T10:37:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc290000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc288112450 con 0x7fc28806a540 2024-04-20T10:37:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc28e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc288107fd0 0x7fc288198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc28cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806aee0 0x7fc28819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc28cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806aee0 0x7fc28819eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49746/0 (socket says 172.21.15.42:49746) 2024-04-20T10:37:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc28cc00640 1 -- 172.21.15.42:0/1340110237 learned_addr learned my addr 172.21.15.42:0/1340110237 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc28d600640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc28806a540 0x7fc28819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc28cc00640 1 -- 172.21.15.42:0/1340110237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc28806a540 msgr2=0x7fc28819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc28cc00640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc28806a540 0x7fc28819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc28cc00640 1 -- 172.21.15.42:0/1340110237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc288107fd0 msgr2=0x7fc288198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.454+0000 7fc28cc00640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc288107fd0 0x7fc288198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.455+0000 7fc28cc00640 1 -- 172.21.15.42:0/1340110237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc288198ef0 con 0x7fc28806aee0 2024-04-20T10:37:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.455+0000 7fc28cc00640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806aee0 0x7fc28819eab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc274009430 tx=0x7fc27402cb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.455+0000 7fc286000640 1 -- 172.21.15.42:0/1340110237 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274041070 con 0x7fc28806aee0 2024-04-20T10:37:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.455+0000 7fc286000640 1 -- 172.21.15.42:0/1340110237 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc274004450 con 0x7fc28806aee0 2024-04-20T10:37:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.455+0000 7fc290000640 1 -- 172.21.15.42:0/1340110237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc288199180 con 0x7fc28806aee0 2024-04-20T10:37:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.455+0000 7fc286000640 1 -- 172.21.15.42:0/1340110237 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274003d00 con 0x7fc28806aee0 2024-04-20T10:37:24.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.455+0000 7fc290000640 1 -- 172.21.15.42:0/1340110237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2881a5310 con 0x7fc28806aee0 2024-04-20T10:37:24.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.459+0000 7fc292db6740 1 Processor -- start 2024-04-20T10:37:24.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.459+0000 7fc286000640 1 -- 172.21.15.42:0/1340110237 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc274044070 con 0x7fc28806aee0 2024-04-20T10:37:24.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.459+0000 7fc292db6740 1 -- start start 2024-04-20T10:37:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.459+0000 7fc292db6740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5611879115b0 0x561187911990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.459+0000 7fc292db6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561187911ed0 0x56118791a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.459+0000 7fc292db6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56118791aab0 0x56118791cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.459+0000 7fc292db6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561187873150 con 0x56118791aab0 2024-04-20T10:37:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.459+0000 7fc292db6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561187819370 con 0x561187911ed0 2024-04-20T10:37:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.459+0000 7fc292db6740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56118786f110 con 0x5611879115b0 2024-04-20T10:37:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.460+0000 7fc28d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561187911ed0 0x56118791a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.460+0000 7fc28d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561187911ed0 0x56118791a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49754/0 (socket says 172.21.15.42:49754) 2024-04-20T10:37:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.460+0000 7fc28d600640 1 -- 172.21.15.42:0/1156646793 learned_addr learned my addr 172.21.15.42:0/1156646793 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.460+0000 7fc28e000640 1 --2- 172.21.15.42:0/1156646793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5611879115b0 0x561187911990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.460+0000 7fc28cc00640 1 --2- 172.21.15.42:0/1156646793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56118791aab0 0x56118791cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.460+0000 7fc28d600640 1 -- 172.21.15.42:0/1156646793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5611879115b0 msgr2=0x561187911990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.460+0000 7fc28d600640 1 --2- 172.21.15.42:0/1156646793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5611879115b0 0x561187911990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.460+0000 7fc28d600640 1 -- 172.21.15.42:0/1156646793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56118791aab0 msgr2=0x56118791cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.460+0000 7fc28d600640 1 --2- 172.21.15.42:0/1156646793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56118791aab0 0x56118791cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.460+0000 7fc28d600640 1 -- 172.21.15.42:0/1156646793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118764f7d0 con 0x561187911ed0 2024-04-20T10:37:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.460+0000 7fc28d600640 1 --2- 172.21.15.42:0/1156646793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561187911ed0 0x56118791a570 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009d20 tx=0x7fc27c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=877a3018aff0c717 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.461+0000 7fc27a600640 1 -- 172.21.15.42:0/1156646793 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c01e070 con 0x561187911ed0 2024-04-20T10:37:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.461+0000 7fc27a600640 1 -- 172.21.15.42:0/1156646793 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc27c019b10 con 0x561187911ed0 2024-04-20T10:37:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.461+0000 7fc27a600640 1 -- 172.21.15.42:0/1156646793 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c00ebf0 con 0x561187911ed0 2024-04-20T10:37:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.461+0000 7fc286000640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc2700a3d20 0x7fc2700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.461+0000 7fc286000640 1 -- 172.21.15.42:0/1340110237 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc2740f2110 con 0x7fc28806aee0 2024-04-20T10:37:24.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.461+0000 7fc28e000640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc2700a3d20 0x7fc2700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.462+0000 7fc292db6740 1 -- 172.21.15.42:0/1156646793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561187911ed0 msgr2=0x56118791a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.462+0000 7fc292db6740 1 --2- 172.21.15.42:0/1156646793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561187911ed0 0x56118791a570 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009d20 tx=0x7fc27c011810 comp rx=0 tx=0).stop 2024-04-20T10:37:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.462+0000 7fc292db6740 1 -- 172.21.15.42:0/1156646793 shutdown_connections 2024-04-20T10:37:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.462+0000 7fc292db6740 1 --2- 172.21.15.42:0/1156646793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56118791aab0 0x56118791cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.462+0000 7fc292db6740 1 --2- 172.21.15.42:0/1156646793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561187911ed0 0x56118791a570 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.462+0000 7fc292db6740 1 --2- 172.21.15.42:0/1156646793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5611879115b0 0x561187911990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.462+0000 7fc292db6740 1 -- 172.21.15.42:0/1156646793 >> 172.21.15.42:0/1156646793 conn(0x56118790bce0 msgr2=0x56118790c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.462+0000 7fc292db6740 1 -- 172.21.15.42:0/1156646793 shutdown_connections 2024-04-20T10:37:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.462+0000 7fc292db6740 1 -- 172.21.15.42:0/1156646793 wait complete. 2024-04-20T10:37:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.462+0000 7fc28e000640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc2700a3d20 0x7fc2700a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc280005fd0 tx=0x7fc280009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.462+0000 7fc292db6740 1 -- start start 2024-04-20T10:37:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.463+0000 7fc292db6740 1 Processor -- start 2024-04-20T10:37:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.463+0000 7fc292db6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 0x561187917af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.463+0000 7fc292db6740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561187911ed0 0x561187918030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.463+0000 7fc292db6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 0x561187918570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.463+0000 7fc292db6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56118791d8a0 con 0x5611879115b0 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.463+0000 7fc292db6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56118782c740 con 0x56118791aab0 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.463+0000 7fc292db6740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56118791da20 con 0x561187911ed0 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.463+0000 7fc292db6740 10 client.0 _ll_register_callbacks cb 0x561187827b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc28d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 0x561187917af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc28e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 0x561187918570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc28e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 0x561187918570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49764/0 (socket says 172.21.15.42:49764) 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc28d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 0x561187917af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55840/0 (socket says 172.21.15.42:55840) 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc28e000640 1 -- 172.21.15.42:0/2913141188 learned_addr learned my addr 172.21.15.42:0/2913141188 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc28cc00640 1 --2- 172.21.15.42:0/2913141188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561187911ed0 0x561187918030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc28e000640 1 -- 172.21.15.42:0/2913141188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561187911ed0 msgr2=0x561187918030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc28e000640 1 --2- 172.21.15.42:0/2913141188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561187911ed0 0x561187918030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc28e000640 1 -- 172.21.15.42:0/2913141188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 msgr2=0x561187917af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc28e000640 1 --2- 172.21.15.42:0/2913141188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 0x561187917af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc28e000640 1 -- 172.21.15.42:0/2913141188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611875853f0 con 0x56118791aab0 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc28e000640 1 --2- 172.21.15.42:0/2913141188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 0x561187918570 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc280005f40 tx=0x7fc280004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d1563c59a74cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.464+0000 7fc26ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.465+0000 7fc26ea00640 1 -- 172.21.15.42:0/2913141188 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28000f810 con 0x56118791aab0 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.465+0000 7fc26ea00640 1 -- 172.21.15.42:0/2913141188 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc28000f9b0 con 0x56118791aab0 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.465+0000 7fc292db6740 1 -- 172.21.15.42:0/2913141188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc27c019b10 con 0x56118791aab0 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.465+0000 7fc26ea00640 1 -- 172.21.15.42:0/2913141188 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc28000fbb0 con 0x56118791aab0 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.465+0000 7fc292db6740 10 client.25820 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.465+0000 7fc292db6740 1 -- 172.21.15.42:0/2913141188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5611875450f0 con 0x56118791aab0 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.465+0000 7fc26ea00640 1 -- 172.21.15.42:0/2913141188 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc27c019b10 con 0x56118791aab0 2024-04-20T10:37:24.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.466+0000 7fc26ea00640 1 -- 172.21.15.42:0/2913141188 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc280027070 con 0x56118791aab0 2024-04-20T10:37:24.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.466+0000 7fc292db6740 10 client.25820 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:24.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.466+0000 7fc292db6740 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:24.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.466+0000 7fc292db6740 20 client.25820 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:24.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.466+0000 7fc292db6740 1 --2- 172.21.15.42:0/2913141188 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56118792af00 0x56118792d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.466+0000 7fc292db6740 4 client.25820 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:37:24.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.466+0000 7fc292db6740 1 -- 172.21.15.42:0/2913141188 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561187880b70 con 0x56118792af00 2024-04-20T10:37:24.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.466+0000 7fc28d600640 1 --2- 172.21.15.42:0/2913141188 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56118792af00 0x56118792d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.467+0000 7fc28d600640 1 --2- 172.21.15.42:0/2913141188 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56118792af00 0x56118792d380 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72bf69ad1e69d0f4 server_cookie=3d8973469fb969f2 in_seq=0 out_seq=0 2024-04-20T10:37:24.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.467+0000 7fc26ea00640 10 client.25820 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:24.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.469+0000 7fc26ea00640 1 -- 172.21.15.42:0/2913141188 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561187880b70 con 0x56118792af00 2024-04-20T10:37:24.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.469+0000 7fc26ea00640 10 client.25820 handle_command_reply: tid=0 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc292db6740 1 client.25820 shutdown 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc292db6740 1 -- 172.21.15.42:0/2913141188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 msgr2=0x561187918570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc292db6740 1 --2- 172.21.15.42:0/2913141188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 0x561187918570 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc280005f40 tx=0x7fc280004ab0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc292db6740 1 -- 172.21.15.42:0/2913141188 shutdown_connections 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc292db6740 1 --2- 172.21.15.42:0/2913141188 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56118792af00 0x56118792d380 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc292db6740 1 --2- 172.21.15.42:0/2913141188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 0x561187918570 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc26ea00640 0 client.25820 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc292db6740 1 --2- 172.21.15.42:0/2913141188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561187911ed0 0x561187918030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc292db6740 1 --2- 172.21.15.42:0/2913141188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 0x561187917af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc292db6740 1 -- 172.21.15.42:0/2913141188 >> 172.21.15.42:0/2913141188 conn(0x56118790bce0 msgr2=0x56118790da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc292db6740 1 -- 172.21.15.42:0/2913141188 shutdown_connections 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc292db6740 1 -- 172.21.15.42:0/2913141188 wait complete. 2024-04-20T10:37:24.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.470+0000 7fc292db6740 20 client.25820 trim_cache size 0 max 16384 2024-04-20T10:37:24.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.491+0000 7fc292db6740 1 Processor -- start 2024-04-20T10:37:24.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.491+0000 7fc292db6740 1 -- start start 2024-04-20T10:37:24.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.491+0000 7fc292db6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611879115b0 0x56118791a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc292db6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561187911ed0 0x561187916a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc292db6740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56118791aab0 0x561187912dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc292db6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc2881a9ae0 con 0x561187911ed0 2024-04-20T10:37:24.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc292db6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2881ac360 con 0x5611879115b0 2024-04-20T10:37:24.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc292db6740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc2881aa150 con 0x56118791aab0 2024-04-20T10:37:24.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc28d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611879115b0 0x56118791a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc28cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561187911ed0 0x561187916a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc28e000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56118791aab0 0x561187912dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc28d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611879115b0 0x56118791a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49766/0 (socket says 172.21.15.42:49766) 2024-04-20T10:37:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc28cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561187911ed0 0x561187916a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55854/0 (socket says 172.21.15.42:55854) 2024-04-20T10:37:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc28d600640 1 -- 172.21.15.42:0/815801882 learned_addr learned my addr 172.21.15.42:0/815801882 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc28e000640 1 -- 172.21.15.42:0/815801882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611879115b0 msgr2=0x56118791a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc28e000640 1 --2- 172.21.15.42:0/815801882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611879115b0 0x56118791a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc28e000640 1 -- 172.21.15.42:0/815801882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561187911ed0 msgr2=0x561187916a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc28e000640 1 --2- 172.21.15.42:0/815801882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561187911ed0 0x561187916a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.492+0000 7fc28e000640 1 -- 172.21.15.42:0/815801882 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561187910930 con 0x56118791aab0 2024-04-20T10:37:24.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.493+0000 7fc28e000640 1 --2- 172.21.15.42:0/815801882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56118791aab0 0x561187912dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc280029790 tx=0x7fc280009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ce94ce33dca0a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:24.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.493+0000 7fc279c00640 1 -- 172.21.15.42:0/815801882 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28000fd50 con 0x56118791aab0 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.493+0000 7fc279c00640 1 -- 172.21.15.42:0/815801882 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc280029810 con 0x56118791aab0 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.493+0000 7fc279c00640 1 -- 172.21.15.42:0/815801882 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280012030 con 0x56118791aab0 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.494+0000 7fc292db6740 1 -- 172.21.15.42:0/815801882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56118791aab0 msgr2=0x561187912dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.494+0000 7fc292db6740 1 --2- 172.21.15.42:0/815801882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56118791aab0 0x561187912dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc280029790 tx=0x7fc280009610 comp rx=0 tx=0).stop 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.494+0000 7fc292db6740 1 -- 172.21.15.42:0/815801882 shutdown_connections 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.494+0000 7fc292db6740 1 --2- 172.21.15.42:0/815801882 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56118791aab0 0x561187912dd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.494+0000 7fc292db6740 1 --2- 172.21.15.42:0/815801882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561187911ed0 0x561187916a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.494+0000 7fc292db6740 1 --2- 172.21.15.42:0/815801882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611879115b0 0x56118791a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.494+0000 7fc292db6740 1 -- 172.21.15.42:0/815801882 >> 172.21.15.42:0/815801882 conn(0x56118790bce0 msgr2=0x561187918340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.494+0000 7fc292db6740 1 -- 172.21.15.42:0/815801882 shutdown_connections 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.494+0000 7fc292db6740 1 -- 172.21.15.42:0/815801882 wait complete. 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.495+0000 7fc292db6740 1 -- start start 2024-04-20T10:37:24.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.495+0000 7fc292db6740 1 Processor -- start 2024-04-20T10:37:24.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc292db6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 0x56118791e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc292db6740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561187911ed0 0x56118791eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc292db6740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 0x56118791f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc292db6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc2881aa820 con 0x5611879115b0 2024-04-20T10:37:24.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc292db6740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc2881abc90 con 0x56118791aab0 2024-04-20T10:37:24.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc292db6740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc2881aaef0 con 0x561187911ed0 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc292db6740 10 client.0 _ll_register_callbacks cb 0x7fc280009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc28d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 0x56118791e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc28e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 0x56118791f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc28d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 0x56118791e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55864/0 (socket says 172.21.15.42:55864) 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc28cc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561187911ed0 0x56118791eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc28d600640 1 -- 172.21.15.42:0/2400667391 learned_addr learned my addr 172.21.15.42:0/2400667391 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc28e000640 1 -- 172.21.15.42:0/2400667391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561187911ed0 msgr2=0x56118791eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc28e000640 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561187911ed0 0x56118791eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc28e000640 1 -- 172.21.15.42:0/2400667391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 msgr2=0x56118791e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc28e000640 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 0x56118791e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc28e000640 1 -- 172.21.15.42:0/2400667391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561187990760 con 0x56118791aab0 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc28cc00640 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561187911ed0 0x56118791eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:24.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.496+0000 7fc28d600640 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 0x56118791e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.497+0000 7fc28e000640 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 0x56118791f400 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc2800295a0 tx=0x7fc280029e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1e8cba2ed81b7bd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.497+0000 7fc26ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.497+0000 7fc26ea00640 1 -- 172.21.15.42:0/2400667391 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28000f810 con 0x56118791aab0 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.497+0000 7fc292db6740 1 -- 172.21.15.42:0/2400667391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc280029810 con 0x56118791aab0 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.497+0000 7fc26ea00640 1 -- 172.21.15.42:0/2400667391 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc28000f9b0 con 0x56118791aab0 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.497+0000 7fc26ea00640 1 -- 172.21.15.42:0/2400667391 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc280049050 con 0x56118791aab0 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.497+0000 7fc292db6740 10 client.25826 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.497+0000 7fc292db6740 1 -- 172.21.15.42:0/2400667391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561187991a70 con 0x56118791aab0 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.498+0000 7fc26ea00640 1 -- 172.21.15.42:0/2400667391 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc280029810 con 0x56118791aab0 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.498+0000 7fc26ea00640 1 -- 172.21.15.42:0/2400667391 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc280027040 con 0x56118791aab0 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.498+0000 7fc292db6740 10 client.25826 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.498+0000 7fc292db6740 10 client.25826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:24.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.498+0000 7fc292db6740 20 client.25826 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:24.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.498+0000 7fc292db6740 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56118792af00 0x561187994080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:24.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.498+0000 7fc292db6740 4 client.25826 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:37:24.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.498+0000 7fc292db6740 1 -- 172.21.15.42:0/2400667391 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56118787a9e0 con 0x56118792af00 2024-04-20T10:37:24.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.498+0000 7fc28d600640 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56118792af00 0x561187994080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:24.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.499+0000 7fc28d600640 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56118792af00 0x561187994080 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8108fe2de6ad118d server_cookie=1d3852314457766f in_seq=0 out_seq=0 2024-04-20T10:37:24.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.499+0000 7fc26ea00640 10 client.25826 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:24.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.502+0000 7fc26ea00640 1 -- 172.21.15.42:0/2400667391 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x56118787a9e0 con 0x56118792af00 2024-04-20T10:37:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.502+0000 7fc26ea00640 10 client.25826 handle_command_reply: tid=0 2024-04-20T10:37:24.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.502+0000 7fc292db6740 1 client.25826 shutdown 2024-04-20T10:37:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.503+0000 7fc292db6740 1 -- 172.21.15.42:0/2400667391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 msgr2=0x56118791f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.503+0000 7fc292db6740 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 0x56118791f400 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc2800295a0 tx=0x7fc280029e20 comp rx=0 tx=0).stop 2024-04-20T10:37:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.503+0000 7fc292db6740 1 -- 172.21.15.42:0/2400667391 shutdown_connections 2024-04-20T10:37:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.503+0000 7fc292db6740 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56118792af00 0x561187994080 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.503+0000 7fc292db6740 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118791aab0 0x56118791f400 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.503+0000 7fc292db6740 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561187911ed0 0x56118791eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.503+0000 7fc292db6740 1 --2- 172.21.15.42:0/2400667391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5611879115b0 0x56118791e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.503+0000 7fc26ea00640 0 client.25826 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.503+0000 7fc292db6740 1 -- 172.21.15.42:0/2400667391 >> 172.21.15.42:0/2400667391 conn(0x56118790bce0 msgr2=0x561187906030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.503+0000 7fc292db6740 1 -- 172.21.15.42:0/2400667391 shutdown_connections 2024-04-20T10:37:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.503+0000 7fc292db6740 1 -- 172.21.15.42:0/2400667391 wait complete. 2024-04-20T10:37:24.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.503+0000 7fc292db6740 20 client.25826 trim_cache size 0 max 16384 2024-04-20T10:37:24.505 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:37:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (1370 inodes in the stack)", 2024-04-20T10:37:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:37:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "8c604d9f-b430-4968-ba98-d4bffb506417": { 2024-04-20T10:37:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:37:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "8c604d9f-b430-4968-ba98-d4bffb506417", 2024-04-20T10:37:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:37:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:37:24.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:37:24.505 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:37:24.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.504+0000 7fc290000640 1 -- 172.21.15.42:0/1340110237 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc2700a3d20 msgr2=0x7fc2700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.504+0000 7fc290000640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc2700a3d20 0x7fc2700a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc280005fd0 tx=0x7fc280009500 comp rx=0 tx=0).stop 2024-04-20T10:37:24.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.504+0000 7fc290000640 1 -- 172.21.15.42:0/1340110237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806aee0 msgr2=0x7fc28819eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:24.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.504+0000 7fc290000640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806aee0 0x7fc28819eab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc274009430 tx=0x7fc27402cb10 comp rx=0 tx=0).stop 2024-04-20T10:37:24.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.504+0000 7fc290000640 1 -- 172.21.15.42:0/1340110237 shutdown_connections 2024-04-20T10:37:24.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.504+0000 7fc290000640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc2700a3d20 0x7fc2700a61e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.504+0000 7fc290000640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc288107fd0 0x7fc288198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.504+0000 7fc290000640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806aee0 0x7fc28819eab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.504+0000 7fc290000640 1 --2- 172.21.15.42:0/1340110237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc28806a540 0x7fc28819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:24.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.504+0000 7fc290000640 1 -- 172.21.15.42:0/1340110237 >> 172.21.15.42:0/1340110237 conn(0x7fc288076f70 msgr2=0x7fc2880ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:24.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.504+0000 7fc290000640 1 -- 172.21.15.42:0/1340110237 shutdown_connections 2024-04-20T10:37:24.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:24.504+0000 7fc290000640 1 -- 172.21.15.42:0/1340110237 wait complete. 2024-04-20T10:37:24.523 INFO:tasks.cephfs.filesystem:scrub status for tag:8c604d9f-b430-4968-ba98-d4bffb506417 - {'path': '/', 'tag': '8c604d9f-b430-4968-ba98-d4bffb506417', 'options': 'recursive,force'} 2024-04-20T10:37:24.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:24 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:37:24.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:24 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:37:24.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:24 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:37:24.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:24 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:37:24.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:24 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:37:24.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:24 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:37:24.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:24 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:37:24.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:24 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:37:24.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:24 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:37:25.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:25 smithi046 ceph-mon[21574]: pgmap v105: 97 pgs: 97 active+clean; 333 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 706 KiB/s wr, 37 op/s 2024-04-20T10:37:25.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:25 smithi042 ceph-mon[17078]: pgmap v105: 97 pgs: 97 active+clean; 333 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 706 KiB/s wr, 37 op/s 2024-04-20T10:37:25.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:25 smithi162 ceph-mon[21854]: pgmap v105: 97 pgs: 97 active+clean; 333 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 706 KiB/s wr, 37 op/s 2024-04-20T10:37:27.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:27 smithi046 ceph-mon[21574]: pgmap v106: 97 pgs: 97 active+clean; 333 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 706 KiB/s wr, 37 op/s 2024-04-20T10:37:27.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:27 smithi042 ceph-mon[17078]: pgmap v106: 97 pgs: 97 active+clean; 333 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 706 KiB/s wr, 37 op/s 2024-04-20T10:37:27.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:27 smithi162 ceph-mon[21854]: pgmap v106: 97 pgs: 97 active+clean; 333 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 706 KiB/s wr, 37 op/s 2024-04-20T10:37:29.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:29 smithi046 ceph-mon[21574]: pgmap v107: 97 pgs: 97 active+clean; 338 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.1 MiB/s wr, 39 op/s 2024-04-20T10:37:29.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:29 smithi042 ceph-mon[17078]: pgmap v107: 97 pgs: 97 active+clean; 338 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.1 MiB/s wr, 39 op/s 2024-04-20T10:37:29.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:29 smithi162 ceph-mon[21854]: pgmap v107: 97 pgs: 97 active+clean; 338 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.1 MiB/s wr, 39 op/s 2024-04-20T10:37:31.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:31 smithi046 ceph-mon[21574]: pgmap v108: 97 pgs: 97 active+clean; 338 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 612 KiB/s wr, 23 op/s 2024-04-20T10:37:31.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:31 smithi042 ceph-mon[17078]: pgmap v108: 97 pgs: 97 active+clean; 338 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 612 KiB/s wr, 23 op/s 2024-04-20T10:37:31.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:31 smithi162 ceph-mon[21854]: pgmap v108: 97 pgs: 97 active+clean; 338 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 612 KiB/s wr, 23 op/s 2024-04-20T10:37:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:33 smithi046 ceph-mon[21574]: pgmap v109: 97 pgs: 97 active+clean; 352 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 144 op/s 2024-04-20T10:37:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:33 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:37:33.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:33 smithi042 ceph-mon[17078]: pgmap v109: 97 pgs: 97 active+clean; 352 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 144 op/s 2024-04-20T10:37:33.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:33 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:37:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:33 smithi162 ceph-mon[21854]: pgmap v109: 97 pgs: 97 active+clean; 352 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 144 op/s 2024-04-20T10:37:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:33 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:37:35.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:35 smithi046 ceph-mon[21574]: pgmap v110: 97 pgs: 97 active+clean; 352 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-04-20T10:37:35.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:35 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:37:35.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:35 smithi042 ceph-mon[17078]: pgmap v110: 97 pgs: 97 active+clean; 352 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-04-20T10:37:35.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:35 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:37:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:35 smithi162 ceph-mon[21854]: pgmap v110: 97 pgs: 97 active+clean; 352 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-04-20T10:37:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:35 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:37:37.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:37 smithi046 ceph-mon[21574]: pgmap v111: 97 pgs: 97 active+clean; 372 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 317 op/s 2024-04-20T10:37:37.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:37 smithi042 ceph-mon[17078]: pgmap v111: 97 pgs: 97 active+clean; 372 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 317 op/s 2024-04-20T10:37:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:37 smithi162 ceph-mon[21854]: pgmap v111: 97 pgs: 97 active+clean; 372 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 317 op/s 2024-04-20T10:37:39.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:39 smithi046 ceph-mon[21574]: pgmap v112: 97 pgs: 97 active+clean; 397 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 691 op/s 2024-04-20T10:37:39.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:39 smithi042 ceph-mon[17078]: pgmap v112: 97 pgs: 97 active+clean; 397 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 691 op/s 2024-04-20T10:37:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:39 smithi162 ceph-mon[21854]: pgmap v112: 97 pgs: 97 active+clean; 397 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 691 op/s 2024-04-20T10:37:41.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:41 smithi046 ceph-mon[21574]: pgmap v113: 97 pgs: 97 active+clean; 397 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.4 MiB/s wr, 685 op/s 2024-04-20T10:37:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:41 smithi042 ceph-mon[17078]: pgmap v113: 97 pgs: 97 active+clean; 397 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.4 MiB/s wr, 685 op/s 2024-04-20T10:37:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:41 smithi162 ceph-mon[21854]: pgmap v113: 97 pgs: 97 active+clean; 397 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.4 MiB/s wr, 685 op/s 2024-04-20T10:37:43.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:43 smithi046 ceph-mon[21574]: pgmap v114: 97 pgs: 97 active+clean; 410 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 699 op/s 2024-04-20T10:37:43.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:43 smithi042 ceph-mon[17078]: pgmap v114: 97 pgs: 97 active+clean; 410 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 699 op/s 2024-04-20T10:37:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:43 smithi162 ceph-mon[21854]: pgmap v114: 97 pgs: 97 active+clean; 410 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 699 op/s 2024-04-20T10:37:45.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:45 smithi046 ceph-mon[21574]: pgmap v115: 97 pgs: 97 active+clean; 410 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 578 op/s 2024-04-20T10:37:45.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:45 smithi042 ceph-mon[17078]: pgmap v115: 97 pgs: 97 active+clean; 410 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 578 op/s 2024-04-20T10:37:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:45 smithi162 ceph-mon[21854]: pgmap v115: 97 pgs: 97 active+clean; 410 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 578 op/s 2024-04-20T10:37:47.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:47 smithi046 ceph-mon[21574]: pgmap v116: 97 pgs: 97 active+clean; 410 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 583 op/s 2024-04-20T10:37:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:47 smithi042 ceph-mon[17078]: pgmap v116: 97 pgs: 97 active+clean; 410 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 583 op/s 2024-04-20T10:37:47.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:47 smithi162 ceph-mon[21854]: pgmap v116: 97 pgs: 97 active+clean; 410 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 583 op/s 2024-04-20T10:37:49.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:49 smithi046 ceph-mon[21574]: pgmap v117: 97 pgs: 97 active+clean; 412 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 403 op/s 2024-04-20T10:37:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:49 smithi042 ceph-mon[17078]: pgmap v117: 97 pgs: 97 active+clean; 412 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 403 op/s 2024-04-20T10:37:49.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:49 smithi162 ceph-mon[21854]: pgmap v117: 97 pgs: 97 active+clean; 412 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 403 op/s 2024-04-20T10:37:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:51 smithi042 ceph-mon[17078]: pgmap v118: 97 pgs: 97 active+clean; 412 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 28 op/s 2024-04-20T10:37:51.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:51 smithi162 ceph-mon[21854]: pgmap v118: 97 pgs: 97 active+clean; 412 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 28 op/s 2024-04-20T10:37:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:51 smithi046 ceph-mon[21574]: pgmap v118: 97 pgs: 97 active+clean; 412 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 28 op/s 2024-04-20T10:37:53.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:53 smithi042 ceph-mon[17078]: pgmap v119: 97 pgs: 97 active+clean; 413 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 39 op/s 2024-04-20T10:37:53.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:53 smithi162 ceph-mon[21854]: pgmap v119: 97 pgs: 97 active+clean; 413 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 39 op/s 2024-04-20T10:37:54.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:53 smithi046 ceph-mon[21574]: pgmap v119: 97 pgs: 97 active+clean; 413 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 39 op/s 2024-04-20T10:37:54.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:37:54.524 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:37:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.758+0000 7f1487200640 1 -- 172.21.15.42:0/32156826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488104280 msgr2=0x7f1488104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.758+0000 7f1487200640 1 --2- 172.21.15.42:0/32156826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488104280 0x7f1488104660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1478009900 tx=0x7f147802bc30 comp rx=0 tx=0).stop 2024-04-20T10:37:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.758+0000 7f1487200640 1 -- 172.21.15.42:0/32156826 shutdown_connections 2024-04-20T10:37:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.758+0000 7f1487200640 1 --2- 172.21.15.42:0/32156826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1488109350 0x7f1488110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.758+0000 7f1487200640 1 --2- 172.21.15.42:0/32156826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1488104c30 0x7f1488108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.758+0000 7f1487200640 1 --2- 172.21.15.42:0/32156826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488104280 0x7f1488104660 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.758+0000 7f1487200640 1 -- 172.21.15.42:0/32156826 >> 172.21.15.42:0/32156826 conn(0x7f14880ffaa0 msgr2=0x7f1488101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.758+0000 7f1487200640 1 -- 172.21.15.42:0/32156826 shutdown_connections 2024-04-20T10:37:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.759+0000 7f1487200640 1 -- 172.21.15.42:0/32156826 wait complete. 2024-04-20T10:37:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.759+0000 7f1487200640 1 Processor -- start 2024-04-20T10:37:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1487200640 1 -- start start 2024-04-20T10:37:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1487200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488104280 0x7f14881a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1487200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1488109350 0x7f14881a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1487200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f148819a730 0x7f148819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1487200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1488114410 con 0x7f148819a730 2024-04-20T10:37:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1487200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1488114290 con 0x7f1488109350 2024-04-20T10:37:54.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1487200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1488114590 con 0x7f1488104280 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1485400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1488109350 0x7f14881a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1485e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488104280 0x7f14881a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1485e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488104280 0x7f14881a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:58460/0 (socket says 172.21.15.42:58460) 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1485400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1488109350 0x7f14881a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46784/0 (socket says 172.21.15.42:46784) 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1485e00640 1 -- 172.21.15.42:0/246733634 learned_addr learned my addr 172.21.15.42:0/246733634 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.760+0000 7f1486800640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f148819a730 0x7f148819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.761+0000 7f1485e00640 1 -- 172.21.15.42:0/246733634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1488109350 msgr2=0x7f14881a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.761+0000 7f1485e00640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1488109350 0x7f14881a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.761+0000 7f1485e00640 1 -- 172.21.15.42:0/246733634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f148819a730 msgr2=0x7f148819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.761+0000 7f1485e00640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f148819a730 0x7f148819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.761+0000 7f1485e00640 1 -- 172.21.15.42:0/246733634 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148819b330 con 0x7f1488104280 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.761+0000 7f1486800640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f148819a730 0x7f148819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.761+0000 7f1485e00640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488104280 0x7f14881a0670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1478006010 tx=0x7f147802cb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.761+0000 7f147ea00640 1 -- 172.21.15.42:0/246733634 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478041070 con 0x7f1488104280 2024-04-20T10:37:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.761+0000 7f1487200640 1 -- 172.21.15.42:0/246733634 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148819b5c0 con 0x7f1488104280 2024-04-20T10:37:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.761+0000 7f147ea00640 1 -- 172.21.15.42:0/246733634 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1478004450 con 0x7f1488104280 2024-04-20T10:37:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.761+0000 7f147ea00640 1 -- 172.21.15.42:0/246733634 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478003e00 con 0x7f1488104280 2024-04-20T10:37:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.762+0000 7f1487200640 1 -- 172.21.15.42:0/246733634 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14881a7750 con 0x7f1488104280 2024-04-20T10:37:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f148fbf3740 1 Processor -- start 2024-04-20T10:37:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f148fbf3740 1 -- start start 2024-04-20T10:37:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f147ea00640 1 -- 172.21.15.42:0/246733634 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1478044070 con 0x7f1488104280 2024-04-20T10:37:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f148fbf3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4380 0x55ce3e7b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f148fbf3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7bd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f148fbf3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7bd880 0x55ce3e7bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f148fbf3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ce3e71a150 con 0x55ce3e7b4ca0 2024-04-20T10:37:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f148fbf3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ce3e6c0370 con 0x55ce3e7bd880 2024-04-20T10:37:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f148fbf3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ce3e716110 con 0x55ce3e7b4380 2024-04-20T10:37:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f1486800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4380 0x55ce3e7b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f1485400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7bd880 0x55ce3e7bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f1486800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4380 0x55ce3e7b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:58470/0 (socket says 172.21.15.42:58470) 2024-04-20T10:37:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f1486800640 1 -- 172.21.15.42:0/2353754175 learned_addr learned my addr 172.21.15.42:0/2353754175 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.765+0000 7f1485e00640 1 --2- 172.21.15.42:0/2353754175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7bd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.766+0000 7f1486800640 1 -- 172.21.15.42:0/2353754175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7bd880 msgr2=0x55ce3e7bfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.766+0000 7f1486800640 1 --2- 172.21.15.42:0/2353754175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7bd880 0x55ce3e7bfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.766+0000 7f1486800640 1 -- 172.21.15.42:0/2353754175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4ca0 msgr2=0x55ce3e7bd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.766+0000 7f1486800640 1 --2- 172.21.15.42:0/2353754175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7bd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.766+0000 7f1486800640 1 -- 172.21.15.42:0/2353754175 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce3e4f67d0 con 0x55ce3e7b4380 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.766+0000 7f1486800640 1 --2- 172.21.15.42:0/2353754175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4380 0x55ce3e7b4760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1480009fd0 tx=0x7f1480010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2337f5bbd27645 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.766+0000 7f1473400640 1 -- 172.21.15.42:0/2353754175 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480011700 con 0x55ce3e7b4380 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.766+0000 7f1473400640 1 -- 172.21.15.42:0/2353754175 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1480011d80 con 0x55ce3e7b4380 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.766+0000 7f1473400640 1 -- 172.21.15.42:0/2353754175 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148000e3c0 con 0x55ce3e7b4380 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.767+0000 7f147ea00640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f146c0a3da0 0x7f146c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.767+0000 7f147ea00640 1 -- 172.21.15.42:0/246733634 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f14780f21a0 con 0x7f1488104280 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.767+0000 7f1485e00640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f146c0a3da0 0x7f146c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.767+0000 7f148fbf3740 1 -- 172.21.15.42:0/2353754175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4380 msgr2=0x55ce3e7b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.767+0000 7f148fbf3740 1 --2- 172.21.15.42:0/2353754175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4380 0x55ce3e7b4760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1480009fd0 tx=0x7f1480010860 comp rx=0 tx=0).stop 2024-04-20T10:37:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.767+0000 7f148fbf3740 1 -- 172.21.15.42:0/2353754175 shutdown_connections 2024-04-20T10:37:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.767+0000 7f148fbf3740 1 --2- 172.21.15.42:0/2353754175 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7bd880 0x55ce3e7bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.767+0000 7f148fbf3740 1 --2- 172.21.15.42:0/2353754175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7bd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.767+0000 7f148fbf3740 1 --2- 172.21.15.42:0/2353754175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4380 0x55ce3e7b4760 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.767+0000 7f148fbf3740 1 -- 172.21.15.42:0/2353754175 >> 172.21.15.42:0/2353754175 conn(0x55ce3e7aeab0 msgr2=0x55ce3e7aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.767+0000 7f148fbf3740 1 -- 172.21.15.42:0/2353754175 shutdown_connections 2024-04-20T10:37:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.768+0000 7f148fbf3740 1 -- 172.21.15.42:0/2353754175 wait complete. 2024-04-20T10:37:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.768+0000 7f1485e00640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f146c0a3da0 0x7f146c0a6260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55ce3e7b3e50 tx=0x7f147802bc30 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.768+0000 7f148fbf3740 1 -- start start 2024-04-20T10:37:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.768+0000 7f148fbf3740 1 Processor -- start 2024-04-20T10:37:54.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f148fbf3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4380 0x55ce3e7ba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f148fbf3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7bae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f148fbf3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7bd880 0x55ce3e7bb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f148fbf3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ce3e7c0670 con 0x55ce3e7b4380 2024-04-20T10:37:54.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f148fbf3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ce3e6d3740 con 0x55ce3e7bd880 2024-04-20T10:37:54.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f148fbf3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ce3e7c07f0 con 0x55ce3e7b4ca0 2024-04-20T10:37:54.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f148fbf3740 10 client.0 _ll_register_callbacks cb 0x55ce3e6ceb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f1486800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4380 0x55ce3e7ba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f1485400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7bae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f1486800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4380 0x55ce3e7ba910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55586/0 (socket says 172.21.15.42:55586) 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f1486800640 1 -- 172.21.15.42:0/408677117 learned_addr learned my addr 172.21.15.42:0/408677117 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f1486800640 1 --2- 172.21.15.42:0/408677117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7bd880 0x55ce3e7bb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f1485400640 1 -- 172.21.15.42:0/408677117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7bd880 msgr2=0x55ce3e7bb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f1485400640 1 --2- 172.21.15.42:0/408677117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7bd880 0x55ce3e7bb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f1485400640 1 -- 172.21.15.42:0/408677117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4380 msgr2=0x55ce3e7ba910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f1485400640 1 --2- 172.21.15.42:0/408677117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4380 0x55ce3e7ba910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.769+0000 7f1485400640 1 -- 172.21.15.42:0/408677117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce3e42c3f0 con 0x55ce3e7b4ca0 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.770+0000 7f1485400640 1 --2- 172.21.15.42:0/408677117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7bae50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f148819bf80 tx=0x7f1474012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae059a4d2b96c0f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.770+0000 7f1467e00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:37:54.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.770+0000 7f1467e00640 1 -- 172.21.15.42:0/408677117 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147400f8f0 con 0x55ce3e7b4ca0 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.770+0000 7f148fbf3740 1 -- 172.21.15.42:0/408677117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1480011d80 con 0x55ce3e7b4ca0 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.770+0000 7f1467e00640 1 -- 172.21.15.42:0/408677117 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f147400fa90 con 0x55ce3e7b4ca0 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.770+0000 7f1467e00640 1 -- 172.21.15.42:0/408677117 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f147400fc90 con 0x55ce3e7b4ca0 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.770+0000 7f148fbf3740 10 client.25858 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.770+0000 7f148fbf3740 1 -- 172.21.15.42:0/408677117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ce3e3ec0f0 con 0x55ce3e7b4ca0 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.771+0000 7f1467e00640 1 -- 172.21.15.42:0/408677117 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1480011d80 con 0x55ce3e7b4ca0 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.771+0000 7f1467e00640 1 -- 172.21.15.42:0/408677117 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f147401c420 con 0x55ce3e7b4ca0 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.771+0000 7f148fbf3740 10 client.25858 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.771+0000 7f148fbf3740 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.771+0000 7f148fbf3740 20 client.25858 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.771+0000 7f148fbf3740 1 --2- 172.21.15.42:0/408677117 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ce3e7cdce0 0x55ce3e7d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.771+0000 7f148fbf3740 4 client.25858 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:37:54.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.771+0000 7f148fbf3740 1 -- 172.21.15.42:0/408677117 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce3e727b70 con 0x55ce3e7cdce0 2024-04-20T10:37:54.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.771+0000 7f1486800640 1 --2- 172.21.15.42:0/408677117 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ce3e7cdce0 0x55ce3e7d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.772+0000 7f1486800640 1 --2- 172.21.15.42:0/408677117 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ce3e7cdce0 0x55ce3e7d0140 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99a88b8faa6a6d server_cookie=b3c96ffd0c8b9bbd in_seq=0 out_seq=0 2024-04-20T10:37:54.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.772+0000 7f1467e00640 10 client.25858 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:54.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.777+0000 7f1467e00640 1 -- 172.21.15.42:0/408677117 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce3e727b70 con 0x55ce3e7cdce0 2024-04-20T10:37:54.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.777+0000 7f1467e00640 10 client.25858 handle_command_reply: tid=0 2024-04-20T10:37:54.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.777+0000 7f148fbf3740 1 client.25858 shutdown 2024-04-20T10:37:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.778+0000 7f148fbf3740 1 -- 172.21.15.42:0/408677117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4ca0 msgr2=0x55ce3e7bae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.778+0000 7f148fbf3740 1 --2- 172.21.15.42:0/408677117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7bae50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f148819bf80 tx=0x7f1474012b60 comp rx=0 tx=0).stop 2024-04-20T10:37:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.778+0000 7f148fbf3740 1 -- 172.21.15.42:0/408677117 shutdown_connections 2024-04-20T10:37:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.778+0000 7f148fbf3740 1 --2- 172.21.15.42:0/408677117 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ce3e7cdce0 0x55ce3e7d0140 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.778+0000 7f148fbf3740 1 --2- 172.21.15.42:0/408677117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7bd880 0x55ce3e7bb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.778+0000 7f148fbf3740 1 --2- 172.21.15.42:0/408677117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7bae50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.778+0000 7f1467e00640 0 client.25858 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.778+0000 7f148fbf3740 1 --2- 172.21.15.42:0/408677117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4380 0x55ce3e7ba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.778+0000 7f148fbf3740 1 -- 172.21.15.42:0/408677117 >> 172.21.15.42:0/408677117 conn(0x55ce3e7aeab0 msgr2=0x55ce3e7b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.778+0000 7f148fbf3740 1 -- 172.21.15.42:0/408677117 shutdown_connections 2024-04-20T10:37:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.778+0000 7f148fbf3740 1 -- 172.21.15.42:0/408677117 wait complete. 2024-04-20T10:37:54.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.778+0000 7f148fbf3740 20 client.25858 trim_cache size 0 max 16384 2024-04-20T10:37:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.798+0000 7f148fbf3740 1 Processor -- start 2024-04-20T10:37:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.798+0000 7f148fbf3740 1 -- start start 2024-04-20T10:37:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.799+0000 7f148fbf3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7aeab0 0x55ce3e7bd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.799+0000 7f148fbf3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 0x55ce3e7b9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.799+0000 7f148fbf3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7c9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.799+0000 7f148fbf3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f14881adcc0 con 0x55ce3e7aeab0 2024-04-20T10:37:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.799+0000 7f148fbf3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14881b0540 con 0x55ce3e7b4380 2024-04-20T10:37:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.799+0000 7f148fbf3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f14881ae330 con 0x55ce3e7b4ca0 2024-04-20T10:37:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.799+0000 7f1485400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 0x55ce3e7b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.799+0000 7f1485400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 0x55ce3e7b9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46822/0 (socket says 172.21.15.42:46822) 2024-04-20T10:37:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.799+0000 7f1485400640 1 -- 172.21.15.42:0/4099367989 learned_addr learned my addr 172.21.15.42:0/4099367989 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.799+0000 7f1486800640 1 --2- 172.21.15.42:0/4099367989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7aeab0 0x55ce3e7bd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.800+0000 7f1485400640 1 -- 172.21.15.42:0/4099367989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4ca0 msgr2=0x55ce3e7c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.800+0000 7f1485400640 1 --2- 172.21.15.42:0/4099367989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.800+0000 7f1485400640 1 -- 172.21.15.42:0/4099367989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7aeab0 msgr2=0x55ce3e7bd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.800+0000 7f1485400640 1 --2- 172.21.15.42:0/4099367989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7aeab0 0x55ce3e7bd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.800+0000 7f1485400640 1 -- 172.21.15.42:0/4099367989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce3e7b4ad0 con 0x55ce3e7b4380 2024-04-20T10:37:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.800+0000 7f1485400640 1 --2- 172.21.15.42:0/4099367989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 0x55ce3e7b9850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1474001eb0 tx=0x7f147400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92941ca4a97ad6f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.800+0000 7f1472a00640 1 -- 172.21.15.42:0/4099367989 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147400fe30 con 0x55ce3e7b4380 2024-04-20T10:37:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.800+0000 7f1472a00640 1 -- 172.21.15.42:0/4099367989 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1474005550 con 0x55ce3e7b4380 2024-04-20T10:37:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.800+0000 7f1472a00640 1 -- 172.21.15.42:0/4099367989 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147401c6a0 con 0x55ce3e7b4380 2024-04-20T10:37:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.801+0000 7f148fbf3740 1 -- 172.21.15.42:0/4099367989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 msgr2=0x55ce3e7b9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.801+0000 7f148fbf3740 1 --2- 172.21.15.42:0/4099367989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 0x55ce3e7b9850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1474001eb0 tx=0x7f147400f790 comp rx=0 tx=0).stop 2024-04-20T10:37:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.801+0000 7f148fbf3740 1 -- 172.21.15.42:0/4099367989 shutdown_connections 2024-04-20T10:37:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.801+0000 7f148fbf3740 1 --2- 172.21.15.42:0/4099367989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.801+0000 7f148fbf3740 1 --2- 172.21.15.42:0/4099367989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 0x55ce3e7b9850 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.801+0000 7f148fbf3740 1 --2- 172.21.15.42:0/4099367989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7aeab0 0x55ce3e7bd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.801+0000 7f148fbf3740 1 -- 172.21.15.42:0/4099367989 >> 172.21.15.42:0/4099367989 conn(0x55ce3e7b62e0 msgr2=0x55ce3e7b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.801+0000 7f148fbf3740 1 -- 172.21.15.42:0/4099367989 shutdown_connections 2024-04-20T10:37:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.801+0000 7f148fbf3740 1 -- 172.21.15.42:0/4099367989 wait complete. 2024-04-20T10:37:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.801+0000 7f148fbf3740 1 -- start start 2024-04-20T10:37:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f148fbf3740 1 Processor -- start 2024-04-20T10:37:54.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f148fbf3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7aeab0 0x55ce3e7c5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f148fbf3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 0x55ce3e7c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f148fbf3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f148fbf3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f14881aea00 con 0x55ce3e7b4ca0 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f148fbf3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14881afe70 con 0x55ce3e7b4380 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f148fbf3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f14881af0d0 con 0x55ce3e7aeab0 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f148fbf3740 10 client.0 _ll_register_callbacks cb 0x7f1474009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f1485400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 0x55ce3e7c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f1485400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 0x55ce3e7c5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46826/0 (socket says 172.21.15.42:46826) 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f1486800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7aeab0 0x55ce3e7c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f1485400640 1 -- 172.21.15.42:0/1697093178 learned_addr learned my addr 172.21.15.42:0/1697093178 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.802+0000 7f1486800640 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.803+0000 7f1485400640 1 -- 172.21.15.42:0/1697093178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7aeab0 msgr2=0x55ce3e7c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.803+0000 7f1485400640 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7aeab0 0x55ce3e7c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.803+0000 7f1485400640 1 -- 172.21.15.42:0/1697093178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4ca0 msgr2=0x55ce3e7c1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.803+0000 7f1485400640 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7c1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.803+0000 7f1485400640 1 -- 172.21.15.42:0/1697093178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce3e7c2490 con 0x55ce3e7b4380 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.803+0000 7f1486800640 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7aeab0 0x55ce3e7c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.803+0000 7f1485400640 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 0x55ce3e7c5d70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f14740092a0 tx=0x7f147401cd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c02259432023a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.803+0000 7f1467e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.803+0000 7f1467e00640 1 -- 172.21.15.42:0/1697093178 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147401c4a0 con 0x55ce3e7b4380 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.803+0000 7f148fbf3740 1 -- 172.21.15.42:0/1697093178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1474005550 con 0x55ce3e7b4380 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.803+0000 7f1467e00640 1 -- 172.21.15.42:0/1697093178 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f147401c640 con 0x55ce3e7b4380 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.804+0000 7f1467e00640 1 -- 172.21.15.42:0/1697093178 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f147403c050 con 0x55ce3e7b4380 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.804+0000 7f148fbf3740 10 client.25844 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.804+0000 7f148fbf3740 1 -- 172.21.15.42:0/1697093178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ce3e834d40 con 0x55ce3e7b4380 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.804+0000 7f1467e00640 1 -- 172.21.15.42:0/1697093178 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1474005550 con 0x55ce3e7b4380 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.804+0000 7f1467e00640 1 -- 172.21.15.42:0/1697093178 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f1474034070 con 0x55ce3e7b4380 2024-04-20T10:37:54.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.804+0000 7f148fbf3740 10 client.25844 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:54.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.804+0000 7f148fbf3740 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:54.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.804+0000 7f148fbf3740 20 client.25844 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:54.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.804+0000 7f148fbf3740 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ce3e7bd880 0x55ce3e837420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:54.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.804+0000 7f148fbf3740 4 client.25844 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:37:54.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.804+0000 7f148fbf3740 1 -- 172.21.15.42:0/1697093178 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce3e7219e0 con 0x55ce3e7bd880 2024-04-20T10:37:54.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.804+0000 7f1486800640 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ce3e7bd880 0x55ce3e837420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:54.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.805+0000 7f1486800640 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ce3e7bd880 0x55ce3e837420 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd9ba1bb72ce0c9 server_cookie=f590ad9904ab4740 in_seq=0 out_seq=0 2024-04-20T10:37:54.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.805+0000 7f1467e00640 10 client.25844 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f1467e00640 1 -- 172.21.15.42:0/1697093178 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ce3e7219e0 con 0x55ce3e7bd880 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f1467e00640 10 client.25844 handle_command_reply: tid=0 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f148fbf3740 1 client.25844 shutdown 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f148fbf3740 1 -- 172.21.15.42:0/1697093178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 msgr2=0x55ce3e7c5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f148fbf3740 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 0x55ce3e7c5d70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f14740092a0 tx=0x7f147401cd40 comp rx=0 tx=0).stop 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f148fbf3740 1 -- 172.21.15.42:0/1697093178 shutdown_connections 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f148fbf3740 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55ce3e7bd880 0x55ce3e837420 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f148fbf3740 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ce3e7b4ca0 0x55ce3e7c1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f1467e00640 0 client.25844 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f148fbf3740 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce3e7b4380 0x55ce3e7c5d70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f148fbf3740 1 --2- 172.21.15.42:0/1697093178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ce3e7aeab0 0x55ce3e7c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f148fbf3740 1 -- 172.21.15.42:0/1697093178 >> 172.21.15.42:0/1697093178 conn(0x55ce3e7b62e0 msgr2=0x55ce3e7ba8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:54.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f148fbf3740 1 -- 172.21.15.42:0/1697093178 shutdown_connections 2024-04-20T10:37:54.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f148fbf3740 1 -- 172.21.15.42:0/1697093178 wait complete. 2024-04-20T10:37:54.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.811+0000 7f148fbf3740 20 client.25844 trim_cache size 0 max 16384 2024-04-20T10:37:54.813 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:37:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:37:54.813 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:37:54.813 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:37:54.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.812+0000 7f1487200640 1 -- 172.21.15.42:0/246733634 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f146c0a3da0 msgr2=0x7f146c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:54.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.812+0000 7f1487200640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f146c0a3da0 0x7f146c0a6260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55ce3e7b3e50 tx=0x7f147802bc30 comp rx=0 tx=0).stop 2024-04-20T10:37:54.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.812+0000 7f1487200640 1 -- 172.21.15.42:0/246733634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488104280 msgr2=0x7f14881a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.812+0000 7f1487200640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488104280 0x7f14881a0670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1478006010 tx=0x7f147802cb30 comp rx=0 tx=0).stop 2024-04-20T10:37:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.812+0000 7f1487200640 1 -- 172.21.15.42:0/246733634 shutdown_connections 2024-04-20T10:37:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.812+0000 7f1487200640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f146c0a3da0 0x7f146c0a6260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.812+0000 7f1487200640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f148819a730 0x7f148819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.812+0000 7f1487200640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1488109350 0x7f14881a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.812+0000 7f1487200640 1 --2- 172.21.15.42:0/246733634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1488104280 0x7f14881a0670 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.812+0000 7f1487200640 1 -- 172.21.15.42:0/246733634 >> 172.21.15.42:0/246733634 conn(0x7f14880ffaa0 msgr2=0x7f1488101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.812+0000 7f1487200640 1 -- 172.21.15.42:0/246733634 shutdown_connections 2024-04-20T10:37:54.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:54.812+0000 7f1487200640 1 -- 172.21.15.42:0/246733634 wait complete. 2024-04-20T10:37:54.829 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:37:54.829 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:37:55.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.021+0000 7f0eb0e00640 1 -- 172.21.15.42:0/4239293289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac109290 msgr2=0x7f0eac110e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.021+0000 7f0eb0e00640 1 --2- 172.21.15.42:0/4239293289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac109290 0x7f0eac110e60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009be0 tx=0x7f0ea402bef0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.021+0000 7f0eb0e00640 1 -- 172.21.15.42:0/4239293289 shutdown_connections 2024-04-20T10:37:55.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.021+0000 7f0eb0e00640 1 --2- 172.21.15.42:0/4239293289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac109290 0x7f0eac110e60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.021+0000 7f0eb0e00640 1 --2- 172.21.15.42:0/4239293289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac104b70 0x7f0eac108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.021+0000 7f0eb0e00640 1 --2- 172.21.15.42:0/4239293289 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac1041c0 0x7f0eac1045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.022+0000 7f0eb0e00640 1 -- 172.21.15.42:0/4239293289 >> 172.21.15.42:0/4239293289 conn(0x7f0eac0ffa00 msgr2=0x7f0eac101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:55.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.022+0000 7f0eb0e00640 1 -- 172.21.15.42:0/4239293289 shutdown_connections 2024-04-20T10:37:55.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.022+0000 7f0eb0e00640 1 -- 172.21.15.42:0/4239293289 wait complete. 2024-04-20T10:37:55.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eb0e00640 1 Processor -- start 2024-04-20T10:37:55.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eb0e00640 1 -- start start 2024-04-20T10:37:55.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eb0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac1041c0 0x7f0eac1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eb0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac109290 0x7f0eac1a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eb0e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac19a640 0x7f0eac19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eb0e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac1143a0 con 0x7f0eac1041c0 2024-04-20T10:37:55.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eb0e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac114220 con 0x7f0eac109290 2024-04-20T10:37:55.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eb0e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac114520 con 0x7f0eac19a640 2024-04-20T10:37:55.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac1041c0 0x7f0eac1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eaaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac109290 0x7f0eac1a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac1041c0 0x7f0eac1a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55622/0 (socket says 172.21.15.42:55622) 2024-04-20T10:37:55.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eab400640 1 -- 172.21.15.42:0/2534046344 learned_addr learned my addr 172.21.15.42:0/2534046344 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:55.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eaaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac109290 0x7f0eac1a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46844/0 (socket says 172.21.15.42:46844) 2024-04-20T10:37:55.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.023+0000 7f0eabe00640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac19a640 0x7f0eac19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.024+0000 7f0eaaa00640 1 -- 172.21.15.42:0/2534046344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac19a640 msgr2=0x7f0eac19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.024+0000 7f0eaaa00640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac19a640 0x7f0eac19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.024+0000 7f0eaaa00640 1 -- 172.21.15.42:0/2534046344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac1041c0 msgr2=0x7f0eac1a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.024+0000 7f0eaaa00640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac1041c0 0x7f0eac1a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.024+0000 7f0eaaa00640 1 -- 172.21.15.42:0/2534046344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac19b240 con 0x7f0eac109290 2024-04-20T10:37:55.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.024+0000 7f0eaaa00640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac109290 0x7f0eac1a0ac0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800e9c0 tx=0x7f0e9800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:55.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.024+0000 7f0ea3e00640 1 -- 172.21.15.42:0/2534046344 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9800cd80 con 0x7f0eac109290 2024-04-20T10:37:55.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.024+0000 7f0ea3e00640 1 -- 172.21.15.42:0/2534046344 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e980035f0 con 0x7f0eac109290 2024-04-20T10:37:55.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.024+0000 7f0ea3e00640 1 -- 172.21.15.42:0/2534046344 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98010690 con 0x7f0eac109290 2024-04-20T10:37:55.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.024+0000 7f0eb0e00640 1 -- 172.21.15.42:0/2534046344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac1a7320 con 0x7f0eac109290 2024-04-20T10:37:55.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.025+0000 7f0eb0e00640 1 -- 172.21.15.42:0/2534046344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac1a7720 con 0x7f0eac109290 2024-04-20T10:37:55.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.026+0000 7f0eb0e00640 1 -- 172.21.15.42:0/2534046344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eac105690 con 0x7f0eac109290 2024-04-20T10:37:55.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.028+0000 7f0ea3e00640 1 -- 172.21.15.42:0/2534046344 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0e980037a0 con 0x7f0eac109290 2024-04-20T10:37:55.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.030+0000 7f0ea3e00640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0e900a3d20 0x7f0e900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.030+0000 7f0ea3e00640 1 -- 172.21.15.42:0/2534046344 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0e980d0a90 con 0x7f0eac109290 2024-04-20T10:37:55.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.030+0000 7f0eab400640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0e900a3d20 0x7f0e900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.031+0000 7f0eab400640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0e900a3d20 0x7f0e900a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009ca0 tx=0x7f0e9c009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:55.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.038+0000 7f0ea3e00640 1 -- 172.21.15.42:0/2534046344 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0e9809d0d0 con 0x7f0eac109290 2024-04-20T10:37:55.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.287+0000 7f0eb0e00640 1 -- 172.21.15.42:0/2534046344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0eac0690f0 con 0x7f0eac109290 2024-04-20T10:37:55.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.288+0000 7f0ea3e00640 1 -- 172.21.15.42:0/2534046344 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f0e9809c9d0 con 0x7f0eac109290 2024-04-20T10:37:55.289 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:37:55.289 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:37:55.289 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:37:55.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.291+0000 7f0eb0e00640 1 -- 172.21.15.42:0/2534046344 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0e900a3d20 msgr2=0x7f0e900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.291+0000 7f0eb0e00640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0e900a3d20 0x7f0e900a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009ca0 tx=0x7f0e9c009260 comp rx=0 tx=0).stop 2024-04-20T10:37:55.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.291+0000 7f0eb0e00640 1 -- 172.21.15.42:0/2534046344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac109290 msgr2=0x7f0eac1a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.291+0000 7f0eb0e00640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac109290 0x7f0eac1a0ac0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800e9c0 tx=0x7f0e9800ee80 comp rx=0 tx=0).stop 2024-04-20T10:37:55.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.291+0000 7f0eb0e00640 1 -- 172.21.15.42:0/2534046344 shutdown_connections 2024-04-20T10:37:55.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.291+0000 7f0eb0e00640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0e900a3d20 0x7f0e900a61e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.291+0000 7f0eb0e00640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac19a640 0x7f0eac19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.291+0000 7f0eb0e00640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac109290 0x7f0eac1a0ac0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.291+0000 7f0eb0e00640 1 --2- 172.21.15.42:0/2534046344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac1041c0 0x7f0eac1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.291+0000 7f0eb0e00640 1 -- 172.21.15.42:0/2534046344 >> 172.21.15.42:0/2534046344 conn(0x7f0eac0ffa00 msgr2=0x7f0eac100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.291+0000 7f0eb0e00640 1 -- 172.21.15.42:0/2534046344 shutdown_connections 2024-04-20T10:37:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.292+0000 7f0eb0e00640 1 -- 172.21.15.42:0/2534046344 wait complete. 2024-04-20T10:37:55.311 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:37:55.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.516+0000 7f6312800640 1 -- 172.21.15.42:0/1215207193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f630c104c20 msgr2=0x7f630c108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.516+0000 7f6312800640 1 --2- 172.21.15.42:0/1215207193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f630c104c20 0x7f630c108c60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009900 tx=0x7f62f802bc30 comp rx=0 tx=0).stop 2024-04-20T10:37:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.517+0000 7f6312800640 1 -- 172.21.15.42:0/1215207193 shutdown_connections 2024-04-20T10:37:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.517+0000 7f6312800640 1 --2- 172.21.15.42:0/1215207193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f630c109340 0x7f630c110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.517+0000 7f6312800640 1 --2- 172.21.15.42:0/1215207193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f630c104c20 0x7f630c108c60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.517+0000 7f6312800640 1 --2- 172.21.15.42:0/1215207193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f630c104270 0x7f630c104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.517+0000 7f6312800640 1 -- 172.21.15.42:0/1215207193 >> 172.21.15.42:0/1215207193 conn(0x7f630c0ffa90 msgr2=0x7f630c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.517+0000 7f6312800640 1 -- 172.21.15.42:0/1215207193 shutdown_connections 2024-04-20T10:37:55.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.517+0000 7f6312800640 1 -- 172.21.15.42:0/1215207193 wait complete. 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.518+0000 7f6312800640 1 Processor -- start 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.518+0000 7f6312800640 1 -- start start 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.518+0000 7f6312800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f630c104270 0x7f630c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.518+0000 7f6312800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f630c109340 0x7f630c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.518+0000 7f6312800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f630c19a7a0 0x7f630c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.518+0000 7f6312800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f630c114400 con 0x7f630c104270 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.518+0000 7f6312800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f630c114280 con 0x7f630c19a7a0 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.518+0000 7f6312800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f630c114580 con 0x7f630c109340 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.519+0000 7f6311400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f630c104270 0x7f630c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.519+0000 7f6310a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f630c109340 0x7f630c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.519+0000 7f6310a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f630c109340 0x7f630c1a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:58526/0 (socket says 172.21.15.42:58526) 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.519+0000 7f6311e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f630c19a7a0 0x7f630c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.519+0000 7f6310a00640 1 -- 172.21.15.42:0/2719588539 learned_addr learned my addr 172.21.15.42:0/2719588539 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:55.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.519+0000 7f6310a00640 1 -- 172.21.15.42:0/2719588539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f630c19a7a0 msgr2=0x7f630c19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.519+0000 7f6310a00640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f630c19a7a0 0x7f630c19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.519+0000 7f6310a00640 1 -- 172.21.15.42:0/2719588539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f630c104270 msgr2=0x7f630c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.519+0000 7f6310a00640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f630c104270 0x7f630c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.519+0000 7f6310a00640 1 -- 172.21.15.42:0/2719588539 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c19b3a0 con 0x7f630c109340 2024-04-20T10:37:55.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.519+0000 7f6311e00640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f630c19a7a0 0x7f630c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:55.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.519+0000 7f6310a00640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f630c109340 0x7f630c1a0bd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62f802c240 tx=0x7f62f8004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:55.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.520+0000 7f630a000640 1 -- 172.21.15.42:0/2719588539 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8043020 con 0x7f630c109340 2024-04-20T10:37:55.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.520+0000 7f630a000640 1 -- 172.21.15.42:0/2719588539 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f80054e0 con 0x7f630c109340 2024-04-20T10:37:55.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.520+0000 7f6312800640 1 -- 172.21.15.42:0/2719588539 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630c19b630 con 0x7f630c109340 2024-04-20T10:37:55.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.520+0000 7f630a000640 1 -- 172.21.15.42:0/2719588539 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8003b80 con 0x7f630c109340 2024-04-20T10:37:55.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.520+0000 7f6312800640 1 -- 172.21.15.42:0/2719588539 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630c1a7780 con 0x7f630c109340 2024-04-20T10:37:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.523+0000 7f63169fb740 1 Processor -- start 2024-04-20T10:37:55.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.523+0000 7f630a000640 1 -- 172.21.15.42:0/2719588539 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f62f8046020 con 0x7f630c109340 2024-04-20T10:37:55.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.523+0000 7f63169fb740 1 -- start start 2024-04-20T10:37:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f63169fb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92d170 0x5616ee92d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f63169fb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92da90 0x5616ee936130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f63169fb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee936670 0x5616ee938a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f63169fb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5616ee892050 con 0x5616ee92d170 2024-04-20T10:37:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f63169fb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5616ee838490 con 0x5616ee92da90 2024-04-20T10:37:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f63169fb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5616ee88e010 con 0x5616ee936670 2024-04-20T10:37:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f6311e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92d170 0x5616ee92d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f6310a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee936670 0x5616ee938a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f6311e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92d170 0x5616ee92d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55646/0 (socket says 172.21.15.42:55646) 2024-04-20T10:37:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f6311e00640 1 -- 172.21.15.42:0/1726906920 learned_addr learned my addr 172.21.15.42:0/1726906920 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:55.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f6311400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92da90 0x5616ee936130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f6311e00640 1 -- 172.21.15.42:0/1726906920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee936670 msgr2=0x5616ee938a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:55.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f6311e00640 1 --2- 172.21.15.42:0/1726906920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee936670 0x5616ee938a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f6311e00640 1 -- 172.21.15.42:0/1726906920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92da90 msgr2=0x5616ee936130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:55.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f6311e00640 1 --2- 172.21.15.42:0/1726906920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92da90 0x5616ee936130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f6311e00640 1 -- 172.21.15.42:0/1726906920 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616ee840070 con 0x5616ee92d170 2024-04-20T10:37:55.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.524+0000 7f6311400640 1 --2- 172.21.15.42:0/1726906920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92da90 0x5616ee936130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:55.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.525+0000 7f6311e00640 1 --2- 172.21.15.42:0/1726906920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92d170 0x5616ee92d550 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6304011790 tx=0x7f6304011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fca457ad206bae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:55.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.525+0000 7f630a000640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f62f40a3d20 0x7f62f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.525+0000 7f62fe600640 1 -- 172.21.15.42:0/1726906920 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304003c30 con 0x5616ee92d170 2024-04-20T10:37:55.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.525+0000 7f62fe600640 1 -- 172.21.15.42:0/1726906920 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f630400fe80 con 0x5616ee92d170 2024-04-20T10:37:55.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.525+0000 7f62fe600640 1 -- 172.21.15.42:0/1726906920 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63040193e0 con 0x5616ee92d170 2024-04-20T10:37:55.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.525+0000 7f630a000640 1 -- 172.21.15.42:0/2719588539 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f62f80f2800 con 0x7f630c109340 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.525+0000 7f6311400640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f62f40a3d20 0x7f62f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.526+0000 7f63169fb740 1 -- 172.21.15.42:0/1726906920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92d170 msgr2=0x5616ee92d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.526+0000 7f63169fb740 1 --2- 172.21.15.42:0/1726906920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92d170 0x5616ee92d550 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6304011790 tx=0x7f6304011c50 comp rx=0 tx=0).stop 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.526+0000 7f63169fb740 1 -- 172.21.15.42:0/1726906920 shutdown_connections 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.526+0000 7f63169fb740 1 --2- 172.21.15.42:0/1726906920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee936670 0x5616ee938a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.526+0000 7f63169fb740 1 --2- 172.21.15.42:0/1726906920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92da90 0x5616ee936130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.526+0000 7f63169fb740 1 --2- 172.21.15.42:0/1726906920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92d170 0x5616ee92d550 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.526+0000 7f63169fb740 1 -- 172.21.15.42:0/1726906920 >> 172.21.15.42:0/1726906920 conn(0x5616ee9278a0 msgr2=0x5616ee927c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.526+0000 7f63169fb740 1 -- 172.21.15.42:0/1726906920 shutdown_connections 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.526+0000 7f6311400640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f62f40a3d20 0x7f62f40a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5616ee92cc40 tx=0x7f630000b410 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.526+0000 7f63169fb740 1 -- 172.21.15.42:0/1726906920 wait complete. 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.527+0000 7f63169fb740 1 -- start start 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.527+0000 7f63169fb740 1 Processor -- start 2024-04-20T10:37:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.527+0000 7f63169fb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee933670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.527+0000 7f63169fb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92da90 0x5616ee933bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.527+0000 7f63169fb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee936670 0x5616ee9340f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.527+0000 7f63169fb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5616ee939460 con 0x5616ee92da90 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.527+0000 7f63169fb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5616ee84b860 con 0x5616ee92d170 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.527+0000 7f63169fb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5616ee9395e0 con 0x5616ee936670 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.527+0000 7f63169fb740 10 client.0 _ll_register_callbacks cb 0x5616ee819fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.528+0000 7f6311400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee933670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.528+0000 7f6310a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92da90 0x5616ee933bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.528+0000 7f6311400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee933670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46874/0 (socket says 172.21.15.42:46874) 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.528+0000 7f6311400640 1 -- 172.21.15.42:0/940718555 learned_addr learned my addr 172.21.15.42:0/940718555 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.528+0000 7f6311e00640 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee936670 0x5616ee9340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.528+0000 7f6311400640 1 -- 172.21.15.42:0/940718555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee936670 msgr2=0x5616ee9340f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.528+0000 7f6311400640 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee936670 0x5616ee9340f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.528+0000 7f6311400640 1 -- 172.21.15.42:0/940718555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92da90 msgr2=0x5616ee933bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.528+0000 7f6311400640 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92da90 0x5616ee933bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.528+0000 7f6311400640 1 -- 172.21.15.42:0/940718555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616ee84f950 con 0x5616ee92d170 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.528+0000 7f6310a00640 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92da90 0x5616ee933bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.528+0000 7f6311400640 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee933670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f630001f4b0 tx=0x7f6300004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93b7d0cdc2fe289d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.529+0000 7f62f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.529+0000 7f62f2a00640 1 -- 172.21.15.42:0/940718555 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630002c060 con 0x5616ee92d170 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.529+0000 7f62f2a00640 1 -- 172.21.15.42:0/940718555 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f6300017030 con 0x5616ee92d170 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.529+0000 7f63169fb740 1 -- 172.21.15.42:0/940718555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5616ee89fa70 con 0x5616ee92d170 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.529+0000 7f62f2a00640 1 -- 172.21.15.42:0/940718555 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f630001c600 con 0x5616ee92d170 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.529+0000 7f63169fb740 10 client.25862 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.529+0000 7f63169fb740 1 -- 172.21.15.42:0/940718555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5616ee81d820 con 0x5616ee92d170 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.529+0000 7f62f2a00640 1 -- 172.21.15.42:0/940718555 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5616ee89fa70 con 0x5616ee92d170 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.530+0000 7f62f2a00640 1 -- 172.21.15.42:0/940718555 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f6300010990 con 0x5616ee92d170 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.530+0000 7f63169fb740 10 client.25862 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.530+0000 7f63169fb740 10 client.25862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:55.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.530+0000 7f63169fb740 20 client.25862 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:55.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.530+0000 7f63169fb740 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5616ee946700 0x5616ee948af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.530+0000 7f63169fb740 4 client.25862 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:37:55.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.530+0000 7f63169fb740 1 -- 172.21.15.42:0/940718555 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616ee8998e0 con 0x5616ee946700 2024-04-20T10:37:55.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.530+0000 7f6310a00640 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5616ee946700 0x5616ee948af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.530+0000 7f6310a00640 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5616ee946700 0x5616ee948af0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e0c32314d372d77 server_cookie=a2240c00d0926d26 in_seq=0 out_seq=0 2024-04-20T10:37:55.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.530+0000 7f62f2a00640 10 client.25862 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:55.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.532+0000 7f62f2a00640 1 -- 172.21.15.42:0/940718555 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616ee8998e0 con 0x5616ee946700 2024-04-20T10:37:55.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.532+0000 7f62f2a00640 10 client.25862 handle_command_reply: tid=0 2024-04-20T10:37:55.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.532+0000 7f63169fb740 1 client.25862 shutdown 2024-04-20T10:37:55.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.533+0000 7f63169fb740 1 -- 172.21.15.42:0/940718555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 msgr2=0x5616ee933670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.533+0000 7f63169fb740 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee933670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f630001f4b0 tx=0x7f6300004730 comp rx=0 tx=0).stop 2024-04-20T10:37:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.533+0000 7f63169fb740 1 -- 172.21.15.42:0/940718555 shutdown_connections 2024-04-20T10:37:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.533+0000 7f63169fb740 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5616ee946700 0x5616ee948af0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.533+0000 7f63169fb740 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee936670 0x5616ee9340f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.533+0000 7f63169fb740 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee92da90 0x5616ee933bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.533+0000 7f62f2a00640 0 client.25862 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.533+0000 7f63169fb740 1 --2- 172.21.15.42:0/940718555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee933670 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.533+0000 7f63169fb740 1 -- 172.21.15.42:0/940718555 >> 172.21.15.42:0/940718555 conn(0x5616ee9278a0 msgr2=0x5616ee9295f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.533+0000 7f63169fb740 1 -- 172.21.15.42:0/940718555 shutdown_connections 2024-04-20T10:37:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.533+0000 7f63169fb740 1 -- 172.21.15.42:0/940718555 wait complete. 2024-04-20T10:37:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.533+0000 7f63169fb740 20 client.25862 trim_cache size 0 max 16384 2024-04-20T10:37:55.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.548+0000 7f63169fb740 1 Processor -- start 2024-04-20T10:37:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.548+0000 7f63169fb740 1 -- start start 2024-04-20T10:37:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f63169fb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee935da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f63169fb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee92da90 0x5616ee942350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f63169fb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee936670 0x5616ee94e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f63169fb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f630c1add40 con 0x5616ee936670 2024-04-20T10:37:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f63169fb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f630c1b05a0 con 0x5616ee92d170 2024-04-20T10:37:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f63169fb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f630c1ae390 con 0x5616ee92da90 2024-04-20T10:37:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f6311400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee935da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f6311400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee935da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46884/0 (socket says 172.21.15.42:46884) 2024-04-20T10:37:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f6311400640 1 -- 172.21.15.42:0/2721176555 learned_addr learned my addr 172.21.15.42:0/2721176555 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:55.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f6311e00640 1 --2- 172.21.15.42:0/2721176555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee936670 0x5616ee94e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f6310a00640 1 --2- 172.21.15.42:0/2721176555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee92da90 0x5616ee942350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f6311400640 1 -- 172.21.15.42:0/2721176555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee92da90 msgr2=0x5616ee942350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f6311400640 1 --2- 172.21.15.42:0/2721176555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee92da90 0x5616ee942350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f6311400640 1 -- 172.21.15.42:0/2721176555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee936670 msgr2=0x5616ee94e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f6311400640 1 --2- 172.21.15.42:0/2721176555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee936670 0x5616ee94e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.549+0000 7f6311400640 1 -- 172.21.15.42:0/2721176555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616ee92d8c0 con 0x5616ee92d170 2024-04-20T10:37:55.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.550+0000 7f6311400640 1 --2- 172.21.15.42:0/2721176555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee935da0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6300011d10 tx=0x7f630000be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eaea970bf333562 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:55.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.550+0000 7f62fdc00640 1 -- 172.21.15.42:0/2721176555 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630001c9a0 con 0x5616ee92d170 2024-04-20T10:37:55.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.550+0000 7f62fdc00640 1 -- 172.21.15.42:0/2721176555 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6300010910 con 0x5616ee92d170 2024-04-20T10:37:55.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.550+0000 7f62fdc00640 1 -- 172.21.15.42:0/2721176555 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300039760 con 0x5616ee92d170 2024-04-20T10:37:55.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.551+0000 7f63169fb740 1 -- 172.21.15.42:0/2721176555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 msgr2=0x5616ee935da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.551+0000 7f63169fb740 1 --2- 172.21.15.42:0/2721176555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee935da0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6300011d10 tx=0x7f630000be00 comp rx=0 tx=0).stop 2024-04-20T10:37:55.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.551+0000 7f63169fb740 1 -- 172.21.15.42:0/2721176555 shutdown_connections 2024-04-20T10:37:55.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.551+0000 7f63169fb740 1 --2- 172.21.15.42:0/2721176555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee936670 0x5616ee94e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.551+0000 7f63169fb740 1 --2- 172.21.15.42:0/2721176555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee92da90 0x5616ee942350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.551+0000 7f63169fb740 1 --2- 172.21.15.42:0/2721176555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee935da0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.551+0000 7f63169fb740 1 -- 172.21.15.42:0/2721176555 >> 172.21.15.42:0/2721176555 conn(0x5616ee9278a0 msgr2=0x5616ee926960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.551+0000 7f63169fb740 1 -- 172.21.15.42:0/2721176555 shutdown_connections 2024-04-20T10:37:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.551+0000 7f63169fb740 1 -- 172.21.15.42:0/2721176555 wait complete. 2024-04-20T10:37:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.551+0000 7f63169fb740 1 -- start start 2024-04-20T10:37:55.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f63169fb740 1 Processor -- start 2024-04-20T10:37:55.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f63169fb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee93a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f63169fb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee92da90 0x5616ee93abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f63169fb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee936670 0x5616ee9b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f63169fb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f630c1aea60 con 0x5616ee936670 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f63169fb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f630c1afed0 con 0x5616ee92d170 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f63169fb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f630c1af130 con 0x5616ee92da90 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f63169fb740 10 client.0 _ll_register_callbacks cb 0x7f630000ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f6311400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee93a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f6310a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee92da90 0x5616ee93abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f6311400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee93a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46896/0 (socket says 172.21.15.42:46896) 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f6311400640 1 -- 172.21.15.42:0/4270793274 learned_addr learned my addr 172.21.15.42:0/4270793274 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.552+0000 7f6311e00640 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee936670 0x5616ee9b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.553+0000 7f6311400640 1 -- 172.21.15.42:0/4270793274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee92da90 msgr2=0x5616ee93abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:55.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.553+0000 7f6311400640 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee92da90 0x5616ee93abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.553+0000 7f6311400640 1 -- 172.21.15.42:0/4270793274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee936670 msgr2=0x5616ee9b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.553+0000 7f6311400640 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee936670 0x5616ee9b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.553+0000 7f6311400640 1 -- 172.21.15.42:0/4270793274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616ee5a43f0 con 0x5616ee92d170 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.553+0000 7f6311e00640 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee936670 0x5616ee9b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.553+0000 7f6311400640 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee93a690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6300018040 tx=0x7f6300010cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=962d96e9bd69c33b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.553+0000 7f62f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.553+0000 7f62f2a00640 1 -- 172.21.15.42:0/4270793274 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630003d020 con 0x5616ee92d170 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f62f2a00640 1 -- 172.21.15.42:0/4270793274 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f630001c7a0 con 0x5616ee92d170 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f63169fb740 1 -- 172.21.15.42:0/4270793274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6300010910 con 0x5616ee92d170 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f62f2a00640 1 -- 172.21.15.42:0/4270793274 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6300048050 con 0x5616ee92d170 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f63169fb740 10 client.25868 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f63169fb740 1 -- 172.21.15.42:0/4270793274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5616ee66e7d0 con 0x5616ee92d170 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f62f2a00640 1 -- 172.21.15.42:0/4270793274 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6300010910 con 0x5616ee92d170 2024-04-20T10:37:55.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f62f2a00640 1 -- 172.21.15.42:0/4270793274 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f6300030410 con 0x5616ee92d170 2024-04-20T10:37:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f63169fb740 10 client.25868 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f63169fb740 10 client.25868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f63169fb740 20 client.25868 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f63169fb740 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5616ee9b5dc0 0x5616ee9b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f63169fb740 4 client.25868 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:37:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.554+0000 7f63169fb740 1 -- 172.21.15.42:0/4270793274 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616ee8155a0 con 0x5616ee9b5dc0 2024-04-20T10:37:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.555+0000 7f6310a00640 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5616ee9b5dc0 0x5616ee9b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.555+0000 7f6310a00640 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5616ee9b5dc0 0x5616ee9b81e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee7d4ec90fa3b75 server_cookie=7023f2694f71ca45 in_seq=0 out_seq=0 2024-04-20T10:37:55.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.555+0000 7f62f2a00640 10 client.25868 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:55.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f62f2a00640 1 -- 172.21.15.42:0/4270793274 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616ee8155a0 con 0x5616ee9b5dc0 2024-04-20T10:37:55.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f62f2a00640 10 client.25868 handle_command_reply: tid=0 2024-04-20T10:37:55.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f63169fb740 1 client.25868 shutdown 2024-04-20T10:37:55.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f63169fb740 1 -- 172.21.15.42:0/4270793274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 msgr2=0x5616ee93a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f63169fb740 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee93a690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6300018040 tx=0x7f6300010cf0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f63169fb740 1 -- 172.21.15.42:0/4270793274 shutdown_connections 2024-04-20T10:37:55.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f63169fb740 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5616ee9b5dc0 0x5616ee9b81e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f63169fb740 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5616ee936670 0x5616ee9b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f63169fb740 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5616ee92da90 0x5616ee93abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f63169fb740 1 --2- 172.21.15.42:0/4270793274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5616ee92d170 0x5616ee93a690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f63169fb740 1 -- 172.21.15.42:0/4270793274 >> 172.21.15.42:0/4270793274 conn(0x5616ee9278a0 msgr2=0x5616ee928ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:55.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f62f2a00640 0 client.25868 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:55.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f63169fb740 1 -- 172.21.15.42:0/4270793274 shutdown_connections 2024-04-20T10:37:55.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.557+0000 7f63169fb740 1 -- 172.21.15.42:0/4270793274 wait complete. 2024-04-20T10:37:55.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f63169fb740 20 client.25868 trim_cache size 0 max 16384 2024-04-20T10:37:55.560 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:37:55.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f6312800640 1 -- 172.21.15.42:0/2719588539 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f62f40a3d20 msgr2=0x7f62f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f6312800640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f62f40a3d20 0x7f62f40a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5616ee92cc40 tx=0x7f630000b410 comp rx=0 tx=0).stop 2024-04-20T10:37:55.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f6312800640 1 -- 172.21.15.42:0/2719588539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f630c109340 msgr2=0x7f630c1a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f6312800640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f630c109340 0x7f630c1a0bd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62f802c240 tx=0x7f62f8004360 comp rx=0 tx=0).stop 2024-04-20T10:37:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f6312800640 1 -- 172.21.15.42:0/2719588539 shutdown_connections 2024-04-20T10:37:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f6312800640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f62f40a3d20 0x7f62f40a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f6312800640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f630c19a7a0 0x7f630c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f6312800640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f630c109340 0x7f630c1a0bd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f6312800640 1 --2- 172.21.15.42:0/2719588539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f630c104270 0x7f630c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f6312800640 1 -- 172.21.15.42:0/2719588539 >> 172.21.15.42:0/2719588539 conn(0x7f630c0ffa90 msgr2=0x7f630c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f6312800640 1 -- 172.21.15.42:0/2719588539 shutdown_connections 2024-04-20T10:37:55.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:55.558+0000 7f6312800640 1 -- 172.21.15.42:0/2719588539 wait complete. 2024-04-20T10:37:55.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:55 smithi042 ceph-mon[17078]: pgmap v120: 97 pgs: 97 active+clean; 413 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 254 KiB/s wr, 25 op/s 2024-04-20T10:37:55.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:55 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2534046344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:37:55.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:55 smithi162 ceph-mon[21854]: pgmap v120: 97 pgs: 97 active+clean; 413 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 254 KiB/s wr, 25 op/s 2024-04-20T10:37:55.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:55 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2534046344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:37:56.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:55 smithi046 ceph-mon[21574]: pgmap v120: 97 pgs: 97 active+clean; 413 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 254 KiB/s wr, 25 op/s 2024-04-20T10:37:56.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:55 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2534046344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:37:56.579 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:37:56.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:37:56.579 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:37:56.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.817+0000 7fbd9fe00640 1 -- 172.21.15.42:0/2211673752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbda0107150 msgr2=0x7fbda010ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:56.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.817+0000 7fbd9fe00640 1 --2- 172.21.15.42:0/2211673752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbda0107150 0x7fbda010ed20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009900 tx=0x7fbd9802bc30 comp rx=0 tx=0).stop 2024-04-20T10:37:56.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.817+0000 7fbd9fe00640 1 -- 172.21.15.42:0/2211673752 shutdown_connections 2024-04-20T10:37:56.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.817+0000 7fbd9fe00640 1 --2- 172.21.15.42:0/2211673752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbda0107150 0x7fbda010ed20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.817+0000 7fbd9fe00640 1 --2- 172.21.15.42:0/2211673752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda0102a30 0x7fbda0106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.817+0000 7fbd9fe00640 1 --2- 172.21.15.42:0/2211673752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda0102080 0x7fbda0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.817+0000 7fbd9fe00640 1 -- 172.21.15.42:0/2211673752 >> 172.21.15.42:0/2211673752 conn(0x7fbda00fd900 msgr2=0x7fbda00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:56.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.817+0000 7fbd9fe00640 1 -- 172.21.15.42:0/2211673752 shutdown_connections 2024-04-20T10:37:56.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.817+0000 7fbd9fe00640 1 -- 172.21.15.42:0/2211673752 wait complete. 2024-04-20T10:37:56.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.818+0000 7fbd9fe00640 1 Processor -- start 2024-04-20T10:37:56.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.818+0000 7fbd9fe00640 1 -- start start 2024-04-20T10:37:56.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.818+0000 7fbd9fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbda0102080 0x7fbda01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.818+0000 7fbd9fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda0107150 0x7fbda01a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda019a6f0 0x7fbda019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbda01121f0 con 0x7fbda019a6f0 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbda0112070 con 0x7fbda0102080 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbda0112370 con 0x7fbda0107150 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbda0102080 0x7fbda01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbda0102080 0x7fbda01a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46904/0 (socket says 172.21.15.42:46904) 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9ea00640 1 -- 172.21.15.42:0/969171721 learned_addr learned my addr 172.21.15.42:0/969171721 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9e000640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda0107150 0x7fbda01a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9f400640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda019a6f0 0x7fbda019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9ea00640 1 -- 172.21.15.42:0/969171721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda0107150 msgr2=0x7fbda01a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9ea00640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda0107150 0x7fbda01a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9ea00640 1 -- 172.21.15.42:0/969171721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda019a6f0 msgr2=0x7fbda019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9ea00640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda019a6f0 0x7fbda019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9ea00640 1 -- 172.21.15.42:0/969171721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda019b2f0 con 0x7fbda0102080 2024-04-20T10:37:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.819+0000 7fbd9f400640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda019a6f0 0x7fbda019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:37:56.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.820+0000 7fbd9ea00640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbda0102080 0x7fbda01a0630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000e9f0 tx=0x7fbd9000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:56.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.820+0000 7fbd97400640 1 -- 172.21.15.42:0/969171721 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000cd80 con 0x7fbda0102080 2024-04-20T10:37:56.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.820+0000 7fbd97400640 1 -- 172.21.15.42:0/969171721 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd90010910 con 0x7fbda0102080 2024-04-20T10:37:56.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.820+0000 7fbd9fe00640 1 -- 172.21.15.42:0/969171721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda01a73d0 con 0x7fbda0102080 2024-04-20T10:37:56.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.820+0000 7fbd97400640 1 -- 172.21.15.42:0/969171721 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90018990 con 0x7fbda0102080 2024-04-20T10:37:56.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.820+0000 7fbd9fe00640 1 -- 172.21.15.42:0/969171721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda01a7830 con 0x7fbda0102080 2024-04-20T10:37:56.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.823+0000 7fbd97400640 1 -- 172.21.15.42:0/969171721 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbd90010ac0 con 0x7fbda0102080 2024-04-20T10:37:56.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.823+0000 7fbda8631740 1 Processor -- start 2024-04-20T10:37:56.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.823+0000 7fbda8631740 1 -- start start 2024-04-20T10:37:56.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbda8631740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffa0f0 0x563c31ffa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbda8631740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c320030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbda8631740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 0x563c320059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbda8631740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563c31f5ed20 con 0x563c31ffaa10 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbda8631740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563c31f053e0 con 0x563c320035f0 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbda8631740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563c31f5ace0 con 0x563c31ffa0f0 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbd9ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c320030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbd9e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 0x563c320059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbd9f400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffa0f0 0x563c31ffa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbd9ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c320030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55698/0 (socket says 172.21.15.42:55698) 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbd9f400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffa0f0 0x563c31ffa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:58600/0 (socket says 172.21.15.42:58600) 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbd9ea00640 1 -- 172.21.15.42:0/3916656824 learned_addr learned my addr 172.21.15.42:0/3916656824 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.824+0000 7fbd9e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 0x563c320059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46918/0 (socket says 172.21.15.42:46918) 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd9e000640 1 -- 172.21.15.42:0/3916656824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffa0f0 msgr2=0x563c31ffa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd9e000640 1 --2- 172.21.15.42:0/3916656824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffa0f0 0x563c31ffa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd9e000640 1 -- 172.21.15.42:0/3916656824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 msgr2=0x563c320030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:56.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd9e000640 1 --2- 172.21.15.42:0/3916656824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c320030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd9e000640 1 -- 172.21.15.42:0/3916656824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c31f24980 con 0x563c320035f0 2024-04-20T10:37:56.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd9f400640 1 --2- 172.21.15.42:0/3916656824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffa0f0 0x563c31ffa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:56.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd9e000640 1 --2- 172.21.15.42:0/3916656824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 0x563c320059e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbda006b540 tx=0x7fbd880118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a7dd2c04b382c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:56.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd97400640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbd800a3da0 0x7fbd800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd8fe00640 1 -- 172.21.15.42:0/3916656824 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88003c30 con 0x563c320035f0 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd97400640 1 -- 172.21.15.42:0/969171721 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbd900d0b90 con 0x7fbda0102080 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd8fe00640 1 -- 172.21.15.42:0/3916656824 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd88003dd0 con 0x563c320035f0 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd8fe00640 1 -- 172.21.15.42:0/3916656824 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8800fb20 con 0x563c320035f0 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.825+0000 7fbd9f400640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbd800a3da0 0x7fbd800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.826+0000 7fbda8631740 1 -- 172.21.15.42:0/3916656824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 msgr2=0x563c320059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.826+0000 7fbd9f400640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbd800a3da0 0x7fbd800a6260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbd98005940 tx=0x7fbd980058b0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.826+0000 7fbda8631740 1 --2- 172.21.15.42:0/3916656824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 0x563c320059e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbda006b540 tx=0x7fbd880118d0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.826+0000 7fbda8631740 1 -- 172.21.15.42:0/3916656824 shutdown_connections 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.826+0000 7fbda8631740 1 --2- 172.21.15.42:0/3916656824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 0x563c320059e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.826+0000 7fbda8631740 1 --2- 172.21.15.42:0/3916656824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c320030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.826+0000 7fbda8631740 1 --2- 172.21.15.42:0/3916656824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffa0f0 0x563c31ffa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.826+0000 7fbda8631740 1 -- 172.21.15.42:0/3916656824 >> 172.21.15.42:0/3916656824 conn(0x563c31ff4820 msgr2=0x563c31ff4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:56.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.827+0000 7fbda8631740 1 -- 172.21.15.42:0/3916656824 shutdown_connections 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.827+0000 7fbda8631740 1 -- 172.21.15.42:0/3916656824 wait complete. 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.827+0000 7fbda8631740 1 -- start start 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.827+0000 7fbda8631740 1 Processor -- start 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbda8631740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffa0f0 0x563c32000630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbda8631740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c32000b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbda8631740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 0x563c320010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbda8631740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563c3200eb60 con 0x563c31ffaa10 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbda8631740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563c31f187a0 con 0x563c320035f0 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbda8631740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563c3200ece0 con 0x563c31ffa0f0 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbda8631740 10 client.0 _ll_register_callbacks cb 0x563c31f1ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbd9e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c32000b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbd9f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 0x563c320010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbd9e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c32000b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55700/0 (socket says 172.21.15.42:55700) 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbd9e000640 1 -- 172.21.15.42:0/2869489984 learned_addr learned my addr 172.21.15.42:0/2869489984 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:56.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbd9ea00640 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffa0f0 0x563c32000630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbd9f400640 1 -- 172.21.15.42:0/2869489984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffa0f0 msgr2=0x563c32000630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.828+0000 7fbd9f400640 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffa0f0 0x563c32000630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.829+0000 7fbd9f400640 1 -- 172.21.15.42:0/2869489984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 msgr2=0x563c32000b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.829+0000 7fbd9f400640 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c32000b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.829+0000 7fbd9f400640 1 -- 172.21.15.42:0/2869489984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c31d397e0 con 0x563c320035f0 2024-04-20T10:37:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.829+0000 7fbd9e000640 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c32000b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.829+0000 7fbd9f400640 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 0x563c320010b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009230 tx=0x7fbd98009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d692d6ed2a95c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.829+0000 7fbd8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:37:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.829+0000 7fbd8cc00640 1 -- 172.21.15.42:0/2869489984 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9804e070 con 0x563c320035f0 2024-04-20T10:37:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.829+0000 7fbd8cc00640 1 -- 172.21.15.42:0/2869489984 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbd98003cc0 con 0x563c320035f0 2024-04-20T10:37:56.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.829+0000 7fbda8631740 1 -- 172.21.15.42:0/2869489984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd88003dd0 con 0x563c320035f0 2024-04-20T10:37:56.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.830+0000 7fbd8cc00640 1 -- 172.21.15.42:0/2869489984 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbd9805a050 con 0x563c320035f0 2024-04-20T10:37:56.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.830+0000 7fbda8631740 10 client.25880 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.830+0000 7fbda8631740 1 -- 172.21.15.42:0/2869489984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563c31d99f70 con 0x563c320035f0 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.830+0000 7fbd8cc00640 1 -- 172.21.15.42:0/2869489984 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd88003dd0 con 0x563c320035f0 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.830+0000 7fbd8cc00640 1 -- 172.21.15.42:0/2869489984 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fbd98034440 con 0x563c320035f0 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.830+0000 7fbda8631740 10 client.25880 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.830+0000 7fbda8631740 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.830+0000 7fbda8631740 20 client.25880 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.830+0000 7fbda8631740 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563c3201c230 0x563c3201e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.830+0000 7fbda8631740 4 client.25880 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.830+0000 7fbda8631740 1 -- 172.21.15.42:0/2869489984 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c31f6c740 con 0x563c3201c230 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.831+0000 7fbd9ea00640 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563c3201c230 0x563c3201e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.831+0000 7fbd9ea00640 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563c3201c230 0x563c3201e6b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc7085520cc354c7 server_cookie=d548ee29f8f7c168 in_seq=0 out_seq=0 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.831+0000 7fbd8cc00640 10 client.25880 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.832+0000 7fbd8cc00640 1 -- 172.21.15.42:0/2869489984 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c31f6c740 con 0x563c3201c230 2024-04-20T10:37:56.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.832+0000 7fbd8cc00640 10 client.25880 handle_command_reply: tid=0 2024-04-20T10:37:56.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.832+0000 7fbda8631740 1 client.25880 shutdown 2024-04-20T10:37:56.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.832+0000 7fbda8631740 1 -- 172.21.15.42:0/2869489984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 msgr2=0x563c320010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:56.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.832+0000 7fbda8631740 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 0x563c320010b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009230 tx=0x7fbd98009260 comp rx=0 tx=0).stop 2024-04-20T10:37:56.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.832+0000 7fbda8631740 1 -- 172.21.15.42:0/2869489984 shutdown_connections 2024-04-20T10:37:56.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.832+0000 7fbda8631740 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563c3201c230 0x563c3201e6b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.832+0000 7fbda8631740 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c320035f0 0x563c320010b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.832+0000 7fbda8631740 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c32000b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.832+0000 7fbda8631740 1 --2- 172.21.15.42:0/2869489984 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffa0f0 0x563c32000630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.832+0000 7fbd8cc00640 0 client.25880 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:56.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.832+0000 7fbda8631740 1 -- 172.21.15.42:0/2869489984 >> 172.21.15.42:0/2869489984 conn(0x563c31ff4820 msgr2=0x563c31ff6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:56.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.833+0000 7fbda8631740 1 -- 172.21.15.42:0/2869489984 shutdown_connections 2024-04-20T10:37:56.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.833+0000 7fbda8631740 1 -- 172.21.15.42:0/2869489984 wait complete. 2024-04-20T10:37:56.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.833+0000 7fbda8631740 20 client.25880 trim_cache size 0 max 16384 2024-04-20T10:37:56.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.854+0000 7fbda8631740 1 Processor -- start 2024-04-20T10:37:56.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.854+0000 7fbda8631740 1 -- start start 2024-04-20T10:37:56.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.854+0000 7fbda8631740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 0x563c32002d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.854+0000 7fbda8631740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c32017b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.854+0000 7fbda8631740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c320035f0 0x563c32023f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.854+0000 7fbda8631740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbda01a9970 con 0x563c31ffaa10 2024-04-20T10:37:56.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.854+0000 7fbda8631740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbda01ac1b0 con 0x563c31ffa0f0 2024-04-20T10:37:56.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.854+0000 7fbda8631740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbda01a9fc0 con 0x563c320035f0 2024-04-20T10:37:56.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.855+0000 7fbd9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 0x563c32002d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.855+0000 7fbd9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 0x563c32002d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46936/0 (socket says 172.21.15.42:46936) 2024-04-20T10:37:56.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.855+0000 7fbd9ea00640 1 -- 172.21.15.42:0/1730991193 learned_addr learned my addr 172.21.15.42:0/1730991193 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:56.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.855+0000 7fbd9e000640 1 --2- 172.21.15.42:0/1730991193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c32017b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.855+0000 7fbd9ea00640 1 -- 172.21.15.42:0/1730991193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c320035f0 msgr2=0x563c32023f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.855+0000 7fbd9f400640 1 --2- 172.21.15.42:0/1730991193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c320035f0 0x563c32023f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.855+0000 7fbd9ea00640 1 --2- 172.21.15.42:0/1730991193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c320035f0 0x563c32023f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.855+0000 7fbd9ea00640 1 -- 172.21.15.42:0/1730991193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 msgr2=0x563c32017b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.855+0000 7fbd9ea00640 1 --2- 172.21.15.42:0/1730991193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c32017b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.855+0000 7fbd9ea00640 1 -- 172.21.15.42:0/1730991193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c31ffa840 con 0x563c31ffa0f0 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.855+0000 7fbd9e000640 1 --2- 172.21.15.42:0/1730991193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c32017b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.855+0000 7fbd9f400640 1 --2- 172.21.15.42:0/1730991193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c320035f0 0x563c32023f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.856+0000 7fbd9ea00640 1 --2- 172.21.15.42:0/1730991193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 0x563c32002d60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbd9009c7d0 tx=0x7fbd900bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b46f8deb3de14ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.856+0000 7fbd8f400640 1 -- 172.21.15.42:0/1730991193 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd900d5040 con 0x563c31ffa0f0 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.856+0000 7fbd8f400640 1 -- 172.21.15.42:0/1730991193 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9009d1d0 con 0x563c31ffa0f0 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.856+0000 7fbd8f400640 1 -- 172.21.15.42:0/1730991193 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9009add0 con 0x563c31ffa0f0 2024-04-20T10:37:56.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.857+0000 7fbda8631740 1 -- 172.21.15.42:0/1730991193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 msgr2=0x563c32002d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:56.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.857+0000 7fbda8631740 1 --2- 172.21.15.42:0/1730991193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 0x563c32002d60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbd9009c7d0 tx=0x7fbd900bd040 comp rx=0 tx=0).stop 2024-04-20T10:37:56.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.857+0000 7fbda8631740 1 -- 172.21.15.42:0/1730991193 shutdown_connections 2024-04-20T10:37:56.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.857+0000 7fbda8631740 1 --2- 172.21.15.42:0/1730991193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c320035f0 0x563c32023f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.857+0000 7fbda8631740 1 --2- 172.21.15.42:0/1730991193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c31ffaa10 0x563c32017b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.857+0000 7fbda8631740 1 --2- 172.21.15.42:0/1730991193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 0x563c32002d60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.857+0000 7fbda8631740 1 -- 172.21.15.42:0/1730991193 >> 172.21.15.42:0/1730991193 conn(0x563c31ff4820 msgr2=0x563c31ff3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:56.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.857+0000 7fbda8631740 1 -- 172.21.15.42:0/1730991193 shutdown_connections 2024-04-20T10:37:56.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.857+0000 7fbda8631740 1 -- 172.21.15.42:0/1730991193 wait complete. 2024-04-20T10:37:56.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.857+0000 7fbda8631740 1 -- start start 2024-04-20T10:37:56.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.857+0000 7fbda8631740 1 Processor -- start 2024-04-20T10:37:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.858+0000 7fbda8631740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 0x563c31ffd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.858+0000 7fbda8631740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffaa10 0x563c31ffd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.858+0000 7fbda8631740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c320035f0 0x563c31ffded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.858+0000 7fbda8631740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbda01aa670 con 0x563c320035f0 2024-04-20T10:37:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.858+0000 7fbda8631740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbda01abae0 con 0x563c31ffa0f0 2024-04-20T10:37:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.858+0000 7fbda8631740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbda01aad40 con 0x563c31ffaa10 2024-04-20T10:37:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.858+0000 7fbda8631740 10 client.0 _ll_register_callbacks cb 0x7fbd880093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.858+0000 7fbd9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 0x563c31ffd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.858+0000 7fbd9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 0x563c31ffd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46944/0 (socket says 172.21.15.42:46944) 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.858+0000 7fbd9ea00640 1 -- 172.21.15.42:0/1223846674 learned_addr learned my addr 172.21.15.42:0/1223846674 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.858+0000 7fbd9f400640 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c320035f0 0x563c31ffded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.858+0000 7fbd9e000640 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffaa10 0x563c31ffd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.859+0000 7fbd9f400640 1 -- 172.21.15.42:0/1223846674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffaa10 msgr2=0x563c31ffd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.859+0000 7fbd9f400640 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffaa10 0x563c31ffd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.859+0000 7fbd9f400640 1 -- 172.21.15.42:0/1223846674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 msgr2=0x563c31ffd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.859+0000 7fbd9f400640 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 0x563c31ffd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.859+0000 7fbd9f400640 1 -- 172.21.15.42:0/1223846674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c31c31170 con 0x563c320035f0 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.859+0000 7fbd9e000640 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffaa10 0x563c31ffd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.859+0000 7fbd9ea00640 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 0x563c31ffd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.859+0000 7fbd9f400640 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c320035f0 0x563c31ffded0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbd98034890 tx=0x7fbd980346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3cf31866047d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.859+0000 7fbd8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.860+0000 7fbd8cc00640 1 -- 172.21.15.42:0/1223846674 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98063060 con 0x563c320035f0 2024-04-20T10:37:56.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.860+0000 7fbda8631740 1 -- 172.21.15.42:0/1223846674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd9009d1d0 con 0x563c320035f0 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.860+0000 7fbd8cc00640 1 -- 172.21.15.42:0/1223846674 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbd980039c0 con 0x563c320035f0 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.860+0000 7fbd8cc00640 1 -- 172.21.15.42:0/1223846674 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbd9804e050 con 0x563c320035f0 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.860+0000 7fbda8631740 10 client.15768 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.860+0000 7fbda8631740 1 -- 172.21.15.42:0/1223846674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563c32082a30 con 0x563c320035f0 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.860+0000 7fbd8cc00640 1 -- 172.21.15.42:0/1223846674 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9009d1d0 con 0x563c320035f0 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.861+0000 7fbd8cc00640 1 -- 172.21.15.42:0/1223846674 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fbd980046c0 con 0x563c320035f0 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.861+0000 7fbda8631740 10 client.15768 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.861+0000 7fbda8631740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.861+0000 7fbda8631740 20 client.15768 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.861+0000 7fbda8631740 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563c320831b0 0x563c320855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.861+0000 7fbda8631740 4 client.15768 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.861+0000 7fbda8631740 1 -- 172.21.15.42:0/1223846674 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563c31f665b0 con 0x563c320831b0 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.861+0000 7fbd9ea00640 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563c320831b0 0x563c320855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.861+0000 7fbd9ea00640 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563c320831b0 0x563c320855d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2323e6ca077abbbc server_cookie=75ed23166f826936 in_seq=0 out_seq=0 2024-04-20T10:37:56.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.861+0000 7fbd8cc00640 10 client.15768 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.863+0000 7fbd8cc00640 1 -- 172.21.15.42:0/1223846674 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563c31f665b0 con 0x563c320831b0 2024-04-20T10:37:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.863+0000 7fbd8cc00640 10 client.15768 handle_command_reply: tid=0 2024-04-20T10:37:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.863+0000 7fbda8631740 1 client.15768 shutdown 2024-04-20T10:37:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.864+0000 7fbda8631740 1 -- 172.21.15.42:0/1223846674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c320035f0 msgr2=0x563c31ffded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.864+0000 7fbda8631740 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c320035f0 0x563c31ffded0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbd98034890 tx=0x7fbd980346f0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.864+0000 7fbda8631740 1 -- 172.21.15.42:0/1223846674 shutdown_connections 2024-04-20T10:37:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.864+0000 7fbda8631740 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563c320831b0 0x563c320855d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.864+0000 7fbda8631740 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c320035f0 0x563c31ffded0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.864+0000 7fbda8631740 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563c31ffaa10 0x563c31ffd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.864+0000 7fbda8631740 1 --2- 172.21.15.42:0/1223846674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c31ffa0f0 0x563c31ffd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.864+0000 7fbd8cc00640 0 client.15768 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.864+0000 7fbda8631740 1 -- 172.21.15.42:0/1223846674 >> 172.21.15.42:0/1223846674 conn(0x563c31ff4820 msgr2=0x563c320005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.864+0000 7fbda8631740 1 -- 172.21.15.42:0/1223846674 shutdown_connections 2024-04-20T10:37:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.864+0000 7fbda8631740 1 -- 172.21.15.42:0/1223846674 wait complete. 2024-04-20T10:37:56.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.864+0000 7fbda8631740 20 client.15768 trim_cache size 0 max 16384 2024-04-20T10:37:56.867 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:37:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:37:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "bc8b157a-d296-45a5-87c7-24c44281e350", 2024-04-20T10:37:56.867 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:37:56.867 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:37:56.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.865+0000 7fbd9fe00640 1 -- 172.21.15.42:0/969171721 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbd800a3da0 msgr2=0x7fbd800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:56.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.865+0000 7fbd9fe00640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbd800a3da0 0x7fbd800a6260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbd98005940 tx=0x7fbd980058b0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.865+0000 7fbd9fe00640 1 -- 172.21.15.42:0/969171721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbda0102080 msgr2=0x7fbda01a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:56.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.865+0000 7fbd9fe00640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbda0102080 0x7fbda01a0630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000e9f0 tx=0x7fbd9000eeb0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.865+0000 7fbd9fe00640 1 -- 172.21.15.42:0/969171721 shutdown_connections 2024-04-20T10:37:56.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.865+0000 7fbd9fe00640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbd800a3da0 0x7fbd800a6260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.865+0000 7fbd9fe00640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda019a6f0 0x7fbda019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.865+0000 7fbd9fe00640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda0107150 0x7fbda01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.865+0000 7fbd9fe00640 1 --2- 172.21.15.42:0/969171721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbda0102080 0x7fbda01a0630 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:56.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.865+0000 7fbd9fe00640 1 -- 172.21.15.42:0/969171721 >> 172.21.15.42:0/969171721 conn(0x7fbda00fd900 msgr2=0x7fbda00ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:56.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.865+0000 7fbd9fe00640 1 -- 172.21.15.42:0/969171721 shutdown_connections 2024-04-20T10:37:56.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:56.865+0000 7fbd9fe00640 1 -- 172.21.15.42:0/969171721 wait complete. 2024-04-20T10:37:56.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:37:56.887 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:37:57.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.098+0000 7fe8c6200640 1 -- 172.21.15.42:0/4215107272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c0102fc0 msgr2=0x7fe8c01033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:57.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.098+0000 7fe8c6200640 1 --2- 172.21.15.42:0/4215107272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c0102fc0 0x7fe8c01033a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009860 tx=0x7fe8b002bc20 comp rx=0 tx=0).stop 2024-04-20T10:37:57.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.098+0000 7fe8c6200640 1 -- 172.21.15.42:0/4215107272 shutdown_connections 2024-04-20T10:37:57.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.098+0000 7fe8c6200640 1 --2- 172.21.15.42:0/4215107272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c010c3e0 0x7fe8c010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.098+0000 7fe8c6200640 1 --2- 172.21.15.42:0/4215107272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c01038e0 0x7fe8c010bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.098+0000 7fe8c6200640 1 --2- 172.21.15.42:0/4215107272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c0102fc0 0x7fe8c01033a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.098+0000 7fe8c6200640 1 -- 172.21.15.42:0/4215107272 >> 172.21.15.42:0/4215107272 conn(0x7fe8c00fc7e0 msgr2=0x7fe8c00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:57.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.098+0000 7fe8c6200640 1 -- 172.21.15.42:0/4215107272 shutdown_connections 2024-04-20T10:37:57.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.099+0000 7fe8c6200640 1 -- 172.21.15.42:0/4215107272 wait complete. 2024-04-20T10:37:57.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8c6200640 1 Processor -- start 2024-04-20T10:37:57.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8c6200640 1 -- start start 2024-04-20T10:37:57.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8c6200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c0102fc0 0x7fe8c01964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8c6200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c01038e0 0x7fe8c0196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8c6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c010c3e0 0x7fe8c019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8c6200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c010ffd0 con 0x7fe8c0102fc0 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8c6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c010fe50 con 0x7fe8c010c3e0 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8c6200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0110150 con 0x7fe8c01038e0 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8c4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c0102fc0 0x7fe8c01964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8c4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c0102fc0 0x7fe8c01964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55756/0 (socket says 172.21.15.42:55756) 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8c4e00640 1 -- 172.21.15.42:0/2317932844 learned_addr learned my addr 172.21.15.42:0/2317932844 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8c5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c010c3e0 0x7fe8c019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.100+0000 7fe8bfe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c01038e0 0x7fe8c0196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.101+0000 7fe8c4e00640 1 -- 172.21.15.42:0/2317932844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c01038e0 msgr2=0x7fe8c0196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.101+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c01038e0 0x7fe8c0196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.101+0000 7fe8c4e00640 1 -- 172.21.15.42:0/2317932844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c010c3e0 msgr2=0x7fe8c019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:57.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.101+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c010c3e0 0x7fe8c019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.101+0000 7fe8c4e00640 1 -- 172.21.15.42:0/2317932844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c019b490 con 0x7fe8c0102fc0 2024-04-20T10:37:57.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.101+0000 7fe8c5800640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c010c3e0 0x7fe8c019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:57.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.101+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c0102fc0 0x7fe8c01964f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0037040 tx=0x7fe8b00344c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:57.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.102+0000 7fe8bd600640 1 -- 172.21.15.42:0/2317932844 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0041020 con 0x7fe8c0102fc0 2024-04-20T10:37:57.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.102+0000 7fe8bd600640 1 -- 172.21.15.42:0/2317932844 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8b002ce50 con 0x7fe8c0102fc0 2024-04-20T10:37:57.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.102+0000 7fe8c6200640 1 -- 172.21.15.42:0/2317932844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c019b720 con 0x7fe8c0102fc0 2024-04-20T10:37:57.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.102+0000 7fe8bd600640 1 -- 172.21.15.42:0/2317932844 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0039040 con 0x7fe8c0102fc0 2024-04-20T10:37:57.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.102+0000 7fe8c6200640 1 -- 172.21.15.42:0/2317932844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c01a3010 con 0x7fe8c0102fc0 2024-04-20T10:37:57.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.105+0000 7fe8bd600640 1 -- 172.21.15.42:0/2317932844 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe8b0044020 con 0x7fe8c0102fc0 2024-04-20T10:37:57.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.105+0000 7fe8ca5e5740 1 Processor -- start 2024-04-20T10:37:57.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.106+0000 7fe8ca5e5740 1 -- start start 2024-04-20T10:37:57.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.106+0000 7fe8ca5e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.106+0000 7fe8ca5e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 0x55bd6dafb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.106+0000 7fe8ca5e5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6dafb880 0x55bd6dafdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.106+0000 7fe8ca5e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bd6da58150 con 0x55bd6daf2380 2024-04-20T10:37:57.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.106+0000 7fe8ca5e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bd6d9fe370 con 0x55bd6daf2ca0 2024-04-20T10:37:57.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.106+0000 7fe8ca5e5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55bd6da54110 con 0x55bd6dafb880 2024-04-20T10:37:57.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.106+0000 7fe8c4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 0x55bd6dafb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.106+0000 7fe8c4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 0x55bd6dafb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46962/0 (socket says 172.21.15.42:46962) 2024-04-20T10:37:57.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.106+0000 7fe8c4e00640 1 -- 172.21.15.42:0/672403032 learned_addr learned my addr 172.21.15.42:0/672403032 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:57.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.106+0000 7fe8c5800640 1 --2- 172.21.15.42:0/672403032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.106+0000 7fe8bfe00640 1 --2- 172.21.15.42:0/672403032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6dafb880 0x55bd6dafdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.107+0000 7fe8c4e00640 1 -- 172.21.15.42:0/672403032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6dafb880 msgr2=0x55bd6dafdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.107+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/672403032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6dafb880 0x55bd6dafdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.107+0000 7fe8c4e00640 1 -- 172.21.15.42:0/672403032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 msgr2=0x55bd6daf2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.107+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/672403032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.107+0000 7fe8c4e00640 1 -- 172.21.15.42:0/672403032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd6d8347d0 con 0x55bd6daf2ca0 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.107+0000 7fe8c5800640 1 --2- 172.21.15.42:0/672403032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.107+0000 7fe8bd600640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe8a00a3d20 0x7fe8a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.107+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/672403032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 0x55bd6dafb340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00ef3c0 tx=0x7fe8b00f5d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc895f3c4b75522a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.107+0000 7fe8bd600640 1 -- 172.21.15.42:0/2317932844 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe8b00f1c80 con 0x7fe8c0102fc0 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.107+0000 7fe8c5800640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe8a00a3d20 0x7fe8a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.107+0000 7fe8b6000640 1 -- 172.21.15.42:0/672403032 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00b9f70 con 0x55bd6daf2ca0 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.108+0000 7fe8b6000640 1 -- 172.21.15.42:0/672403032 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8b00be3d0 con 0x55bd6daf2ca0 2024-04-20T10:37:57.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.108+0000 7fe8b6000640 1 -- 172.21.15.42:0/672403032 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00bd630 con 0x55bd6daf2ca0 2024-04-20T10:37:57.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.108+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/672403032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 msgr2=0x55bd6dafb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:57.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.108+0000 7fe8c5800640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe8a00a3d20 0x7fe8a00a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8005fd0 tx=0x7fe8b8011040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:57.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.108+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/672403032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 0x55bd6dafb340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00ef3c0 tx=0x7fe8b00f5d20 comp rx=0 tx=0).stop 2024-04-20T10:37:57.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.109+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/672403032 shutdown_connections 2024-04-20T10:37:57.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.109+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/672403032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6dafb880 0x55bd6dafdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.109+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/672403032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 0x55bd6dafb340 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.109+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/672403032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.109+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/672403032 >> 172.21.15.42:0/672403032 conn(0x55bd6daecab0 msgr2=0x55bd6daece90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.109+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/672403032 shutdown_connections 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.109+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/672403032 wait complete. 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.109+0000 7fe8ca5e5740 1 -- start start 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.110+0000 7fe8ca5e5740 1 Processor -- start 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.110+0000 7fe8ca5e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.110+0000 7fe8ca5e5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.110+0000 7fe8ca5e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6dafb880 0x55bd6daf9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.110+0000 7fe8ca5e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bd6dafe430 con 0x55bd6daf2380 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.110+0000 7fe8ca5e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bd6da11740 con 0x55bd6dafb880 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.110+0000 7fe8ca5e5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55bd6dafe5b0 con 0x55bd6daf2ca0 2024-04-20T10:37:57.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.110+0000 7fe8ca5e5740 10 client.0 _ll_register_callbacks cb 0x55bd6da0cb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.110+0000 7fe8c4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.110+0000 7fe8c4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55760/0 (socket says 172.21.15.42:55760) 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.110+0000 7fe8c4e00640 1 -- 172.21.15.42:0/4045154645 learned_addr learned my addr 172.21.15.42:0/4045154645 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.110+0000 7fe8c5800640 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6dafb880 0x55bd6daf9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.111+0000 7fe8bfe00640 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.111+0000 7fe8c5800640 1 -- 172.21.15.42:0/4045154645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6daf2ca0 msgr2=0x55bd6daf8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.111+0000 7fe8c5800640 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.111+0000 7fe8c5800640 1 -- 172.21.15.42:0/4045154645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 msgr2=0x55bd6daf88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.111+0000 7fe8c5800640 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.111+0000 7fe8c5800640 1 -- 172.21.15.42:0/4045154645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd6d76a3f0 con 0x55bd6dafb880 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.111+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.111+0000 7fe8bfe00640 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.111+0000 7fe8c5800640 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6dafb880 0x55bd6daf9360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800e8e0 tx=0x7fe8b8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3732b046e5050e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.111+0000 7fe8aea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:37:57.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.112+0000 7fe8aea00640 1 -- 172.21.15.42:0/4045154645 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b802e060 con 0x55bd6dafb880 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.112+0000 7fe8aea00640 1 -- 172.21.15.42:0/4045154645 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe8b801ebf0 con 0x55bd6dafb880 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.112+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/4045154645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b00be3d0 con 0x55bd6dafb880 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.112+0000 7fe8aea00640 1 -- 172.21.15.42:0/4045154645 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe8b801edf0 con 0x55bd6dafb880 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.112+0000 7fe8ca5e5740 10 client.25898 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.112+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/4045154645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bd6d72a0f0 con 0x55bd6dafb880 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.112+0000 7fe8aea00640 1 -- 172.21.15.42:0/4045154645 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8b00be3d0 con 0x55bd6dafb880 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.112+0000 7fe8aea00640 1 -- 172.21.15.42:0/4045154645 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fe8b8029430 con 0x55bd6dafb880 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.113+0000 7fe8ca5e5740 10 client.25898 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.113+0000 7fe8ca5e5740 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.113+0000 7fe8ca5e5740 20 client.25898 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.113+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bd6db0b9c0 0x55bd6db0dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.113+0000 7fe8ca5e5740 4 client.25898 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.113+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/4045154645 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd6da65b70 con 0x55bd6db0b9c0 2024-04-20T10:37:57.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.113+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bd6db0b9c0 0x55bd6db0dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.113+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bd6db0b9c0 0x55bd6db0dde0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b21fde5fcc1dd08 server_cookie=c11f7a29ee9fe314 in_seq=0 out_seq=0 2024-04-20T10:37:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.113+0000 7fe8aea00640 10 client.25898 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.115+0000 7fe8aea00640 1 -- 172.21.15.42:0/4045154645 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd6da65b70 con 0x55bd6db0b9c0 2024-04-20T10:37:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.115+0000 7fe8aea00640 10 client.25898 handle_command_reply: tid=0 2024-04-20T10:37:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.115+0000 7fe8ca5e5740 1 client.25898 shutdown 2024-04-20T10:37:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.115+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/4045154645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6dafb880 msgr2=0x55bd6daf9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.115+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6dafb880 0x55bd6daf9360 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800e8e0 tx=0x7fe8b8004730 comp rx=0 tx=0).stop 2024-04-20T10:37:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.115+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/4045154645 shutdown_connections 2024-04-20T10:37:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.115+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bd6db0b9c0 0x55bd6db0dde0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.115+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6dafb880 0x55bd6daf9360 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.115+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.115+0000 7fe8aea00640 0 client.25898 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:57.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.115+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/4045154645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.115+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/4045154645 >> 172.21.15.42:0/4045154645 conn(0x55bd6daecab0 msgr2=0x55bd6daee800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:57.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.116+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/4045154645 shutdown_connections 2024-04-20T10:37:57.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.116+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/4045154645 wait complete. 2024-04-20T10:37:57.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.116+0000 7fe8ca5e5740 20 client.25898 trim_cache size 0 max 16384 2024-04-20T10:37:57.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.138+0000 7fe8ca5e5740 1 Processor -- start 2024-04-20T10:37:57.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.139+0000 7fe8ca5e5740 1 -- start start 2024-04-20T10:37:57.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.139+0000 7fe8ca5e5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6daf2380 0x55bd6dafb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.139+0000 7fe8ca5e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.139+0000 7fe8ca5e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6dafb880 0x55bd6db07260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.139+0000 7fe8ca5e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01a9900 con 0x55bd6daf2ca0 2024-04-20T10:37:57.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.139+0000 7fe8ca5e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01ac180 con 0x55bd6dafb880 2024-04-20T10:37:57.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.139+0000 7fe8ca5e5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01a9f70 con 0x55bd6daf2380 2024-04-20T10:37:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.139+0000 7fe8bfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.139+0000 7fe8c4e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6daf2380 0x55bd6dafb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.139+0000 7fe8bfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55770/0 (socket says 172.21.15.42:55770) 2024-04-20T10:37:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.139+0000 7fe8bfe00640 1 -- 172.21.15.42:0/560953852 learned_addr learned my addr 172.21.15.42:0/560953852 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.139+0000 7fe8c5800640 1 --2- 172.21.15.42:0/560953852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6dafb880 0x55bd6db07260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.140+0000 7fe8c4e00640 1 -- 172.21.15.42:0/560953852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6dafb880 msgr2=0x55bd6db07260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.140+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/560953852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6dafb880 0x55bd6db07260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.140+0000 7fe8c4e00640 1 -- 172.21.15.42:0/560953852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2ca0 msgr2=0x55bd6daf7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.140+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/560953852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.140+0000 7fe8c4e00640 1 -- 172.21.15.42:0/560953852 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd6daec810 con 0x55bd6daf2380 2024-04-20T10:37:57.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.140+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/560953852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6daf2380 0x55bd6dafb010 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00fa740 tx=0x7fe8b002bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0ba62a7965f9119 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.141+0000 7fe8b5600640 1 -- 172.21.15.42:0/560953852 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00bd810 con 0x55bd6daf2380 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.141+0000 7fe8b5600640 1 -- 172.21.15.42:0/560953852 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8b00be3d0 con 0x55bd6daf2380 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.141+0000 7fe8b5600640 1 -- 172.21.15.42:0/560953852 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00bdaa0 con 0x55bd6daf2380 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.142+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/560953852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6daf2380 msgr2=0x55bd6dafb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.142+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/560953852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6daf2380 0x55bd6dafb010 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00fa740 tx=0x7fe8b002bc20 comp rx=0 tx=0).stop 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.142+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/560953852 shutdown_connections 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.142+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/560953852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6dafb880 0x55bd6db07260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.142+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/560953852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.142+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/560953852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6daf2380 0x55bd6dafb010 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.142+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/560953852 >> 172.21.15.42:0/560953852 conn(0x55bd6daecab0 msgr2=0x55bd6daebb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.142+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/560953852 shutdown_connections 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.142+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/560953852 wait complete. 2024-04-20T10:37:57.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.142+0000 7fe8ca5e5740 1 -- start start 2024-04-20T10:37:57.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8ca5e5740 1 Processor -- start 2024-04-20T10:37:57.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8ca5e5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8ca5e5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8ca5e5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6dafb880 0x55bd6db70f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8ca5e5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01aa640 con 0x55bd6daf2380 2024-04-20T10:37:57.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8ca5e5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01abab0 con 0x55bd6daf2ca0 2024-04-20T10:37:57.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8ca5e5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01aad10 con 0x55bd6dafb880 2024-04-20T10:37:57.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8ca5e5740 10 client.0 _ll_register_callbacks cb 0x7fe8b800fea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:37:57.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8c4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:46992/0 (socket says 172.21.15.42:46992) 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8bfe00640 1 -- 172.21.15.42:0/2789534820 learned_addr learned my addr 172.21.15.42:0/2789534820 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.143+0000 7fe8c4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55778/0 (socket says 172.21.15.42:55778) 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.144+0000 7fe8c5800640 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6dafb880 0x55bd6db70f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.144+0000 7fe8c4e00640 1 -- 172.21.15.42:0/2789534820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6dafb880 msgr2=0x55bd6db70f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.144+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6dafb880 0x55bd6db70f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.144+0000 7fe8c4e00640 1 -- 172.21.15.42:0/2789534820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 msgr2=0x55bd6daf5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.144+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.144+0000 7fe8c4e00640 1 -- 172.21.15.42:0/2789534820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd6daf67c0 con 0x55bd6daf2380 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.144+0000 7fe8c5800640 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6dafb880 0x55bd6db70f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.144+0000 7fe8bfe00640 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.144+0000 7fe8c4e00640 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf59f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00f5a20 tx=0x7fe8b0005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8e738ef28a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.144+0000 7fe8aea00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8aea00640 1 -- 172.21.15.42:0/2789534820 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00bdaa0 con 0x55bd6daf2380 2024-04-20T10:37:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8aea00640 1 -- 172.21.15.42:0/2789534820 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe8b00bbfd0 con 0x55bd6daf2380 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/2789534820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b00be3d0 con 0x55bd6daf2380 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8aea00640 1 -- 172.21.15.42:0/2789534820 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe8b00bd630 con 0x55bd6daf2380 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8ca5e5740 10 client.15786 fetch_fsmap learned FSMap version 19 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/2789534820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bd6db72540 con 0x55bd6daf2380 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8aea00640 1 -- 172.21.15.42:0/2789534820 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8b00be3d0 con 0x55bd6daf2380 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8aea00640 1 -- 172.21.15.42:0/2789534820 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fe8b00f9040 con 0x55bd6daf2380 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8ca5e5740 10 client.15786 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8ca5e5740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8ca5e5740 20 client.15786 populate_metadata read hostname 'smithi042' 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bd6db72880 0x55bd6db74ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8ca5e5740 4 client.15786 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.145+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/2789534820 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bd6da5f9e0 con 0x55bd6db72880 2024-04-20T10:37:57.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.146+0000 7fe8bfe00640 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bd6db72880 0x55bd6db74ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:37:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.146+0000 7fe8bfe00640 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bd6db72880 0x55bd6db74ca0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1536f57741ab8e0 server_cookie=72f00ad58ef0205b in_seq=0 out_seq=0 2024-04-20T10:37:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.146+0000 7fe8aea00640 10 client.15786 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.148+0000 7fe8aea00640 1 -- 172.21.15.42:0/2789534820 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55bd6da5f9e0 con 0x55bd6db72880 2024-04-20T10:37:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.148+0000 7fe8aea00640 10 client.15786 handle_command_reply: tid=0 2024-04-20T10:37:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.148+0000 7fe8ca5e5740 1 client.15786 shutdown 2024-04-20T10:37:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.149+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/2789534820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 msgr2=0x55bd6daf59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.149+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf59f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00f5a20 tx=0x7fe8b0005a90 comp rx=0 tx=0).stop 2024-04-20T10:37:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.149+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/2789534820 shutdown_connections 2024-04-20T10:37:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.149+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bd6db72880 0x55bd6db74ca0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.149+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd6dafb880 0x55bd6db70f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.149+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bd6daf2ca0 0x55bd6daf5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.149+0000 7fe8ca5e5740 1 --2- 172.21.15.42:0/2789534820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bd6daf2380 0x55bd6daf59f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.149+0000 7fe8aea00640 0 client.15786 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:37:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.149+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/2789534820 >> 172.21.15.42:0/2789534820 conn(0x55bd6daecab0 msgr2=0x55bd6daf88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.149+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/2789534820 shutdown_connections 2024-04-20T10:37:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.149+0000 7fe8ca5e5740 1 -- 172.21.15.42:0/2789534820 wait complete. 2024-04-20T10:37:57.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.149+0000 7fe8ca5e5740 20 client.15786 trim_cache size 0 max 16384 2024-04-20T10:37:57.151 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:37:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (382 inodes in the stack)", 2024-04-20T10:37:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:37:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "bc8b157a-d296-45a5-87c7-24c44281e350": { 2024-04-20T10:37:57.151 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:37:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "bc8b157a-d296-45a5-87c7-24c44281e350", 2024-04-20T10:37:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:37:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:37:57.152 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:37:57.152 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:37:57.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.150+0000 7fe8c6200640 1 -- 172.21.15.42:0/2317932844 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe8a00a3d20 msgr2=0x7fe8a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.150+0000 7fe8c6200640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe8a00a3d20 0x7fe8a00a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8005fd0 tx=0x7fe8b8011040 comp rx=0 tx=0).stop 2024-04-20T10:37:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.150+0000 7fe8c6200640 1 -- 172.21.15.42:0/2317932844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c0102fc0 msgr2=0x7fe8c01964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:37:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.150+0000 7fe8c6200640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c0102fc0 0x7fe8c01964f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0037040 tx=0x7fe8b00344c0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.150+0000 7fe8c6200640 1 -- 172.21.15.42:0/2317932844 shutdown_connections 2024-04-20T10:37:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.150+0000 7fe8c6200640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe8a00a3d20 0x7fe8a00a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.150+0000 7fe8c6200640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c010c3e0 0x7fe8c019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.150+0000 7fe8c6200640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c01038e0 0x7fe8c0196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.150+0000 7fe8c6200640 1 --2- 172.21.15.42:0/2317932844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c0102fc0 0x7fe8c01964f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:37:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.150+0000 7fe8c6200640 1 -- 172.21.15.42:0/2317932844 >> 172.21.15.42:0/2317932844 conn(0x7fe8c00fc7e0 msgr2=0x7fe8c00fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:37:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.151+0000 7fe8c6200640 1 -- 172.21.15.42:0/2317932844 shutdown_connections 2024-04-20T10:37:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:37:57.151+0000 7fe8c6200640 1 -- 172.21.15.42:0/2317932844 wait complete. 2024-04-20T10:37:57.171 INFO:tasks.cephfs.filesystem:scrub status for tag:bc8b157a-d296-45a5-87c7-24c44281e350 - {'path': '/', 'tag': 'bc8b157a-d296-45a5-87c7-24c44281e350', 'options': 'recursive,force'} 2024-04-20T10:37:57.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:57 smithi042 ceph-mon[17078]: pgmap v121: 97 pgs: 97 active+clean; 418 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 771 KiB/s wr, 61 op/s 2024-04-20T10:37:57.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:57 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:37:57.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:57 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:37:57.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:57 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:37:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:57 smithi162 ceph-mon[21854]: pgmap v121: 97 pgs: 97 active+clean; 418 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 771 KiB/s wr, 61 op/s 2024-04-20T10:37:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:57 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:37:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:57 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:37:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:57 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:37:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:57 smithi046 ceph-mon[21574]: pgmap v121: 97 pgs: 97 active+clean; 418 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 771 KiB/s wr, 61 op/s 2024-04-20T10:37:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:57 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:37:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:57 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:37:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:57 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:37:59.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:37:59 smithi042 ceph-mon[17078]: pgmap v122: 97 pgs: 97 active+clean; 426 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-04-20T10:37:59.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:37:59 smithi162 ceph-mon[21854]: pgmap v122: 97 pgs: 97 active+clean; 426 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-04-20T10:38:00.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:37:59 smithi046 ceph-mon[21574]: pgmap v122: 97 pgs: 97 active+clean; 426 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-04-20T10:38:01.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:01 smithi042 ceph-mon[17078]: pgmap v123: 97 pgs: 97 active+clean; 426 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-04-20T10:38:01.854 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:01 smithi046 ceph-mon[21574]: pgmap v123: 97 pgs: 97 active+clean; 426 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-04-20T10:38:01.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:01 smithi162 ceph-mon[21854]: pgmap v123: 97 pgs: 97 active+clean; 426 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-04-20T10:38:03.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:03 smithi162 ceph-mon[21854]: pgmap v124: 97 pgs: 97 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 5.9 MiB/s wr, 410 op/s 2024-04-20T10:38:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:03 smithi046 ceph-mon[21574]: pgmap v124: 97 pgs: 97 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 5.9 MiB/s wr, 410 op/s 2024-04-20T10:38:04.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:03 smithi042 ceph-mon[17078]: pgmap v124: 97 pgs: 97 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 5.9 MiB/s wr, 410 op/s 2024-04-20T10:38:05.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:05 smithi162 ceph-mon[21854]: pgmap v125: 97 pgs: 97 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 5.9 MiB/s wr, 399 op/s 2024-04-20T10:38:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:05 smithi046 ceph-mon[21574]: pgmap v125: 97 pgs: 97 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 5.9 MiB/s wr, 399 op/s 2024-04-20T10:38:06.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:05 smithi042 ceph-mon[17078]: pgmap v125: 97 pgs: 97 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 5.9 MiB/s wr, 399 op/s 2024-04-20T10:38:07.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:07 smithi162 ceph-mon[21854]: pgmap v126: 97 pgs: 97 active+clean; 493 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.9 MiB/s wr, 408 op/s 2024-04-20T10:38:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:07 smithi046 ceph-mon[21574]: pgmap v126: 97 pgs: 97 active+clean; 493 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.9 MiB/s wr, 408 op/s 2024-04-20T10:38:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:07 smithi042 ceph-mon[17078]: pgmap v126: 97 pgs: 97 active+clean; 493 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.9 MiB/s wr, 408 op/s 2024-04-20T10:38:09.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:09 smithi162 ceph-mon[21854]: pgmap v127: 97 pgs: 97 active+clean; 493 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 388 op/s 2024-04-20T10:38:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:09 smithi046 ceph-mon[21574]: pgmap v127: 97 pgs: 97 active+clean; 493 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 388 op/s 2024-04-20T10:38:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:09 smithi042 ceph-mon[17078]: pgmap v127: 97 pgs: 97 active+clean; 493 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 388 op/s 2024-04-20T10:38:10.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:10 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:38:11.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:10 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:38:11.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:10 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:38:11.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:11 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:38:11.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:11 smithi162 ceph-mon[21854]: pgmap v128: 97 pgs: 97 active+clean; 493 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 324 op/s 2024-04-20T10:38:12.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:11 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:38:12.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:11 smithi046 ceph-mon[21574]: pgmap v128: 97 pgs: 97 active+clean; 493 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 324 op/s 2024-04-20T10:38:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:11 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:38:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:11 smithi042 ceph-mon[17078]: pgmap v128: 97 pgs: 97 active+clean; 493 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 324 op/s 2024-04-20T10:38:13.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:13 smithi162 ceph-mon[21854]: pgmap v129: 97 pgs: 97 active+clean; 495 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 343 op/s 2024-04-20T10:38:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:13 smithi046 ceph-mon[21574]: pgmap v129: 97 pgs: 97 active+clean; 495 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 343 op/s 2024-04-20T10:38:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:13 smithi042 ceph-mon[17078]: pgmap v129: 97 pgs: 97 active+clean; 495 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 343 op/s 2024-04-20T10:38:15.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:15 smithi162 ceph-mon[21854]: pgmap v130: 97 pgs: 97 active+clean; 495 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 44 op/s 2024-04-20T10:38:15.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:15 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:38:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:15 smithi046 ceph-mon[21574]: pgmap v130: 97 pgs: 97 active+clean; 495 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 44 op/s 2024-04-20T10:38:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:15 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:38:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:15 smithi042 ceph-mon[17078]: pgmap v130: 97 pgs: 97 active+clean; 495 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 44 op/s 2024-04-20T10:38:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:15 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:38:16.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:16 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:38:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:16 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:38:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:16 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:38:17.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:16 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:38:17.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:16 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:38:17.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:16 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:38:17.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:16 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:38:17.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:16 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:38:17.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:16 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:38:17.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:17 smithi162 ceph-mon[21854]: pgmap v131: 97 pgs: 97 active+clean; 504 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 75 op/s 2024-04-20T10:38:18.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:17 smithi046 ceph-mon[21574]: pgmap v131: 97 pgs: 97 active+clean; 504 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 75 op/s 2024-04-20T10:38:18.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:17 smithi042 ceph-mon[17078]: pgmap v131: 97 pgs: 97 active+clean; 504 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 75 op/s 2024-04-20T10:38:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:19 smithi046 ceph-mon[21574]: pgmap v132: 97 pgs: 97 active+clean; 519 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 124 op/s 2024-04-20T10:38:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:19 smithi042 ceph-mon[17078]: pgmap v132: 97 pgs: 97 active+clean; 519 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 124 op/s 2024-04-20T10:38:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:19 smithi162 ceph-mon[21854]: pgmap v132: 97 pgs: 97 active+clean; 519 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 124 op/s 2024-04-20T10:38:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:21 smithi046 ceph-mon[21574]: pgmap v133: 97 pgs: 97 active+clean; 519 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 2.3 MiB/s wr, 108 op/s 2024-04-20T10:38:22.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:21 smithi042 ceph-mon[17078]: pgmap v133: 97 pgs: 97 active+clean; 519 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 2.3 MiB/s wr, 108 op/s 2024-04-20T10:38:22.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:21 smithi162 ceph-mon[21854]: pgmap v133: 97 pgs: 97 active+clean; 519 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 2.3 MiB/s wr, 108 op/s 2024-04-20T10:38:24.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:23 smithi046 ceph-mon[21574]: pgmap v134: 97 pgs: 97 active+clean; 728 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 20 MiB/s wr, 389 op/s 2024-04-20T10:38:24.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:23 smithi042 ceph-mon[17078]: pgmap v134: 97 pgs: 97 active+clean; 728 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 20 MiB/s wr, 389 op/s 2024-04-20T10:38:24.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:23 smithi162 ceph-mon[21854]: pgmap v134: 97 pgs: 97 active+clean; 728 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 20 MiB/s wr, 389 op/s 2024-04-20T10:38:25.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:24 smithi046 ceph-mon[21574]: pgmap v135: 97 pgs: 97 active+clean; 728 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 20 MiB/s wr, 370 op/s 2024-04-20T10:38:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:24 smithi042 ceph-mon[17078]: pgmap v135: 97 pgs: 97 active+clean; 728 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 20 MiB/s wr, 370 op/s 2024-04-20T10:38:25.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:24 smithi162 ceph-mon[21854]: pgmap v135: 97 pgs: 97 active+clean; 728 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 20 MiB/s wr, 370 op/s 2024-04-20T10:38:27.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:38:27.172 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:38:27.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.394+0000 7f0baec00640 1 -- 172.21.15.42:0/3178204575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba810d3d0 msgr2=0x7f0ba810d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.394+0000 7f0baec00640 1 --2- 172.21.15.42:0/3178204575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba810d3d0 0x7f0ba810d7b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009900 tx=0x7f0ba002bcd0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.395+0000 7f0baec00640 1 -- 172.21.15.42:0/3178204575 shutdown_connections 2024-04-20T10:38:27.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.395+0000 7f0baec00640 1 --2- 172.21.15.42:0/3178204575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ba8108ea0 0x7f0ba81152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.395+0000 7f0baec00640 1 --2- 172.21.15.42:0/3178204575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ba8108500 0x7f0ba8108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.395+0000 7f0baec00640 1 --2- 172.21.15.42:0/3178204575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba810d3d0 0x7f0ba810d7b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.395+0000 7f0baec00640 1 -- 172.21.15.42:0/3178204575 >> 172.21.15.42:0/3178204575 conn(0x7f0ba8077d80 msgr2=0x7f0ba8106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:27.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.395+0000 7f0baec00640 1 -- 172.21.15.42:0/3178204575 shutdown_connections 2024-04-20T10:38:27.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.395+0000 7f0baec00640 1 -- 172.21.15.42:0/3178204575 wait complete. 2024-04-20T10:38:27.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.396+0000 7f0baec00640 1 Processor -- start 2024-04-20T10:38:27.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.396+0000 7f0baec00640 1 -- start start 2024-04-20T10:38:27.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.396+0000 7f0baec00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ba8108500 0x7f0ba819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.396+0000 7f0baec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ba8108ea0 0x7f0ba819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.396+0000 7f0baec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba810d3d0 0x7f0ba81a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.396+0000 7f0baec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba8118860 con 0x7f0ba8108ea0 2024-04-20T10:38:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.396+0000 7f0baec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81186e0 con 0x7f0ba810d3d0 2024-04-20T10:38:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.396+0000 7f0baec00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81189e0 con 0x7f0ba8108500 2024-04-20T10:38:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.397+0000 7f0bae200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba810d3d0 0x7f0ba81a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.397+0000 7f0bae200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba810d3d0 0x7f0ba81a3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:38466/0 (socket says 172.21.15.42:38466) 2024-04-20T10:38:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.397+0000 7f0bae200640 1 -- 172.21.15.42:0/1300014436 learned_addr learned my addr 172.21.15.42:0/1300014436 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.397+0000 7f0bad800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ba8108500 0x7f0ba819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.397+0000 7f0bace00640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ba8108ea0 0x7f0ba819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.397+0000 7f0bae200640 1 -- 172.21.15.42:0/1300014436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ba8108500 msgr2=0x7f0ba819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.397+0000 7f0bae200640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ba8108500 0x7f0ba819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.397+0000 7f0bae200640 1 -- 172.21.15.42:0/1300014436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ba8108ea0 msgr2=0x7f0ba819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.397+0000 7f0bae200640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ba8108ea0 0x7f0ba819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.397+0000 7f0bae200640 1 -- 172.21.15.42:0/1300014436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba81a3d50 con 0x7f0ba810d3d0 2024-04-20T10:38:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.397+0000 7f0bae200640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba810d3d0 0x7f0ba81a3620 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00d680 tx=0x7f0b9c00db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.398+0000 7f0ba6000640 1 -- 172.21.15.42:0/1300014436 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c017070 con 0x7f0ba810d3d0 2024-04-20T10:38:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.398+0000 7f0ba6000640 1 -- 172.21.15.42:0/1300014436 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b9c009820 con 0x7f0ba810d3d0 2024-04-20T10:38:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.398+0000 7f0ba6000640 1 -- 172.21.15.42:0/1300014436 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c004d00 con 0x7f0ba810d3d0 2024-04-20T10:38:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.398+0000 7f0baec00640 1 -- 172.21.15.42:0/1300014436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba81a4040 con 0x7f0ba810d3d0 2024-04-20T10:38:27.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.398+0000 7f0baec00640 1 -- 172.21.15.42:0/1300014436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba81ab8d0 con 0x7f0ba810d3d0 2024-04-20T10:38:27.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.401+0000 7f0ba6000640 1 -- 172.21.15.42:0/1300014436 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0b9c0099d0 con 0x7f0ba810d3d0 2024-04-20T10:38:27.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.401+0000 7f0bb2fcc740 1 Processor -- start 2024-04-20T10:38:27.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.401+0000 7f0bb2fcc740 1 -- start start 2024-04-20T10:38:27.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.402+0000 7f0bb2fcc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2380 0x558c872f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.402+0000 7f0bb2fcc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2ca0 0x558c872fb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.402+0000 7f0bb2fcc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872fb880 0x558c872fdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.402+0000 7f0bb2fcc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558c87258150 con 0x558c872f2380 2024-04-20T10:38:27.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.402+0000 7f0bb2fcc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558c871fe370 con 0x558c872fb880 2024-04-20T10:38:27.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.402+0000 7f0bb2fcc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x558c87254110 con 0x558c872f2ca0 2024-04-20T10:38:27.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.402+0000 7f0bace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2380 0x558c872f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.402+0000 7f0bace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2380 0x558c872f2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50894/0 (socket says 172.21.15.42:50894) 2024-04-20T10:38:27.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.402+0000 7f0bace00640 1 -- 172.21.15.42:0/2018200691 learned_addr learned my addr 172.21.15.42:0/2018200691 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:27.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.402+0000 7f0bad800640 1 --2- 172.21.15.42:0/2018200691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2ca0 0x558c872fb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.402+0000 7f0bace00640 1 --2- 172.21.15.42:0/2018200691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872fb880 0x558c872fdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.403+0000 7f0bad800640 1 -- 172.21.15.42:0/2018200691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872fb880 msgr2=0x558c872fdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:27.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.403+0000 7f0bad800640 1 --2- 172.21.15.42:0/2018200691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872fb880 0x558c872fdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.403+0000 7f0bad800640 1 -- 172.21.15.42:0/2018200691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2380 msgr2=0x558c872f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:27.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.403+0000 7f0bad800640 1 --2- 172.21.15.42:0/2018200691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2380 0x558c872f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.403+0000 7f0bad800640 1 -- 172.21.15.42:0/2018200691 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c870347d0 con 0x558c872f2ca0 2024-04-20T10:38:27.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.403+0000 7f0bace00640 1 --2- 172.21.15.42:0/2018200691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2380 0x558c872f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:38:27.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.403+0000 7f0ba6000640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0b900a3da0 0x7f0b900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.403+0000 7f0ba6000640 1 -- 172.21.15.42:0/1300014436 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0b9c0d0070 con 0x7f0ba810d3d0 2024-04-20T10:38:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.403+0000 7f0bad800640 1 --2- 172.21.15.42:0/2018200691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2ca0 0x558c872fb340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0ba002c210 tx=0x7f0ba002c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6f58e736ae406ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.403+0000 7f0bace00640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0b900a3da0 0x7f0b900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.404+0000 7f0b96a00640 1 -- 172.21.15.42:0/2018200691 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0043070 con 0x558c872f2ca0 2024-04-20T10:38:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.404+0000 7f0b96a00640 1 -- 172.21.15.42:0/2018200691 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba002c920 con 0x558c872f2ca0 2024-04-20T10:38:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.404+0000 7f0b96a00640 1 -- 172.21.15.42:0/2018200691 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0004b50 con 0x558c872f2ca0 2024-04-20T10:38:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.405+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2018200691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2ca0 msgr2=0x558c872fb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.405+0000 7f0bace00640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0b900a3da0 0x7f0b900a6260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x558c872edc40 tx=0x7f0b98009b80 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.405+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/2018200691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2ca0 0x558c872fb340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0ba002c210 tx=0x7f0ba002c670 comp rx=0 tx=0).stop 2024-04-20T10:38:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.405+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2018200691 shutdown_connections 2024-04-20T10:38:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.405+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/2018200691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872fb880 0x558c872fdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.405+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/2018200691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2ca0 0x558c872fb340 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.405+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/2018200691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2380 0x558c872f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.405+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2018200691 >> 172.21.15.42:0/2018200691 conn(0x558c872ecab0 msgr2=0x558c872ece90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.405+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2018200691 shutdown_connections 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.405+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2018200691 wait complete. 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.405+0000 7f0bb2fcc740 1 -- start start 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.405+0000 7f0bb2fcc740 1 Processor -- start 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.406+0000 7f0bb2fcc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2380 0x558c872f8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.406+0000 7f0bb2fcc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2ca0 0x558c872f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.406+0000 7f0bb2fcc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872fb880 0x558c872f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.406+0000 7f0bb2fcc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558c872fe610 con 0x558c872f2ca0 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.406+0000 7f0bb2fcc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558c87211740 con 0x558c872f2380 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.406+0000 7f0bb2fcc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x558c872fe790 con 0x558c872fb880 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.406+0000 7f0bb2fcc740 10 client.0 _ll_register_callbacks cb 0x558c8720cb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:38:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.406+0000 7f0bad800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2380 0x558c872f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.406+0000 7f0bace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2ca0 0x558c872f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.406+0000 7f0bad800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2380 0x558c872f8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:38486/0 (socket says 172.21.15.42:38486) 2024-04-20T10:38:27.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.406+0000 7f0bae200640 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872fb880 0x558c872f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.406+0000 7f0bad800640 1 -- 172.21.15.42:0/3862934799 learned_addr learned my addr 172.21.15.42:0/3862934799 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:27.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.407+0000 7f0bad800640 1 -- 172.21.15.42:0/3862934799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872fb880 msgr2=0x558c872f92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:27.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.407+0000 7f0bad800640 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872fb880 0x558c872f92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.407+0000 7f0bad800640 1 -- 172.21.15.42:0/3862934799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2ca0 msgr2=0x558c872f8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:27.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.407+0000 7f0bad800640 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2ca0 0x558c872f8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.407+0000 7f0bad800640 1 -- 172.21.15.42:0/3862934799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c86f6a3f0 con 0x558c872f2380 2024-04-20T10:38:27.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.407+0000 7f0bae200640 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872fb880 0x558c872f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:38:27.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.407+0000 7f0bad800640 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2380 0x558c872f8830 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0004b80 tx=0x7f0ba0004bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a79fb3b32d27eec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:27.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.407+0000 7f0b8b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:38:27.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.407+0000 7f0b8b400640 1 -- 172.21.15.42:0/3862934799 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba004c070 con 0x558c872f2380 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.407+0000 7f0b8b400640 1 -- 172.21.15.42:0/3862934799 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0ba00052f0 con 0x558c872f2380 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.407+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/3862934799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ba002c920 con 0x558c872f2380 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.408+0000 7f0b8b400640 1 -- 172.21.15.42:0/3862934799 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0ba0058050 con 0x558c872f2380 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.408+0000 7f0bb2fcc740 10 client.25916 fetch_fsmap learned FSMap version 19 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.408+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/3862934799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558c86f2a0f0 con 0x558c872f2380 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.408+0000 7f0b8b400640 1 -- 172.21.15.42:0/3862934799 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba002c920 con 0x558c872f2380 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.408+0000 7f0b8b400640 1 -- 172.21.15.42:0/3862934799 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f0ba003f440 con 0x558c872f2380 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.408+0000 7f0bb2fcc740 10 client.25916 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.408+0000 7f0bb2fcc740 10 client.25916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.408+0000 7f0bb2fcc740 20 client.25916 populate_metadata read hostname 'smithi042' 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.408+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558c8730b9e0 0x558c8730ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.408+0000 7f0bb2fcc740 4 client.25916 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.408+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/3862934799 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c87265b70 con 0x558c8730b9e0 2024-04-20T10:38:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.409+0000 7f0bace00640 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558c8730b9e0 0x558c8730ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.409+0000 7f0b8b400640 10 client.25916 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.409+0000 7f0bace00640 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558c8730b9e0 0x558c8730ddd0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da725f9d2c00ac97 server_cookie=e63a14184515d53a in_seq=0 out_seq=0 2024-04-20T10:38:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0b8b400640 1 -- 172.21.15.42:0/3862934799 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c87265b70 con 0x558c8730b9e0 2024-04-20T10:38:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0b8b400640 10 client.25916 handle_command_reply: tid=0 2024-04-20T10:38:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0bb2fcc740 1 client.25916 shutdown 2024-04-20T10:38:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/3862934799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2380 msgr2=0x558c872f8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2380 0x558c872f8830 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0004b80 tx=0x7f0ba0004bb0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/3862934799 shutdown_connections 2024-04-20T10:38:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558c8730b9e0 0x558c8730ddd0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872fb880 0x558c872f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2ca0 0x558c872f8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0b8b400640 0 client.25916 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/3862934799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2380 0x558c872f8830 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/3862934799 >> 172.21.15.42:0/3862934799 conn(0x558c872ecab0 msgr2=0x558c872ee800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/3862934799 shutdown_connections 2024-04-20T10:38:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/3862934799 wait complete. 2024-04-20T10:38:27.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.411+0000 7f0bb2fcc740 20 client.25916 trim_cache size 0 max 16384 2024-04-20T10:38:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.431+0000 7f0bb2fcc740 1 Processor -- start 2024-04-20T10:38:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.431+0000 7f0bb2fcc740 1 -- start start 2024-04-20T10:38:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bb2fcc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2380 0x558c872faf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bb2fcc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2ca0 0x558c87307500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bb2fcc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872fb880 0x558c872f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bb2fcc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81b2140 con 0x558c872f2ca0 2024-04-20T10:38:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bb2fcc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81b49c0 con 0x558c872fb880 2024-04-20T10:38:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bb2fcc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81b27b0 con 0x558c872f2380 2024-04-20T10:38:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2ca0 0x558c87307500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bae200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872fb880 0x558c872f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bae200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872fb880 0x558c872f1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:38500/0 (socket says 172.21.15.42:38500) 2024-04-20T10:38:27.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bae200640 1 -- 172.21.15.42:0/177642192 learned_addr learned my addr 172.21.15.42:0/177642192 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:27.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bad800640 1 --2- 172.21.15.42:0/177642192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2380 0x558c872faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bae200640 1 -- 172.21.15.42:0/177642192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2380 msgr2=0x558c872faf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:27.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bae200640 1 --2- 172.21.15.42:0/177642192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2380 0x558c872faf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bae200640 1 -- 172.21.15.42:0/177642192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2ca0 msgr2=0x558c87307500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:27.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bae200640 1 --2- 172.21.15.42:0/177642192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2ca0 0x558c87307500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.432+0000 7f0bae200640 1 -- 172.21.15.42:0/177642192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c872fe770 con 0x558c872fb880 2024-04-20T10:38:27.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.433+0000 7f0bae200640 1 --2- 172.21.15.42:0/177642192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872fb880 0x558c872f1b20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0cd7c0 tx=0x7f0b9c09bac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f652aa1180efec5c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:27.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.433+0000 7f0b96000640 1 -- 172.21.15.42:0/177642192 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c0b4070 con 0x558c872fb880 2024-04-20T10:38:27.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.433+0000 7f0b96000640 1 -- 172.21.15.42:0/177642192 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b9c0a4c20 con 0x558c872fb880 2024-04-20T10:38:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.433+0000 7f0b96000640 1 -- 172.21.15.42:0/177642192 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c020040 con 0x558c872fb880 2024-04-20T10:38:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.434+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/177642192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872fb880 msgr2=0x558c872f1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.434+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/177642192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872fb880 0x558c872f1b20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0cd7c0 tx=0x7f0b9c09bac0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.434+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/177642192 shutdown_connections 2024-04-20T10:38:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.434+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/177642192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872fb880 0x558c872f1b20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.434+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/177642192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872f2ca0 0x558c87307500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.434+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/177642192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2380 0x558c872faf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.434+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/177642192 >> 172.21.15.42:0/177642192 conn(0x558c872ecab0 msgr2=0x558c872f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.434+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/177642192 shutdown_connections 2024-04-20T10:38:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.434+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/177642192 wait complete. 2024-04-20T10:38:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.434+0000 7f0bb2fcc740 1 -- start start 2024-04-20T10:38:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bb2fcc740 1 Processor -- start 2024-04-20T10:38:27.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bb2fcc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2380 0x558c87303cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bb2fcc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2ca0 0x558c872ff030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bb2fcc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872fb880 0x558c872ff570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bb2fcc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81b2e80 con 0x558c872fb880 2024-04-20T10:38:27.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bb2fcc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81b42f0 con 0x558c872f2ca0 2024-04-20T10:38:27.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bb2fcc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81b3550 con 0x558c872f2380 2024-04-20T10:38:27.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bb2fcc740 10 client.0 _ll_register_callbacks cb 0x7f0b98009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:38:27.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bae200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872fb880 0x558c872ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bace00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2ca0 0x558c872ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bace00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2ca0 0x558c872ff030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:38510/0 (socket says 172.21.15.42:38510) 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bad800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2380 0x558c87303cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.435+0000 7f0bace00640 1 -- 172.21.15.42:0/2454826026 learned_addr learned my addr 172.21.15.42:0/2454826026 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.436+0000 7f0bace00640 1 -- 172.21.15.42:0/2454826026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2380 msgr2=0x558c87303cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.436+0000 7f0bace00640 1 --2- 172.21.15.42:0/2454826026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2380 0x558c87303cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.436+0000 7f0bace00640 1 -- 172.21.15.42:0/2454826026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872fb880 msgr2=0x558c872ff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.436+0000 7f0bace00640 1 --2- 172.21.15.42:0/2454826026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872fb880 0x558c872ff570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.436+0000 7f0bace00640 1 -- 172.21.15.42:0/2454826026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c872ffc60 con 0x558c872f2ca0 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.436+0000 7f0bace00640 1 --2- 172.21.15.42:0/2454826026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2ca0 0x558c872ff030 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0b98005c10 tx=0x7f0b98009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ee9d2a73673764 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.436+0000 7f0b8b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.436+0000 7f0b8b400640 1 -- 172.21.15.42:0/2454826026 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b98030070 con 0x558c872f2ca0 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.436+0000 7f0b8b400640 1 -- 172.21.15.42:0/2454826026 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0b9801b030 con 0x558c872f2ca0 2024-04-20T10:38:27.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.436+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2454826026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b9c0a4c20 con 0x558c872f2ca0 2024-04-20T10:38:27.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.437+0000 7f0b8b400640 1 -- 172.21.15.42:0/2454826026 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0b98004b20 con 0x558c872f2ca0 2024-04-20T10:38:27.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.437+0000 7f0bb2fcc740 10 client.25922 fetch_fsmap learned FSMap version 19 2024-04-20T10:38:27.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.437+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2454826026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558c87371c10 con 0x558c872f2ca0 2024-04-20T10:38:27.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.437+0000 7f0b8b400640 1 -- 172.21.15.42:0/2454826026 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b9c0a4c20 con 0x558c872f2ca0 2024-04-20T10:38:27.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.437+0000 7f0b8b400640 1 -- 172.21.15.42:0/2454826026 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f0b98010c20 con 0x558c872f2ca0 2024-04-20T10:38:27.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.437+0000 7f0bb2fcc740 10 client.25922 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:38:27.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.437+0000 7f0bb2fcc740 10 client.25922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:38:27.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.437+0000 7f0bb2fcc740 20 client.25922 populate_metadata read hostname 'smithi042' 2024-04-20T10:38:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.437+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/2454826026 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558c873723c0 0x558c873747e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.437+0000 7f0bb2fcc740 4 client.25922 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:38:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.437+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2454826026 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c8725f9e0 con 0x558c873723c0 2024-04-20T10:38:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.438+0000 7f0bad800640 1 --2- 172.21.15.42:0/2454826026 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558c873723c0 0x558c873747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.438+0000 7f0bad800640 1 --2- 172.21.15.42:0/2454826026 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558c873723c0 0x558c873747e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1db6cbc5214f9ae server_cookie=cbcb4bff70ab17e3 in_seq=0 out_seq=0 2024-04-20T10:38:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.438+0000 7f0b8b400640 10 client.25922 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.439+0000 7f0b8b400640 1 -- 172.21.15.42:0/2454826026 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558c8725f9e0 con 0x558c873723c0 2024-04-20T10:38:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.439+0000 7f0b8b400640 10 client.25922 handle_command_reply: tid=0 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.439+0000 7f0bb2fcc740 1 client.25922 shutdown 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.440+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2454826026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2ca0 msgr2=0x558c872ff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.440+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/2454826026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2ca0 0x558c872ff030 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0b98005c10 tx=0x7f0b98009760 comp rx=0 tx=0).stop 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.440+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2454826026 shutdown_connections 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.440+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/2454826026 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x558c873723c0 0x558c873747e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.440+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/2454826026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558c872fb880 0x558c872ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.440+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/2454826026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c872f2ca0 0x558c872ff030 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.440+0000 7f0bb2fcc740 1 --2- 172.21.15.42:0/2454826026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558c872f2380 0x558c87303cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.440+0000 7f0b8b400640 0 client.25922 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.440+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2454826026 >> 172.21.15.42:0/2454826026 conn(0x558c872ecab0 msgr2=0x558c872f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.440+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2454826026 shutdown_connections 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.440+0000 7f0bb2fcc740 1 -- 172.21.15.42:0/2454826026 wait complete. 2024-04-20T10:38:27.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.440+0000 7f0bb2fcc740 20 client.25922 trim_cache size 0 max 16384 2024-04-20T10:38:27.445 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:38:27.445 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:38:27.445 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:38:27.445 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:38:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.441+0000 7f0baec00640 1 -- 172.21.15.42:0/1300014436 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0b900a3da0 msgr2=0x7f0b900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.441+0000 7f0baec00640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0b900a3da0 0x7f0b900a6260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x558c872edc40 tx=0x7f0b98009b80 comp rx=0 tx=0).stop 2024-04-20T10:38:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.441+0000 7f0baec00640 1 -- 172.21.15.42:0/1300014436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba810d3d0 msgr2=0x7f0ba81a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.441+0000 7f0baec00640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba810d3d0 0x7f0ba81a3620 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00d680 tx=0x7f0b9c00db40 comp rx=0 tx=0).stop 2024-04-20T10:38:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.441+0000 7f0baec00640 1 -- 172.21.15.42:0/1300014436 shutdown_connections 2024-04-20T10:38:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.441+0000 7f0baec00640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0b900a3da0 0x7f0b900a6260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.441+0000 7f0baec00640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba810d3d0 0x7f0ba81a3620 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.441+0000 7f0baec00640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ba8108ea0 0x7f0ba819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.441+0000 7f0baec00640 1 --2- 172.21.15.42:0/1300014436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ba8108500 0x7f0ba819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.441+0000 7f0baec00640 1 -- 172.21.15.42:0/1300014436 >> 172.21.15.42:0/1300014436 conn(0x7f0ba8077d80 msgr2=0x7f0ba80783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:27.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.441+0000 7f0baec00640 1 -- 172.21.15.42:0/1300014436 shutdown_connections 2024-04-20T10:38:27.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.441+0000 7f0baec00640 1 -- 172.21.15.42:0/1300014436 wait complete. 2024-04-20T10:38:27.461 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:38:27.461 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:38:27.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.686+0000 7f822a200640 1 -- 172.21.15.42:0/1377191652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8224112c00 msgr2=0x7f8224114ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.686+0000 7f822a200640 1 --2- 172.21.15.42:0/1377191652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8224112c00 0x7f8224114ff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f821c009cd0 tx=0x7f821c02c0e0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.686+0000 7f822a200640 1 -- 172.21.15.42:0/1377191652 shutdown_connections 2024-04-20T10:38:27.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.686+0000 7f822a200640 1 --2- 172.21.15.42:0/1377191652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8224112c00 0x7f8224114ff0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.686+0000 7f822a200640 1 --2- 172.21.15.42:0/1377191652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8224076db0 0x7f82241126c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.686+0000 7f822a200640 1 --2- 172.21.15.42:0/1377191652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8224076490 0x7f8224076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.686+0000 7f822a200640 1 -- 172.21.15.42:0/1377191652 >> 172.21.15.42:0/1377191652 conn(0x7f82240ff9f0 msgr2=0x7f8224101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:27.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.687+0000 7f822a200640 1 -- 172.21.15.42:0/1377191652 shutdown_connections 2024-04-20T10:38:27.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.687+0000 7f822a200640 1 -- 172.21.15.42:0/1377191652 wait complete. 2024-04-20T10:38:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.688+0000 7f822a200640 1 Processor -- start 2024-04-20T10:38:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.688+0000 7f822a200640 1 -- start start 2024-04-20T10:38:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.688+0000 7f822a200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8224076490 0x7f82241a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.688+0000 7f822a200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8224076db0 0x7f82241a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.688+0000 7f822a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8224112c00 0x7f822419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.688+0000 7f822a200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8224118740 con 0x7f8224076490 2024-04-20T10:38:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.688+0000 7f822a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82241185c0 con 0x7f8224112c00 2024-04-20T10:38:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.688+0000 7f822a200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f82241188c0 con 0x7f8224076db0 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.689+0000 7f8228e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8224076490 0x7f82241a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.689+0000 7f8229800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8224112c00 0x7f822419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.689+0000 7f8228e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8224076490 0x7f82241a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50938/0 (socket says 172.21.15.42:50938) 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.689+0000 7f8229800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8224112c00 0x7f822419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:38540/0 (socket says 172.21.15.42:38540) 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.689+0000 7f8228e00640 1 -- 172.21.15.42:0/3933401858 learned_addr learned my addr 172.21.15.42:0/3933401858 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.689+0000 7f8223e00640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8224076db0 0x7f82241a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.689+0000 7f8229800640 1 -- 172.21.15.42:0/3933401858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8224076db0 msgr2=0x7f82241a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.689+0000 7f8229800640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8224076db0 0x7f82241a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.689+0000 7f8229800640 1 -- 172.21.15.42:0/3933401858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8224076490 msgr2=0x7f82241a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.689+0000 7f8229800640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8224076490 0x7f82241a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.689+0000 7f8229800640 1 -- 172.21.15.42:0/3933401858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822419f260 con 0x7f8224112c00 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.689+0000 7f8228e00640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8224076490 0x7f82241a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.690+0000 7f8229800640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8224112c00 0x7f822419eae0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f821c02c6f0 tx=0x7f821c032460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.690+0000 7f8221600640 1 -- 172.21.15.42:0/3933401858 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c039040 con 0x7f8224112c00 2024-04-20T10:38:27.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.690+0000 7f8221600640 1 -- 172.21.15.42:0/3933401858 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821c044070 con 0x7f8224112c00 2024-04-20T10:38:27.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.690+0000 7f8221600640 1 -- 172.21.15.42:0/3933401858 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c0042a0 con 0x7f8224112c00 2024-04-20T10:38:27.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.690+0000 7f822a200640 1 -- 172.21.15.42:0/3933401858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822419f4f0 con 0x7f8224112c00 2024-04-20T10:38:27.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.690+0000 7f822a200640 1 -- 172.21.15.42:0/3933401858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82241ab7a0 con 0x7f8224112c00 2024-04-20T10:38:27.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.691+0000 7f822a200640 1 -- 172.21.15.42:0/3933401858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8224077bd0 con 0x7f8224112c00 2024-04-20T10:38:27.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.693+0000 7f8221600640 1 -- 172.21.15.42:0/3933401858 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f821c003970 con 0x7f8224112c00 2024-04-20T10:38:27.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.695+0000 7f8221600640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f820c0a3da0 0x7f820c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:27.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.695+0000 7f8221600640 1 -- 172.21.15.42:0/3933401858 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f821c0f2a80 con 0x7f8224112c00 2024-04-20T10:38:27.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.695+0000 7f8228e00640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f820c0a3da0 0x7f820c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:27.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.696+0000 7f8228e00640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f820c0a3da0 0x7f820c0a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82241151f0 tx=0x7f8218005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:27.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.702+0000 7f8221600640 1 -- 172.21.15.42:0/3933401858 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f821c0f4e20 con 0x7f8224112c00 2024-04-20T10:38:27.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:27 smithi046 ceph-mon[21574]: pgmap v136: 97 pgs: 97 active+clean; 732 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 378 op/s 2024-04-20T10:38:27.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:27 smithi042 ceph-mon[17078]: pgmap v136: 97 pgs: 97 active+clean; 732 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 378 op/s 2024-04-20T10:38:27.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:27 smithi162 ceph-mon[21854]: pgmap v136: 97 pgs: 97 active+clean; 732 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 378 op/s 2024-04-20T10:38:27.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.967+0000 7f822a200640 1 -- 172.21.15.42:0/3933401858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f822410dd90 con 0x7f8224112c00 2024-04-20T10:38:27.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.968+0000 7f8221600640 1 -- 172.21.15.42:0/3933401858 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f821c0bf260 con 0x7f8224112c00 2024-04-20T10:38:27.969 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:38:27.970 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:38:27.970 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:38:27.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.971+0000 7f822a200640 1 -- 172.21.15.42:0/3933401858 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f820c0a3da0 msgr2=0x7f820c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.971+0000 7f822a200640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f820c0a3da0 0x7f820c0a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82241151f0 tx=0x7f8218005f90 comp rx=0 tx=0).stop 2024-04-20T10:38:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.972+0000 7f822a200640 1 -- 172.21.15.42:0/3933401858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8224112c00 msgr2=0x7f822419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.972+0000 7f822a200640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8224112c00 0x7f822419eae0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f821c02c6f0 tx=0x7f821c032460 comp rx=0 tx=0).stop 2024-04-20T10:38:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.972+0000 7f822a200640 1 -- 172.21.15.42:0/3933401858 shutdown_connections 2024-04-20T10:38:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.972+0000 7f822a200640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f820c0a3da0 0x7f820c0a6260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.972+0000 7f822a200640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8224112c00 0x7f822419eae0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.972+0000 7f822a200640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8224076db0 0x7f82241a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.972+0000 7f822a200640 1 --2- 172.21.15.42:0/3933401858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8224076490 0x7f82241a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.972+0000 7f822a200640 1 -- 172.21.15.42:0/3933401858 >> 172.21.15.42:0/3933401858 conn(0x7f82240ff9f0 msgr2=0x7f82241142c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.972+0000 7f822a200640 1 -- 172.21.15.42:0/3933401858 shutdown_connections 2024-04-20T10:38:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:27.972+0000 7f822a200640 1 -- 172.21.15.42:0/3933401858 wait complete. 2024-04-20T10:38:27.992 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:38:28.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.213+0000 7f4ca0600640 1 -- 172.21.15.42:0/3820879400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c98074230 msgr2=0x7f4c98074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:28.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.213+0000 7f4ca0600640 1 --2- 172.21.15.42:0/3820879400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c98074230 0x7f4c98074610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009900 tx=0x7f4c8c02bcc0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.213+0000 7f4ca0600640 1 -- 172.21.15.42:0/3820879400 shutdown_connections 2024-04-20T10:38:28.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.213+0000 7f4ca0600640 1 --2- 172.21.15.42:0/3820879400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c9810e840 0x7f4c98110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.213+0000 7f4ca0600640 1 --2- 172.21.15.42:0/3820879400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c98074b50 0x7f4c9810e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.213+0000 7f4ca0600640 1 --2- 172.21.15.42:0/3820879400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c98074230 0x7f4c98074610 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.213+0000 7f4ca0600640 1 -- 172.21.15.42:0/3820879400 >> 172.21.15.42:0/3820879400 conn(0x7f4c980793b0 msgr2=0x7f4c980797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:28.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.214+0000 7f4ca0600640 1 -- 172.21.15.42:0/3820879400 shutdown_connections 2024-04-20T10:38:28.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.214+0000 7f4ca0600640 1 -- 172.21.15.42:0/3820879400 wait complete. 2024-04-20T10:38:28.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.215+0000 7f4ca0600640 1 Processor -- start 2024-04-20T10:38:28.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.215+0000 7f4ca0600640 1 -- start start 2024-04-20T10:38:28.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.216+0000 7f4ca0600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c98074230 0x7f4c981a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.216+0000 7f4ca0600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c98074b50 0x7f4c981a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.216+0000 7f4ca0600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9810e840 0x7f4c9819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.216+0000 7f4ca0600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c98114530 con 0x7f4c98074230 2024-04-20T10:38:28.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.216+0000 7f4ca0600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c981143b0 con 0x7f4c9810e840 2024-04-20T10:38:28.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.216+0000 7f4ca0600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4c981146b0 con 0x7f4c98074b50 2024-04-20T10:38:28.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.216+0000 7f4c9dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c98074230 0x7f4c981a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.216+0000 7f4c9e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9810e840 0x7f4c9819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.216+0000 7f4c9e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9810e840 0x7f4c9819a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:38546/0 (socket says 172.21.15.42:38546) 2024-04-20T10:38:28.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.216+0000 7f4c9e600640 1 -- 172.21.15.42:0/3423420936 learned_addr learned my addr 172.21.15.42:0/3423420936 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:28.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.216+0000 7f4c9dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c98074230 0x7f4c981a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50958/0 (socket says 172.21.15.42:50958) 2024-04-20T10:38:28.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.216+0000 7f4c9d200640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c98074b50 0x7f4c981a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.217+0000 7f4c9e600640 1 -- 172.21.15.42:0/3423420936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c98074b50 msgr2=0x7f4c981a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:28.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.217+0000 7f4c9e600640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c98074b50 0x7f4c981a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.217+0000 7f4c9e600640 1 -- 172.21.15.42:0/3423420936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c98074230 msgr2=0x7f4c981a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:28.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.217+0000 7f4c9e600640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c98074230 0x7f4c981a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.217+0000 7f4c9e600640 1 -- 172.21.15.42:0/3423420936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9819b070 con 0x7f4c9810e840 2024-04-20T10:38:28.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.217+0000 7f4c9e600640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9810e840 0x7f4c9819a8f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000e9d0 tx=0x7f4c9000ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:28.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.217+0000 7f4c96000640 1 -- 172.21.15.42:0/3423420936 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9000cd80 con 0x7f4c9810e840 2024-04-20T10:38:28.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.217+0000 7f4c96000640 1 -- 172.21.15.42:0/3423420936 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c90005020 con 0x7f4c9810e840 2024-04-20T10:38:28.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.217+0000 7f4c96000640 1 -- 172.21.15.42:0/3423420936 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c900116d0 con 0x7f4c9810e840 2024-04-20T10:38:28.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.217+0000 7f4ca0600640 1 -- 172.21.15.42:0/3423420936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9819b360 con 0x7f4c9810e840 2024-04-20T10:38:28.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.217+0000 7f4ca0600640 1 -- 172.21.15.42:0/3423420936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c981a7600 con 0x7f4c9810e840 2024-04-20T10:38:28.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.221+0000 7f4ca3377740 1 Processor -- start 2024-04-20T10:38:28.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.222+0000 7f4c96000640 1 -- 172.21.15.42:0/3423420936 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4c900051d0 con 0x7f4c9810e840 2024-04-20T10:38:28.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.222+0000 7f4ca3377740 1 -- start start 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.222+0000 7f4ca3377740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e420fb0 0x56077e421390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.222+0000 7f4ca3377740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 0x56077e429f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.222+0000 7f4ca3377740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e42c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.222+0000 7f4ca3377740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56077e386050 con 0x56077e42a4b0 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.222+0000 7f4ca3377740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56077e32c490 con 0x56077e4218d0 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.222+0000 7f4ca3377740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56077e382010 con 0x56077e420fb0 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.223+0000 7f4c9dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 0x56077e429f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.223+0000 7f4c9dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 0x56077e429f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:38552/0 (socket says 172.21.15.42:38552) 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.223+0000 7f4c9dc00640 1 -- 172.21.15.42:0/3025477833 learned_addr learned my addr 172.21.15.42:0/3025477833 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.223+0000 7f4c9d200640 1 --2- 172.21.15.42:0/3025477833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e42c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.223+0000 7f4c9d200640 1 --2- 172.21.15.42:0/3025477833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e420fb0 0x56077e421390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.223+0000 7f4c9dc00640 1 -- 172.21.15.42:0/3025477833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e420fb0 msgr2=0x56077e421390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.223+0000 7f4c9dc00640 1 --2- 172.21.15.42:0/3025477833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e420fb0 0x56077e421390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.223+0000 7f4c9dc00640 1 -- 172.21.15.42:0/3025477833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 msgr2=0x56077e42c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:28.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.223+0000 7f4c9dc00640 1 --2- 172.21.15.42:0/3025477833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e42c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.223+0000 7f4c9dc00640 1 -- 172.21.15.42:0/3025477833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56077e334070 con 0x56077e4218d0 2024-04-20T10:38:28.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.223+0000 7f4c9d200640 1 --2- 172.21.15.42:0/3025477833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e42c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:38:28.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.223+0000 7f4c9dc00640 1 --2- 172.21.15.42:0/3025477833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 0x56077e429f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009230 tx=0x7f4c8c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1088533c8a005799 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:28.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.224+0000 7f4c96000640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c800a3e20 0x7f4c800a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.224+0000 7f4c8aa00640 1 -- 172.21.15.42:0/3025477833 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c045070 con 0x56077e4218d0 2024-04-20T10:38:28.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.224+0000 7f4c8aa00640 1 -- 172.21.15.42:0/3025477833 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c8c004040 con 0x56077e4218d0 2024-04-20T10:38:28.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.224+0000 7f4c8aa00640 1 -- 172.21.15.42:0/3025477833 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c004ef0 con 0x56077e4218d0 2024-04-20T10:38:28.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.224+0000 7f4c96000640 1 -- 172.21.15.42:0/3423420936 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4c900d0c60 con 0x7f4c9810e840 2024-04-20T10:38:28.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.224+0000 7f4c9d200640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c800a3e20 0x7f4c800a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.225+0000 7f4ca3377740 1 -- 172.21.15.42:0/3025477833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 msgr2=0x56077e429f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:28.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.225+0000 7f4ca3377740 1 --2- 172.21.15.42:0/3025477833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 0x56077e429f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009230 tx=0x7f4c8c005e80 comp rx=0 tx=0).stop 2024-04-20T10:38:28.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.225+0000 7f4ca3377740 1 -- 172.21.15.42:0/3025477833 shutdown_connections 2024-04-20T10:38:28.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.225+0000 7f4ca3377740 1 --2- 172.21.15.42:0/3025477833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e42c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.225+0000 7f4ca3377740 1 --2- 172.21.15.42:0/3025477833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 0x56077e429f70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.225+0000 7f4ca3377740 1 --2- 172.21.15.42:0/3025477833 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e420fb0 0x56077e421390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.225+0000 7f4ca3377740 1 -- 172.21.15.42:0/3025477833 >> 172.21.15.42:0/3025477833 conn(0x56077e41b6e0 msgr2=0x56077e41bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:28.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.225+0000 7f4c9d200640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c800a3e20 0x7f4c800a62e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x56077e41c870 tx=0x7f4c84004730 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:28.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.225+0000 7f4ca3377740 1 -- 172.21.15.42:0/3025477833 shutdown_connections 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.225+0000 7f4ca3377740 1 -- 172.21.15.42:0/3025477833 wait complete. 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.225+0000 7f4ca3377740 1 -- start start 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.226+0000 7f4ca3377740 1 Processor -- start 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.226+0000 7f4ca3377740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 0x56077e427510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.226+0000 7f4ca3377740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e4218d0 0x56077e427a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.226+0000 7f4ca3377740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e427f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.226+0000 7f4ca3377740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56077e42d240 con 0x56077e42a4b0 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.226+0000 7f4ca3377740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56077e33f860 con 0x56077e420fb0 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.226+0000 7f4ca3377740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56077e42d3c0 con 0x56077e4218d0 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.226+0000 7f4ca3377740 10 client.0 _ll_register_callbacks cb 0x56077e30dfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 0x56077e427510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 0x56077e427510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:38566/0 (socket says 172.21.15.42:38566) 2024-04-20T10:38:28.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9dc00640 1 -- 172.21.15.42:0/3191670499 learned_addr learned my addr 172.21.15.42:0/3191670499 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9e600640 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e427f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9d200640 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e4218d0 0x56077e427a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9dc00640 1 -- 172.21.15.42:0/3191670499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e4218d0 msgr2=0x56077e427a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9dc00640 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e4218d0 0x56077e427a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9dc00640 1 -- 172.21.15.42:0/3191670499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 msgr2=0x56077e427f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9dc00640 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e427f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9dc00640 1 -- 172.21.15.42:0/3191670499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56077e343950 con 0x56077e420fb0 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9d200640 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e4218d0 0x56077e427a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9e600640 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e427f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c9dc00640 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 0x56077e427510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c005c40 tx=0x7f4c8c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=246fae822e95a040 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.227+0000 7f4c7f400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.228+0000 7f4c7f400640 1 -- 172.21.15.42:0/3191670499 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c045040 con 0x56077e420fb0 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.228+0000 7f4c7f400640 1 -- 172.21.15.42:0/3191670499 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4c8c02cb10 con 0x56077e420fb0 2024-04-20T10:38:28.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.228+0000 7f4ca3377740 1 -- 172.21.15.42:0/3191670499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c8c004040 con 0x56077e420fb0 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.228+0000 7f4c7f400640 1 -- 172.21.15.42:0/3191670499 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4c8c05a050 con 0x56077e420fb0 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.228+0000 7f4ca3377740 10 client.25940 fetch_fsmap learned FSMap version 19 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.228+0000 7f4ca3377740 1 -- 172.21.15.42:0/3191670499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56077e311820 con 0x56077e420fb0 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.228+0000 7f4c7f400640 1 -- 172.21.15.42:0/3191670499 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c8c004040 con 0x56077e420fb0 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.229+0000 7f4c7f400640 1 -- 172.21.15.42:0/3191670499 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f4c8c003980 con 0x56077e420fb0 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.229+0000 7f4ca3377740 10 client.25940 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.229+0000 7f4ca3377740 10 client.25940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.229+0000 7f4ca3377740 20 client.25940 populate_metadata read hostname 'smithi042' 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.229+0000 7f4ca3377740 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56077e43a360 0x56077e43c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.229+0000 7f4ca3377740 4 client.25940 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.229+0000 7f4ca3377740 1 -- 172.21.15.42:0/3191670499 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56077e393a70 con 0x56077e43a360 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.229+0000 7f4c9d200640 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56077e43a360 0x56077e43c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.230+0000 7f4c7f400640 10 client.25940 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.230+0000 7f4c9d200640 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56077e43a360 0x56077e43c750 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f2d92f3f587882 server_cookie=779adeaad2b7a15c in_seq=0 out_seq=0 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.231+0000 7f4c7f400640 1 -- 172.21.15.42:0/3191670499 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56077e393a70 con 0x56077e43a360 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.231+0000 7f4c7f400640 10 client.25940 handle_command_reply: tid=0 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.231+0000 7f4ca3377740 1 client.25940 shutdown 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.231+0000 7f4ca3377740 1 -- 172.21.15.42:0/3191670499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 msgr2=0x56077e427510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.231+0000 7f4ca3377740 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 0x56077e427510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c005c40 tx=0x7f4c8c009260 comp rx=0 tx=0).stop 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.231+0000 7f4ca3377740 1 -- 172.21.15.42:0/3191670499 shutdown_connections 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.231+0000 7f4ca3377740 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56077e43a360 0x56077e43c750 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.231+0000 7f4ca3377740 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e427f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.231+0000 7f4ca3377740 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e4218d0 0x56077e427a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.231+0000 7f4ca3377740 1 --2- 172.21.15.42:0/3191670499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 0x56077e427510 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.231+0000 7f4ca3377740 1 -- 172.21.15.42:0/3191670499 >> 172.21.15.42:0/3191670499 conn(0x56077e41b6e0 msgr2=0x56077e41d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.231+0000 7f4c7f400640 0 client.25940 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.232+0000 7f4ca3377740 1 -- 172.21.15.42:0/3191670499 shutdown_connections 2024-04-20T10:38:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.232+0000 7f4ca3377740 1 -- 172.21.15.42:0/3191670499 wait complete. 2024-04-20T10:38:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.232+0000 7f4ca3377740 20 client.25940 trim_cache size 0 max 16384 2024-04-20T10:38:28.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.249+0000 7f4ca3377740 1 Processor -- start 2024-04-20T10:38:28.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.249+0000 7f4ca3377740 1 -- start start 2024-04-20T10:38:28.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.249+0000 7f4ca3377740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 0x56077e429c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.249+0000 7f4ca3377740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e4218d0 0x56077e436160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.249+0000 7f4ca3377740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e441db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.249+0000 7f4ca3377740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c981ae010 con 0x56077e42a4b0 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.249+0000 7f4ca3377740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c981b0890 con 0x56077e420fb0 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.249+0000 7f4ca3377740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4c981ae680 con 0x56077e4218d0 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.250+0000 7f4c9dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 0x56077e429c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.250+0000 7f4c9dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 0x56077e429c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:38578/0 (socket says 172.21.15.42:38578) 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.250+0000 7f4c9dc00640 1 -- 172.21.15.42:0/341026784 learned_addr learned my addr 172.21.15.42:0/341026784 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.250+0000 7f4c9d200640 1 --2- 172.21.15.42:0/341026784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e4218d0 0x56077e436160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.250+0000 7f4c9e600640 1 --2- 172.21.15.42:0/341026784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e441db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.250+0000 7f4c9d200640 1 -- 172.21.15.42:0/341026784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 msgr2=0x56077e429c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.250+0000 7f4c9d200640 1 --2- 172.21.15.42:0/341026784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 0x56077e429c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.250+0000 7f4c9d200640 1 -- 172.21.15.42:0/341026784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 msgr2=0x56077e441db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:28.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.250+0000 7f4c9d200640 1 --2- 172.21.15.42:0/341026784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e441db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.250+0000 7f4c9d200640 1 -- 172.21.15.42:0/341026784 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56077e0983f0 con 0x56077e4218d0 2024-04-20T10:38:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.251+0000 7f4c9d200640 1 --2- 172.21.15.42:0/341026784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e4218d0 0x56077e436160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4c84012fd0 tx=0x7f4c84014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a60e1be3b85c31c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.251+0000 7f4c8a000640 1 -- 172.21.15.42:0/341026784 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84031070 con 0x56077e4218d0 2024-04-20T10:38:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.251+0000 7f4c8a000640 1 -- 172.21.15.42:0/341026784 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c840114a0 con 0x56077e4218d0 2024-04-20T10:38:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.251+0000 7f4c8a000640 1 -- 172.21.15.42:0/341026784 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84022680 con 0x56077e4218d0 2024-04-20T10:38:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.252+0000 7f4ca3377740 1 -- 172.21.15.42:0/341026784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e4218d0 msgr2=0x56077e436160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.252+0000 7f4ca3377740 1 --2- 172.21.15.42:0/341026784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e4218d0 0x56077e436160 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4c84012fd0 tx=0x7f4c84014040 comp rx=0 tx=0).stop 2024-04-20T10:38:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.252+0000 7f4ca3377740 1 -- 172.21.15.42:0/341026784 shutdown_connections 2024-04-20T10:38:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.252+0000 7f4ca3377740 1 --2- 172.21.15.42:0/341026784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e42a4b0 0x56077e441db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.252+0000 7f4ca3377740 1 --2- 172.21.15.42:0/341026784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e4218d0 0x56077e436160 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.252+0000 7f4ca3377740 1 --2- 172.21.15.42:0/341026784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e420fb0 0x56077e429c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.252+0000 7f4ca3377740 1 -- 172.21.15.42:0/341026784 >> 172.21.15.42:0/341026784 conn(0x56077e41b6e0 msgr2=0x56077e41a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.252+0000 7f4ca3377740 1 -- 172.21.15.42:0/341026784 shutdown_connections 2024-04-20T10:38:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.252+0000 7f4ca3377740 1 -- 172.21.15.42:0/341026784 wait complete. 2024-04-20T10:38:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.252+0000 7f4ca3377740 1 -- start start 2024-04-20T10:38:28.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.253+0000 7f4ca3377740 1 Processor -- start 2024-04-20T10:38:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.253+0000 7f4ca3377740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e420fb0 0x56077e4242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.253+0000 7f4ca3377740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 0x56077e424830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.253+0000 7f4ca3377740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e42a4b0 0x56077e424d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.253+0000 7f4ca3377740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c981aed50 con 0x56077e420fb0 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.253+0000 7f4ca3377740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c981b01c0 con 0x56077e4218d0 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.253+0000 7f4ca3377740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4c981af420 con 0x56077e42a4b0 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.253+0000 7f4ca3377740 10 client.0 _ll_register_callbacks cb 0x7f4c8c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.253+0000 7f4c9dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e420fb0 0x56077e4242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c9d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 0x56077e424830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c9dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e420fb0 0x56077e4242f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50996/0 (socket says 172.21.15.42:50996) 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c9d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 0x56077e424830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:38592/0 (socket says 172.21.15.42:38592) 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c9dc00640 1 -- 172.21.15.42:0/908345850 learned_addr learned my addr 172.21.15.42:0/908345850 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c9e600640 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e42a4b0 0x56077e424d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c9dc00640 1 -- 172.21.15.42:0/908345850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e42a4b0 msgr2=0x56077e424d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c9dc00640 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e42a4b0 0x56077e424d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c9dc00640 1 -- 172.21.15.42:0/908345850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 msgr2=0x56077e424830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c9dc00640 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 0x56077e424830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c9dc00640 1 -- 172.21.15.42:0/908345850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56077e1627d0 con 0x56077e420fb0 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c9d200640 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 0x56077e424830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c9dc00640 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e420fb0 0x56077e4242f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009230 tx=0x7f4c8c0042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dfc01e1202be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.254+0000 7f4c7f400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4c7f400640 1 -- 172.21.15.42:0/908345850 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c063060 con 0x56077e420fb0 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4c7f400640 1 -- 172.21.15.42:0/908345850 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f4c8c02cd20 con 0x56077e420fb0 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4ca3377740 1 -- 172.21.15.42:0/908345850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c840114a0 con 0x56077e420fb0 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4c7f400640 1 -- 172.21.15.42:0/908345850 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4c8c045040 con 0x56077e420fb0 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4ca3377740 10 client.15828 fetch_fsmap learned FSMap version 19 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4ca3377740 1 -- 172.21.15.42:0/908345850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56077e4a0860 con 0x56077e420fb0 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4c7f400640 1 -- 172.21.15.42:0/908345850 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c840114a0 con 0x56077e420fb0 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4c7f400640 1 -- 172.21.15.42:0/908345850 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f4c8c005400 con 0x56077e420fb0 2024-04-20T10:38:28.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4ca3377740 10 client.15828 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:38:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4ca3377740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:38:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4ca3377740 20 client.15828 populate_metadata read hostname 'smithi042' 2024-04-20T10:38:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4ca3377740 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56077e4a0fe0 0x56077e4a3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4ca3377740 4 client.15828 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:38:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.255+0000 7f4ca3377740 1 -- 172.21.15.42:0/908345850 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56077e38d8e0 con 0x56077e4a0fe0 2024-04-20T10:38:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.256+0000 7f4c9d200640 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56077e4a0fe0 0x56077e4a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.256+0000 7f4c9d200640 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56077e4a0fe0 0x56077e4a3400 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c91ea70ed26fd37 server_cookie=66c155a6a2b8a746 in_seq=0 out_seq=0 2024-04-20T10:38:28.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.256+0000 7f4c7f400640 10 client.15828 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:28.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4c7f400640 1 -- 172.21.15.42:0/908345850 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56077e38d8e0 con 0x56077e4a0fe0 2024-04-20T10:38:28.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4c7f400640 10 client.15828 handle_command_reply: tid=0 2024-04-20T10:38:28.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4ca3377740 1 client.15828 shutdown 2024-04-20T10:38:28.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4ca3377740 1 -- 172.21.15.42:0/908345850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e420fb0 msgr2=0x56077e4242f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:28.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4ca3377740 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e420fb0 0x56077e4242f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009230 tx=0x7f4c8c0042c0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4ca3377740 1 -- 172.21.15.42:0/908345850 shutdown_connections 2024-04-20T10:38:28.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4ca3377740 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56077e4a0fe0 0x56077e4a3400 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4ca3377740 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56077e42a4b0 0x56077e424d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4ca3377740 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e4218d0 0x56077e424830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4ca3377740 1 --2- 172.21.15.42:0/908345850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56077e420fb0 0x56077e4242f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4ca3377740 1 -- 172.21.15.42:0/908345850 >> 172.21.15.42:0/908345850 conn(0x56077e41b6e0 msgr2=0x56077e419eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:28.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4c7f400640 0 client.15828 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:28.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4ca3377740 1 -- 172.21.15.42:0/908345850 shutdown_connections 2024-04-20T10:38:28.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4ca3377740 1 -- 172.21.15.42:0/908345850 wait complete. 2024-04-20T10:38:28.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.258+0000 7f4ca3377740 20 client.15828 trim_cache size 0 max 16384 2024-04-20T10:38:28.261 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:38:28.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.259+0000 7f4ca0600640 1 -- 172.21.15.42:0/3423420936 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c800a3e20 msgr2=0x7f4c800a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:28.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.259+0000 7f4ca0600640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c800a3e20 0x7f4c800a62e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x56077e41c870 tx=0x7f4c84004730 comp rx=0 tx=0).stop 2024-04-20T10:38:28.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.259+0000 7f4ca0600640 1 -- 172.21.15.42:0/3423420936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9810e840 msgr2=0x7f4c9819a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:28.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.259+0000 7f4ca0600640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9810e840 0x7f4c9819a8f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000e9d0 tx=0x7f4c9000ee90 comp rx=0 tx=0).stop 2024-04-20T10:38:28.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.259+0000 7f4ca0600640 1 -- 172.21.15.42:0/3423420936 shutdown_connections 2024-04-20T10:38:28.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.259+0000 7f4ca0600640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c800a3e20 0x7f4c800a62e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.259+0000 7f4ca0600640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9810e840 0x7f4c9819a8f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.259+0000 7f4ca0600640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c98074b50 0x7f4c981a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.259+0000 7f4ca0600640 1 --2- 172.21.15.42:0/3423420936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c98074230 0x7f4c981a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:28.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.259+0000 7f4ca0600640 1 -- 172.21.15.42:0/3423420936 >> 172.21.15.42:0/3423420936 conn(0x7f4c980793b0 msgr2=0x7f4c980feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:28.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.260+0000 7f4ca0600640 1 -- 172.21.15.42:0/3423420936 shutdown_connections 2024-04-20T10:38:28.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:28.260+0000 7f4ca0600640 1 -- 172.21.15.42:0/3423420936 wait complete. 2024-04-20T10:38:28.603 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:28 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3933401858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:38:28.807 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:28 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3933401858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:38:28.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:28 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3933401858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:38:29.280 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:38:29.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:38:29.281 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:38:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.509+0000 7fc71ea00640 1 -- 172.21.15.42:0/3995103960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718074c20 msgr2=0x7fc7181128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.509+0000 7fc71ea00640 1 --2- 172.21.15.42:0/3995103960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718074c20 0x7fc7181128d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc704009900 tx=0x7fc70402bcd0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.509+0000 7fc71ea00640 1 -- 172.21.15.42:0/3995103960 shutdown_connections 2024-04-20T10:38:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.509+0000 7fc71ea00640 1 --2- 172.21.15.42:0/3995103960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc718112e10 0x7fc718115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.509+0000 7fc71ea00640 1 --2- 172.21.15.42:0/3995103960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718074c20 0x7fc7181128d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.509+0000 7fc71ea00640 1 --2- 172.21.15.42:0/3995103960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc718074300 0x7fc7180746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.509+0000 7fc71ea00640 1 -- 172.21.15.42:0/3995103960 >> 172.21.15.42:0/3995103960 conn(0x7fc7180fd930 msgr2=0x7fc7180ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.509+0000 7fc71ea00640 1 -- 172.21.15.42:0/3995103960 shutdown_connections 2024-04-20T10:38:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.510+0000 7fc71ea00640 1 -- 172.21.15.42:0/3995103960 wait complete. 2024-04-20T10:38:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.511+0000 7fc71ea00640 1 Processor -- start 2024-04-20T10:38:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.511+0000 7fc71ea00640 1 -- start start 2024-04-20T10:38:29.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.512+0000 7fc71ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc718074300 0x7fc71819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.512+0000 7fc71ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc718074c20 0x7fc71819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.512+0000 7fc71ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718112e10 0x7fc7181a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.512+0000 7fc71ea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc718118850 con 0x7fc718074300 2024-04-20T10:38:29.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.512+0000 7fc71ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181186d0 con 0x7fc718112e10 2024-04-20T10:38:29.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.512+0000 7fc71ea00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181189d0 con 0x7fc718074c20 2024-04-20T10:38:29.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.512+0000 7fc71e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718112e10 0x7fc7181a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.512+0000 7fc71d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc718074300 0x7fc71819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.512+0000 7fc71e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718112e10 0x7fc7181a3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:33916/0 (socket says 172.21.15.42:33916) 2024-04-20T10:38:29.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.512+0000 7fc71e000640 1 -- 172.21.15.42:0/1881264073 learned_addr learned my addr 172.21.15.42:0/1881264073 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:29.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.513+0000 7fc71cc00640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc718074c20 0x7fc71819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.513+0000 7fc71e000640 1 -- 172.21.15.42:0/1881264073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc718074c20 msgr2=0x7fc71819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.513+0000 7fc71e000640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc718074c20 0x7fc71819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.513+0000 7fc71e000640 1 -- 172.21.15.42:0/1881264073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc718074300 msgr2=0x7fc71819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.513+0000 7fc71e000640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc718074300 0x7fc71819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.513+0000 7fc71e000640 1 -- 172.21.15.42:0/1881264073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7181a3d40 con 0x7fc718112e10 2024-04-20T10:38:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.513+0000 7fc71cc00640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc718074c20 0x7fc71819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:38:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.513+0000 7fc71e000640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718112e10 0x7fc7181a3610 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc71000eb40 tx=0x7fc71000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.513+0000 7fc716000640 1 -- 172.21.15.42:0/1881264073 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710004b20 con 0x7fc718112e10 2024-04-20T10:38:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.513+0000 7fc716000640 1 -- 172.21.15.42:0/1881264073 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc710004cc0 con 0x7fc718112e10 2024-04-20T10:38:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.513+0000 7fc71ea00640 1 -- 172.21.15.42:0/1881264073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7181a4030 con 0x7fc718112e10 2024-04-20T10:38:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.514+0000 7fc716000640 1 -- 172.21.15.42:0/1881264073 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710010850 con 0x7fc718112e10 2024-04-20T10:38:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.514+0000 7fc71ea00640 1 -- 172.21.15.42:0/1881264073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7181ab8c0 con 0x7fc718112e10 2024-04-20T10:38:29.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.517+0000 7fc716000640 1 -- 172.21.15.42:0/1881264073 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc710005020 con 0x7fc718112e10 2024-04-20T10:38:29.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.517+0000 7fc722e08740 1 Processor -- start 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.517+0000 7fc722e08740 1 -- start start 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.517+0000 7fc722e08740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073a0f0 0x56055073a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.517+0000 7fc722e08740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073aa10 0x5605507430b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc722e08740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5605507435f0 0x5605507459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc722e08740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56055069ed20 con 0x56055073aa10 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc722e08740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5605506453e0 con 0x56055073a0f0 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc722e08740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56055069ace0 con 0x5605507435f0 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc71cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073a0f0 0x56055073a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc71cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073a0f0 0x56055073a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:33930/0 (socket says 172.21.15.42:33930) 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc71cc00640 1 -- 172.21.15.42:0/2294939611 learned_addr learned my addr 172.21.15.42:0/2294939611 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc71d600640 1 --2- 172.21.15.42:0/2294939611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073aa10 0x5605507430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc71cc00640 1 --2- 172.21.15.42:0/2294939611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5605507435f0 0x5605507459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc71cc00640 1 -- 172.21.15.42:0/2294939611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5605507435f0 msgr2=0x5605507459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc71cc00640 1 --2- 172.21.15.42:0/2294939611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5605507435f0 0x5605507459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc71cc00640 1 -- 172.21.15.42:0/2294939611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073aa10 msgr2=0x5605507430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc71cc00640 1 --2- 172.21.15.42:0/2294939611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073aa10 0x5605507430b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.518+0000 7fc71cc00640 1 -- 172.21.15.42:0/2294939611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560550664980 con 0x56055073a0f0 2024-04-20T10:38:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.519+0000 7fc71cc00640 1 --2- 172.21.15.42:0/2294939611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073a0f0 0x56055073a4d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc704000c00 tx=0x7fc704005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e8c1ebbc83ce065 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:29.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.520+0000 7fc716000640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7000a3da0 0x7fc7000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.520+0000 7fc716000640 1 -- 172.21.15.42:0/1881264073 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc7100d0ff0 con 0x7fc718112e10 2024-04-20T10:38:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.520+0000 7fc71d600640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7000a3da0 0x7fc7000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.521+0000 7fc71d600640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7000a3da0 0x7fc7000a6260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x560550739bc0 tx=0x7fc70c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.522+0000 7fc70a600640 1 -- 172.21.15.42:0/2294939611 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704048070 con 0x56055073a0f0 2024-04-20T10:38:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.522+0000 7fc70a600640 1 -- 172.21.15.42:0/2294939611 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc70402ce00 con 0x56055073a0f0 2024-04-20T10:38:29.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.522+0000 7fc70a600640 1 -- 172.21.15.42:0/2294939611 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704004900 con 0x56055073a0f0 2024-04-20T10:38:29.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.523+0000 7fc722e08740 1 -- 172.21.15.42:0/2294939611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073a0f0 msgr2=0x56055073a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.523+0000 7fc722e08740 1 --2- 172.21.15.42:0/2294939611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073a0f0 0x56055073a4d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc704000c00 tx=0x7fc704005be0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.523+0000 7fc722e08740 1 -- 172.21.15.42:0/2294939611 shutdown_connections 2024-04-20T10:38:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.523+0000 7fc722e08740 1 --2- 172.21.15.42:0/2294939611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5605507435f0 0x5605507459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.523+0000 7fc722e08740 1 --2- 172.21.15.42:0/2294939611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073aa10 0x5605507430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.523+0000 7fc722e08740 1 --2- 172.21.15.42:0/2294939611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073a0f0 0x56055073a4d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.523+0000 7fc722e08740 1 -- 172.21.15.42:0/2294939611 >> 172.21.15.42:0/2294939611 conn(0x560550734820 msgr2=0x560550734c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.523+0000 7fc722e08740 1 -- 172.21.15.42:0/2294939611 shutdown_connections 2024-04-20T10:38:29.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.523+0000 7fc722e08740 1 -- 172.21.15.42:0/2294939611 wait complete. 2024-04-20T10:38:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.524+0000 7fc722e08740 1 -- start start 2024-04-20T10:38:29.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.524+0000 7fc722e08740 1 Processor -- start 2024-04-20T10:38:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.525+0000 7fc722e08740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073a0f0 0x560550740640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.525+0000 7fc722e08740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073aa10 0x560550740b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.525+0000 7fc722e08740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605507435f0 0x5605507410c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.525+0000 7fc722e08740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560550746380 con 0x5605507435f0 2024-04-20T10:38:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.525+0000 7fc722e08740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5605506587a0 con 0x56055073aa10 2024-04-20T10:38:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.525+0000 7fc722e08740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x560550746500 con 0x56055073a0f0 2024-04-20T10:38:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.525+0000 7fc722e08740 10 client.0 _ll_register_callbacks cb 0x56055065ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:38:29.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.525+0000 7fc71cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073aa10 0x560550740b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.525+0000 7fc71e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605507435f0 0x5605507410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.525+0000 7fc71d600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073a0f0 0x560550740640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.525+0000 7fc71cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073aa10 0x560550740b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:33936/0 (socket says 172.21.15.42:33936) 2024-04-20T10:38:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.525+0000 7fc71cc00640 1 -- 172.21.15.42:0/4125280132 learned_addr learned my addr 172.21.15.42:0/4125280132 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.526+0000 7fc71cc00640 1 -- 172.21.15.42:0/4125280132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073a0f0 msgr2=0x560550740640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.526+0000 7fc71cc00640 1 --2- 172.21.15.42:0/4125280132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073a0f0 0x560550740640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.526+0000 7fc71cc00640 1 -- 172.21.15.42:0/4125280132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605507435f0 msgr2=0x5605507410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.526+0000 7fc71cc00640 1 --2- 172.21.15.42:0/4125280132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605507435f0 0x5605507410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.526+0000 7fc71cc00640 1 -- 172.21.15.42:0/4125280132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605504797e0 con 0x56055073aa10 2024-04-20T10:38:29.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.526+0000 7fc71cc00640 1 --2- 172.21.15.42:0/4125280132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073aa10 0x560550740b80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc70402bd10 tx=0x7fc7040044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d2ab64c49daa1e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:29.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.526+0000 7fc6fea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:38:29.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.526+0000 7fc6fea00640 1 -- 172.21.15.42:0/4125280132 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704048040 con 0x56055073aa10 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.526+0000 7fc6fea00640 1 -- 172.21.15.42:0/4125280132 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc70402cb20 con 0x56055073aa10 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.526+0000 7fc722e08740 1 -- 172.21.15.42:0/4125280132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70402ce00 con 0x56055073aa10 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.527+0000 7fc6fea00640 1 -- 172.21.15.42:0/4125280132 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc70405a050 con 0x56055073aa10 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.527+0000 7fc722e08740 10 client.25958 fetch_fsmap learned FSMap version 19 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.527+0000 7fc722e08740 1 -- 172.21.15.42:0/4125280132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5605504d9f70 con 0x56055073aa10 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.527+0000 7fc6fea00640 1 -- 172.21.15.42:0/4125280132 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc70402ce00 con 0x56055073aa10 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.527+0000 7fc6fea00640 1 -- 172.21.15.42:0/4125280132 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc70404b070 con 0x56055073aa10 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.527+0000 7fc722e08740 10 client.25958 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.527+0000 7fc722e08740 10 client.25958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.527+0000 7fc722e08740 20 client.25958 populate_metadata read hostname 'smithi042' 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.527+0000 7fc722e08740 1 --2- 172.21.15.42:0/4125280132 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5605507537f0 0x560550755be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.527+0000 7fc722e08740 4 client.25958 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:38:29.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.527+0000 7fc722e08740 1 -- 172.21.15.42:0/4125280132 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605506ac740 con 0x5605507537f0 2024-04-20T10:38:29.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.528+0000 7fc71d600640 1 --2- 172.21.15.42:0/4125280132 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5605507537f0 0x560550755be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.528+0000 7fc71d600640 1 --2- 172.21.15.42:0/4125280132 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5605507537f0 0x560550755be0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77f6b2c0eaca3216 server_cookie=28e29ba9c72968bb in_seq=0 out_seq=0 2024-04-20T10:38:29.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.528+0000 7fc6fea00640 10 client.25958 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc6fea00640 1 -- 172.21.15.42:0/4125280132 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605506ac740 con 0x5605507537f0 2024-04-20T10:38:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc6fea00640 10 client.25958 handle_command_reply: tid=0 2024-04-20T10:38:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc722e08740 1 client.25958 shutdown 2024-04-20T10:38:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc722e08740 1 -- 172.21.15.42:0/4125280132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073aa10 msgr2=0x560550740b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc722e08740 1 --2- 172.21.15.42:0/4125280132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073aa10 0x560550740b80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc70402bd10 tx=0x7fc7040044d0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc722e08740 1 -- 172.21.15.42:0/4125280132 shutdown_connections 2024-04-20T10:38:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc722e08740 1 --2- 172.21.15.42:0/4125280132 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5605507537f0 0x560550755be0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc722e08740 1 --2- 172.21.15.42:0/4125280132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5605507435f0 0x5605507410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc722e08740 1 --2- 172.21.15.42:0/4125280132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56055073aa10 0x560550740b80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc722e08740 1 --2- 172.21.15.42:0/4125280132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073a0f0 0x560550740640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc6fea00640 0 client.25958 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:29.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc722e08740 1 -- 172.21.15.42:0/4125280132 >> 172.21.15.42:0/4125280132 conn(0x560550734820 msgr2=0x560550736570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:29.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc722e08740 1 -- 172.21.15.42:0/4125280132 shutdown_connections 2024-04-20T10:38:29.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.530+0000 7fc722e08740 1 -- 172.21.15.42:0/4125280132 wait complete. 2024-04-20T10:38:29.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.531+0000 7fc722e08740 20 client.25958 trim_cache size 0 max 16384 2024-04-20T10:38:29.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.551+0000 7fc722e08740 1 Processor -- start 2024-04-20T10:38:29.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.551+0000 7fc722e08740 1 -- start start 2024-04-20T10:38:29.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.551+0000 7fc722e08740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073a0f0 0x560550742d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.551+0000 7fc722e08740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073aa10 0x560550733880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.551+0000 7fc722e08740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 0x560550758c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.551+0000 7fc722e08740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181b20c0 con 0x56055073aa10 2024-04-20T10:38:29.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.551+0000 7fc722e08740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181b4940 con 0x5605507435f0 2024-04-20T10:38:29.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.551+0000 7fc722e08740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181b2730 con 0x56055073a0f0 2024-04-20T10:38:29.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.552+0000 7fc71d600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073a0f0 0x560550742d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.552+0000 7fc71d600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073a0f0 0x560550742d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:47194/0 (socket says 172.21.15.42:47194) 2024-04-20T10:38:29.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.552+0000 7fc71d600640 1 -- 172.21.15.42:0/514937854 learned_addr learned my addr 172.21.15.42:0/514937854 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:29.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.552+0000 7fc71e000640 1 --2- 172.21.15.42:0/514937854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 0x560550758c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.552+0000 7fc71cc00640 1 --2- 172.21.15.42:0/514937854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073aa10 0x560550733880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.552+0000 7fc71d600640 1 -- 172.21.15.42:0/514937854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 msgr2=0x560550758c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.552+0000 7fc71d600640 1 --2- 172.21.15.42:0/514937854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 0x560550758c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.552+0000 7fc71d600640 1 -- 172.21.15.42:0/514937854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073aa10 msgr2=0x560550733880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.552+0000 7fc71d600640 1 --2- 172.21.15.42:0/514937854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073aa10 0x560550733880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.552+0000 7fc71d600640 1 -- 172.21.15.42:0/514937854 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560550371170 con 0x56055073a0f0 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.552+0000 7fc71e000640 1 --2- 172.21.15.42:0/514937854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 0x560550758c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.552+0000 7fc71d600640 1 --2- 172.21.15.42:0/514937854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073a0f0 0x560550742d70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc70c012cb0 tx=0x7fc70c01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9cf876b684cf9f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.553+0000 7fc709c00640 1 -- 172.21.15.42:0/514937854 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0043b0 con 0x56055073a0f0 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.553+0000 7fc709c00640 1 -- 172.21.15.42:0/514937854 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc70c02b900 con 0x56055073a0f0 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.553+0000 7fc709c00640 1 -- 172.21.15.42:0/514937854 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0047a0 con 0x56055073a0f0 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.553+0000 7fc722e08740 1 -- 172.21.15.42:0/514937854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073a0f0 msgr2=0x560550742d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.553+0000 7fc722e08740 1 --2- 172.21.15.42:0/514937854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073a0f0 0x560550742d70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc70c012cb0 tx=0x7fc70c01b000 comp rx=0 tx=0).stop 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.554+0000 7fc722e08740 1 -- 172.21.15.42:0/514937854 shutdown_connections 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.554+0000 7fc722e08740 1 --2- 172.21.15.42:0/514937854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 0x560550758c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.554+0000 7fc722e08740 1 --2- 172.21.15.42:0/514937854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073aa10 0x560550733880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.554+0000 7fc722e08740 1 --2- 172.21.15.42:0/514937854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073a0f0 0x560550742d70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.554+0000 7fc722e08740 1 -- 172.21.15.42:0/514937854 >> 172.21.15.42:0/514937854 conn(0x560550734820 msgr2=0x560550733170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:29.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.554+0000 7fc722e08740 1 -- 172.21.15.42:0/514937854 shutdown_connections 2024-04-20T10:38:29.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.554+0000 7fc722e08740 1 -- 172.21.15.42:0/514937854 wait complete. 2024-04-20T10:38:29.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.554+0000 7fc722e08740 1 -- start start 2024-04-20T10:38:29.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.554+0000 7fc722e08740 1 Processor -- start 2024-04-20T10:38:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc722e08740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073a0f0 0x560550756660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc722e08740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073aa10 0x560550756ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc722e08740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 0x5605507570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc722e08740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181b2e00 con 0x56055073a0f0 2024-04-20T10:38:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc722e08740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181b4270 con 0x5605507435f0 2024-04-20T10:38:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc722e08740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc7181b34d0 con 0x56055073aa10 2024-04-20T10:38:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc722e08740 10 client.0 _ll_register_callbacks cb 0x7fc704046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:38:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc71d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073a0f0 0x560550756660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc71e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 0x5605507570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc71e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 0x5605507570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:33952/0 (socket says 172.21.15.42:33952) 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc71e000640 1 -- 172.21.15.42:0/2848601969 learned_addr learned my addr 172.21.15.42:0/2848601969 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc71cc00640 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073aa10 0x560550756ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc71e000640 1 -- 172.21.15.42:0/2848601969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073aa10 msgr2=0x560550756ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc71e000640 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073aa10 0x560550756ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc71e000640 1 -- 172.21.15.42:0/2848601969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073a0f0 msgr2=0x560550756660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc71e000640 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073a0f0 0x560550756660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc71e000640 1 -- 172.21.15.42:0/2848601969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605507b89d0 con 0x5605507435f0 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc71cc00640 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073aa10 0x560550756ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.555+0000 7fc71d600640 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073a0f0 0x560550756660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.556+0000 7fc71e000640 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 0x5605507570e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc7100993d0 tx=0x7fc71009d730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=161c66676c9ce8c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.556+0000 7fc6fea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:38:29.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.556+0000 7fc6fea00640 1 -- 172.21.15.42:0/2848601969 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7100a5be0 con 0x5605507435f0 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.556+0000 7fc6fea00640 1 -- 172.21.15.42:0/2848601969 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc7100a5750 con 0x5605507435f0 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.556+0000 7fc722e08740 1 -- 172.21.15.42:0/2848601969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70c02b900 con 0x5605507435f0 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.556+0000 7fc6fea00640 1 -- 172.21.15.42:0/2848601969 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc7100a5950 con 0x5605507435f0 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.556+0000 7fc722e08740 10 client.25964 fetch_fsmap learned FSMap version 19 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.556+0000 7fc722e08740 1 -- 172.21.15.42:0/2848601969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5605507b9d20 con 0x5605507435f0 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.556+0000 7fc6fea00640 1 -- 172.21.15.42:0/2848601969 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc70c02b900 con 0x5605507435f0 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.557+0000 7fc6fea00640 1 -- 172.21.15.42:0/2848601969 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc7100b6370 con 0x5605507435f0 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.557+0000 7fc722e08740 10 client.25964 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.557+0000 7fc722e08740 10 client.25964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.557+0000 7fc722e08740 20 client.25964 populate_metadata read hostname 'smithi042' 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.557+0000 7fc722e08740 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5605507ba420 0x5605507bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.557+0000 7fc722e08740 4 client.25964 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.557+0000 7fc722e08740 1 -- 172.21.15.42:0/2848601969 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5605506a65b0 con 0x5605507ba420 2024-04-20T10:38:29.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.557+0000 7fc71d600640 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5605507ba420 0x5605507bc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.558+0000 7fc71d600640 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5605507ba420 0x5605507bc840 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d0d0986b1dceba server_cookie=be72a17135c28fda in_seq=0 out_seq=0 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.558+0000 7fc6fea00640 10 client.25964 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.558+0000 7fc6fea00640 1 -- 172.21.15.42:0/2848601969 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5605506a65b0 con 0x5605507ba420 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.558+0000 7fc6fea00640 10 client.25964 handle_command_reply: tid=0 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc722e08740 1 client.25964 shutdown 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc722e08740 1 -- 172.21.15.42:0/2848601969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 msgr2=0x5605507570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc722e08740 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 0x5605507570e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc7100993d0 tx=0x7fc71009d730 comp rx=0 tx=0).stop 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc722e08740 1 -- 172.21.15.42:0/2848601969 shutdown_connections 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc722e08740 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5605507ba420 0x5605507bc840 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc722e08740 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605507435f0 0x5605507570e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc722e08740 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56055073aa10 0x560550756ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc722e08740 1 --2- 172.21.15.42:0/2848601969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56055073a0f0 0x560550756660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc722e08740 1 -- 172.21.15.42:0/2848601969 >> 172.21.15.42:0/2848601969 conn(0x560550734820 msgr2=0x56055074f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc6fea00640 0 client.25964 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:29.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc722e08740 1 -- 172.21.15.42:0/2848601969 shutdown_connections 2024-04-20T10:38:29.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc722e08740 1 -- 172.21.15.42:0/2848601969 wait complete. 2024-04-20T10:38:29.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.559+0000 7fc722e08740 20 client.25964 trim_cache size 0 max 16384 2024-04-20T10:38:29.564 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:38:29.564 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:38:29.564 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "7ff36b7c-c0dc-42b3-ae42-515ab490a7a8", 2024-04-20T10:38:29.564 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:38:29.564 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:38:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.560+0000 7fc71ea00640 1 -- 172.21.15.42:0/1881264073 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7000a3da0 msgr2=0x7fc7000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.560+0000 7fc71ea00640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7000a3da0 0x7fc7000a6260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x560550739bc0 tx=0x7fc70c005f90 comp rx=0 tx=0).stop 2024-04-20T10:38:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.560+0000 7fc71ea00640 1 -- 172.21.15.42:0/1881264073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718112e10 msgr2=0x7fc7181a3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.560+0000 7fc71ea00640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718112e10 0x7fc7181a3610 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc71000eb40 tx=0x7fc71000c2d0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.560+0000 7fc71ea00640 1 -- 172.21.15.42:0/1881264073 shutdown_connections 2024-04-20T10:38:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.560+0000 7fc71ea00640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7000a3da0 0x7fc7000a6260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.560+0000 7fc71ea00640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc718112e10 0x7fc7181a3610 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.560+0000 7fc71ea00640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc718074c20 0x7fc71819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.560+0000 7fc71ea00640 1 --2- 172.21.15.42:0/1881264073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc718074300 0x7fc71819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.560+0000 7fc71ea00640 1 -- 172.21.15.42:0/1881264073 >> 172.21.15.42:0/1881264073 conn(0x7fc7180fd930 msgr2=0x7fc7180fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:29.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.560+0000 7fc71ea00640 1 -- 172.21.15.42:0/1881264073 shutdown_connections 2024-04-20T10:38:29.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.560+0000 7fc71ea00640 1 -- 172.21.15.42:0/1881264073 wait complete. 2024-04-20T10:38:29.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:38:29.583 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:38:29.657 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:29 smithi042 ceph-mon[17078]: pgmap v137: 97 pgs: 97 active+clean; 733 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 19 MiB/s wr, 353 op/s 2024-04-20T10:38:29.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:29 smithi046 ceph-mon[21574]: pgmap v137: 97 pgs: 97 active+clean; 733 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 19 MiB/s wr, 353 op/s 2024-04-20T10:38:29.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.809+0000 7fc738e00640 1 -- 172.21.15.42:0/1233614099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc734103040 msgr2=0x7fc734103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.809+0000 7fc738e00640 1 --2- 172.21.15.42:0/1233614099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc734103040 0x7fc734103420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc724009900 tx=0x7fc72402bcc0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.809+0000 7fc738e00640 1 -- 172.21.15.42:0/1233614099 shutdown_connections 2024-04-20T10:38:29.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.809+0000 7fc738e00640 1 --2- 172.21.15.42:0/1233614099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc73410c410 0x7fc73410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.809+0000 7fc738e00640 1 --2- 172.21.15.42:0/1233614099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc734103960 0x7fc73410be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.809+0000 7fc738e00640 1 --2- 172.21.15.42:0/1233614099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc734103040 0x7fc734103420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.809+0000 7fc738e00640 1 -- 172.21.15.42:0/1233614099 >> 172.21.15.42:0/1233614099 conn(0x7fc7340fc7e0 msgr2=0x7fc7340fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:29.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.809+0000 7fc738e00640 1 -- 172.21.15.42:0/1233614099 shutdown_connections 2024-04-20T10:38:29.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.809+0000 7fc738e00640 1 -- 172.21.15.42:0/1233614099 wait complete. 2024-04-20T10:38:29.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.810+0000 7fc738e00640 1 Processor -- start 2024-04-20T10:38:29.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.810+0000 7fc738e00640 1 -- start start 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.810+0000 7fc738e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc734103040 0x7fc734100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.810+0000 7fc738e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc734103960 0x7fc7341011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc738e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc73410c410 0x7fc734101710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc738e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341121f0 con 0x7fc73410c410 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc738e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc734112070 con 0x7fc734103960 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc738e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc734112370 con 0x7fc734103040 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc733400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc734103040 0x7fc734100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc72be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc734103960 0x7fc7341011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc733400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc734103040 0x7fc734100c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:47238/0 (socket says 172.21.15.42:47238) 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc733400640 1 -- 172.21.15.42:0/2064169894 learned_addr learned my addr 172.21.15.42:0/2064169894 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc733e00640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc73410c410 0x7fc734101710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc733400640 1 -- 172.21.15.42:0/2064169894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc734103960 msgr2=0x7fc7341011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc733400640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc734103960 0x7fc7341011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc733400640 1 -- 172.21.15.42:0/2064169894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc73410c410 msgr2=0x7fc734101710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc733400640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc73410c410 0x7fc734101710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc733400640 1 -- 172.21.15.42:0/2064169894 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7341a6bb0 con 0x7fc734103040 2024-04-20T10:38:29.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.811+0000 7fc733400640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc734103040 0x7fc734100c90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc724000c00 tx=0x7fc724004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.812+0000 7fc730c00640 1 -- 172.21.15.42:0/2064169894 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724042020 con 0x7fc734103040 2024-04-20T10:38:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.812+0000 7fc730c00640 1 -- 172.21.15.42:0/2064169894 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc724005000 con 0x7fc734103040 2024-04-20T10:38:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.812+0000 7fc730c00640 1 -- 172.21.15.42:0/2064169894 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724003b20 con 0x7fc734103040 2024-04-20T10:38:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.812+0000 7fc738e00640 1 -- 172.21.15.42:0/2064169894 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7341a6e40 con 0x7fc734103040 2024-04-20T10:38:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.812+0000 7fc738e00640 1 -- 172.21.15.42:0/2064169894 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7341a72b0 con 0x7fc734103040 2024-04-20T10:38:29.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.816+0000 7fc730c00640 1 -- 172.21.15.42:0/2064169894 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc724045020 con 0x7fc734103040 2024-04-20T10:38:29.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.816+0000 7fc73d032740 1 Processor -- start 2024-04-20T10:38:29.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.816+0000 7fc73d032740 1 -- start start 2024-04-20T10:38:29.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.816+0000 7fc73d032740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a606380 0x55f10a606760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.816+0000 7fc73d032740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a60f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.816+0000 7fc73d032740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a60f880 0x55f10a611c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.816+0000 7fc73d032740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f10a56c150 con 0x55f10a60f880 2024-04-20T10:38:29.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.816+0000 7fc73d032740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f10a512370 con 0x55f10a606ca0 2024-04-20T10:38:29.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.816+0000 7fc73d032740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f10a568110 con 0x55f10a606380 2024-04-20T10:38:29.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.816+0000 7fc733400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a60f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.816+0000 7fc733e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a606380 0x55f10a606760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.816+0000 7fc733400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a60f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:33986/0 (socket says 172.21.15.42:33986) 2024-04-20T10:38:29.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.817+0000 7fc733400640 1 -- 172.21.15.42:0/2991289539 learned_addr learned my addr 172.21.15.42:0/2991289539 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:29.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.817+0000 7fc72be00640 1 --2- 172.21.15.42:0/2991289539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a60f880 0x55f10a611c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.817+0000 7fc733400640 1 -- 172.21.15.42:0/2991289539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a606380 msgr2=0x55f10a606760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.817+0000 7fc733400640 1 --2- 172.21.15.42:0/2991289539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a606380 0x55f10a606760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.817+0000 7fc733400640 1 -- 172.21.15.42:0/2991289539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a60f880 msgr2=0x55f10a611c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.817+0000 7fc733400640 1 --2- 172.21.15.42:0/2991289539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a60f880 0x55f10a611c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.817+0000 7fc733400640 1 -- 172.21.15.42:0/2991289539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f10a3487d0 con 0x55f10a606ca0 2024-04-20T10:38:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.817+0000 7fc733e00640 1 --2- 172.21.15.42:0/2991289539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a606380 0x55f10a606760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:38:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.817+0000 7fc733400640 1 --2- 172.21.15.42:0/2991289539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a60f340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc7240ef980 tx=0x7fc7240f5f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=705c622240edc0f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.818+0000 7fc728c00640 1 -- 172.21.15.42:0/2991289539 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240f40d0 con 0x55f10a606ca0 2024-04-20T10:38:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.818+0000 7fc728c00640 1 -- 172.21.15.42:0/2991289539 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7240ff030 con 0x55f10a606ca0 2024-04-20T10:38:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.818+0000 7fc728c00640 1 -- 172.21.15.42:0/2991289539 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7241035d0 con 0x55f10a606ca0 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.818+0000 7fc730c00640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7180a3dd0 0x7fc7180a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.818+0000 7fc730c00640 1 -- 172.21.15.42:0/2064169894 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc7240f21b0 con 0x7fc734103040 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.818+0000 7fc733e00640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7180a3dd0 0x7fc7180a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.818+0000 7fc73d032740 1 -- 172.21.15.42:0/2991289539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 msgr2=0x55f10a60f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.818+0000 7fc73d032740 1 --2- 172.21.15.42:0/2991289539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a60f340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc7240ef980 tx=0x7fc7240f5f90 comp rx=0 tx=0).stop 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.819+0000 7fc73d032740 1 -- 172.21.15.42:0/2991289539 shutdown_connections 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.819+0000 7fc73d032740 1 --2- 172.21.15.42:0/2991289539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a60f880 0x55f10a611c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.819+0000 7fc73d032740 1 --2- 172.21.15.42:0/2991289539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a60f340 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.819+0000 7fc73d032740 1 --2- 172.21.15.42:0/2991289539 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a606380 0x55f10a606760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.819+0000 7fc73d032740 1 -- 172.21.15.42:0/2991289539 >> 172.21.15.42:0/2991289539 conn(0x55f10a600ab0 msgr2=0x55f10a600e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.819+0000 7fc73d032740 1 -- 172.21.15.42:0/2991289539 shutdown_connections 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.819+0000 7fc73d032740 1 -- 172.21.15.42:0/2991289539 wait complete. 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.819+0000 7fc733e00640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7180a3dd0 0x7fc7180a6290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009fd0 tx=0x7fc72c009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.819+0000 7fc73d032740 1 -- start start 2024-04-20T10:38:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.820+0000 7fc73d032740 1 Processor -- start 2024-04-20T10:38:29.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.820+0000 7fc73d032740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a606380 0x55f10a60c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.820+0000 7fc73d032740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a60ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.820+0000 7fc73d032740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a60f880 0x55f10a60d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.820+0000 7fc73d032740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f10a612430 con 0x55f10a60f880 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.820+0000 7fc73d032740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f10a525740 con 0x55f10a606ca0 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.820+0000 7fc73d032740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f10a6125b0 con 0x55f10a606380 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.820+0000 7fc73d032740 10 client.0 _ll_register_callbacks cb 0x55f10a520b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.820+0000 7fc72be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a60ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.820+0000 7fc72be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a60ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:33988/0 (socket says 172.21.15.42:33988) 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.820+0000 7fc733400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a606380 0x55f10a60c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.820+0000 7fc72be00640 1 -- 172.21.15.42:0/1991612307 learned_addr learned my addr 172.21.15.42:0/1991612307 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.821+0000 7fc733e00640 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a60f880 0x55f10a60d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.821+0000 7fc733400640 1 -- 172.21.15.42:0/1991612307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 msgr2=0x55f10a60ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.821+0000 7fc733400640 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a60ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.821+0000 7fc733400640 1 -- 172.21.15.42:0/1991612307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a60f880 msgr2=0x55f10a60d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.821+0000 7fc733400640 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a60f880 0x55f10a60d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.821+0000 7fc733400640 1 -- 172.21.15.42:0/1991612307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f10a27e3f0 con 0x55f10a606380 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.821+0000 7fc72be00640 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a60ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.821+0000 7fc733400640 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a606380 0x55f10a60c8e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc724005ab0 tx=0x7fc724005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e53868ba9fe972ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.821+0000 7fc721600640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.821+0000 7fc721600640 1 -- 172.21.15.42:0/1991612307 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240ba500 con 0x55f10a606380 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.821+0000 7fc721600640 1 -- 172.21.15.42:0/1991612307 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc7240051b0 con 0x55f10a606380 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.821+0000 7fc73d032740 1 -- 172.21.15.42:0/1991612307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7240ff030 con 0x55f10a606380 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.822+0000 7fc721600640 1 -- 172.21.15.42:0/1991612307 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc7240d4e60 con 0x55f10a606380 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.822+0000 7fc73d032740 10 client.25990 fetch_fsmap learned FSMap version 19 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.822+0000 7fc73d032740 1 -- 172.21.15.42:0/1991612307 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f10a23e0f0 con 0x55f10a606380 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.822+0000 7fc721600640 1 -- 172.21.15.42:0/1991612307 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7240ff030 con 0x55f10a606380 2024-04-20T10:38:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.822+0000 7fc721600640 1 -- 172.21.15.42:0/1991612307 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc7240ba8d0 con 0x55f10a606380 2024-04-20T10:38:29.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.822+0000 7fc73d032740 10 client.25990 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:38:29.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.822+0000 7fc73d032740 10 client.25990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:38:29.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.822+0000 7fc73d032740 20 client.25990 populate_metadata read hostname 'smithi042' 2024-04-20T10:38:29.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.822+0000 7fc73d032740 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f10a61f7f0 0x55f10a621be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.822+0000 7fc73d032740 4 client.25990 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:38:29.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.822+0000 7fc73d032740 1 -- 172.21.15.42:0/1991612307 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f10a579b70 con 0x55f10a61f7f0 2024-04-20T10:38:29.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.823+0000 7fc72be00640 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f10a61f7f0 0x55f10a621be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.823+0000 7fc72be00640 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f10a61f7f0 0x55f10a621be0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b41d3f2287c5ad6a server_cookie=fe8b90baff6b5e4a in_seq=0 out_seq=0 2024-04-20T10:38:29.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.823+0000 7fc721600640 10 client.25990 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:29.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.827+0000 7fc721600640 1 -- 172.21.15.42:0/1991612307 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f10a579b70 con 0x55f10a61f7f0 2024-04-20T10:38:29.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.827+0000 7fc721600640 10 client.25990 handle_command_reply: tid=0 2024-04-20T10:38:29.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.827+0000 7fc73d032740 1 client.25990 shutdown 2024-04-20T10:38:29.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.827+0000 7fc73d032740 1 -- 172.21.15.42:0/1991612307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a606380 msgr2=0x55f10a60c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.827+0000 7fc73d032740 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a606380 0x55f10a60c8e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc724005ab0 tx=0x7fc724005d50 comp rx=0 tx=0).stop 2024-04-20T10:38:29.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.828+0000 7fc73d032740 1 -- 172.21.15.42:0/1991612307 shutdown_connections 2024-04-20T10:38:29.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.828+0000 7fc73d032740 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f10a61f7f0 0x55f10a621be0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.828+0000 7fc73d032740 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a60f880 0x55f10a60d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.828+0000 7fc73d032740 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a60ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.828+0000 7fc73d032740 1 --2- 172.21.15.42:0/1991612307 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a606380 0x55f10a60c8e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.828+0000 7fc721600640 0 client.25990 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:29.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.828+0000 7fc73d032740 1 -- 172.21.15.42:0/1991612307 >> 172.21.15.42:0/1991612307 conn(0x55f10a600ab0 msgr2=0x55f10a602800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:29.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.828+0000 7fc73d032740 1 -- 172.21.15.42:0/1991612307 shutdown_connections 2024-04-20T10:38:29.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.828+0000 7fc73d032740 1 -- 172.21.15.42:0/1991612307 wait complete. 2024-04-20T10:38:29.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.828+0000 7fc73d032740 20 client.25990 trim_cache size 0 max 16384 2024-04-20T10:38:29.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.851+0000 7fc73d032740 1 Processor -- start 2024-04-20T10:38:29.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.852+0000 7fc73d032740 1 -- start start 2024-04-20T10:38:29.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.852+0000 7fc73d032740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a606380 0x55f10a60f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.852+0000 7fc73d032740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a600e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.852+0000 7fc73d032740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a60f880 0x55f10a61b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.852+0000 7fc73d032740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341adcb0 con 0x55f10a606380 2024-04-20T10:38:29.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.852+0000 7fc73d032740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341b0530 con 0x55f10a606ca0 2024-04-20T10:38:29.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.852+0000 7fc73d032740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ae320 con 0x55f10a60f880 2024-04-20T10:38:29.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.852+0000 7fc733e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a60f880 0x55f10a61b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.852+0000 7fc733400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a606380 0x55f10a60f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.853+0000 7fc733e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a60f880 0x55f10a61b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:47258/0 (socket says 172.21.15.42:47258) 2024-04-20T10:38:29.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.853+0000 7fc733e00640 1 -- 172.21.15.42:0/3450297316 learned_addr learned my addr 172.21.15.42:0/3450297316 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:29.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.853+0000 7fc72be00640 1 --2- 172.21.15.42:0/3450297316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a600e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.853+0000 7fc733e00640 1 -- 172.21.15.42:0/3450297316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 msgr2=0x55f10a600e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.853+0000 7fc733e00640 1 --2- 172.21.15.42:0/3450297316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a600e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.853+0000 7fc733e00640 1 -- 172.21.15.42:0/3450297316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a606380 msgr2=0x55f10a60f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.853+0000 7fc733e00640 1 --2- 172.21.15.42:0/3450297316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a606380 0x55f10a60f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.853+0000 7fc733e00640 1 -- 172.21.15.42:0/3450297316 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f10a611c70 con 0x55f10a60f880 2024-04-20T10:38:29.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.853+0000 7fc733e00640 1 --2- 172.21.15.42:0/3450297316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a60f880 0x55f10a61b1a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00f670 tx=0x7fc72c019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad2fc90003add8d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:29.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.854+0000 7fc723e00640 1 -- 172.21.15.42:0/3450297316 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c003a30 con 0x55f10a60f880 2024-04-20T10:38:29.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.854+0000 7fc723e00640 1 -- 172.21.15.42:0/3450297316 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c003bd0 con 0x55f10a60f880 2024-04-20T10:38:29.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.854+0000 7fc723e00640 1 -- 172.21.15.42:0/3450297316 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c010ab0 con 0x55f10a60f880 2024-04-20T10:38:29.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.854+0000 7fc73d032740 1 -- 172.21.15.42:0/3450297316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a60f880 msgr2=0x55f10a61b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.854+0000 7fc73d032740 1 --2- 172.21.15.42:0/3450297316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a60f880 0x55f10a61b1a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00f670 tx=0x7fc72c019000 comp rx=0 tx=0).stop 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.855+0000 7fc73d032740 1 -- 172.21.15.42:0/3450297316 shutdown_connections 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.855+0000 7fc73d032740 1 --2- 172.21.15.42:0/3450297316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a60f880 0x55f10a61b1a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.855+0000 7fc73d032740 1 --2- 172.21.15.42:0/3450297316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a600e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.855+0000 7fc73d032740 1 --2- 172.21.15.42:0/3450297316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a606380 0x55f10a60f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.855+0000 7fc73d032740 1 -- 172.21.15.42:0/3450297316 >> 172.21.15.42:0/3450297316 conn(0x55f10a600ab0 msgr2=0x55f10a601e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.855+0000 7fc73d032740 1 -- 172.21.15.42:0/3450297316 shutdown_connections 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.855+0000 7fc73d032740 1 -- 172.21.15.42:0/3450297316 wait complete. 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.855+0000 7fc73d032740 1 -- start start 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.855+0000 7fc73d032740 1 Processor -- start 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.856+0000 7fc73d032740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a606380 0x55f10a6178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.856+0000 7fc73d032740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a609190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.856+0000 7fc73d032740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a60f880 0x55f10a6096d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.856+0000 7fc73d032740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ae9f0 con 0x55f10a606380 2024-04-20T10:38:29.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.856+0000 7fc73d032740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341afe60 con 0x55f10a606ca0 2024-04-20T10:38:29.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.856+0000 7fc73d032740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341af0c0 con 0x55f10a60f880 2024-04-20T10:38:29.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.856+0000 7fc73d032740 10 client.0 _ll_register_callbacks cb 0x7fc72c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:38:29.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.856+0000 7fc72be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a609190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.856+0000 7fc72be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a609190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:34006/0 (socket says 172.21.15.42:34006) 2024-04-20T10:38:29.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.856+0000 7fc72be00640 1 -- 172.21.15.42:0/1746290397 learned_addr learned my addr 172.21.15.42:0/1746290397 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:38:29.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.856+0000 7fc733e00640 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a60f880 0x55f10a6096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.856+0000 7fc733400640 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a606380 0x55f10a6178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.857+0000 7fc72be00640 1 -- 172.21.15.42:0/1746290397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a60f880 msgr2=0x55f10a6096d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.857+0000 7fc72be00640 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a60f880 0x55f10a6096d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.857+0000 7fc72be00640 1 -- 172.21.15.42:0/1746290397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a606380 msgr2=0x55f10a6178a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.857+0000 7fc72be00640 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a606380 0x55f10a6178a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.857+0000 7fc72be00640 1 -- 172.21.15.42:0/1746290397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f10a609c10 con 0x55f10a606ca0 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.857+0000 7fc72be00640 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a609190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc71c000c00 tx=0x7fc71c012aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fbe58c39474640f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.857+0000 7fc721600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.857+0000 7fc721600640 1 -- 172.21.15.42:0/1746290397 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c020130 con 0x55f10a606ca0 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.857+0000 7fc721600640 1 -- 172.21.15.42:0/1746290397 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc71c0202d0 con 0x55f10a606ca0 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.857+0000 7fc73d032740 1 -- 172.21.15.42:0/1746290397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc72c003bd0 con 0x55f10a606ca0 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.858+0000 7fc721600640 1 -- 172.21.15.42:0/1746290397 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc71c030050 con 0x55f10a606ca0 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.858+0000 7fc73d032740 10 client.25982 fetch_fsmap learned FSMap version 19 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.858+0000 7fc73d032740 1 -- 172.21.15.42:0/1746290397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f10a685870 con 0x55f10a606ca0 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.858+0000 7fc721600640 1 -- 172.21.15.42:0/1746290397 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c003bd0 con 0x55f10a606ca0 2024-04-20T10:38:29.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.858+0000 7fc721600640 1 -- 172.21.15.42:0/1746290397 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc71c010430 con 0x55f10a606ca0 2024-04-20T10:38:29.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.858+0000 7fc73d032740 10 client.25982 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:38:29.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.858+0000 7fc73d032740 10 client.25982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:38:29.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.858+0000 7fc73d032740 20 client.25982 populate_metadata read hostname 'smithi042' 2024-04-20T10:38:29.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.858+0000 7fc73d032740 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f10a685f40 0x55f10a688360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:38:29.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.858+0000 7fc73d032740 4 client.25982 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:38:29.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.858+0000 7fc73d032740 1 -- 172.21.15.42:0/1746290397 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f10a5739e0 con 0x55f10a685f40 2024-04-20T10:38:29.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.859+0000 7fc733400640 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f10a685f40 0x55f10a688360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:38:29.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.859+0000 7fc733400640 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f10a685f40 0x55f10a688360 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f5f359009f2871 server_cookie=f38078fb210f3576 in_seq=0 out_seq=0 2024-04-20T10:38:29.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.859+0000 7fc721600640 10 client.25982 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:29.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.860+0000 7fc721600640 1 -- 172.21.15.42:0/1746290397 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55f10a5739e0 con 0x55f10a685f40 2024-04-20T10:38:29.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.860+0000 7fc721600640 10 client.25982 handle_command_reply: tid=0 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.860+0000 7fc73d032740 1 client.25982 shutdown 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.861+0000 7fc73d032740 1 -- 172.21.15.42:0/1746290397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 msgr2=0x55f10a609190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.861+0000 7fc73d032740 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a609190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc71c000c00 tx=0x7fc71c012aa0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.861+0000 7fc73d032740 1 -- 172.21.15.42:0/1746290397 shutdown_connections 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.861+0000 7fc73d032740 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f10a685f40 0x55f10a688360 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.861+0000 7fc73d032740 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f10a60f880 0x55f10a6096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.861+0000 7fc721600640 0 client.25982 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.861+0000 7fc73d032740 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f10a606ca0 0x55f10a609190 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.861+0000 7fc73d032740 1 --2- 172.21.15.42:0/1746290397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f10a606380 0x55f10a6178a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.861+0000 7fc73d032740 1 -- 172.21.15.42:0/1746290397 >> 172.21.15.42:0/1746290397 conn(0x55f10a600ab0 msgr2=0x55f10a5ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.861+0000 7fc73d032740 1 -- 172.21.15.42:0/1746290397 shutdown_connections 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.861+0000 7fc73d032740 1 -- 172.21.15.42:0/1746290397 wait complete. 2024-04-20T10:38:29.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.861+0000 7fc73d032740 20 client.25982 trim_cache size 0 max 16384 2024-04-20T10:38:29.864 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:38:29.864 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (3754 inodes in the stack)", 2024-04-20T10:38:29.864 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:38:29.864 INFO:teuthology.orchestra.run.smithi042.stdout: "7ff36b7c-c0dc-42b3-ae42-515ab490a7a8": { 2024-04-20T10:38:29.864 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:38:29.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "7ff36b7c-c0dc-42b3-ae42-515ab490a7a8", 2024-04-20T10:38:29.864 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:38:29.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:38:29.864 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:38:29.864 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:38:29.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.862+0000 7fc738e00640 1 -- 172.21.15.42:0/2064169894 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7180a3dd0 msgr2=0x7fc7180a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.862+0000 7fc738e00640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7180a3dd0 0x7fc7180a6290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009fd0 tx=0x7fc72c009500 comp rx=0 tx=0).stop 2024-04-20T10:38:29.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.862+0000 7fc738e00640 1 -- 172.21.15.42:0/2064169894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc734103040 msgr2=0x7fc734100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:38:29.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.862+0000 7fc738e00640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc734103040 0x7fc734100c90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc724000c00 tx=0x7fc724004360 comp rx=0 tx=0).stop 2024-04-20T10:38:29.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.862+0000 7fc738e00640 1 -- 172.21.15.42:0/2064169894 shutdown_connections 2024-04-20T10:38:29.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.862+0000 7fc738e00640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7180a3dd0 0x7fc7180a6290 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.862+0000 7fc738e00640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc73410c410 0x7fc734101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.862+0000 7fc738e00640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc734103960 0x7fc7341011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.862+0000 7fc738e00640 1 --2- 172.21.15.42:0/2064169894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc734103040 0x7fc734100c90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:38:29.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.862+0000 7fc738e00640 1 -- 172.21.15.42:0/2064169894 >> 172.21.15.42:0/2064169894 conn(0x7fc7340fc7e0 msgr2=0x7fc7340fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:38:29.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.862+0000 7fc738e00640 1 -- 172.21.15.42:0/2064169894 shutdown_connections 2024-04-20T10:38:29.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:38:29.862+0000 7fc738e00640 1 -- 172.21.15.42:0/2064169894 wait complete. 2024-04-20T10:38:29.883 INFO:tasks.cephfs.filesystem:scrub status for tag:7ff36b7c-c0dc-42b3-ae42-515ab490a7a8 - {'path': '/', 'tag': '7ff36b7c-c0dc-42b3-ae42-515ab490a7a8', 'options': 'recursive,force'} 2024-04-20T10:38:29.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:29 smithi162 ceph-mon[21854]: pgmap v137: 97 pgs: 97 active+clean; 733 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 19 MiB/s wr, 353 op/s 2024-04-20T10:38:30.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:30 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:38:30.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:30 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:38:30.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:30 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:38:30.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:30 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:38:30.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:30 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:38:30.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:30 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:38:30.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:30 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:38:30.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:30 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:38:30.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:30 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:38:31.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:31 smithi046 ceph-mon[21574]: pgmap v138: 97 pgs: 97 active+clean; 733 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 18 MiB/s wr, 296 op/s 2024-04-20T10:38:31.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:31 smithi042 ceph-mon[17078]: pgmap v138: 97 pgs: 97 active+clean; 733 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 18 MiB/s wr, 296 op/s 2024-04-20T10:38:31.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:31 smithi162 ceph-mon[21854]: pgmap v138: 97 pgs: 97 active+clean; 733 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 18 MiB/s wr, 296 op/s 2024-04-20T10:38:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:33 smithi046 ceph-mon[21574]: pgmap v139: 97 pgs: 97 active+clean; 739 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 19 MiB/s wr, 307 op/s 2024-04-20T10:38:33.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:33 smithi042 ceph-mon[17078]: pgmap v139: 97 pgs: 97 active+clean; 739 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 19 MiB/s wr, 307 op/s 2024-04-20T10:38:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:33 smithi162 ceph-mon[21854]: pgmap v139: 97 pgs: 97 active+clean; 739 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 19 MiB/s wr, 307 op/s 2024-04-20T10:38:35.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:35 smithi046 ceph-mon[21574]: pgmap v140: 97 pgs: 97 active+clean; 739 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.0 MiB/s wr, 26 op/s 2024-04-20T10:38:35.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:35 smithi042 ceph-mon[17078]: pgmap v140: 97 pgs: 97 active+clean; 739 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.0 MiB/s wr, 26 op/s 2024-04-20T10:38:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:35 smithi162 ceph-mon[21854]: pgmap v140: 97 pgs: 97 active+clean; 739 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.0 MiB/s wr, 26 op/s 2024-04-20T10:38:37.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:37 smithi046 ceph-mon[21574]: pgmap v141: 97 pgs: 97 active+clean; 743 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.4 MiB/s wr, 28 op/s 2024-04-20T10:38:37.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:37 smithi042 ceph-mon[17078]: pgmap v141: 97 pgs: 97 active+clean; 743 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.4 MiB/s wr, 28 op/s 2024-04-20T10:38:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:37 smithi162 ceph-mon[21854]: pgmap v141: 97 pgs: 97 active+clean; 743 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.4 MiB/s wr, 28 op/s 2024-04-20T10:38:38.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:38 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:38:38.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:38 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:38:39.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:38 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:38:39.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:39 smithi042 ceph-mon[17078]: pgmap v142: 97 pgs: 97 active+clean; 765 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.9 MiB/s wr, 29 op/s 2024-04-20T10:38:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:39 smithi162 ceph-mon[21854]: pgmap v142: 97 pgs: 97 active+clean; 765 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.9 MiB/s wr, 29 op/s 2024-04-20T10:38:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:39 smithi046 ceph-mon[21574]: pgmap v142: 97 pgs: 97 active+clean; 765 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.9 MiB/s wr, 29 op/s 2024-04-20T10:38:40.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:40 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:38:40.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:40 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:38:41.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:40 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:38:41.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:41 smithi042 ceph-mon[17078]: pgmap v143: 97 pgs: 97 active+clean; 765 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.8 MiB/s wr, 22 op/s 2024-04-20T10:38:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:41 smithi162 ceph-mon[21854]: pgmap v143: 97 pgs: 97 active+clean; 765 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.8 MiB/s wr, 22 op/s 2024-04-20T10:38:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:41 smithi046 ceph-mon[21574]: pgmap v143: 97 pgs: 97 active+clean; 765 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 2.8 MiB/s wr, 22 op/s 2024-04-20T10:38:43.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:43 smithi042 ceph-mon[17078]: pgmap v144: 97 pgs: 97 active+clean; 789 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 4.7 MiB/s wr, 28 op/s 2024-04-20T10:38:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:43 smithi162 ceph-mon[21854]: pgmap v144: 97 pgs: 97 active+clean; 789 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 4.7 MiB/s wr, 28 op/s 2024-04-20T10:38:44.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:43 smithi046 ceph-mon[21574]: pgmap v144: 97 pgs: 97 active+clean; 789 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 4.7 MiB/s wr, 28 op/s 2024-04-20T10:38:45.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:45 smithi042 ceph-mon[17078]: pgmap v145: 97 pgs: 97 active+clean; 789 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 4.2 MiB/s wr, 17 op/s 2024-04-20T10:38:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:45 smithi162 ceph-mon[21854]: pgmap v145: 97 pgs: 97 active+clean; 789 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 4.2 MiB/s wr, 17 op/s 2024-04-20T10:38:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:45 smithi046 ceph-mon[21574]: pgmap v145: 97 pgs: 97 active+clean; 789 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 4.2 MiB/s wr, 17 op/s 2024-04-20T10:38:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:47 smithi042 ceph-mon[17078]: pgmap v146: 97 pgs: 97 active+clean; 892 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 13 MiB/s wr, 183 op/s 2024-04-20T10:38:47.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:47 smithi162 ceph-mon[21854]: pgmap v146: 97 pgs: 97 active+clean; 892 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 13 MiB/s wr, 183 op/s 2024-04-20T10:38:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:47 smithi046 ceph-mon[21574]: pgmap v146: 97 pgs: 97 active+clean; 892 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 13 MiB/s wr, 183 op/s 2024-04-20T10:38:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:49 smithi042 ceph-mon[17078]: pgmap v147: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 25 MiB/s wr, 428 op/s 2024-04-20T10:38:49.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:49 smithi162 ceph-mon[21854]: pgmap v147: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 25 MiB/s wr, 428 op/s 2024-04-20T10:38:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:49 smithi046 ceph-mon[21574]: pgmap v147: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 25 MiB/s wr, 428 op/s 2024-04-20T10:38:51.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:51 smithi162 ceph-mon[21854]: pgmap v148: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 23 MiB/s wr, 419 op/s 2024-04-20T10:38:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:51 smithi046 ceph-mon[21574]: pgmap v148: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 23 MiB/s wr, 419 op/s 2024-04-20T10:38:52.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:51 smithi042 ceph-mon[17078]: pgmap v148: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 23 MiB/s wr, 419 op/s 2024-04-20T10:38:53.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:53 smithi162 ceph-mon[21854]: pgmap v149: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 23 MiB/s wr, 433 op/s 2024-04-20T10:38:54.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:53 smithi046 ceph-mon[21574]: pgmap v149: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 23 MiB/s wr, 433 op/s 2024-04-20T10:38:54.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:53 smithi042 ceph-mon[17078]: pgmap v149: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 23 MiB/s wr, 433 op/s 2024-04-20T10:38:55.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:55 smithi162 ceph-mon[21854]: pgmap v150: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 426 op/s 2024-04-20T10:38:56.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:55 smithi046 ceph-mon[21574]: pgmap v150: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 426 op/s 2024-04-20T10:38:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:55 smithi042 ceph-mon[17078]: pgmap v150: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 426 op/s 2024-04-20T10:38:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:57 smithi162 ceph-mon[21854]: pgmap v151: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 22 MiB/s wr, 435 op/s 2024-04-20T10:38:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:57 smithi046 ceph-mon[21574]: pgmap v151: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 22 MiB/s wr, 435 op/s 2024-04-20T10:38:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:57 smithi042 ceph-mon[17078]: pgmap v151: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 22 MiB/s wr, 435 op/s 2024-04-20T10:38:59.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:38:59.884 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:38:59.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:38:59 smithi162 ceph-mon[21854]: pgmap v152: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 13 MiB/s wr, 271 op/s 2024-04-20T10:38:59.970 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:38:59 smithi042 ceph-mon[17078]: pgmap v152: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 13 MiB/s wr, 271 op/s 2024-04-20T10:39:00.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:38:59 smithi046 ceph-mon[21574]: pgmap v152: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 13 MiB/s wr, 271 op/s 2024-04-20T10:39:00.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.122+0000 7f019e800640 1 -- 172.21.15.42:0/159222906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0198102a20 msgr2=0x7f0198106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.122+0000 7f019e800640 1 --2- 172.21.15.42:0/159222906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0198102a20 0x7f0198106a60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0184009900 tx=0x7f018402bc30 comp rx=0 tx=0).stop 2024-04-20T10:39:00.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.122+0000 7f019e800640 1 -- 172.21.15.42:0/159222906 shutdown_connections 2024-04-20T10:39:00.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.122+0000 7f019e800640 1 --2- 172.21.15.42:0/159222906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0198107140 0x7f019810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.122+0000 7f019e800640 1 --2- 172.21.15.42:0/159222906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0198102a20 0x7f0198106a60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.122+0000 7f019e800640 1 --2- 172.21.15.42:0/159222906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0198102070 0x7f0198102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.122+0000 7f019e800640 1 -- 172.21.15.42:0/159222906 >> 172.21.15.42:0/159222906 conn(0x7f01980fd930 msgr2=0x7f01980ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.122+0000 7f019e800640 1 -- 172.21.15.42:0/159222906 shutdown_connections 2024-04-20T10:39:00.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.123+0000 7f019e800640 1 -- 172.21.15.42:0/159222906 wait complete. 2024-04-20T10:39:00.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.123+0000 7f019e800640 1 Processor -- start 2024-04-20T10:39:00.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.124+0000 7f019e800640 1 -- start start 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.124+0000 7f019e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0198102070 0x7f01981a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.124+0000 7f019e800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0198107140 0x7f01981a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.124+0000 7f019e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f019819a6e0 0x7f019819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.124+0000 7f019e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0198112200 con 0x7f0198102070 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.124+0000 7f019e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0198112080 con 0x7f019819a6e0 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.124+0000 7f019e800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0198112380 con 0x7f0198107140 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.124+0000 7f019ca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0198107140 0x7f01981a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.124+0000 7f019de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f019819a6e0 0x7f019819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.125+0000 7f019d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0198102070 0x7f01981a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.125+0000 7f019ca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0198107140 0x7f01981a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:51178/0 (socket says 172.21.15.42:51178) 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.125+0000 7f019ca00640 1 -- 172.21.15.42:0/3554788859 learned_addr learned my addr 172.21.15.42:0/3554788859 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.125+0000 7f019de00640 1 -- 172.21.15.42:0/3554788859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0198107140 msgr2=0x7f01981a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.125+0000 7f019de00640 1 --2- 172.21.15.42:0/3554788859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0198107140 0x7f01981a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.125+0000 7f019de00640 1 -- 172.21.15.42:0/3554788859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0198102070 msgr2=0x7f01981a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.125+0000 7f019de00640 1 --2- 172.21.15.42:0/3554788859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0198102070 0x7f01981a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.125+0000 7f019de00640 1 -- 172.21.15.42:0/3554788859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019819b2e0 con 0x7f019819a6e0 2024-04-20T10:39:00.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.125+0000 7f019de00640 1 --2- 172.21.15.42:0/3554788859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f019819a6e0 0x7f019819ab60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0190009fd0 tx=0x7f019000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.126+0000 7f0196000640 1 -- 172.21.15.42:0/3554788859 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019000ce20 con 0x7f019819a6e0 2024-04-20T10:39:00.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.126+0000 7f0196000640 1 -- 172.21.15.42:0/3554788859 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0190005020 con 0x7f019819a6e0 2024-04-20T10:39:00.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.126+0000 7f0196000640 1 -- 172.21.15.42:0/3554788859 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190010720 con 0x7f019819a6e0 2024-04-20T10:39:00.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.126+0000 7f019e800640 1 -- 172.21.15.42:0/3554788859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01981a73c0 con 0x7f019819a6e0 2024-04-20T10:39:00.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.126+0000 7f019e800640 1 -- 172.21.15.42:0/3554788859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01981a77c0 con 0x7f019819a6e0 2024-04-20T10:39:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.129+0000 7f01a2bec740 1 Processor -- start 2024-04-20T10:39:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.129+0000 7f0196000640 1 -- 172.21.15.42:0/3554788859 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0190005540 con 0x7f019819a6e0 2024-04-20T10:39:00.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.130+0000 7f01a2bec740 1 -- start start 2024-04-20T10:39:00.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.130+0000 7f01a2bec740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78380 0x555d20c78760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.130+0000 7f01a2bec740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 0x555d20c81340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.130+0000 7f01a2bec740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 0x555d20c83c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.130+0000 7f01a2bec740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555d20bde150 con 0x555d20c81880 2024-04-20T10:39:00.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.130+0000 7f01a2bec740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d20b84370 con 0x555d20c78ca0 2024-04-20T10:39:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.130+0000 7f01a2bec740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x555d20bda110 con 0x555d20c78380 2024-04-20T10:39:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.131+0000 7f019d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 0x555d20c81340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.131+0000 7f019d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 0x555d20c81340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47666/0 (socket says 172.21.15.42:47666) 2024-04-20T10:39:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.131+0000 7f019d400640 1 -- 172.21.15.42:0/2316461225 learned_addr learned my addr 172.21.15.42:0/2316461225 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:00.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.131+0000 7f019ca00640 1 --2- 172.21.15.42:0/2316461225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78380 0x555d20c78760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.131+0000 7f019ca00640 1 --2- 172.21.15.42:0/2316461225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 0x555d20c83c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.131+0000 7f019d400640 1 -- 172.21.15.42:0/2316461225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78380 msgr2=0x555d20c78760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.131+0000 7f019d400640 1 --2- 172.21.15.42:0/2316461225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78380 0x555d20c78760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.131+0000 7f019d400640 1 -- 172.21.15.42:0/2316461225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 msgr2=0x555d20c83c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.131+0000 7f019d400640 1 --2- 172.21.15.42:0/2316461225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 0x555d20c83c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.131+0000 7f019d400640 1 -- 172.21.15.42:0/2316461225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d209ba7d0 con 0x555d20c78ca0 2024-04-20T10:39:00.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.131+0000 7f019d400640 1 --2- 172.21.15.42:0/2316461225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 0x555d20c81340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f018c009cf0 tx=0x7f018c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6557ea3348437bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.132+0000 7f018a600640 1 -- 172.21.15.42:0/2316461225 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c004230 con 0x555d20c78ca0 2024-04-20T10:39:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.132+0000 7f0196000640 1 --2- 172.21.15.42:0/3554788859 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f01800a3da0 0x7f01800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.132+0000 7f018a600640 1 -- 172.21.15.42:0/2316461225 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f018c0043d0 con 0x555d20c78ca0 2024-04-20T10:39:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.132+0000 7f018a600640 1 -- 172.21.15.42:0/2316461225 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c019960 con 0x555d20c78ca0 2024-04-20T10:39:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.132+0000 7f0196000640 1 -- 172.21.15.42:0/3554788859 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f01900d0e30 con 0x7f019819a6e0 2024-04-20T10:39:00.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.132+0000 7f019ca00640 1 --2- 172.21.15.42:0/3554788859 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f01800a3da0 0x7f01800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.132+0000 7f01a2bec740 1 -- 172.21.15.42:0/2316461225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 msgr2=0x555d20c81340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.132+0000 7f01a2bec740 1 --2- 172.21.15.42:0/2316461225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 0x555d20c81340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f018c009cf0 tx=0x7f018c011810 comp rx=0 tx=0).stop 2024-04-20T10:39:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.133+0000 7f01a2bec740 1 -- 172.21.15.42:0/2316461225 shutdown_connections 2024-04-20T10:39:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.133+0000 7f01a2bec740 1 --2- 172.21.15.42:0/2316461225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 0x555d20c83c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.133+0000 7f01a2bec740 1 --2- 172.21.15.42:0/2316461225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 0x555d20c81340 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.133+0000 7f01a2bec740 1 --2- 172.21.15.42:0/2316461225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78380 0x555d20c78760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.133+0000 7f01a2bec740 1 -- 172.21.15.42:0/2316461225 >> 172.21.15.42:0/2316461225 conn(0x555d20c72ab0 msgr2=0x555d20c72e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.133+0000 7f01a2bec740 1 -- 172.21.15.42:0/2316461225 shutdown_connections 2024-04-20T10:39:00.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.133+0000 7f019ca00640 1 --2- 172.21.15.42:0/3554788859 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f01800a3da0 0x7f01800a6260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x555d20c73c40 tx=0x7f018402df90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.133+0000 7f01a2bec740 1 -- 172.21.15.42:0/2316461225 wait complete. 2024-04-20T10:39:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.133+0000 7f01a2bec740 1 -- start start 2024-04-20T10:39:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.134+0000 7f01a2bec740 1 Processor -- start 2024-04-20T10:39:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.134+0000 7f01a2bec740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 0x555d20c7e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.134+0000 7f01a2bec740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78ca0 0x555d20c7ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.134+0000 7f01a2bec740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 0x555d20c7f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.134+0000 7f01a2bec740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555d20c84670 con 0x555d20c81880 2024-04-20T10:39:00.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.134+0000 7f01a2bec740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d20b97740 con 0x555d20c78380 2024-04-20T10:39:00.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.134+0000 7f01a2bec740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x555d20c847f0 con 0x555d20c78ca0 2024-04-20T10:39:00.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.134+0000 7f01a2bec740 10 client.0 _ll_register_callbacks cb 0x555d20b92b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:39:00.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.134+0000 7f019d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 0x555d20c7e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.134+0000 7f019d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 0x555d20c7e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47672/0 (socket says 172.21.15.42:47672) 2024-04-20T10:39:00.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.134+0000 7f019d400640 1 -- 172.21.15.42:0/2102792593 learned_addr learned my addr 172.21.15.42:0/2102792593 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:00.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.134+0000 7f019de00640 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 0x555d20c7f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.135+0000 7f019ca00640 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78ca0 0x555d20c7ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.135+0000 7f019d400640 1 -- 172.21.15.42:0/2102792593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78ca0 msgr2=0x555d20c7ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.135+0000 7f019d400640 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78ca0 0x555d20c7ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.135+0000 7f019d400640 1 -- 172.21.15.42:0/2102792593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 msgr2=0x555d20c7f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.135+0000 7f019d400640 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 0x555d20c7f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.135+0000 7f019d400640 1 -- 172.21.15.42:0/2102792593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d208f03f0 con 0x555d20c78380 2024-04-20T10:39:00.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.135+0000 7f019d400640 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 0x555d20c7e8c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f018c0195c0 tx=0x7f018c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7792315b5f85d9d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.135+0000 7f017ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:39:00.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.135+0000 7f017ea00640 1 -- 172.21.15.42:0/2102792593 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c027070 con 0x555d20c78380 2024-04-20T10:39:00.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.136+0000 7f01a2bec740 1 -- 172.21.15.42:0/2102792593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f018c0043d0 con 0x555d20c78380 2024-04-20T10:39:00.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.136+0000 7f017ea00640 1 -- 172.21.15.42:0/2102792593 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f018c005240 con 0x555d20c78380 2024-04-20T10:39:00.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.136+0000 7f017ea00640 1 -- 172.21.15.42:0/2102792593 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f018c033050 con 0x555d20c78380 2024-04-20T10:39:00.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.136+0000 7f01a2bec740 10 client.25994 fetch_fsmap learned FSMap version 19 2024-04-20T10:39:00.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.136+0000 7f01a2bec740 1 -- 172.21.15.42:0/2102792593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555d208b00f0 con 0x555d20c78380 2024-04-20T10:39:00.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.136+0000 7f017ea00640 1 -- 172.21.15.42:0/2102792593 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f018c0043d0 con 0x555d20c78380 2024-04-20T10:39:00.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.136+0000 7f017ea00640 1 -- 172.21.15.42:0/2102792593 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f018c004ba0 con 0x555d20c78380 2024-04-20T10:39:00.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.136+0000 7f01a2bec740 10 client.25994 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:39:00.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.136+0000 7f01a2bec740 10 client.25994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:39:00.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.136+0000 7f01a2bec740 20 client.25994 populate_metadata read hostname 'smithi042' 2024-04-20T10:39:00.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.136+0000 7f01a2bec740 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x555d20c91ba0 0x555d20c93f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.137+0000 7f01a2bec740 4 client.25994 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:39:00.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.137+0000 7f01a2bec740 1 -- 172.21.15.42:0/2102792593 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d20bebb70 con 0x555d20c91ba0 2024-04-20T10:39:00.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.137+0000 7f019ca00640 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x555d20c91ba0 0x555d20c93f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.137+0000 7f019ca00640 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x555d20c91ba0 0x555d20c93f90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f6977b9ccc566e server_cookie=a08c2ca7a36150ca in_seq=0 out_seq=0 2024-04-20T10:39:00.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.137+0000 7f017ea00640 10 client.25994 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:00.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.138+0000 7f017ea00640 1 -- 172.21.15.42:0/2102792593 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d20bebb70 con 0x555d20c91ba0 2024-04-20T10:39:00.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.138+0000 7f017ea00640 10 client.25994 handle_command_reply: tid=0 2024-04-20T10:39:00.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.138+0000 7f01a2bec740 1 client.25994 shutdown 2024-04-20T10:39:00.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.138+0000 7f01a2bec740 1 -- 172.21.15.42:0/2102792593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 msgr2=0x555d20c7e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.138+0000 7f01a2bec740 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 0x555d20c7e8c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f018c0195c0 tx=0x7f018c003820 comp rx=0 tx=0).stop 2024-04-20T10:39:00.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.139+0000 7f01a2bec740 1 -- 172.21.15.42:0/2102792593 shutdown_connections 2024-04-20T10:39:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.139+0000 7f01a2bec740 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x555d20c91ba0 0x555d20c93f90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.139+0000 7f01a2bec740 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 0x555d20c7f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.139+0000 7f01a2bec740 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78ca0 0x555d20c7ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.139+0000 7f01a2bec740 1 --2- 172.21.15.42:0/2102792593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 0x555d20c7e8c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.139+0000 7f01a2bec740 1 -- 172.21.15.42:0/2102792593 >> 172.21.15.42:0/2102792593 conn(0x555d20c72ab0 msgr2=0x555d20c74800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.139+0000 7f017ea00640 0 client.25994 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.139+0000 7f01a2bec740 1 -- 172.21.15.42:0/2102792593 shutdown_connections 2024-04-20T10:39:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.139+0000 7f01a2bec740 1 -- 172.21.15.42:0/2102792593 wait complete. 2024-04-20T10:39:00.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.139+0000 7f01a2bec740 20 client.25994 trim_cache size 0 max 16384 2024-04-20T10:39:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.162+0000 7f01a2bec740 1 Processor -- start 2024-04-20T10:39:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.162+0000 7f01a2bec740 1 -- start start 2024-04-20T10:39:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.163+0000 7f01a2bec740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 0x555d20c80ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.163+0000 7f01a2bec740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78ca0 0x555d20c8d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.163+0000 7f01a2bec740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 0x555d20c745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.163+0000 7f01a2bec740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f01981adce0 con 0x555d20c81880 2024-04-20T10:39:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.163+0000 7f01a2bec740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01981b0560 con 0x555d20c78380 2024-04-20T10:39:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.163+0000 7f01a2bec740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f01981ae350 con 0x555d20c78ca0 2024-04-20T10:39:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.163+0000 7f019d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 0x555d20c80ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.163+0000 7f019ca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78ca0 0x555d20c8d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.163+0000 7f019d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 0x555d20c80ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47688/0 (socket says 172.21.15.42:47688) 2024-04-20T10:39:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.163+0000 7f019d400640 1 -- 172.21.15.42:0/3122134385 learned_addr learned my addr 172.21.15.42:0/3122134385 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.163+0000 7f019de00640 1 --2- 172.21.15.42:0/3122134385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 0x555d20c745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.164+0000 7f019ca00640 1 -- 172.21.15.42:0/3122134385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 msgr2=0x555d20c80ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.164+0000 7f019ca00640 1 --2- 172.21.15.42:0/3122134385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 0x555d20c80ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.164+0000 7f019ca00640 1 -- 172.21.15.42:0/3122134385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 msgr2=0x555d20c745c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.164+0000 7f019ca00640 1 --2- 172.21.15.42:0/3122134385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 0x555d20c745c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.164+0000 7f019ca00640 1 -- 172.21.15.42:0/3122134385 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d20c72290 con 0x555d20c78ca0 2024-04-20T10:39:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.164+0000 7f019d400640 1 --2- 172.21.15.42:0/3122134385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 0x555d20c80ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:39:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.164+0000 7f019ca00640 1 --2- 172.21.15.42:0/3122134385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78ca0 0x555d20c8d560 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0184006010 tx=0x7f0184045000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7cd2a80de11ba23 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.164+0000 7f0189c00640 1 -- 172.21.15.42:0/3122134385 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018405d070 con 0x555d20c78ca0 2024-04-20T10:39:00.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.164+0000 7f0189c00640 1 -- 172.21.15.42:0/3122134385 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0184048040 con 0x555d20c78ca0 2024-04-20T10:39:00.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.164+0000 7f0189c00640 1 -- 172.21.15.42:0/3122134385 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0184057550 con 0x555d20c78ca0 2024-04-20T10:39:00.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.165+0000 7f01a2bec740 1 -- 172.21.15.42:0/3122134385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78ca0 msgr2=0x555d20c8d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.165+0000 7f01a2bec740 1 --2- 172.21.15.42:0/3122134385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78ca0 0x555d20c8d560 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0184006010 tx=0x7f0184045000 comp rx=0 tx=0).stop 2024-04-20T10:39:00.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.166+0000 7f01a2bec740 1 -- 172.21.15.42:0/3122134385 shutdown_connections 2024-04-20T10:39:00.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.166+0000 7f01a2bec740 1 --2- 172.21.15.42:0/3122134385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c81880 0x555d20c745c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.166+0000 7f01a2bec740 1 --2- 172.21.15.42:0/3122134385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c78ca0 0x555d20c8d560 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.166+0000 7f01a2bec740 1 --2- 172.21.15.42:0/3122134385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78380 0x555d20c80ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.166+0000 7f01a2bec740 1 -- 172.21.15.42:0/3122134385 >> 172.21.15.42:0/3122134385 conn(0x555d20c72ab0 msgr2=0x555d20c73eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.166+0000 7f01a2bec740 1 -- 172.21.15.42:0/3122134385 shutdown_connections 2024-04-20T10:39:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.166+0000 7f01a2bec740 1 -- 172.21.15.42:0/3122134385 wait complete. 2024-04-20T10:39:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.166+0000 7f01a2bec740 1 -- start start 2024-04-20T10:39:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f01a2bec740 1 Processor -- start 2024-04-20T10:39:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f01a2bec740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c78380 0x555d20c84a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f01a2bec740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 0x555d20c84f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f01a2bec740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c81880 0x555d20cf6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f01a2bec740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f01981aea20 con 0x555d20c78380 2024-04-20T10:39:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f01a2bec740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01981afe90 con 0x555d20c78ca0 2024-04-20T10:39:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f01a2bec740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f01981af0f0 con 0x555d20c81880 2024-04-20T10:39:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f01a2bec740 10 client.0 _ll_register_callbacks cb 0x7f018c009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:39:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f019ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 0x555d20c84f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f019ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 0x555d20c84f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47692/0 (socket says 172.21.15.42:47692) 2024-04-20T10:39:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f019ca00640 1 -- 172.21.15.42:0/3803712859 learned_addr learned my addr 172.21.15.42:0/3803712859 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f019d400640 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c78380 0x555d20c84a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.167+0000 7f019de00640 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c81880 0x555d20cf6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.168+0000 7f019ca00640 1 -- 172.21.15.42:0/3803712859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c81880 msgr2=0x555d20cf6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.168+0000 7f019ca00640 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c81880 0x555d20cf6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.168+0000 7f019ca00640 1 -- 172.21.15.42:0/3803712859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c78380 msgr2=0x555d20c84a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.168+0000 7f019ca00640 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c78380 0x555d20c84a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.168+0000 7f019ca00640 1 -- 172.21.15.42:0/3803712859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d20c85800 con 0x555d20c78ca0 2024-04-20T10:39:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.168+0000 7f019de00640 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c81880 0x555d20cf6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:39:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.168+0000 7f019ca00640 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 0x555d20c84f70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f018402c240 tx=0x7f018402c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4e3c46f375f5201 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.168+0000 7f017ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:39:00.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.169+0000 7f017ea00640 1 -- 172.21.15.42:0/3803712859 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0184066060 con 0x555d20c78ca0 2024-04-20T10:39:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.169+0000 7f017ea00640 1 -- 172.21.15.42:0/3803712859 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f01840046c0 con 0x555d20c78ca0 2024-04-20T10:39:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.169+0000 7f01a2bec740 1 -- 172.21.15.42:0/3803712859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0184048040 con 0x555d20c78ca0 2024-04-20T10:39:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.169+0000 7f017ea00640 1 -- 172.21.15.42:0/3803712859 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f018405d030 con 0x555d20c78ca0 2024-04-20T10:39:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.169+0000 7f01a2bec740 10 client.26000 fetch_fsmap learned FSMap version 19 2024-04-20T10:39:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.169+0000 7f01a2bec740 1 -- 172.21.15.42:0/3803712859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555d20cf7620 con 0x555d20c78ca0 2024-04-20T10:39:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.169+0000 7f017ea00640 1 -- 172.21.15.42:0/3803712859 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0184048040 con 0x555d20c78ca0 2024-04-20T10:39:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.169+0000 7f017ea00640 1 -- 172.21.15.42:0/3803712859 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f018404d040 con 0x555d20c78ca0 2024-04-20T10:39:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.170+0000 7f01a2bec740 10 client.26000 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:39:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.170+0000 7f01a2bec740 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:39:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.170+0000 7f01a2bec740 20 client.26000 populate_metadata read hostname 'smithi042' 2024-04-20T10:39:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.170+0000 7f01a2bec740 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x555d20cf7d50 0x555d20cfa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.170+0000 7f01a2bec740 4 client.26000 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:39:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.170+0000 7f01a2bec740 1 -- 172.21.15.42:0/3803712859 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555d20be59e0 con 0x555d20cf7d50 2024-04-20T10:39:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.170+0000 7f019d400640 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x555d20cf7d50 0x555d20cfa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.170+0000 7f019d400640 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x555d20cf7d50 0x555d20cfa170 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c6927274589b30 server_cookie=3cbd1a13d3028762 in_seq=0 out_seq=0 2024-04-20T10:39:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.170+0000 7f017ea00640 10 client.26000 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.171+0000 7f017ea00640 1 -- 172.21.15.42:0/3803712859 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555d20be59e0 con 0x555d20cf7d50 2024-04-20T10:39:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f017ea00640 10 client.26000 handle_command_reply: tid=0 2024-04-20T10:39:00.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f01a2bec740 1 client.26000 shutdown 2024-04-20T10:39:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f01a2bec740 1 -- 172.21.15.42:0/3803712859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 msgr2=0x555d20c84f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f01a2bec740 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 0x555d20c84f70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f018402c240 tx=0x7f018402c880 comp rx=0 tx=0).stop 2024-04-20T10:39:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f01a2bec740 1 -- 172.21.15.42:0/3803712859 shutdown_connections 2024-04-20T10:39:00.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f01a2bec740 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x555d20cf7d50 0x555d20cfa170 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f01a2bec740 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555d20c81880 0x555d20cf6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f01a2bec740 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d20c78ca0 0x555d20c84f70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f01a2bec740 1 --2- 172.21.15.42:0/3803712859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d20c78380 0x555d20c84a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f01a2bec740 1 -- 172.21.15.42:0/3803712859 >> 172.21.15.42:0/3803712859 conn(0x555d20c72ab0 msgr2=0x555d20c71b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f017ea00640 0 client.26000 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f01a2bec740 1 -- 172.21.15.42:0/3803712859 shutdown_connections 2024-04-20T10:39:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f01a2bec740 1 -- 172.21.15.42:0/3803712859 wait complete. 2024-04-20T10:39:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.172+0000 7f01a2bec740 20 client.26000 trim_cache size 0 max 16384 2024-04-20T10:39:00.178 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:39:00.178 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:39:00.178 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:39:00.178 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:39:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.173+0000 7f019e800640 1 -- 172.21.15.42:0/3554788859 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f01800a3da0 msgr2=0x7f01800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.173+0000 7f019e800640 1 --2- 172.21.15.42:0/3554788859 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f01800a3da0 0x7f01800a6260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x555d20c73c40 tx=0x7f018402df90 comp rx=0 tx=0).stop 2024-04-20T10:39:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.173+0000 7f019e800640 1 -- 172.21.15.42:0/3554788859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f019819a6e0 msgr2=0x7f019819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.173+0000 7f019e800640 1 --2- 172.21.15.42:0/3554788859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f019819a6e0 0x7f019819ab60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0190009fd0 tx=0x7f019000ef10 comp rx=0 tx=0).stop 2024-04-20T10:39:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.173+0000 7f019e800640 1 -- 172.21.15.42:0/3554788859 shutdown_connections 2024-04-20T10:39:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.173+0000 7f019e800640 1 --2- 172.21.15.42:0/3554788859 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f01800a3da0 0x7f01800a6260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.173+0000 7f019e800640 1 --2- 172.21.15.42:0/3554788859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f019819a6e0 0x7f019819ab60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.173+0000 7f019e800640 1 --2- 172.21.15.42:0/3554788859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0198107140 0x7f01981a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.173+0000 7f019e800640 1 --2- 172.21.15.42:0/3554788859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0198102070 0x7f01981a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.173+0000 7f019e800640 1 -- 172.21.15.42:0/3554788859 >> 172.21.15.42:0/3554788859 conn(0x7f01980fd930 msgr2=0x7f01980ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.173+0000 7f019e800640 1 -- 172.21.15.42:0/3554788859 shutdown_connections 2024-04-20T10:39:00.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.173+0000 7f019e800640 1 -- 172.21.15.42:0/3554788859 wait complete. 2024-04-20T10:39:00.190 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:39:00.190 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:39:00.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.401+0000 7fce4f200640 1 -- 172.21.15.42:0/3724785743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce48105c90 msgr2=0x7fce48106070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.401+0000 7fce4f200640 1 --2- 172.21.15.42:0/3724785743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce48105c90 0x7fce48106070 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce44009cd0 tx=0x7fce4402c180 comp rx=0 tx=0).stop 2024-04-20T10:39:00.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.401+0000 7fce4f200640 1 -- 172.21.15.42:0/3724785743 shutdown_connections 2024-04-20T10:39:00.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.401+0000 7fce4f200640 1 --2- 172.21.15.42:0/3724785743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48101910 0x7fce4810db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.401+0000 7fce4f200640 1 --2- 172.21.15.42:0/3724785743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce48100f50 0x7fce481013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.401+0000 7fce4f200640 1 --2- 172.21.15.42:0/3724785743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce48105c90 0x7fce48106070 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.401+0000 7fce4f200640 1 -- 172.21.15.42:0/3724785743 >> 172.21.15.42:0/3724785743 conn(0x7fce480767a0 msgr2=0x7fce48076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.401+0000 7fce4f200640 1 -- 172.21.15.42:0/3724785743 shutdown_connections 2024-04-20T10:39:00.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.401+0000 7fce4f200640 1 -- 172.21.15.42:0/3724785743 wait complete. 2024-04-20T10:39:00.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.402+0000 7fce4f200640 1 Processor -- start 2024-04-20T10:39:00.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.402+0000 7fce4f200640 1 -- start start 2024-04-20T10:39:00.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.403+0000 7fce4f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48100f50 0x7fce481a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.403+0000 7fce4f200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce48101910 0x7fce481a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.403+0000 7fce4f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce48105c90 0x7fce4819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.403+0000 7fce4f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fce4810ff70 con 0x7fce48100f50 2024-04-20T10:39:00.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.403+0000 7fce4f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce4810fdf0 con 0x7fce48105c90 2024-04-20T10:39:00.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.403+0000 7fce4f200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fce481100f0 con 0x7fce48101910 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.403+0000 7fce4de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48100f50 0x7fce481a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.403+0000 7fce4e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce48105c90 0x7fce4819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.403+0000 7fce4e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce48105c90 0x7fce4819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47710/0 (socket says 172.21.15.42:47710) 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.403+0000 7fce4de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48100f50 0x7fce481a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45002/0 (socket says 172.21.15.42:45002) 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.403+0000 7fce4e800640 1 -- 172.21.15.42:0/2534385620 learned_addr learned my addr 172.21.15.42:0/2534385620 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.403+0000 7fce4d400640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce48101910 0x7fce481a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.404+0000 7fce4e800640 1 -- 172.21.15.42:0/2534385620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce48101910 msgr2=0x7fce481a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.404+0000 7fce4e800640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce48101910 0x7fce481a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.404+0000 7fce4e800640 1 -- 172.21.15.42:0/2534385620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48100f50 msgr2=0x7fce481a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.404+0000 7fce4e800640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48100f50 0x7fce481a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.404+0000 7fce4e800640 1 -- 172.21.15.42:0/2534385620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4819ada0 con 0x7fce48105c90 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.404+0000 7fce4de00640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48100f50 0x7fce481a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:39:00.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.404+0000 7fce4e800640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce48105c90 0x7fce4819a620 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce4000e970 tx=0x7fce4000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.404+0000 7fce3ea00640 1 -- 172.21.15.42:0/2534385620 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4000ccc0 con 0x7fce48105c90 2024-04-20T10:39:00.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.404+0000 7fce3ea00640 1 -- 172.21.15.42:0/2534385620 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce4000ce60 con 0x7fce48105c90 2024-04-20T10:39:00.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.404+0000 7fce4f200640 1 -- 172.21.15.42:0/2534385620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4819b090 con 0x7fce48105c90 2024-04-20T10:39:00.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.404+0000 7fce3ea00640 1 -- 172.21.15.42:0/2534385620 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40010490 con 0x7fce48105c90 2024-04-20T10:39:00.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.404+0000 7fce4f200640 1 -- 172.21.15.42:0/2534385620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce481a7330 con 0x7fce48105c90 2024-04-20T10:39:00.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.406+0000 7fce4f200640 1 -- 172.21.15.42:0/2534385620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce48068ca0 con 0x7fce48105c90 2024-04-20T10:39:00.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.408+0000 7fce3ea00640 1 -- 172.21.15.42:0/2534385620 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fce40005020 con 0x7fce48105c90 2024-04-20T10:39:00.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.409+0000 7fce3ea00640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fce340a3da0 0x7fce340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.409+0000 7fce3ea00640 1 -- 172.21.15.42:0/2534385620 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fce400d0c50 con 0x7fce48105c90 2024-04-20T10:39:00.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.409+0000 7fce4de00640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fce340a3da0 0x7fce340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.410+0000 7fce4de00640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fce340a3da0 0x7fce340a6260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce4402c6c0 tx=0x7fce44009800 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.413+0000 7fce3ea00640 1 -- 172.21.15.42:0/2534385620 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fce4009d290 con 0x7fce48105c90 2024-04-20T10:39:00.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.700+0000 7fce4f200640 1 -- 172.21.15.42:0/2534385620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce4802d5d0 con 0x7fce48105c90 2024-04-20T10:39:00.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.701+0000 7fce3ea00640 1 -- 172.21.15.42:0/2534385620 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7fce4009cb90 con 0x7fce48105c90 2024-04-20T10:39:00.701 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:39:00.701 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:39:00.702 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:39:00.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.704+0000 7fce4f200640 1 -- 172.21.15.42:0/2534385620 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fce340a3da0 msgr2=0x7fce340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.704+0000 7fce4f200640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fce340a3da0 0x7fce340a6260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce4402c6c0 tx=0x7fce44009800 comp rx=0 tx=0).stop 2024-04-20T10:39:00.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.704+0000 7fce4f200640 1 -- 172.21.15.42:0/2534385620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce48105c90 msgr2=0x7fce4819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.704+0000 7fce4f200640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce48105c90 0x7fce4819a620 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce4000e970 tx=0x7fce4000ed70 comp rx=0 tx=0).stop 2024-04-20T10:39:00.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.705+0000 7fce4f200640 1 -- 172.21.15.42:0/2534385620 shutdown_connections 2024-04-20T10:39:00.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.705+0000 7fce4f200640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fce340a3da0 0x7fce340a6260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.705+0000 7fce4f200640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce48105c90 0x7fce4819a620 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.705+0000 7fce4f200640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce48101910 0x7fce481a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.705+0000 7fce4f200640 1 --2- 172.21.15.42:0/2534385620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48100f50 0x7fce481a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.705+0000 7fce4f200640 1 -- 172.21.15.42:0/2534385620 >> 172.21.15.42:0/2534385620 conn(0x7fce480767a0 msgr2=0x7fce480fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.705+0000 7fce4f200640 1 -- 172.21.15.42:0/2534385620 shutdown_connections 2024-04-20T10:39:00.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.705+0000 7fce4f200640 1 -- 172.21.15.42:0/2534385620 wait complete. 2024-04-20T10:39:00.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:39:00.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.933+0000 7fc742600640 1 -- 172.21.15.42:0/2970818332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73c113010 msgr2=0x7fc73c115400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.933+0000 7fc742600640 1 --2- 172.21.15.42:0/2970818332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73c113010 0x7fc73c115400 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc734009be0 tx=0x7fc73402bf80 comp rx=0 tx=0).stop 2024-04-20T10:39:00.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.933+0000 7fc742600640 1 -- 172.21.15.42:0/2970818332 shutdown_connections 2024-04-20T10:39:00.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.933+0000 7fc742600640 1 --2- 172.21.15.42:0/2970818332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73c113010 0x7fc73c115400 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.933+0000 7fc742600640 1 --2- 172.21.15.42:0/2970818332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc73c074b50 0x7fc73c112ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.933+0000 7fc742600640 1 --2- 172.21.15.42:0/2970818332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73c074230 0x7fc73c074610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.933+0000 7fc742600640 1 -- 172.21.15.42:0/2970818332 >> 172.21.15.42:0/2970818332 conn(0x7fc73c0793b0 msgr2=0x7fc73c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.933+0000 7fc742600640 1 -- 172.21.15.42:0/2970818332 shutdown_connections 2024-04-20T10:39:00.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.933+0000 7fc742600640 1 -- 172.21.15.42:0/2970818332 wait complete. 2024-04-20T10:39:00.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.934+0000 7fc742600640 1 Processor -- start 2024-04-20T10:39:00.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.934+0000 7fc742600640 1 -- start start 2024-04-20T10:39:00.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.935+0000 7fc742600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73c074230 0x7fc73c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.935+0000 7fc742600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc73c074b50 0x7fc73c19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.935+0000 7fc742600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73c113010 0x7fc73c1a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.935+0000 7fc742600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1188f0 con 0x7fc73c074b50 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.935+0000 7fc742600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c118770 con 0x7fc73c074230 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.935+0000 7fc742600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c118a70 con 0x7fc73c113010 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.935+0000 7fc73b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73c074230 0x7fc73c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.935+0000 7fc73aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc73c074b50 0x7fc73c19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.935+0000 7fc73b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73c074230 0x7fc73c19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47726/0 (socket says 172.21.15.42:47726) 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.935+0000 7fc73be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73c113010 0x7fc73c1a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.935+0000 7fc73b400640 1 -- 172.21.15.42:0/1854777127 learned_addr learned my addr 172.21.15.42:0/1854777127 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.936+0000 7fc73be00640 1 -- 172.21.15.42:0/1854777127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73c074230 msgr2=0x7fc73c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.936+0000 7fc73be00640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73c074230 0x7fc73c19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.936+0000 7fc73be00640 1 -- 172.21.15.42:0/1854777127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc73c074b50 msgr2=0x7fc73c19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.936+0000 7fc73be00640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc73c074b50 0x7fc73c19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.936+0000 7fc73be00640 1 -- 172.21.15.42:0/1854777127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73c1a3e80 con 0x7fc73c113010 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.936+0000 7fc73b400640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73c074230 0x7fc73c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:39:00.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.936+0000 7fc73be00640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73c113010 0x7fc73c1a3750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc73402c4c0 tx=0x7fc734033750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.936+0000 7fc733e00640 1 -- 172.21.15.42:0/1854777127 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734043020 con 0x7fc73c113010 2024-04-20T10:39:00.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.936+0000 7fc742600640 1 -- 172.21.15.42:0/1854777127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73c1a4110 con 0x7fc73c113010 2024-04-20T10:39:00.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.936+0000 7fc733e00640 1 -- 172.21.15.42:0/1854777127 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc734033940 con 0x7fc73c113010 2024-04-20T10:39:00.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.936+0000 7fc733e00640 1 -- 172.21.15.42:0/1854777127 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73403e6b0 con 0x7fc73c113010 2024-04-20T10:39:00.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.937+0000 7fc742600640 1 -- 172.21.15.42:0/1854777127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73c1aba00 con 0x7fc73c113010 2024-04-20T10:39:00.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.940+0000 7fc733e00640 1 -- 172.21.15.42:0/1854777127 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc734046020 con 0x7fc73c113010 2024-04-20T10:39:00.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.940+0000 7fc745251740 1 Processor -- start 2024-04-20T10:39:00.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.940+0000 7fc745251740 1 -- start start 2024-04-20T10:39:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.940+0000 7fc745251740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b880fb0 0x56242b881390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.940+0000 7fc745251740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b889f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc745251740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b88a4b0 0x56242b88c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc745251740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56242b7e6050 con 0x56242b88a4b0 2024-04-20T10:39:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc745251740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56242b78c490 con 0x56242b8818d0 2024-04-20T10:39:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc745251740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56242b7e2010 con 0x56242b880fb0 2024-04-20T10:39:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc73b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b889f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc73b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b889f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47738/0 (socket says 172.21.15.42:47738) 2024-04-20T10:39:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc73b400640 1 -- 172.21.15.42:0/443804263 learned_addr learned my addr 172.21.15.42:0/443804263 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc73aa00640 1 --2- 172.21.15.42:0/443804263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b880fb0 0x56242b881390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc73aa00640 1 --2- 172.21.15.42:0/443804263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b88a4b0 0x56242b88c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc73b400640 1 -- 172.21.15.42:0/443804263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b880fb0 msgr2=0x56242b881390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc73b400640 1 --2- 172.21.15.42:0/443804263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b880fb0 0x56242b881390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc73b400640 1 -- 172.21.15.42:0/443804263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b88a4b0 msgr2=0x56242b88c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc73b400640 1 --2- 172.21.15.42:0/443804263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b88a4b0 0x56242b88c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc73b400640 1 -- 172.21.15.42:0/443804263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242b794070 con 0x56242b8818d0 2024-04-20T10:39:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.941+0000 7fc73aa00640 1 --2- 172.21.15.42:0/443804263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b880fb0 0x56242b881390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:39:00.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.942+0000 7fc73b400640 1 --2- 172.21.15.42:0/443804263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b889f70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc72c011650 tx=0x7fc72c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fd0599ad016a747 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.942+0000 7fc733e00640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7200a3d20 0x7fc7200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.942+0000 7fc733e00640 1 -- 172.21.15.42:0/1854777127 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc7340f2d00 con 0x7fc73c113010 2024-04-20T10:39:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.942+0000 7fc730c00640 1 -- 172.21.15.42:0/443804263 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c01e070 con 0x56242b8818d0 2024-04-20T10:39:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.942+0000 7fc730c00640 1 -- 172.21.15.42:0/443804263 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c00fba0 con 0x56242b8818d0 2024-04-20T10:39:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.942+0000 7fc730c00640 1 -- 172.21.15.42:0/443804263 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c019cd0 con 0x56242b8818d0 2024-04-20T10:39:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.942+0000 7fc73aa00640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7200a3d20 0x7fc7200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.943+0000 7fc745251740 1 -- 172.21.15.42:0/443804263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 msgr2=0x56242b889f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.943+0000 7fc73aa00640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7200a3d20 0x7fc7200a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x56242b87c870 tx=0x7fc728009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.943+0000 7fc745251740 1 --2- 172.21.15.42:0/443804263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b889f70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc72c011650 tx=0x7fc72c009f90 comp rx=0 tx=0).stop 2024-04-20T10:39:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.943+0000 7fc745251740 1 -- 172.21.15.42:0/443804263 shutdown_connections 2024-04-20T10:39:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.943+0000 7fc745251740 1 --2- 172.21.15.42:0/443804263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b88a4b0 0x56242b88c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.943+0000 7fc745251740 1 --2- 172.21.15.42:0/443804263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b889f70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.943+0000 7fc745251740 1 --2- 172.21.15.42:0/443804263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b880fb0 0x56242b881390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.943+0000 7fc745251740 1 -- 172.21.15.42:0/443804263 >> 172.21.15.42:0/443804263 conn(0x56242b87b6e0 msgr2=0x56242b87bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.943+0000 7fc745251740 1 -- 172.21.15.42:0/443804263 shutdown_connections 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.943+0000 7fc745251740 1 -- 172.21.15.42:0/443804263 wait complete. 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.943+0000 7fc745251740 1 -- start start 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.944+0000 7fc745251740 1 Processor -- start 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.944+0000 7fc745251740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b880fb0 0x56242b887560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.944+0000 7fc745251740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b887aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.944+0000 7fc745251740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b887fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.944+0000 7fc745251740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56242b88d2a0 con 0x56242b880fb0 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.944+0000 7fc745251740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56242b79f860 con 0x56242b8818d0 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.944+0000 7fc745251740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56242b88d420 con 0x56242b88a4b0 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.944+0000 7fc745251740 10 client.0 _ll_register_callbacks cb 0x56242b76dfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.944+0000 7fc73aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b887aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.944+0000 7fc73b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b880fb0 0x56242b887560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.945+0000 7fc73aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b887aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47746/0 (socket says 172.21.15.42:47746) 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.945+0000 7fc73aa00640 1 -- 172.21.15.42:0/1044573854 learned_addr learned my addr 172.21.15.42:0/1044573854 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.945+0000 7fc73be00640 1 --2- 172.21.15.42:0/1044573854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b887fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.945+0000 7fc73aa00640 1 -- 172.21.15.42:0/1044573854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 msgr2=0x56242b887fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.945+0000 7fc73aa00640 1 --2- 172.21.15.42:0/1044573854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b887fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.945+0000 7fc73aa00640 1 -- 172.21.15.42:0/1044573854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b880fb0 msgr2=0x56242b887560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.945+0000 7fc73aa00640 1 --2- 172.21.15.42:0/1044573854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b880fb0 0x56242b887560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.945+0000 7fc73aa00640 1 -- 172.21.15.42:0/1044573854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242b7a3950 con 0x56242b8818d0 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.945+0000 7fc73aa00640 1 --2- 172.21.15.42:0/1044573854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b887aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc728004010 tx=0x7fc728004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51fa768bdd114328 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.945+0000 7fc725600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.946+0000 7fc725600640 1 -- 172.21.15.42:0/1044573854 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728013cc0 con 0x56242b8818d0 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.946+0000 7fc725600640 1 -- 172.21.15.42:0/1044573854 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc728009040 con 0x56242b8818d0 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.946+0000 7fc745251740 1 -- 172.21.15.42:0/1044573854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc72c00fba0 con 0x56242b8818d0 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.946+0000 7fc725600640 1 -- 172.21.15.42:0/1044573854 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc728013e60 con 0x56242b8818d0 2024-04-20T10:39:00.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.946+0000 7fc745251740 10 client.26018 fetch_fsmap learned FSMap version 19 2024-04-20T10:39:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.946+0000 7fc745251740 1 -- 172.21.15.42:0/1044573854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56242b771820 con 0x56242b8818d0 2024-04-20T10:39:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.946+0000 7fc725600640 1 -- 172.21.15.42:0/1044573854 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c00fba0 con 0x56242b8818d0 2024-04-20T10:39:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.946+0000 7fc725600640 1 -- 172.21.15.42:0/1044573854 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc72802b590 con 0x56242b8818d0 2024-04-20T10:39:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.947+0000 7fc745251740 10 client.26018 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:39:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.947+0000 7fc745251740 10 client.26018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:39:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.947+0000 7fc745251740 20 client.26018 populate_metadata read hostname 'smithi042' 2024-04-20T10:39:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.947+0000 7fc745251740 1 --2- 172.21.15.42:0/1044573854 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56242b89a4e0 0x56242b89c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.947+0000 7fc745251740 4 client.26018 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:39:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.947+0000 7fc745251740 1 -- 172.21.15.42:0/1044573854 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56242b7f3a70 con 0x56242b89a4e0 2024-04-20T10:39:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.947+0000 7fc73b400640 1 --2- 172.21.15.42:0/1044573854 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56242b89a4e0 0x56242b89c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.947+0000 7fc73b400640 1 --2- 172.21.15.42:0/1044573854 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56242b89a4e0 0x56242b89c910 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9cc1f54e90cfd77 server_cookie=fcb933b5d302e9fa in_seq=0 out_seq=0 2024-04-20T10:39:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.947+0000 7fc725600640 10 client.26018 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:00.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc725600640 1 -- 172.21.15.42:0/1044573854 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56242b7f3a70 con 0x56242b89a4e0 2024-04-20T10:39:00.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc725600640 10 client.26018 handle_command_reply: tid=0 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc745251740 1 client.26018 shutdown 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc745251740 1 -- 172.21.15.42:0/1044573854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 msgr2=0x56242b887aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc745251740 1 --2- 172.21.15.42:0/1044573854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b887aa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc728004010 tx=0x7fc728004730 comp rx=0 tx=0).stop 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc745251740 1 -- 172.21.15.42:0/1044573854 shutdown_connections 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc745251740 1 --2- 172.21.15.42:0/1044573854 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56242b89a4e0 0x56242b89c910 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc745251740 1 --2- 172.21.15.42:0/1044573854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b887fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc745251740 1 --2- 172.21.15.42:0/1044573854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b887aa0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc745251740 1 --2- 172.21.15.42:0/1044573854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b880fb0 0x56242b887560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc725600640 0 client.26018 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc745251740 1 -- 172.21.15.42:0/1044573854 >> 172.21.15.42:0/1044573854 conn(0x56242b87b6e0 msgr2=0x56242b87d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc745251740 1 -- 172.21.15.42:0/1044573854 shutdown_connections 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.949+0000 7fc745251740 1 -- 172.21.15.42:0/1044573854 wait complete. 2024-04-20T10:39:00.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.950+0000 7fc745251740 20 client.26018 trim_cache size 0 max 16384 2024-04-20T10:39:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.965+0000 7fc745251740 1 Processor -- start 2024-04-20T10:39:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.965+0000 7fc745251740 1 -- start start 2024-04-20T10:39:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.965+0000 7fc745251740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b880fb0 0x56242b889c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.965+0000 7fc745251740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b8818d0 0x56242b896160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.965+0000 7fc745251740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b8a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.965+0000 7fc745251740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1b22f0 con 0x56242b8818d0 2024-04-20T10:39:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.965+0000 7fc745251740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1b4b70 con 0x56242b880fb0 2024-04-20T10:39:00.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.965+0000 7fc745251740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1b2960 con 0x56242b88a4b0 2024-04-20T10:39:00.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.966+0000 7fc73b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b880fb0 0x56242b889c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.966+0000 7fc73b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b880fb0 0x56242b889c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47760/0 (socket says 172.21.15.42:47760) 2024-04-20T10:39:00.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.966+0000 7fc73aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b8818d0 0x56242b896160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.966+0000 7fc73b400640 1 -- 172.21.15.42:0/2044671537 learned_addr learned my addr 172.21.15.42:0/2044671537 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:00.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.966+0000 7fc73be00640 1 --2- 172.21.15.42:0/2044671537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b8a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.966+0000 7fc73b400640 1 -- 172.21.15.42:0/2044671537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 msgr2=0x56242b8a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.966+0000 7fc73b400640 1 --2- 172.21.15.42:0/2044671537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b8a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.966+0000 7fc73b400640 1 -- 172.21.15.42:0/2044671537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b8818d0 msgr2=0x56242b896160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.966+0000 7fc73b400640 1 --2- 172.21.15.42:0/2044671537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b8818d0 0x56242b896160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.966+0000 7fc73b400640 1 -- 172.21.15.42:0/2044671537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242b881700 con 0x56242b880fb0 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.966+0000 7fc73be00640 1 --2- 172.21.15.42:0/2044671537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b8a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.967+0000 7fc73b400640 1 --2- 172.21.15.42:0/2044671537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b880fb0 0x56242b889c90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc72c019880 tx=0x7fc72c019770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cd8257231a52c4f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.967+0000 7fc727e00640 1 -- 172.21.15.42:0/2044671537 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c02f070 con 0x56242b880fb0 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.967+0000 7fc727e00640 1 -- 172.21.15.42:0/2044671537 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c019410 con 0x56242b880fb0 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.967+0000 7fc727e00640 1 -- 172.21.15.42:0/2044671537 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c02a070 con 0x56242b880fb0 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.968+0000 7fc745251740 1 -- 172.21.15.42:0/2044671537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b880fb0 msgr2=0x56242b889c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.968+0000 7fc745251740 1 --2- 172.21.15.42:0/2044671537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b880fb0 0x56242b889c90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc72c019880 tx=0x7fc72c019770 comp rx=0 tx=0).stop 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.968+0000 7fc745251740 1 -- 172.21.15.42:0/2044671537 shutdown_connections 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.968+0000 7fc745251740 1 --2- 172.21.15.42:0/2044671537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b8a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.968+0000 7fc745251740 1 --2- 172.21.15.42:0/2044671537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b8818d0 0x56242b896160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.968+0000 7fc745251740 1 --2- 172.21.15.42:0/2044671537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b880fb0 0x56242b889c90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.968+0000 7fc745251740 1 -- 172.21.15.42:0/2044671537 >> 172.21.15.42:0/2044671537 conn(0x56242b87b6e0 msgr2=0x56242b87a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.968+0000 7fc745251740 1 -- 172.21.15.42:0/2044671537 shutdown_connections 2024-04-20T10:39:00.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.968+0000 7fc745251740 1 -- 172.21.15.42:0/2044671537 wait complete. 2024-04-20T10:39:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.968+0000 7fc745251740 1 -- start start 2024-04-20T10:39:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.968+0000 7fc745251740 1 Processor -- start 2024-04-20T10:39:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.969+0000 7fc745251740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b880fb0 0x56242b884670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.969+0000 7fc745251740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b884bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.969+0000 7fc745251740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b8ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.969+0000 7fc745251740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1b3030 con 0x56242b880fb0 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.969+0000 7fc745251740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1b44a0 con 0x56242b8818d0 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.969+0000 7fc745251740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1b3700 con 0x56242b88a4b0 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.969+0000 7fc745251740 10 client.0 _ll_register_callbacks cb 0x7fc728013970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.969+0000 7fc73b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b880fb0 0x56242b884670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.969+0000 7fc73be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b8ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.969+0000 7fc73aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b884bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.969+0000 7fc73be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b8ffa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:51286/0 (socket says 172.21.15.42:51286) 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.969+0000 7fc73aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b884bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47762/0 (socket says 172.21.15.42:47762) 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.970+0000 7fc73be00640 1 -- 172.21.15.42:0/2022597993 learned_addr learned my addr 172.21.15.42:0/2022597993 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.970+0000 7fc73aa00640 1 -- 172.21.15.42:0/2022597993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 msgr2=0x56242b8ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.970+0000 7fc73aa00640 1 --2- 172.21.15.42:0/2022597993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b8ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.970+0000 7fc73aa00640 1 -- 172.21.15.42:0/2022597993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b880fb0 msgr2=0x56242b884670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.970+0000 7fc73aa00640 1 --2- 172.21.15.42:0/2022597993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b880fb0 0x56242b884670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.970+0000 7fc73aa00640 1 -- 172.21.15.42:0/2022597993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56242b4f83f0 con 0x56242b8818d0 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.970+0000 7fc73aa00640 1 --2- 172.21.15.42:0/2022597993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b884bb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc72802b9e0 tx=0x7fc72802b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4cb208e94630aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.970+0000 7fc725600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.971+0000 7fc725600640 1 -- 172.21.15.42:0/2022597993 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728043060 con 0x56242b8818d0 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.971+0000 7fc745251740 1 -- 172.21.15.42:0/2022597993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc72c019410 con 0x56242b8818d0 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.971+0000 7fc725600640 1 -- 172.21.15.42:0/2022597993 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc72803e450 con 0x56242b8818d0 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.971+0000 7fc725600640 1 -- 172.21.15.42:0/2022597993 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc72803a070 con 0x56242b8818d0 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.971+0000 7fc745251740 10 client.26024 fetch_fsmap learned FSMap version 19 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.971+0000 7fc745251740 1 -- 172.21.15.42:0/2022597993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56242b5c27d0 con 0x56242b8818d0 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.971+0000 7fc725600640 1 -- 172.21.15.42:0/2022597993 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c019410 con 0x56242b8818d0 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.971+0000 7fc725600640 1 -- 172.21.15.42:0/2022597993 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc72803eb30 con 0x56242b8818d0 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.972+0000 7fc745251740 10 client.26024 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:39:00.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.972+0000 7fc745251740 10 client.26024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:39:00.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.972+0000 7fc745251740 20 client.26024 populate_metadata read hostname 'smithi042' 2024-04-20T10:39:00.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.972+0000 7fc745251740 1 --2- 172.21.15.42:0/2022597993 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56242b9014b0 0x56242b9038d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:00.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.972+0000 7fc745251740 4 client.26024 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:39:00.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.972+0000 7fc745251740 1 -- 172.21.15.42:0/2022597993 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56242b7ed8e0 con 0x56242b9014b0 2024-04-20T10:39:00.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.973+0000 7fc73b400640 1 --2- 172.21.15.42:0/2022597993 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56242b9014b0 0x56242b9038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:00.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.976+0000 7fc73b400640 1 --2- 172.21.15.42:0/2022597993 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56242b9014b0 0x56242b9038d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c92789396342ec server_cookie=f58f5a6356b4d948 in_seq=0 out_seq=0 2024-04-20T10:39:00.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.976+0000 7fc725600640 10 client.26024 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.977+0000 7fc725600640 1 -- 172.21.15.42:0/2022597993 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56242b7ed8e0 con 0x56242b9014b0 2024-04-20T10:39:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.977+0000 7fc725600640 10 client.26024 handle_command_reply: tid=0 2024-04-20T10:39:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.977+0000 7fc745251740 1 client.26024 shutdown 2024-04-20T10:39:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.977+0000 7fc745251740 1 -- 172.21.15.42:0/2022597993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 msgr2=0x56242b884bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.977+0000 7fc745251740 1 --2- 172.21.15.42:0/2022597993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b884bb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc72802b9e0 tx=0x7fc72802b720 comp rx=0 tx=0).stop 2024-04-20T10:39:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.977+0000 7fc745251740 1 -- 172.21.15.42:0/2022597993 shutdown_connections 2024-04-20T10:39:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.977+0000 7fc745251740 1 --2- 172.21.15.42:0/2022597993 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56242b9014b0 0x56242b9038d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.977+0000 7fc745251740 1 --2- 172.21.15.42:0/2022597993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56242b88a4b0 0x56242b8ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.977+0000 7fc745251740 1 --2- 172.21.15.42:0/2022597993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56242b8818d0 0x56242b884bb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.977+0000 7fc725600640 0 client.26024 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.977+0000 7fc745251740 1 --2- 172.21.15.42:0/2022597993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56242b880fb0 0x56242b884670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.978+0000 7fc745251740 1 -- 172.21.15.42:0/2022597993 >> 172.21.15.42:0/2022597993 conn(0x56242b87b6e0 msgr2=0x56242b887500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.978+0000 7fc745251740 1 -- 172.21.15.42:0/2022597993 shutdown_connections 2024-04-20T10:39:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.978+0000 7fc745251740 1 -- 172.21.15.42:0/2022597993 wait complete. 2024-04-20T10:39:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.978+0000 7fc745251740 20 client.26024 trim_cache size 0 max 16384 2024-04-20T10:39:00.978 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:39:00.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.979+0000 7fc742600640 1 -- 172.21.15.42:0/1854777127 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7200a3d20 msgr2=0x7fc7200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.979+0000 7fc742600640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7200a3d20 0x7fc7200a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x56242b87c870 tx=0x7fc728009260 comp rx=0 tx=0).stop 2024-04-20T10:39:00.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.979+0000 7fc742600640 1 -- 172.21.15.42:0/1854777127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73c113010 msgr2=0x7fc73c1a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:00.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.979+0000 7fc742600640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73c113010 0x7fc73c1a3750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc73402c4c0 tx=0x7fc734033750 comp rx=0 tx=0).stop 2024-04-20T10:39:00.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.979+0000 7fc742600640 1 -- 172.21.15.42:0/1854777127 shutdown_connections 2024-04-20T10:39:00.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.979+0000 7fc742600640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc7200a3d20 0x7fc7200a61e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.979+0000 7fc742600640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc73c113010 0x7fc73c1a3750 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.979+0000 7fc742600640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc73c074b50 0x7fc73c19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.979+0000 7fc742600640 1 --2- 172.21.15.42:0/1854777127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73c074230 0x7fc73c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:00.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.979+0000 7fc742600640 1 -- 172.21.15.42:0/1854777127 >> 172.21.15.42:0/1854777127 conn(0x7fc73c0793b0 msgr2=0x7fc73c100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:00.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.979+0000 7fc742600640 1 -- 172.21.15.42:0/1854777127 shutdown_connections 2024-04-20T10:39:00.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:00.979+0000 7fc742600640 1 -- 172.21.15.42:0/1854777127 wait complete. 2024-04-20T10:39:01.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:01 smithi162 ceph-mon[21854]: pgmap v153: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 706 KiB/s wr, 24 op/s 2024-04-20T10:39:01.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:01 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2534385620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:39:01.999 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:39:02.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:39:02.000 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:39:02.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:01 smithi046 ceph-mon[21574]: pgmap v153: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 706 KiB/s wr, 24 op/s 2024-04-20T10:39:02.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:01 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2534385620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:39:02.077 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:01 smithi042 ceph-mon[17078]: pgmap v153: 97 pgs: 97 active+clean; 1.0 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 706 KiB/s wr, 24 op/s 2024-04-20T10:39:02.078 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:01 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2534385620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:39:02.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.223+0000 7f36eca00640 1 -- 172.21.15.42:0/377009791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e8104c30 msgr2=0x7f36e8108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.223+0000 7f36eca00640 1 --2- 172.21.15.42:0/377009791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e8104c30 0x7f36e8108c70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009900 tx=0x7f36d002bc40 comp rx=0 tx=0).stop 2024-04-20T10:39:02.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.223+0000 7f36eca00640 1 -- 172.21.15.42:0/377009791 shutdown_connections 2024-04-20T10:39:02.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.223+0000 7f36eca00640 1 --2- 172.21.15.42:0/377009791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e8109350 0x7f36e8110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.223+0000 7f36eca00640 1 --2- 172.21.15.42:0/377009791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e8104c30 0x7f36e8108c70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.223+0000 7f36eca00640 1 --2- 172.21.15.42:0/377009791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36e8104280 0x7f36e8104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.223+0000 7f36eca00640 1 -- 172.21.15.42:0/377009791 >> 172.21.15.42:0/377009791 conn(0x7f36e80ffae0 msgr2=0x7f36e8101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:02.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.223+0000 7f36eca00640 1 -- 172.21.15.42:0/377009791 shutdown_connections 2024-04-20T10:39:02.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.223+0000 7f36eca00640 1 -- 172.21.15.42:0/377009791 wait complete. 2024-04-20T10:39:02.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.224+0000 7f36eca00640 1 Processor -- start 2024-04-20T10:39:02.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.225+0000 7f36eca00640 1 -- start start 2024-04-20T10:39:02.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.225+0000 7f36eca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e8104280 0x7f36e81a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.225+0000 7f36eca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e8109350 0x7f36e81a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.225+0000 7f36eca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36e819a7e0 0x7f36e819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.225+0000 7f36eca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81143e0 con 0x7f36e8109350 2024-04-20T10:39:02.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.225+0000 7f36eca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36e8114260 con 0x7f36e8104280 2024-04-20T10:39:02.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.225+0000 7f36eca00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f36e8114560 con 0x7f36e819a7e0 2024-04-20T10:39:02.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.225+0000 7f36e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e8104280 0x7f36e81a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.225+0000 7f36dde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e8109350 0x7f36e81a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.225+0000 7f36e7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36e819a7e0 0x7f36e819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.225+0000 7f36e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e8104280 0x7f36e81a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47772/0 (socket says 172.21.15.42:47772) 2024-04-20T10:39:02.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.225+0000 7f36e6800640 1 -- 172.21.15.42:0/140886111 learned_addr learned my addr 172.21.15.42:0/140886111 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:02.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.226+0000 7f36e6800640 1 -- 172.21.15.42:0/140886111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36e819a7e0 msgr2=0x7f36e819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.226+0000 7f36e6800640 1 --2- 172.21.15.42:0/140886111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36e819a7e0 0x7f36e819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.226+0000 7f36e6800640 1 -- 172.21.15.42:0/140886111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e8109350 msgr2=0x7f36e81a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.226+0000 7f36e6800640 1 --2- 172.21.15.42:0/140886111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e8109350 0x7f36e81a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.226+0000 7f36e6800640 1 -- 172.21.15.42:0/140886111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e819b370 con 0x7f36e8104280 2024-04-20T10:39:02.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.226+0000 7f36e6800640 1 --2- 172.21.15.42:0/140886111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e8104280 0x7f36e81a0670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f36d800e9f0 tx=0x7f36d800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:02.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.226+0000 7f36dfe00640 1 -- 172.21.15.42:0/140886111 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d800cd80 con 0x7f36e8104280 2024-04-20T10:39:02.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.226+0000 7f36dfe00640 1 -- 172.21.15.42:0/140886111 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d8005500 con 0x7f36e8104280 2024-04-20T10:39:02.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.226+0000 7f36dfe00640 1 -- 172.21.15.42:0/140886111 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8010690 con 0x7f36e8104280 2024-04-20T10:39:02.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.226+0000 7f36eca00640 1 -- 172.21.15.42:0/140886111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e819b660 con 0x7f36e8104280 2024-04-20T10:39:02.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.227+0000 7f36eca00640 1 -- 172.21.15.42:0/140886111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e81a77c0 con 0x7f36e8104280 2024-04-20T10:39:02.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.230+0000 7f36dfe00640 1 -- 172.21.15.42:0/140886111 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f36d80046c0 con 0x7f36e8104280 2024-04-20T10:39:02.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.230+0000 7f36f01e7740 1 Processor -- start 2024-04-20T10:39:02.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.230+0000 7f36f01e7740 1 -- start start 2024-04-20T10:39:02.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.230+0000 7f36f01e7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 0x56434ed204d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.230+0000 7f36f01e7740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed290b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36f01e7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed2b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36f01e7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56434ec84d20 con 0x56434ed200f0 2024-04-20T10:39:02.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36f01e7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56434ec2b3e0 con 0x56434ed295f0 2024-04-20T10:39:02.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36f01e7740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56434ec80ce0 con 0x56434ed20a10 2024-04-20T10:39:02.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed2b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36e7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 0x56434ed204d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed2b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47788/0 (socket says 172.21.15.42:47788) 2024-04-20T10:39:02.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36dde00640 1 -- 172.21.15.42:0/4075050085 learned_addr learned my addr 172.21.15.42:0/4075050085 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:02.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36e6800640 1 --2- 172.21.15.42:0/4075050085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed290b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36dde00640 1 -- 172.21.15.42:0/4075050085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 msgr2=0x56434ed290b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36dde00640 1 --2- 172.21.15.42:0/4075050085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed290b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36dde00640 1 -- 172.21.15.42:0/4075050085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 msgr2=0x56434ed204d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36dde00640 1 --2- 172.21.15.42:0/4075050085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 0x56434ed204d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36dde00640 1 -- 172.21.15.42:0/4075050085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434ec4a980 con 0x56434ed295f0 2024-04-20T10:39:02.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.231+0000 7f36e7200640 1 --2- 172.21.15.42:0/4075050085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 0x56434ed204d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:39:02.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.232+0000 7f36dfe00640 1 --2- 172.21.15.42:0/140886111 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f36cc0a3d20 0x7f36cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.232+0000 7f36dde00640 1 --2- 172.21.15.42:0/4075050085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed2b9e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009900 tx=0x7f36d0005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87e52a988d0cb085 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:02.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.232+0000 7f36dfe00640 1 -- 172.21.15.42:0/140886111 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f36d80d0c70 con 0x7f36e8104280 2024-04-20T10:39:02.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.232+0000 7f36e7200640 1 --2- 172.21.15.42:0/140886111 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f36cc0a3d20 0x7f36cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.232+0000 7f36d7400640 1 -- 172.21.15.42:0/4075050085 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0042070 con 0x56434ed295f0 2024-04-20T10:39:02.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.232+0000 7f36d7400640 1 -- 172.21.15.42:0/4075050085 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d0004030 con 0x56434ed295f0 2024-04-20T10:39:02.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.232+0000 7f36d7400640 1 -- 172.21.15.42:0/4075050085 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0004f00 con 0x56434ed295f0 2024-04-20T10:39:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.233+0000 7f36e7200640 1 --2- 172.21.15.42:0/140886111 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f36cc0a3d20 0x7f36cc0a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009fd0 tx=0x7f36e0009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.233+0000 7f36f01e7740 1 -- 172.21.15.42:0/4075050085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 msgr2=0x56434ed2b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.233+0000 7f36f01e7740 1 --2- 172.21.15.42:0/4075050085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed2b9e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009900 tx=0x7f36d0005be0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.233+0000 7f36f01e7740 1 -- 172.21.15.42:0/4075050085 shutdown_connections 2024-04-20T10:39:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.233+0000 7f36f01e7740 1 --2- 172.21.15.42:0/4075050085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed2b9e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.233+0000 7f36f01e7740 1 --2- 172.21.15.42:0/4075050085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed290b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.233+0000 7f36f01e7740 1 --2- 172.21.15.42:0/4075050085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 0x56434ed204d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.233+0000 7f36f01e7740 1 -- 172.21.15.42:0/4075050085 >> 172.21.15.42:0/4075050085 conn(0x56434ed1a820 msgr2=0x56434ed1ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.233+0000 7f36f01e7740 1 -- 172.21.15.42:0/4075050085 shutdown_connections 2024-04-20T10:39:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.233+0000 7f36f01e7740 1 -- 172.21.15.42:0/4075050085 wait complete. 2024-04-20T10:39:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.234+0000 7f36f01e7740 1 -- start start 2024-04-20T10:39:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.234+0000 7f36f01e7740 1 Processor -- start 2024-04-20T10:39:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36f01e7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed200f0 0x56434ed26640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36f01e7740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed26b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36f01e7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed295f0 0x56434ed270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36f01e7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56434ed2c380 con 0x56434ed295f0 2024-04-20T10:39:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36f01e7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56434ec3e7a0 con 0x56434ed200f0 2024-04-20T10:39:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36f01e7740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56434ed2c500 con 0x56434ed20a10 2024-04-20T10:39:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36f01e7740 10 client.0 _ll_register_callbacks cb 0x56434ec40b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:39:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36e7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed295f0 0x56434ed270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36dde00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed26b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed200f0 0x56434ed26640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36dde00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed26b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:51306/0 (socket says 172.21.15.42:51306) 2024-04-20T10:39:02.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed200f0 0x56434ed26640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47794/0 (socket says 172.21.15.42:47794) 2024-04-20T10:39:02.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.235+0000 7f36dde00640 1 -- 172.21.15.42:0/3038477313 learned_addr learned my addr 172.21.15.42:0/3038477313 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:02.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36e6800640 1 -- 172.21.15.42:0/3038477313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 msgr2=0x56434ed26b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36e6800640 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed26b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36e6800640 1 -- 172.21.15.42:0/3038477313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed295f0 msgr2=0x56434ed270c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36e6800640 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed295f0 0x56434ed270c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36e6800640 1 -- 172.21.15.42:0/3038477313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434ea5f7e0 con 0x56434ed200f0 2024-04-20T10:39:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36dde00640 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed26b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:39:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36e7200640 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed295f0 0x56434ed270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:39:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36e6800640 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed200f0 0x56434ed26640 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f36d80ce360 tx=0x7f36d8004ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3cb2178cadc1220 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:39:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36cbe00640 1 -- 172.21.15.42:0/3038477313 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d809c510 con 0x56434ed200f0 2024-04-20T10:39:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36cbe00640 1 -- 172.21.15.42:0/3038477313 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f36d80ce5e0 con 0x56434ed200f0 2024-04-20T10:39:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36f01e7740 1 -- 172.21.15.42:0/3038477313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d0004030 con 0x56434ed200f0 2024-04-20T10:39:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36cbe00640 1 -- 172.21.15.42:0/3038477313 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f36d80ce7e0 con 0x56434ed200f0 2024-04-20T10:39:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36f01e7740 10 client.26036 fetch_fsmap learned FSMap version 19 2024-04-20T10:39:02.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.236+0000 7f36f01e7740 1 -- 172.21.15.42:0/3038477313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56434eabff70 con 0x56434ed200f0 2024-04-20T10:39:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.237+0000 7f36cbe00640 1 -- 172.21.15.42:0/3038477313 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d0004030 con 0x56434ed200f0 2024-04-20T10:39:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.237+0000 7f36cbe00640 1 -- 172.21.15.42:0/3038477313 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f36d80a5390 con 0x56434ed200f0 2024-04-20T10:39:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.237+0000 7f36f01e7740 10 client.26036 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:39:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.237+0000 7f36f01e7740 10 client.26036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:39:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.237+0000 7f36f01e7740 20 client.26036 populate_metadata read hostname 'smithi042' 2024-04-20T10:39:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.237+0000 7f36f01e7740 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56434ed39a10 0x56434ed3be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.237+0000 7f36f01e7740 4 client.26036 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:39:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.237+0000 7f36f01e7740 1 -- 172.21.15.42:0/3038477313 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56434ec92740 con 0x56434ed39a10 2024-04-20T10:39:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.237+0000 7f36dde00640 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56434ed39a10 0x56434ed3be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.238+0000 7f36cbe00640 10 client.26036 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.238+0000 7f36dde00640 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56434ed39a10 0x56434ed3be00 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ace6031dae4d57 server_cookie=f1dd115609c9cf16 in_seq=0 out_seq=0 2024-04-20T10:39:02.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.239+0000 7f36cbe00640 1 -- 172.21.15.42:0/3038477313 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56434ec92740 con 0x56434ed39a10 2024-04-20T10:39:02.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.239+0000 7f36cbe00640 10 client.26036 handle_command_reply: tid=0 2024-04-20T10:39:02.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.239+0000 7f36f01e7740 1 client.26036 shutdown 2024-04-20T10:39:02.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.240+0000 7f36f01e7740 1 -- 172.21.15.42:0/3038477313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed200f0 msgr2=0x56434ed26640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.240+0000 7f36f01e7740 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed200f0 0x56434ed26640 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f36d80ce360 tx=0x7f36d8004ba0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.240+0000 7f36f01e7740 1 -- 172.21.15.42:0/3038477313 shutdown_connections 2024-04-20T10:39:02.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.240+0000 7f36f01e7740 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56434ed39a10 0x56434ed3be00 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.240+0000 7f36f01e7740 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed295f0 0x56434ed270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.240+0000 7f36f01e7740 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed26b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.240+0000 7f36f01e7740 1 --2- 172.21.15.42:0/3038477313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed200f0 0x56434ed26640 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.240+0000 7f36cbe00640 0 client.26036 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:02.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.240+0000 7f36f01e7740 1 -- 172.21.15.42:0/3038477313 >> 172.21.15.42:0/3038477313 conn(0x56434ed1a820 msgr2=0x56434ed1c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:02.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.240+0000 7f36f01e7740 1 -- 172.21.15.42:0/3038477313 shutdown_connections 2024-04-20T10:39:02.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.240+0000 7f36f01e7740 1 -- 172.21.15.42:0/3038477313 wait complete. 2024-04-20T10:39:02.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.240+0000 7f36f01e7740 20 client.26036 trim_cache size 0 max 16384 2024-04-20T10:39:02.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.261+0000 7f36f01e7740 1 Processor -- start 2024-04-20T10:39:02.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.262+0000 7f36f01e7740 1 -- start start 2024-04-20T10:39:02.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.262+0000 7f36f01e7740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed200f0 0x56434ed28d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.262+0000 7f36f01e7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed20a10 0x56434ed35270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.262+0000 7f36f01e7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed41690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.262+0000 7f36f01e7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81adcb0 con 0x56434ed20a10 2024-04-20T10:39:02.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.262+0000 7f36f01e7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81b0530 con 0x56434ed295f0 2024-04-20T10:39:02.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.262+0000 7f36f01e7740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81ae320 con 0x56434ed200f0 2024-04-20T10:39:02.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.262+0000 7f36e7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed41690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.262+0000 7f36e7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed41690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47798/0 (socket says 172.21.15.42:47798) 2024-04-20T10:39:02.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.262+0000 7f36e7200640 1 -- 172.21.15.42:0/4256266784 learned_addr learned my addr 172.21.15.42:0/4256266784 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:02.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.262+0000 7f36dde00640 1 --2- 172.21.15.42:0/4256266784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed20a10 0x56434ed35270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.262+0000 7f36e6800640 1 --2- 172.21.15.42:0/4256266784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed200f0 0x56434ed28d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.263+0000 7f36e6800640 1 -- 172.21.15.42:0/4256266784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 msgr2=0x56434ed41690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.263+0000 7f36e6800640 1 --2- 172.21.15.42:0/4256266784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed41690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.263+0000 7f36e6800640 1 -- 172.21.15.42:0/4256266784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed20a10 msgr2=0x56434ed35270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.263+0000 7f36e6800640 1 --2- 172.21.15.42:0/4256266784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed20a10 0x56434ed35270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.263+0000 7f36e6800640 1 -- 172.21.15.42:0/4256266784 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434e957170 con 0x56434ed200f0 2024-04-20T10:39:02.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.263+0000 7f36e6800640 1 --2- 172.21.15.42:0/4256266784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed200f0 0x56434ed28d70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f36d800c6a0 tx=0x7f36d80b6260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c12beb795035594 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:02.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.263+0000 7f36d6a00640 1 -- 172.21.15.42:0/4256266784 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d809c980 con 0x56434ed200f0 2024-04-20T10:39:02.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.263+0000 7f36d6a00640 1 -- 172.21.15.42:0/4256266784 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d809d2b0 con 0x56434ed200f0 2024-04-20T10:39:02.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.263+0000 7f36d6a00640 1 -- 172.21.15.42:0/4256266784 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8099220 con 0x56434ed200f0 2024-04-20T10:39:02.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.264+0000 7f36f01e7740 1 -- 172.21.15.42:0/4256266784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed200f0 msgr2=0x56434ed28d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.264+0000 7f36f01e7740 1 --2- 172.21.15.42:0/4256266784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed200f0 0x56434ed28d70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f36d800c6a0 tx=0x7f36d80b6260 comp rx=0 tx=0).stop 2024-04-20T10:39:02.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.265+0000 7f36f01e7740 1 -- 172.21.15.42:0/4256266784 shutdown_connections 2024-04-20T10:39:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.265+0000 7f36f01e7740 1 --2- 172.21.15.42:0/4256266784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed41690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.265+0000 7f36f01e7740 1 --2- 172.21.15.42:0/4256266784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed20a10 0x56434ed35270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.265+0000 7f36f01e7740 1 --2- 172.21.15.42:0/4256266784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed200f0 0x56434ed28d70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.265+0000 7f36f01e7740 1 -- 172.21.15.42:0/4256266784 >> 172.21.15.42:0/4256266784 conn(0x56434ed1a820 msgr2=0x56434ed1c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.265+0000 7f36f01e7740 1 -- 172.21.15.42:0/4256266784 shutdown_connections 2024-04-20T10:39:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.265+0000 7f36f01e7740 1 -- 172.21.15.42:0/4256266784 wait complete. 2024-04-20T10:39:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.265+0000 7f36f01e7740 1 -- start start 2024-04-20T10:39:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.265+0000 7f36f01e7740 1 Processor -- start 2024-04-20T10:39:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36f01e7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 0x56434ed23290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36f01e7740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36f01e7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed23d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36f01e7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81ae9f0 con 0x56434ed200f0 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36f01e7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81afe60 con 0x56434ed295f0 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36f01e7740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f36e81af0c0 con 0x56434ed20a10 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36f01e7740 10 client.0 _ll_register_callbacks cb 0x7f36d80d4410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36e6800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 0x56434ed23290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36e7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed23d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36dde00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36e6800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 0x56434ed23290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45126/0 (socket says 172.21.15.42:45126) 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36e6800640 1 -- 172.21.15.42:0/3495019595 learned_addr learned my addr 172.21.15.42:0/3495019595 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.266+0000 7f36e7200640 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed23d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47800/0 (socket says 172.21.15.42:47800) 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.267+0000 7f36e7200640 1 -- 172.21.15.42:0/3495019595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 msgr2=0x56434ed237d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.267+0000 7f36e7200640 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed237d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.267+0000 7f36e7200640 1 -- 172.21.15.42:0/3495019595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 msgr2=0x56434ed23290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.267+0000 7f36e7200640 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 0x56434ed23290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.267+0000 7f36e7200640 1 -- 172.21.15.42:0/3495019595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434eda7710 con 0x56434ed295f0 2024-04-20T10:39:02.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.267+0000 7f36e6800640 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 0x56434ed23290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:39:02.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.267+0000 7f36dde00640 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:39:02.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.267+0000 7f36e7200640 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed23d10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x56434ed38860 tx=0x7f36e0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6effdd5b047b14b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:02.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.267+0000 7f36cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:39:02.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.267+0000 7f36cbe00640 1 -- 172.21.15.42:0/3495019595 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e002e060 con 0x56434ed295f0 2024-04-20T10:39:02.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.267+0000 7f36cbe00640 1 -- 172.21.15.42:0/3495019595 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f36e0003b80 con 0x56434ed295f0 2024-04-20T10:39:02.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.267+0000 7f36f01e7740 1 -- 172.21.15.42:0/3495019595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d809d2b0 con 0x56434ed295f0 2024-04-20T10:39:02.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.268+0000 7f36cbe00640 1 -- 172.21.15.42:0/3495019595 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f36e000fe30 con 0x56434ed295f0 2024-04-20T10:39:02.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.268+0000 7f36f01e7740 10 client.26042 fetch_fsmap learned FSMap version 19 2024-04-20T10:39:02.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.268+0000 7f36f01e7740 1 -- 172.21.15.42:0/3495019595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56434eda8a10 con 0x56434ed295f0 2024-04-20T10:39:02.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.268+0000 7f36cbe00640 1 -- 172.21.15.42:0/3495019595 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d809d2b0 con 0x56434ed295f0 2024-04-20T10:39:02.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.268+0000 7f36cbe00640 1 -- 172.21.15.42:0/3495019595 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f36e0028030 con 0x56434ed295f0 2024-04-20T10:39:02.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.268+0000 7f36f01e7740 10 client.26042 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:39:02.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.268+0000 7f36f01e7740 10 client.26042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:39:02.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.268+0000 7f36f01e7740 20 client.26042 populate_metadata read hostname 'smithi042' 2024-04-20T10:39:02.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.268+0000 7f36f01e7740 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56434eda9190 0x56434edab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.268+0000 7f36f01e7740 4 client.26042 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:39:02.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.268+0000 7f36f01e7740 1 -- 172.21.15.42:0/3495019595 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56434ec8c5b0 con 0x56434eda9190 2024-04-20T10:39:02.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.268+0000 7f36e6800640 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56434eda9190 0x56434edab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.269+0000 7f36e6800640 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56434eda9190 0x56434edab5b0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0026ef5bf7f08e8 server_cookie=bb368e3a088f97fd in_seq=0 out_seq=0 2024-04-20T10:39:02.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.269+0000 7f36cbe00640 10 client.26042 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:02.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36cbe00640 1 -- 172.21.15.42:0/3495019595 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56434ec8c5b0 con 0x56434eda9190 2024-04-20T10:39:02.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36cbe00640 10 client.26042 handle_command_reply: tid=0 2024-04-20T10:39:02.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36f01e7740 1 client.26042 shutdown 2024-04-20T10:39:02.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36f01e7740 1 -- 172.21.15.42:0/3495019595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 msgr2=0x56434ed23d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36f01e7740 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed23d10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x56434ed38860 tx=0x7f36e0004ab0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36f01e7740 1 -- 172.21.15.42:0/3495019595 shutdown_connections 2024-04-20T10:39:02.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36f01e7740 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56434eda9190 0x56434edab5b0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36f01e7740 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56434ed295f0 0x56434ed23d10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36cbe00640 0 client.26042 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:02.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36f01e7740 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434ed20a10 0x56434ed237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36f01e7740 1 --2- 172.21.15.42:0/3495019595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56434ed200f0 0x56434ed23290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36f01e7740 1 -- 172.21.15.42:0/3495019595 >> 172.21.15.42:0/3495019595 conn(0x56434ed1a820 msgr2=0x56434ed19170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:02.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36f01e7740 1 -- 172.21.15.42:0/3495019595 shutdown_connections 2024-04-20T10:39:02.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36f01e7740 1 -- 172.21.15.42:0/3495019595 wait complete. 2024-04-20T10:39:02.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.270+0000 7f36f01e7740 20 client.26042 trim_cache size 0 max 16384 2024-04-20T10:39:02.275 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:39:02.275 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:39:02.275 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "98a2be70-7f71-4a57-befe-368fa414aeaf", 2024-04-20T10:39:02.275 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:39:02.275 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:39:02.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.271+0000 7f36eca00640 1 -- 172.21.15.42:0/140886111 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f36cc0a3d20 msgr2=0x7f36cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.271+0000 7f36eca00640 1 --2- 172.21.15.42:0/140886111 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f36cc0a3d20 0x7f36cc0a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009fd0 tx=0x7f36e0009500 comp rx=0 tx=0).stop 2024-04-20T10:39:02.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.271+0000 7f36eca00640 1 -- 172.21.15.42:0/140886111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e8104280 msgr2=0x7f36e81a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.271+0000 7f36eca00640 1 --2- 172.21.15.42:0/140886111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e8104280 0x7f36e81a0670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f36d800e9f0 tx=0x7f36d800edf0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.271+0000 7f36eca00640 1 -- 172.21.15.42:0/140886111 shutdown_connections 2024-04-20T10:39:02.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.271+0000 7f36eca00640 1 --2- 172.21.15.42:0/140886111 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f36cc0a3d20 0x7f36cc0a61e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.271+0000 7f36eca00640 1 --2- 172.21.15.42:0/140886111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36e819a7e0 0x7f36e819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.271+0000 7f36eca00640 1 --2- 172.21.15.42:0/140886111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e8109350 0x7f36e81a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.271+0000 7f36eca00640 1 --2- 172.21.15.42:0/140886111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e8104280 0x7f36e81a0670 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.271+0000 7f36eca00640 1 -- 172.21.15.42:0/140886111 >> 172.21.15.42:0/140886111 conn(0x7f36e80ffae0 msgr2=0x7f36e8101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:02.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.271+0000 7f36eca00640 1 -- 172.21.15.42:0/140886111 shutdown_connections 2024-04-20T10:39:02.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.271+0000 7f36eca00640 1 -- 172.21.15.42:0/140886111 wait complete. 2024-04-20T10:39:02.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:39:02.289 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:39:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.509+0000 7f2788a00640 1 -- 172.21.15.42:0/1264149631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2784104c30 msgr2=0x7f2784108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.509+0000 7f2788a00640 1 --2- 172.21.15.42:0/1264149631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2784104c30 0x7f2784108c70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f276c009900 tx=0x7f276c02bc40 comp rx=0 tx=0).stop 2024-04-20T10:39:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.509+0000 7f2788a00640 1 -- 172.21.15.42:0/1264149631 shutdown_connections 2024-04-20T10:39:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.509+0000 7f2788a00640 1 --2- 172.21.15.42:0/1264149631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2784109350 0x7f2784110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.509+0000 7f2788a00640 1 --2- 172.21.15.42:0/1264149631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2784104c30 0x7f2784108c70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.509+0000 7f2788a00640 1 --2- 172.21.15.42:0/1264149631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2784104280 0x7f2784104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.509+0000 7f2788a00640 1 -- 172.21.15.42:0/1264149631 >> 172.21.15.42:0/1264149631 conn(0x7f27840ffaa0 msgr2=0x7f2784101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:02.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.509+0000 7f2788a00640 1 -- 172.21.15.42:0/1264149631 shutdown_connections 2024-04-20T10:39:02.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.509+0000 7f2788a00640 1 -- 172.21.15.42:0/1264149631 wait complete. 2024-04-20T10:39:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.510+0000 7f2788a00640 1 Processor -- start 2024-04-20T10:39:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.511+0000 7f2788a00640 1 -- start start 2024-04-20T10:39:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.511+0000 7f2788a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2784104280 0x7f27841a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.511+0000 7f2788a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2784109350 0x7f27841a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.511+0000 7f2788a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f278419a730 0x7f278419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.511+0000 7f2788a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2784114430 con 0x7f2784109350 2024-04-20T10:39:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.511+0000 7f2788a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27841142b0 con 0x7f2784104280 2024-04-20T10:39:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.511+0000 7f2788a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f27841145b0 con 0x7f278419a730 2024-04-20T10:39:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.511+0000 7f2782800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2784104280 0x7f27841a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.511+0000 7f2782800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2784104280 0x7f27841a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47822/0 (socket says 172.21.15.42:47822) 2024-04-20T10:39:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.511+0000 7f2782800640 1 -- 172.21.15.42:0/267880959 learned_addr learned my addr 172.21.15.42:0/267880959 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.511+0000 7f2779e00640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2784109350 0x7f27841a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.511+0000 7f2783200640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f278419a730 0x7f278419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.512+0000 7f2782800640 1 -- 172.21.15.42:0/267880959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f278419a730 msgr2=0x7f278419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.512+0000 7f2782800640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f278419a730 0x7f278419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.512+0000 7f2782800640 1 -- 172.21.15.42:0/267880959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2784109350 msgr2=0x7f27841a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.512+0000 7f2782800640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2784109350 0x7f27841a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.512+0000 7f2782800640 1 -- 172.21.15.42:0/267880959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278419b330 con 0x7f2784104280 2024-04-20T10:39:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.512+0000 7f2779e00640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2784109350 0x7f27841a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:39:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.512+0000 7f2783200640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f278419a730 0x7f278419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:39:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.512+0000 7f2782800640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2784104280 0x7f27841a0670 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f277400d950 tx=0x7f277400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:02.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.513+0000 7f277be00640 1 -- 172.21.15.42:0/267880959 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774017030 con 0x7f2784104280 2024-04-20T10:39:02.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.513+0000 7f277be00640 1 -- 172.21.15.42:0/267880959 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2774004020 con 0x7f2784104280 2024-04-20T10:39:02.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.513+0000 7f2788a00640 1 -- 172.21.15.42:0/267880959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27841a7410 con 0x7f2784104280 2024-04-20T10:39:02.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.513+0000 7f277be00640 1 -- 172.21.15.42:0/267880959 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27740047a0 con 0x7f2784104280 2024-04-20T10:39:02.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.513+0000 7f2788a00640 1 -- 172.21.15.42:0/267880959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27841a77c0 con 0x7f2784104280 2024-04-20T10:39:02.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.517+0000 7f277be00640 1 -- 172.21.15.42:0/267880959 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f27740041d0 con 0x7f2784104280 2024-04-20T10:39:02.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.517+0000 7f278c211740 1 Processor -- start 2024-04-20T10:39:02.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.517+0000 7f278c211740 1 -- start start 2024-04-20T10:39:02.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.518+0000 7f278c211740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 0x55d052596760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.518+0000 7f278c211740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596ca0 0x55d05259f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.518+0000 7f278c211740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 0x55d0525a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.518+0000 7f278c211740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d0524fc150 con 0x55d05259f880 2024-04-20T10:39:02.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.518+0000 7f278c211740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d0524a2370 con 0x55d052596380 2024-04-20T10:39:02.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.518+0000 7f278c211740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d0524f8110 con 0x55d052596ca0 2024-04-20T10:39:02.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.518+0000 7f2783200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 0x55d052596760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.518+0000 7f2783200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 0x55d052596760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47836/0 (socket says 172.21.15.42:47836) 2024-04-20T10:39:02.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.518+0000 7f2783200640 1 -- 172.21.15.42:0/1410065027 learned_addr learned my addr 172.21.15.42:0/1410065027 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:02.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.518+0000 7f2782800640 1 --2- 172.21.15.42:0/1410065027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596ca0 0x55d05259f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.519+0000 7f2783200640 1 -- 172.21.15.42:0/1410065027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596ca0 msgr2=0x55d05259f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.519+0000 7f2783200640 1 --2- 172.21.15.42:0/1410065027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596ca0 0x55d05259f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.519+0000 7f2783200640 1 -- 172.21.15.42:0/1410065027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 msgr2=0x55d0525a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.519+0000 7f2783200640 1 --2- 172.21.15.42:0/1410065027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 0x55d0525a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.519+0000 7f2783200640 1 -- 172.21.15.42:0/1410065027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0522d87d0 con 0x55d052596380 2024-04-20T10:39:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.519+0000 7f2783200640 1 --2- 172.21.15.42:0/1410065027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 0x55d052596760 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f277c011790 tx=0x7f277c011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0063940d11c35a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.519+0000 7f277be00640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f27680a3d20 0x7f27680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.519+0000 7f277be00640 1 -- 172.21.15.42:0/267880959 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f27740d0280 con 0x7f2784104280 2024-04-20T10:39:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.519+0000 7f2773400640 1 -- 172.21.15.42:0/1410065027 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c003c30 con 0x55d052596380 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.520+0000 7f2773400640 1 -- 172.21.15.42:0/1410065027 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f277c00fe80 con 0x55d052596380 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.520+0000 7f2773400640 1 -- 172.21.15.42:0/1410065027 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c0193e0 con 0x55d052596380 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.520+0000 7f2782800640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f27680a3d20 0x7f27680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.521+0000 7f278c211740 1 -- 172.21.15.42:0/1410065027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 msgr2=0x55d052596760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.521+0000 7f2782800640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f27680a3d20 0x7f27680a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55d052595e50 tx=0x7f27740d3d30 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.521+0000 7f278c211740 1 --2- 172.21.15.42:0/1410065027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 0x55d052596760 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f277c011790 tx=0x7f277c011c50 comp rx=0 tx=0).stop 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.521+0000 7f278c211740 1 -- 172.21.15.42:0/1410065027 shutdown_connections 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.521+0000 7f278c211740 1 --2- 172.21.15.42:0/1410065027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 0x55d0525a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.521+0000 7f278c211740 1 --2- 172.21.15.42:0/1410065027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596ca0 0x55d05259f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.521+0000 7f278c211740 1 --2- 172.21.15.42:0/1410065027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 0x55d052596760 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.521+0000 7f278c211740 1 -- 172.21.15.42:0/1410065027 >> 172.21.15.42:0/1410065027 conn(0x55d052590ab0 msgr2=0x55d052590e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.521+0000 7f278c211740 1 -- 172.21.15.42:0/1410065027 shutdown_connections 2024-04-20T10:39:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.521+0000 7f278c211740 1 -- 172.21.15.42:0/1410065027 wait complete. 2024-04-20T10:39:02.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.522+0000 7f278c211740 1 -- start start 2024-04-20T10:39:02.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.522+0000 7f278c211740 1 Processor -- start 2024-04-20T10:39:02.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.522+0000 7f278c211740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 0x55d05259c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f278c211740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d052596ca0 0x55d05259ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f278c211740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d05259f880 0x55d05259d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f278c211740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d0525a2670 con 0x55d052596ca0 2024-04-20T10:39:02.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f278c211740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d0524b5740 con 0x55d052596380 2024-04-20T10:39:02.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f278c211740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d0525a27f0 con 0x55d05259f880 2024-04-20T10:39:02.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f278c211740 10 client.0 _ll_register_callbacks cb 0x55d0524b0b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:39:02.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f2783200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 0x55d05259c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f2783200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 0x55d05259c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47840/0 (socket says 172.21.15.42:47840) 2024-04-20T10:39:02.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f2783200640 1 -- 172.21.15.42:0/31776846 learned_addr learned my addr 172.21.15.42:0/31776846 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:02.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f2779e00640 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d052596ca0 0x55d05259ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f2783200640 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d05259f880 0x55d05259d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f2783200640 1 -- 172.21.15.42:0/31776846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d05259f880 msgr2=0x55d05259d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f2783200640 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d05259f880 0x55d05259d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f2783200640 1 -- 172.21.15.42:0/31776846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d052596ca0 msgr2=0x55d05259ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f2783200640 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d052596ca0 0x55d05259ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.523+0000 7f2783200640 1 -- 172.21.15.42:0/31776846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d05220e3f0 con 0x55d052596380 2024-04-20T10:39:02.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.524+0000 7f2783200640 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 0x55d05259c920 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f277c0115a0 tx=0x7f277c004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4e63e67bc805ae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:02.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.524+0000 7f2767e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:39:02.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.524+0000 7f2767e00640 1 -- 172.21.15.42:0/31776846 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c004260 con 0x55d052596380 2024-04-20T10:39:02.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.524+0000 7f2767e00640 1 -- 172.21.15.42:0/31776846 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f277c011050 con 0x55d052596380 2024-04-20T10:39:02.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.524+0000 7f278c211740 1 -- 172.21.15.42:0/31776846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d052509b70 con 0x55d052596380 2024-04-20T10:39:02.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.524+0000 7f2767e00640 1 -- 172.21.15.42:0/31776846 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f277c011250 con 0x55d052596380 2024-04-20T10:39:02.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.524+0000 7f278c211740 10 client.26054 fetch_fsmap learned FSMap version 19 2024-04-20T10:39:02.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.524+0000 7f278c211740 1 -- 172.21.15.42:0/31776846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d0521ce0f0 con 0x55d052596380 2024-04-20T10:39:02.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.525+0000 7f2767e00640 1 -- 172.21.15.42:0/31776846 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d052509b70 con 0x55d052596380 2024-04-20T10:39:02.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.525+0000 7f2767e00640 1 -- 172.21.15.42:0/31776846 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f277c018070 con 0x55d052596380 2024-04-20T10:39:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.525+0000 7f278c211740 10 client.26054 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:39:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.525+0000 7f278c211740 10 client.26054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:39:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.525+0000 7f278c211740 20 client.26054 populate_metadata read hostname 'smithi042' 2024-04-20T10:39:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.525+0000 7f278c211740 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d0525afc00 0x55d0525b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.525+0000 7f278c211740 4 client.26054 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:39:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.525+0000 7f278c211740 1 -- 172.21.15.42:0/31776846 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0525039e0 con 0x55d0525afc00 2024-04-20T10:39:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.525+0000 7f2779e00640 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d0525afc00 0x55d0525b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.526+0000 7f2779e00640 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d0525afc00 0x55d0525b1ff0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e182c0431847e54 server_cookie=2bf933d2bb8e62c0 in_seq=0 out_seq=0 2024-04-20T10:39:02.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.526+0000 7f2767e00640 10 client.26054 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:02.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.527+0000 7f2767e00640 1 -- 172.21.15.42:0/31776846 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0525039e0 con 0x55d0525afc00 2024-04-20T10:39:02.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.527+0000 7f2767e00640 10 client.26054 handle_command_reply: tid=0 2024-04-20T10:39:02.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.527+0000 7f278c211740 1 client.26054 shutdown 2024-04-20T10:39:02.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.527+0000 7f278c211740 1 -- 172.21.15.42:0/31776846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 msgr2=0x55d05259c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.527+0000 7f278c211740 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 0x55d05259c920 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f277c0115a0 tx=0x7f277c004530 comp rx=0 tx=0).stop 2024-04-20T10:39:02.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.527+0000 7f278c211740 1 -- 172.21.15.42:0/31776846 shutdown_connections 2024-04-20T10:39:02.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.527+0000 7f278c211740 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d0525afc00 0x55d0525b1ff0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.527+0000 7f278c211740 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d05259f880 0x55d05259d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.528+0000 7f278c211740 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d052596ca0 0x55d05259ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.528+0000 7f278c211740 1 --2- 172.21.15.42:0/31776846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596380 0x55d05259c920 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.528+0000 7f278c211740 1 -- 172.21.15.42:0/31776846 >> 172.21.15.42:0/31776846 conn(0x55d052590ab0 msgr2=0x55d052592800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:02.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.528+0000 7f2767e00640 0 client.26054 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:02.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.528+0000 7f278c211740 1 -- 172.21.15.42:0/31776846 shutdown_connections 2024-04-20T10:39:02.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.528+0000 7f278c211740 1 -- 172.21.15.42:0/31776846 wait complete. 2024-04-20T10:39:02.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.528+0000 7f278c211740 20 client.26054 trim_cache size 0 max 16384 2024-04-20T10:39:02.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.551+0000 7f278c211740 1 Processor -- start 2024-04-20T10:39:02.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.551+0000 7f278c211740 1 -- start start 2024-04-20T10:39:02.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.551+0000 7f278c211740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596380 0x55d052591f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.551+0000 7f278c211740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 0x55d05259c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.551+0000 7f278c211740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 0x55d0525ab530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.551+0000 7f278c211740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f27841adde0 con 0x55d05259f880 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.551+0000 7f278c211740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b0660 con 0x55d052596ca0 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.551+0000 7f278c211740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f27841ae450 con 0x55d052596380 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.551+0000 7f2779e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 0x55d05259c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.551+0000 7f2779e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 0x55d05259c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47844/0 (socket says 172.21.15.42:47844) 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.551+0000 7f2779e00640 1 -- 172.21.15.42:0/1140566920 learned_addr learned my addr 172.21.15.42:0/1140566920 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.551+0000 7f2783200640 1 --2- 172.21.15.42:0/1140566920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 0x55d0525ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.552+0000 7f2783200640 1 --2- 172.21.15.42:0/1140566920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596380 0x55d052591f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.552+0000 7f2779e00640 1 -- 172.21.15.42:0/1140566920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596380 msgr2=0x55d052591f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.552+0000 7f2779e00640 1 --2- 172.21.15.42:0/1140566920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596380 0x55d052591f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.552+0000 7f2779e00640 1 -- 172.21.15.42:0/1140566920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 msgr2=0x55d0525ab530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.552+0000 7f2779e00640 1 --2- 172.21.15.42:0/1140566920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 0x55d0525ab530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.552+0000 7f2779e00640 1 -- 172.21.15.42:0/1140566920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d052592db0 con 0x55d052596ca0 2024-04-20T10:39:02.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.552+0000 7f2779e00640 1 --2- 172.21.15.42:0/1140566920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 0x55d05259c200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f276c009900 tx=0x7f276c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ea21d016fb5e0a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:02.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.552+0000 7f2772a00640 1 -- 172.21.15.42:0/1140566920 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c04c070 con 0x55d052596ca0 2024-04-20T10:39:02.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.552+0000 7f2772a00640 1 -- 172.21.15.42:0/1140566920 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f276c046c80 con 0x55d052596ca0 2024-04-20T10:39:02.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.552+0000 7f2772a00640 1 -- 172.21.15.42:0/1140566920 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c0465d0 con 0x55d052596ca0 2024-04-20T10:39:02.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.553+0000 7f278c211740 1 -- 172.21.15.42:0/1140566920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 msgr2=0x55d05259c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.553+0000 7f278c211740 1 --2- 172.21.15.42:0/1140566920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 0x55d05259c200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f276c009900 tx=0x7f276c000bc0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.553+0000 7f278c211740 1 -- 172.21.15.42:0/1140566920 shutdown_connections 2024-04-20T10:39:02.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.553+0000 7f278c211740 1 --2- 172.21.15.42:0/1140566920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 0x55d0525ab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.553+0000 7f278c211740 1 --2- 172.21.15.42:0/1140566920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 0x55d05259c200 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.553+0000 7f278c211740 1 --2- 172.21.15.42:0/1140566920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596380 0x55d052591f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.553+0000 7f278c211740 1 -- 172.21.15.42:0/1140566920 >> 172.21.15.42:0/1140566920 conn(0x55d052590ab0 msgr2=0x55d05258fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.553+0000 7f278c211740 1 -- 172.21.15.42:0/1140566920 shutdown_connections 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.553+0000 7f278c211740 1 -- 172.21.15.42:0/1140566920 wait complete. 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.554+0000 7f278c211740 1 -- start start 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.554+0000 7f278c211740 1 Processor -- start 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f278c211740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596380 0x55d0525a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f278c211740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 0x55d0525a4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f278c211740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 0x55d052614f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f278c211740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f27841aeb20 con 0x55d05259f880 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f278c211740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27841aff90 con 0x55d052596ca0 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f278c211740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f27841af1f0 con 0x55d052596380 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f278c211740 10 client.0 _ll_register_callbacks cb 0x7f27740d3c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:39:02.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f2783200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596380 0x55d0525a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f2779e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 0x55d0525a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f2779e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 0x55d0525a4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:47860/0 (socket says 172.21.15.42:47860) 2024-04-20T10:39:02.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f2779e00640 1 -- 172.21.15.42:0/3731289038 learned_addr learned my addr 172.21.15.42:0/3731289038 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:02.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f2783200640 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 0x55d052614f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f2779e00640 1 -- 172.21.15.42:0/3731289038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596380 msgr2=0x55d0525a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f2779e00640 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596380 0x55d0525a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f2779e00640 1 -- 172.21.15.42:0/3731289038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 msgr2=0x55d052614f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:02.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f2779e00640 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 0x55d052614f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f2779e00640 1 -- 172.21.15.42:0/3731289038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0526154c0 con 0x55d052596ca0 2024-04-20T10:39:02.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.555+0000 7f2783200640 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 0x55d052614f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:39:02.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.556+0000 7f2779e00640 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 0x55d0525a4080 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f276c02ccd0 tx=0x7f276c02cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1ad76241f9c5c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:02.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.556+0000 7f2767e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:39:02.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.556+0000 7f2767e00640 1 -- 172.21.15.42:0/3731289038 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c04c070 con 0x55d052596ca0 2024-04-20T10:39:02.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.556+0000 7f2767e00640 1 -- 172.21.15.42:0/3731289038 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f276c004020 con 0x55d052596ca0 2024-04-20T10:39:02.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.556+0000 7f278c211740 1 -- 172.21.15.42:0/3731289038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f276c046c80 con 0x55d052596ca0 2024-04-20T10:39:02.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.556+0000 7f2767e00640 1 -- 172.21.15.42:0/3731289038 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f276c004240 con 0x55d052596ca0 2024-04-20T10:39:02.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.556+0000 7f278c211740 10 client.26060 fetch_fsmap learned FSMap version 19 2024-04-20T10:39:02.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.556+0000 7f278c211740 1 -- 172.21.15.42:0/3731289038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d052616740 con 0x55d052596ca0 2024-04-20T10:39:02.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.557+0000 7f2767e00640 1 -- 172.21.15.42:0/3731289038 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f276c046c80 con 0x55d052596ca0 2024-04-20T10:39:02.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.557+0000 7f2767e00640 1 -- 172.21.15.42:0/3731289038 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f276c003710 con 0x55d052596ca0 2024-04-20T10:39:02.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.557+0000 7f278c211740 10 client.26060 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:39:02.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.557+0000 7f278c211740 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:39:02.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.557+0000 7f278c211740 20 client.26060 populate_metadata read hostname 'smithi042' 2024-04-20T10:39:02.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.557+0000 7f278c211740 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d052616e00 0x55d052619220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.557+0000 7f278c211740 4 client.26060 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.557+0000 7f278c211740 1 -- 172.21.15.42:0/3731289038 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d05247f450 con 0x55d052616e00 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.558+0000 7f2783200640 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d052616e00 0x55d052619220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.558+0000 7f2783200640 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d052616e00 0x55d052619220 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77689a550b38200 server_cookie=f27988c84823ee50 in_seq=0 out_seq=0 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.558+0000 7f2767e00640 10 client.26060 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.558+0000 7f2767e00640 1 -- 172.21.15.42:0/3731289038 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55d05247f450 con 0x55d052616e00 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.558+0000 7f2767e00640 10 client.26060 handle_command_reply: tid=0 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.558+0000 7f278c211740 1 client.26060 shutdown 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.559+0000 7f278c211740 1 -- 172.21.15.42:0/3731289038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 msgr2=0x55d0525a4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.559+0000 7f278c211740 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 0x55d0525a4080 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f276c02ccd0 tx=0x7f276c02cdd0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.559+0000 7f278c211740 1 -- 172.21.15.42:0/3731289038 shutdown_connections 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.559+0000 7f278c211740 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d052616e00 0x55d052619220 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.559+0000 7f278c211740 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d05259f880 0x55d052614f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.559+0000 7f278c211740 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d052596ca0 0x55d0525a4080 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.559+0000 7f278c211740 1 --2- 172.21.15.42:0/3731289038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d052596380 0x55d0525a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.559+0000 7f278c211740 1 -- 172.21.15.42:0/3731289038 >> 172.21.15.42:0/3731289038 conn(0x55d052590ab0 msgr2=0x55d05259c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:02.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.559+0000 7f2767e00640 0 client.26060 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:02.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.559+0000 7f278c211740 1 -- 172.21.15.42:0/3731289038 shutdown_connections 2024-04-20T10:39:02.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.559+0000 7f278c211740 1 -- 172.21.15.42:0/3731289038 wait complete. 2024-04-20T10:39:02.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.559+0000 7f278c211740 20 client.26060 trim_cache size 0 max 16384 2024-04-20T10:39:02.563 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:39:02.564 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (4748 inodes in the stack)", 2024-04-20T10:39:02.564 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:39:02.564 INFO:teuthology.orchestra.run.smithi042.stdout: "98a2be70-7f71-4a57-befe-368fa414aeaf": { 2024-04-20T10:39:02.564 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:39:02.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "98a2be70-7f71-4a57-befe-368fa414aeaf", 2024-04-20T10:39:02.564 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:39:02.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:39:02.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:39:02.564 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:39:02.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.560+0000 7f2788a00640 1 -- 172.21.15.42:0/267880959 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f27680a3d20 msgr2=0x7f27680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.560+0000 7f2788a00640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f27680a3d20 0x7f27680a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55d052595e50 tx=0x7f27740d3d30 comp rx=0 tx=0).stop 2024-04-20T10:39:02.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.560+0000 7f2788a00640 1 -- 172.21.15.42:0/267880959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2784104280 msgr2=0x7f27841a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:02.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.560+0000 7f2788a00640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2784104280 0x7f27841a0670 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f277400d950 tx=0x7f277400de10 comp rx=0 tx=0).stop 2024-04-20T10:39:02.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.560+0000 7f2788a00640 1 -- 172.21.15.42:0/267880959 shutdown_connections 2024-04-20T10:39:02.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.560+0000 7f2788a00640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f27680a3d20 0x7f27680a61e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.560+0000 7f2788a00640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f278419a730 0x7f278419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.560+0000 7f2788a00640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2784109350 0x7f27841a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.560+0000 7f2788a00640 1 --2- 172.21.15.42:0/267880959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2784104280 0x7f27841a0670 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:02.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.560+0000 7f2788a00640 1 -- 172.21.15.42:0/267880959 >> 172.21.15.42:0/267880959 conn(0x7f27840ffaa0 msgr2=0x7f2784101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:02.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.560+0000 7f2788a00640 1 -- 172.21.15.42:0/267880959 shutdown_connections 2024-04-20T10:39:02.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:02.560+0000 7f2788a00640 1 -- 172.21.15.42:0/267880959 wait complete. 2024-04-20T10:39:02.581 INFO:tasks.cephfs.filesystem:scrub status for tag:98a2be70-7f71-4a57-befe-368fa414aeaf - {'path': '/', 'tag': '98a2be70-7f71-4a57-befe-368fa414aeaf', 'options': 'recursive,force'} 2024-04-20T10:39:02.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:02 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:39:02.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:02 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:39:02.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:02 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:39:03.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:02 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:39:03.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:02 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:39:03.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:02 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:39:03.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:02 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:39:03.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:02 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:39:03.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:02 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:39:03.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:03 smithi162 ceph-mon[21854]: pgmap v154: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 46 op/s 2024-04-20T10:39:04.005 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:03 smithi042 ceph-mon[17078]: pgmap v154: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 46 op/s 2024-04-20T10:39:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:03 smithi046 ceph-mon[21574]: pgmap v154: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 46 op/s 2024-04-20T10:39:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:05 smithi046 ceph-mon[21574]: pgmap v155: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 33 op/s 2024-04-20T10:39:06.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:05 smithi042 ceph-mon[17078]: pgmap v155: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 33 op/s 2024-04-20T10:39:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:05 smithi162 ceph-mon[21854]: pgmap v155: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 33 op/s 2024-04-20T10:39:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:07 smithi046 ceph-mon[21574]: pgmap v156: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 39 op/s 2024-04-20T10:39:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:07 smithi042 ceph-mon[17078]: pgmap v156: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 39 op/s 2024-04-20T10:39:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:07 smithi162 ceph-mon[21854]: pgmap v156: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 39 op/s 2024-04-20T10:39:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:09 smithi046 ceph-mon[21574]: pgmap v157: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 34 op/s 2024-04-20T10:39:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:09 smithi042 ceph-mon[17078]: pgmap v157: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 34 op/s 2024-04-20T10:39:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:09 smithi162 ceph-mon[21854]: pgmap v157: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 34 op/s 2024-04-20T10:39:12.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:11 smithi046 ceph-mon[21574]: pgmap v158: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 31 op/s 2024-04-20T10:39:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:11 smithi042 ceph-mon[17078]: pgmap v158: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 31 op/s 2024-04-20T10:39:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:11 smithi162 ceph-mon[21854]: pgmap v158: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 31 op/s 2024-04-20T10:39:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:13 smithi046 ceph-mon[21574]: pgmap v159: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 65 op/s 2024-04-20T10:39:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:13 smithi042 ceph-mon[17078]: pgmap v159: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 65 op/s 2024-04-20T10:39:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:13 smithi162 ceph-mon[21854]: pgmap v159: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 65 op/s 2024-04-20T10:39:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:15 smithi046 ceph-mon[21574]: pgmap v160: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 921 KiB/s wr, 43 op/s 2024-04-20T10:39:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:15 smithi042 ceph-mon[17078]: pgmap v160: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 921 KiB/s wr, 43 op/s 2024-04-20T10:39:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:15 smithi162 ceph-mon[21854]: pgmap v160: 97 pgs: 97 active+clean; 1.0 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 921 KiB/s wr, 43 op/s 2024-04-20T10:39:17.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:16 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:39:17.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:16 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:39:17.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:16 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:39:18.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:17 smithi046 ceph-mon[21574]: pgmap v161: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 325 op/s 2024-04-20T10:39:18.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:17 smithi042 ceph-mon[17078]: pgmap v161: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 325 op/s 2024-04-20T10:39:18.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:17 smithi162 ceph-mon[21854]: pgmap v161: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 325 op/s 2024-04-20T10:39:19.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:18 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:39:19.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:18 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:39:19.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:18 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:39:19.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:18 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:39:19.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:18 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:39:19.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:18 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:39:19.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:18 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:39:19.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:18 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:39:19.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:18 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:39:20.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:19 smithi042 ceph-mon[17078]: pgmap v162: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 761 op/s 2024-04-20T10:39:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:19 smithi162 ceph-mon[21854]: pgmap v162: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 761 op/s 2024-04-20T10:39:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:19 smithi046 ceph-mon[21574]: pgmap v162: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 761 op/s 2024-04-20T10:39:22.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:21 smithi042 ceph-mon[17078]: pgmap v163: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 3.6 MiB/s wr, 757 op/s 2024-04-20T10:39:22.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:21 smithi162 ceph-mon[21854]: pgmap v163: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 3.6 MiB/s wr, 757 op/s 2024-04-20T10:39:22.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:21 smithi046 ceph-mon[21574]: pgmap v163: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 3.6 MiB/s wr, 757 op/s 2024-04-20T10:39:24.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:23 smithi162 ceph-mon[21854]: pgmap v164: 97 pgs: 97 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 15 MiB/s wr, 2.25k op/s 2024-04-20T10:39:24.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:23 smithi046 ceph-mon[21574]: pgmap v164: 97 pgs: 97 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 15 MiB/s wr, 2.25k op/s 2024-04-20T10:39:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:23 smithi042 ceph-mon[17078]: pgmap v164: 97 pgs: 97 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 15 MiB/s wr, 2.25k op/s 2024-04-20T10:39:26.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:25 smithi162 ceph-mon[21854]: pgmap v165: 97 pgs: 97 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 14 MiB/s wr, 2.21k op/s 2024-04-20T10:39:26.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:25 smithi046 ceph-mon[21574]: pgmap v165: 97 pgs: 97 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 14 MiB/s wr, 2.21k op/s 2024-04-20T10:39:26.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:25 smithi042 ceph-mon[17078]: pgmap v165: 97 pgs: 97 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 14 MiB/s wr, 2.21k op/s 2024-04-20T10:39:28.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:27 smithi162 ceph-mon[21854]: pgmap v166: 97 pgs: 97 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 16 MiB/s wr, 2.55k op/s 2024-04-20T10:39:28.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:27 smithi046 ceph-mon[21574]: pgmap v166: 97 pgs: 97 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 16 MiB/s wr, 2.55k op/s 2024-04-20T10:39:28.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:27 smithi042 ceph-mon[17078]: pgmap v166: 97 pgs: 97 active+clean; 1.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 16 MiB/s wr, 2.55k op/s 2024-04-20T10:39:30.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:29 smithi162 ceph-mon[21854]: pgmap v167: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 15 MiB/s wr, 2.69k op/s 2024-04-20T10:39:30.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:29 smithi046 ceph-mon[21574]: pgmap v167: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 15 MiB/s wr, 2.69k op/s 2024-04-20T10:39:30.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:29 smithi042 ceph-mon[17078]: pgmap v167: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 15 MiB/s wr, 2.69k op/s 2024-04-20T10:39:32.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:31 smithi162 ceph-mon[21854]: pgmap v168: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 14 MiB/s wr, 2.25k op/s 2024-04-20T10:39:32.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:31 smithi046 ceph-mon[21574]: pgmap v168: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 14 MiB/s wr, 2.25k op/s 2024-04-20T10:39:32.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:31 smithi042 ceph-mon[17078]: pgmap v168: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 14 MiB/s wr, 2.25k op/s 2024-04-20T10:39:32.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:39:32.583 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:39:32.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.803+0000 7fe547200640 1 -- 172.21.15.42:0/32551130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54006b180 msgr2=0x7fe540106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:32.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.803+0000 7fe547200640 1 --2- 172.21.15.42:0/32551130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54006b180 0x7fe540106bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe528009900 tx=0x7fe52802bc50 comp rx=0 tx=0).stop 2024-04-20T10:39:32.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.803+0000 7fe547200640 1 -- 172.21.15.42:0/32551130 shutdown_connections 2024-04-20T10:39:32.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.803+0000 7fe547200640 1 --2- 172.21.15.42:0/32551130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe540107290 0x7fe54010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.803+0000 7fe547200640 1 --2- 172.21.15.42:0/32551130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54006b180 0x7fe540106bb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.803+0000 7fe547200640 1 --2- 172.21.15.42:0/32551130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe54006a7d0 0x7fe54006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.803+0000 7fe547200640 1 -- 172.21.15.42:0/32551130 >> 172.21.15.42:0/32551130 conn(0x7fe540076c60 msgr2=0x7fe540077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:32.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.803+0000 7fe547200640 1 -- 172.21.15.42:0/32551130 shutdown_connections 2024-04-20T10:39:32.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.803+0000 7fe547200640 1 -- 172.21.15.42:0/32551130 wait complete. 2024-04-20T10:39:32.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.804+0000 7fe547200640 1 Processor -- start 2024-04-20T10:39:32.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.804+0000 7fe547200640 1 -- start start 2024-04-20T10:39:32.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.805+0000 7fe547200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54006a7d0 0x7fe5401a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.805+0000 7fe547200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe540107290 0x7fe5401a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.805+0000 7fe547200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe54019a850 0x7fe54019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.805+0000 7fe547200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe540112330 con 0x7fe54006a7d0 2024-04-20T10:39:32.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.805+0000 7fe547200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401121b0 con 0x7fe54019a850 2024-04-20T10:39:32.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.805+0000 7fe547200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401124b0 con 0x7fe540107290 2024-04-20T10:39:32.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.805+0000 7fe53fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54006a7d0 0x7fe5401a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.805+0000 7fe545200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe54019a850 0x7fe54019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.805+0000 7fe53fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54006a7d0 0x7fe5401a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50974/0 (socket says 172.21.15.42:50974) 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.805+0000 7fe53fe00640 1 -- 172.21.15.42:0/1203131519 learned_addr learned my addr 172.21.15.42:0/1203131519 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.805+0000 7fe53f400640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe540107290 0x7fe5401a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.806+0000 7fe545200640 1 -- 172.21.15.42:0/1203131519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe540107290 msgr2=0x7fe5401a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.806+0000 7fe545200640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe540107290 0x7fe5401a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.806+0000 7fe545200640 1 -- 172.21.15.42:0/1203131519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54006a7d0 msgr2=0x7fe5401a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.806+0000 7fe545200640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54006a7d0 0x7fe5401a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.806+0000 7fe545200640 1 -- 172.21.15.42:0/1203131519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54019b3e0 con 0x7fe54019a850 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.806+0000 7fe53fe00640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54006a7d0 0x7fe5401a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.806+0000 7fe545200640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe54019a850 0x7fe54019ad00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe53800ec50 tx=0x7fe53800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.806+0000 7fe53cc00640 1 -- 172.21.15.42:0/1203131519 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538003b70 con 0x7fe54019a850 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.806+0000 7fe53cc00640 1 -- 172.21.15.42:0/1203131519 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe538003d10 con 0x7fe54019a850 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.806+0000 7fe53cc00640 1 -- 172.21.15.42:0/1203131519 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538010890 con 0x7fe54019a850 2024-04-20T10:39:32.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.806+0000 7fe547200640 1 -- 172.21.15.42:0/1203131519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54019b6d0 con 0x7fe54019a850 2024-04-20T10:39:32.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.807+0000 7fe547200640 1 -- 172.21.15.42:0/1203131519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5401a77e0 con 0x7fe54019a850 2024-04-20T10:39:32.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.810+0000 7fe53cc00640 1 -- 172.21.15.42:0/1203131519 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe538003fc0 con 0x7fe54019a850 2024-04-20T10:39:32.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.810+0000 7fe54a034740 1 Processor -- start 2024-04-20T10:39:32.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.810+0000 7fe54a034740 1 -- start start 2024-04-20T10:39:32.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.811+0000 7fe54a034740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.811+0000 7fe54a034740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0bc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.811+0000 7fe54a034740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.811+0000 7fe54a034740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a0bc019150 con 0x55a0bc0b3ac0 2024-04-20T10:39:32.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.811+0000 7fe54a034740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a0bbfbf370 con 0x55a0bc0b31a0 2024-04-20T10:39:32.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.811+0000 7fe54a034740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a0bc015110 con 0x55a0bc0bc6a0 2024-04-20T10:39:32.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.811+0000 7fe53f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.811+0000 7fe53f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:41772/0 (socket says 172.21.15.42:41772) 2024-04-20T10:39:32.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.811+0000 7fe53f400640 1 -- 172.21.15.42:0/1733917870 learned_addr learned my addr 172.21.15.42:0/1733917870 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:32.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.811+0000 7fe53fe00640 1 --2- 172.21.15.42:0/1733917870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0bc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.811+0000 7fe53f400640 1 --2- 172.21.15.42:0/1733917870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.812+0000 7fe53fe00640 1 -- 172.21.15.42:0/1733917870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0bc6a0 msgr2=0x55a0bc0bea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:32.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.812+0000 7fe53fe00640 1 --2- 172.21.15.42:0/1733917870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0bea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.812+0000 7fe53fe00640 1 -- 172.21.15.42:0/1733917870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b31a0 msgr2=0x55a0bc0b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.812+0000 7fe53fe00640 1 --2- 172.21.15.42:0/1733917870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.812+0000 7fe53fe00640 1 -- 172.21.15.42:0/1733917870 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0bbdf57d0 con 0x55a0bc0b3ac0 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.812+0000 7fe53f400640 1 --2- 172.21.15.42:0/1733917870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0bea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.812+0000 7fe53cc00640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe5240a3d20 0x7fe5240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.812+0000 7fe53cc00640 1 -- 172.21.15.42:0/1203131519 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe5380d0df0 con 0x7fe54019a850 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.812+0000 7fe53fe00640 1 --2- 172.21.15.42:0/1733917870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0bc160 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe5300114f0 tx=0x7fe5300119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe187fedfc99189 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.812+0000 7fe53f400640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe5240a3d20 0x7fe5240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.812+0000 7fe535600640 1 -- 172.21.15.42:0/1733917870 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53001e070 con 0x55a0bc0b3ac0 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.812+0000 7fe535600640 1 -- 172.21.15.42:0/1733917870 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5300035f0 con 0x55a0bc0b3ac0 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.813+0000 7fe535600640 1 -- 172.21.15.42:0/1733917870 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53000f820 con 0x55a0bc0b3ac0 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.813+0000 7fe54a034740 1 -- 172.21.15.42:0/1733917870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b3ac0 msgr2=0x55a0bc0bc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.813+0000 7fe53f400640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe5240a3d20 0x7fe5240a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe52802c840 tx=0x7fe528005fd0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:32.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.813+0000 7fe54a034740 1 --2- 172.21.15.42:0/1733917870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0bc160 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe5300114f0 tx=0x7fe5300119b0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.813+0000 7fe54a034740 1 -- 172.21.15.42:0/1733917870 shutdown_connections 2024-04-20T10:39:32.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.813+0000 7fe54a034740 1 --2- 172.21.15.42:0/1733917870 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0bea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.813+0000 7fe54a034740 1 --2- 172.21.15.42:0/1733917870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0bc160 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.813+0000 7fe54a034740 1 --2- 172.21.15.42:0/1733917870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.813+0000 7fe54a034740 1 -- 172.21.15.42:0/1733917870 >> 172.21.15.42:0/1733917870 conn(0x55a0bc0ad8d0 msgr2=0x55a0bc0adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:32.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.813+0000 7fe54a034740 1 -- 172.21.15.42:0/1733917870 shutdown_connections 2024-04-20T10:39:32.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.813+0000 7fe54a034740 1 -- 172.21.15.42:0/1733917870 wait complete. 2024-04-20T10:39:32.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.814+0000 7fe54a034740 1 -- start start 2024-04-20T10:39:32.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.814+0000 7fe54a034740 1 Processor -- start 2024-04-20T10:39:32.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe54a034740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe54a034740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe54a034740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe54a034740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a0bc0bf490 con 0x55a0bc0b31a0 2024-04-20T10:39:32.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe54a034740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a0bbfd2740 con 0x55a0bc0bc6a0 2024-04-20T10:39:32.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe54a034740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a0bc0bf610 con 0x55a0bc0b3ac0 2024-04-20T10:39:32.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe54a034740 10 client.0 _ll_register_callbacks cb 0x55a0bbfcdb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:39:32.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe53fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe53fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0b96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51002/0 (socket says 172.21.15.42:51002) 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe53fe00640 1 -- 172.21.15.42:0/1093965094 learned_addr learned my addr 172.21.15.42:0/1093965094 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe53f400640 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe545200640 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe545200640 1 -- 172.21.15.42:0/1093965094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b3ac0 msgr2=0x55a0bc0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe545200640 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe545200640 1 -- 172.21.15.42:0/1093965094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b31a0 msgr2=0x55a0bc0b96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe545200640 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0b96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe545200640 1 -- 172.21.15.42:0/1093965094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0bbd2b3f0 con 0x55a0bc0bc6a0 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.815+0000 7fe53fe00640 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.816+0000 7fe545200640 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0ba130 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe5380ce4b0 tx=0x7fe53809c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9f2eddceff873c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.816+0000 7fe52e000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.816+0000 7fe52e000640 1 -- 172.21.15.42:0/1093965094 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53809cd30 con 0x55a0bc0bc6a0 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.816+0000 7fe52e000640 1 -- 172.21.15.42:0/1093965094 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe5380a5510 con 0x55a0bc0bc6a0 2024-04-20T10:39:32.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.816+0000 7fe54a034740 1 -- 172.21.15.42:0/1093965094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5300035f0 con 0x55a0bc0bc6a0 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.816+0000 7fe52e000640 1 -- 172.21.15.42:0/1093965094 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe5380a10e0 con 0x55a0bc0bc6a0 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.816+0000 7fe54a034740 10 client.26072 fetch_fsmap learned FSMap version 19 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.816+0000 7fe54a034740 1 -- 172.21.15.42:0/1093965094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a0bbceb0f0 con 0x55a0bc0bc6a0 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.817+0000 7fe52e000640 1 -- 172.21.15.42:0/1093965094 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5300035f0 con 0x55a0bc0bc6a0 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.817+0000 7fe52e000640 1 -- 172.21.15.42:0/1093965094 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fe53807bb50 con 0x55a0bc0bc6a0 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.817+0000 7fe54a034740 10 client.26072 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.817+0000 7fe54a034740 10 client.26072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.817+0000 7fe54a034740 20 client.26072 populate_metadata read hostname 'smithi042' 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.817+0000 7fe54a034740 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a0bc0cca00 0x55a0bc0cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.817+0000 7fe54a034740 4 client.26072 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.817+0000 7fe54a034740 1 -- 172.21.15.42:0/1093965094 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0bc026b70 con 0x55a0bc0cca00 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.817+0000 7fe53fe00640 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a0bc0cca00 0x55a0bc0cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.818+0000 7fe53fe00640 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a0bc0cca00 0x55a0bc0cedf0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2eac5d33b1be2e3 server_cookie=f368766250ca6bc9 in_seq=0 out_seq=0 2024-04-20T10:39:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.818+0000 7fe52e000640 10 client.26072 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:32.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.820+0000 7fe52e000640 1 -- 172.21.15.42:0/1093965094 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0bc026b70 con 0x55a0bc0cca00 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.820+0000 7fe52e000640 10 client.26072 handle_command_reply: tid=0 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.820+0000 7fe54a034740 1 client.26072 shutdown 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.820+0000 7fe54a034740 1 -- 172.21.15.42:0/1093965094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0bc6a0 msgr2=0x55a0bc0ba130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.820+0000 7fe54a034740 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0ba130 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe5380ce4b0 tx=0x7fe53809c930 comp rx=0 tx=0).stop 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.821+0000 7fe54a034740 1 -- 172.21.15.42:0/1093965094 shutdown_connections 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.821+0000 7fe54a034740 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a0bc0cca00 0x55a0bc0cedf0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.821+0000 7fe54a034740 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0ba130 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.821+0000 7fe54a034740 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.821+0000 7fe52e000640 0 client.26072 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.821+0000 7fe54a034740 1 --2- 172.21.15.42:0/1093965094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.821+0000 7fe54a034740 1 -- 172.21.15.42:0/1093965094 >> 172.21.15.42:0/1093965094 conn(0x55a0bc0ad8d0 msgr2=0x55a0bc0af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.821+0000 7fe54a034740 1 -- 172.21.15.42:0/1093965094 shutdown_connections 2024-04-20T10:39:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.821+0000 7fe54a034740 1 -- 172.21.15.42:0/1093965094 wait complete. 2024-04-20T10:39:32.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.821+0000 7fe54a034740 20 client.26072 trim_cache size 0 max 16384 2024-04-20T10:39:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.841+0000 7fe54a034740 1 Processor -- start 2024-04-20T10:39:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.841+0000 7fe54a034740 1 -- start start 2024-04-20T10:39:32.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.841+0000 7fe54a034740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0bbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.841+0000 7fe54a034740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.841+0000 7fe54a034740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0b8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.841+0000 7fe54a034740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401ade50 con 0x55a0bc0bc6a0 2024-04-20T10:39:32.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.841+0000 7fe54a034740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401b06d0 con 0x55a0bc0b3ac0 2024-04-20T10:39:32.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.841+0000 7fe54a034740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401ae4c0 con 0x55a0bc0b31a0 2024-04-20T10:39:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.841+0000 7fe53f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.841+0000 7fe53f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:41784/0 (socket says 172.21.15.42:41784) 2024-04-20T10:39:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.841+0000 7fe53f400640 1 -- 172.21.15.42:0/1216301433 learned_addr learned my addr 172.21.15.42:0/1216301433 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.841+0000 7fe545200640 1 --2- 172.21.15.42:0/1216301433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.842+0000 7fe53fe00640 1 --2- 172.21.15.42:0/1216301433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0bbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.842+0000 7fe53f400640 1 -- 172.21.15.42:0/1216301433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b31a0 msgr2=0x55a0bc0bbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.842+0000 7fe53f400640 1 --2- 172.21.15.42:0/1216301433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0bbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.842+0000 7fe53f400640 1 -- 172.21.15.42:0/1216301433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0bc6a0 msgr2=0x55a0bc0b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.842+0000 7fe53f400640 1 --2- 172.21.15.42:0/1216301433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.842+0000 7fe53f400640 1 -- 172.21.15.42:0/1216301433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0bc0a9190 con 0x55a0bc0b3ac0 2024-04-20T10:39:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.842+0000 7fe53f400640 1 --2- 172.21.15.42:0/1216301433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0c8350 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe528036040 tx=0x7fe528009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46948db56b7625f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.843+0000 7fe534c00640 1 -- 172.21.15.42:0/1216301433 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528054020 con 0x55a0bc0b3ac0 2024-04-20T10:39:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.843+0000 7fe534c00640 1 -- 172.21.15.42:0/1216301433 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe528004ee0 con 0x55a0bc0b3ac0 2024-04-20T10:39:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.843+0000 7fe534c00640 1 -- 172.21.15.42:0/1216301433 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528003950 con 0x55a0bc0b3ac0 2024-04-20T10:39:32.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.843+0000 7fe54a034740 1 -- 172.21.15.42:0/1216301433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b3ac0 msgr2=0x55a0bc0c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:32.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.843+0000 7fe54a034740 1 --2- 172.21.15.42:0/1216301433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0c8350 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe528036040 tx=0x7fe528009260 comp rx=0 tx=0).stop 2024-04-20T10:39:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.843+0000 7fe54a034740 1 -- 172.21.15.42:0/1216301433 shutdown_connections 2024-04-20T10:39:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.843+0000 7fe54a034740 1 --2- 172.21.15.42:0/1216301433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.843+0000 7fe54a034740 1 --2- 172.21.15.42:0/1216301433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0c8350 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.843+0000 7fe54a034740 1 --2- 172.21.15.42:0/1216301433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0bbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.843+0000 7fe54a034740 1 -- 172.21.15.42:0/1216301433 >> 172.21.15.42:0/1216301433 conn(0x55a0bc0ad8d0 msgr2=0x55a0bc0ac030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.844+0000 7fe54a034740 1 -- 172.21.15.42:0/1216301433 shutdown_connections 2024-04-20T10:39:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.844+0000 7fe54a034740 1 -- 172.21.15.42:0/1216301433 wait complete. 2024-04-20T10:39:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.844+0000 7fe54a034740 1 -- start start 2024-04-20T10:39:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.844+0000 7fe54a034740 1 Processor -- start 2024-04-20T10:39:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe54a034740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe54a034740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0bfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe54a034740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0c0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe54a034740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401aeb90 con 0x55a0bc0b3ac0 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe54a034740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401b0000 con 0x55a0bc0bc6a0 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe54a034740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe5401af260 con 0x55a0bc0b31a0 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe54a034740 10 client.0 _ll_register_callbacks cb 0x7fe53800cf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe53fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe53fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:50554/0 (socket says 172.21.15.42:50554) 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe53fe00640 1 -- 172.21.15.42:0/1362822519 learned_addr learned my addr 172.21.15.42:0/1362822519 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe53f400640 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0bfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe545200640 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0c0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe53fe00640 1 -- 172.21.15.42:0/1362822519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0bc6a0 msgr2=0x55a0bc0c0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe53fe00640 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0c0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe53fe00640 1 -- 172.21.15.42:0/1362822519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b3ac0 msgr2=0x55a0bc0bfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe53fe00640 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0bfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.845+0000 7fe53fe00640 1 -- 172.21.15.42:0/1362822519 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0bc0c0910 con 0x55a0bc0b31a0 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.846+0000 7fe53fe00640 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0c4a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe5300114f0 tx=0x7fe530003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8fef322d01faa02 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.846+0000 7fe52e000640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.846+0000 7fe52e000640 1 -- 172.21.15.42:0/1362822519 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53001e040 con 0x55a0bc0b31a0 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.846+0000 7fe52e000640 1 -- 172.21.15.42:0/1362822519 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fe5300051b0 con 0x55a0bc0b31a0 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.846+0000 7fe54a034740 1 -- 172.21.15.42:0/1362822519 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe528004ee0 con 0x55a0bc0b31a0 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.846+0000 7fe52e000640 1 -- 172.21.15.42:0/1362822519 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe5300053d0 con 0x55a0bc0b31a0 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.846+0000 7fe54a034740 10 client.26089 fetch_fsmap learned FSMap version 19 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.846+0000 7fe54a034740 1 -- 172.21.15.42:0/1362822519 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a0bc132cf0 con 0x55a0bc0b31a0 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.847+0000 7fe52e000640 1 -- 172.21.15.42:0/1362822519 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe528004ee0 con 0x55a0bc0b31a0 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.847+0000 7fe52e000640 1 -- 172.21.15.42:0/1362822519 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fe53000f6e0 con 0x55a0bc0b31a0 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.847+0000 7fe54a034740 10 client.26089 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.847+0000 7fe54a034740 10 client.26089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.847+0000 7fe54a034740 20 client.26089 populate_metadata read hostname 'smithi042' 2024-04-20T10:39:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.847+0000 7fe54a034740 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a0bc1332d0 0x55a0bc1356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:39:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.847+0000 7fe54a034740 4 client.26089 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:39:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.847+0000 7fe54a034740 1 -- 172.21.15.42:0/1362822519 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0bc0209e0 con 0x55a0bc1332d0 2024-04-20T10:39:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.847+0000 7fe53f400640 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a0bc1332d0 0x55a0bc1356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:39:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.848+0000 7fe53f400640 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a0bc1332d0 0x55a0bc1356f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=187e40f52fbcc23e server_cookie=df5ddefa9a405b2a in_seq=0 out_seq=0 2024-04-20T10:39:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.848+0000 7fe52e000640 10 client.26089 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.851+0000 7fe52e000640 1 -- 172.21.15.42:0/1362822519 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55a0bc0209e0 con 0x55a0bc1332d0 2024-04-20T10:39:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.851+0000 7fe52e000640 10 client.26089 handle_command_reply: tid=0 2024-04-20T10:39:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.851+0000 7fe54a034740 1 client.26089 shutdown 2024-04-20T10:39:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.851+0000 7fe54a034740 1 -- 172.21.15.42:0/1362822519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b31a0 msgr2=0x55a0bc0c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.851+0000 7fe54a034740 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0c4a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe5300114f0 tx=0x7fe530003680 comp rx=0 tx=0).stop 2024-04-20T10:39:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.851+0000 7fe54a034740 1 -- 172.21.15.42:0/1362822519 shutdown_connections 2024-04-20T10:39:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.851+0000 7fe54a034740 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a0bc1332d0 0x55a0bc1356f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.851+0000 7fe54a034740 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0bc0bc6a0 0x55a0bc0c0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.851+0000 7fe54a034740 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0bc0b3ac0 0x55a0bc0bfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.851+0000 7fe54a034740 1 --2- 172.21.15.42:0/1362822519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a0bc0b31a0 0x55a0bc0c4a20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.851+0000 7fe54a034740 1 -- 172.21.15.42:0/1362822519 >> 172.21.15.42:0/1362822519 conn(0x55a0bc0ad8d0 msgr2=0x55a0bc0ac030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:32.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.851+0000 7fe52e000640 0 client.26089 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:39:32.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.852+0000 7fe54a034740 1 -- 172.21.15.42:0/1362822519 shutdown_connections 2024-04-20T10:39:32.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.852+0000 7fe54a034740 1 -- 172.21.15.42:0/1362822519 wait complete. 2024-04-20T10:39:32.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.852+0000 7fe54a034740 20 client.26089 trim_cache size 0 max 16384 2024-04-20T10:39:32.855 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:39:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (171 inodes in the stack)", 2024-04-20T10:39:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:39:32.855 INFO:teuthology.orchestra.run.smithi042.stdout: "98a2be70-7f71-4a57-befe-368fa414aeaf": { 2024-04-20T10:39:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:39:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "98a2be70-7f71-4a57-befe-368fa414aeaf", 2024-04-20T10:39:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:39:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:39:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:39:32.856 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:39:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.852+0000 7fe547200640 1 -- 172.21.15.42:0/1203131519 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe5240a3d20 msgr2=0x7fe5240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.852+0000 7fe547200640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe5240a3d20 0x7fe5240a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe52802c840 tx=0x7fe528005fd0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.852+0000 7fe547200640 1 -- 172.21.15.42:0/1203131519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe54019a850 msgr2=0x7fe54019ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:39:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.852+0000 7fe547200640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe54019a850 0x7fe54019ad00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe53800ec50 tx=0x7fe53800c6a0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.853+0000 7fe547200640 1 -- 172.21.15.42:0/1203131519 shutdown_connections 2024-04-20T10:39:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.853+0000 7fe547200640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe5240a3d20 0x7fe5240a61e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.853+0000 7fe547200640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe54019a850 0x7fe54019ad00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.853+0000 7fe547200640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe540107290 0x7fe5401a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.853+0000 7fe547200640 1 --2- 172.21.15.42:0/1203131519 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe54006a7d0 0x7fe5401a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:39:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.853+0000 7fe547200640 1 -- 172.21.15.42:0/1203131519 >> 172.21.15.42:0/1203131519 conn(0x7fe540076c60 msgr2=0x7fe54010d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:39:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.853+0000 7fe547200640 1 -- 172.21.15.42:0/1203131519 shutdown_connections 2024-04-20T10:39:32.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:39:32.853+0000 7fe547200640 1 -- 172.21.15.42:0/1203131519 wait complete. 2024-04-20T10:39:32.871 INFO:tasks.cephfs.filesystem:scrub status for tag:98a2be70-7f71-4a57-befe-368fa414aeaf - {'path': '/', 'tag': '98a2be70-7f71-4a57-befe-368fa414aeaf', 'options': 'recursive,force'} 2024-04-20T10:39:34.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:33 smithi162 ceph-mon[21854]: pgmap v169: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 15 MiB/s wr, 2.43k op/s 2024-04-20T10:39:34.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:33 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:39:34.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:33 smithi046 ceph-mon[21574]: pgmap v169: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 15 MiB/s wr, 2.43k op/s 2024-04-20T10:39:34.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:33 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:39:34.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:33 smithi042 ceph-mon[17078]: pgmap v169: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 15 MiB/s wr, 2.43k op/s 2024-04-20T10:39:34.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:33 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:39:36.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:35 smithi162 ceph-mon[21854]: pgmap v170: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 937 op/s 2024-04-20T10:39:36.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:35 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:39:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:35 smithi046 ceph-mon[21574]: pgmap v170: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 937 op/s 2024-04-20T10:39:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:35 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:39:36.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:35 smithi042 ceph-mon[17078]: pgmap v170: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 937 op/s 2024-04-20T10:39:36.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:35 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:39:38.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:37 smithi162 ceph-mon[21854]: pgmap v171: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 965 op/s 2024-04-20T10:39:38.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:37 smithi046 ceph-mon[21574]: pgmap v171: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 965 op/s 2024-04-20T10:39:38.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:37 smithi042 ceph-mon[17078]: pgmap v171: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 965 op/s 2024-04-20T10:39:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:39 smithi162 ceph-mon[21854]: pgmap v172: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 646 op/s 2024-04-20T10:39:40.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:39 smithi046 ceph-mon[21574]: pgmap v172: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 646 op/s 2024-04-20T10:39:40.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:39 smithi042 ceph-mon[17078]: pgmap v172: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 646 op/s 2024-04-20T10:39:42.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:41 smithi046 ceph-mon[21574]: pgmap v173: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-04-20T10:39:42.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:41 smithi042 ceph-mon[17078]: pgmap v173: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-04-20T10:39:42.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:41 smithi162 ceph-mon[21854]: pgmap v173: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-04-20T10:39:43.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:42 smithi046 ceph-mon[21574]: pgmap v174: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 267 op/s 2024-04-20T10:39:43.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:42 smithi042 ceph-mon[17078]: pgmap v174: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 267 op/s 2024-04-20T10:39:43.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:42 smithi162 ceph-mon[21854]: pgmap v174: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 267 op/s 2024-04-20T10:39:45.804 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:45 smithi046 ceph-mon[21574]: pgmap v175: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-04-20T10:39:45.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:45 smithi042 ceph-mon[17078]: pgmap v175: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-04-20T10:39:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:45 smithi162 ceph-mon[21854]: pgmap v175: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-04-20T10:39:47.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:47 smithi042 ceph-mon[17078]: pgmap v176: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-04-20T10:39:47.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:47 smithi162 ceph-mon[21854]: pgmap v176: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-04-20T10:39:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:47 smithi046 ceph-mon[21574]: pgmap v176: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-04-20T10:39:49.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:49 smithi042 ceph-mon[17078]: pgmap v177: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-04-20T10:39:49.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:49 smithi162 ceph-mon[21854]: pgmap v177: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-04-20T10:39:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:49 smithi046 ceph-mon[21574]: pgmap v177: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-04-20T10:39:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:51 smithi042 ceph-mon[17078]: pgmap v178: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 3.8 MiB/s wr, 221 op/s 2024-04-20T10:39:51.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:51 smithi162 ceph-mon[21854]: pgmap v178: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 3.8 MiB/s wr, 221 op/s 2024-04-20T10:39:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:51 smithi046 ceph-mon[21574]: pgmap v178: 97 pgs: 97 active+clean; 1.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 3.8 MiB/s wr, 221 op/s 2024-04-20T10:39:53.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:53 smithi042 ceph-mon[17078]: pgmap v179: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 1.36k op/s 2024-04-20T10:39:53.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:53 smithi162 ceph-mon[21854]: pgmap v179: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 1.36k op/s 2024-04-20T10:39:54.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:53 smithi046 ceph-mon[21574]: pgmap v179: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 1.36k op/s 2024-04-20T10:39:55.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:55 smithi042 ceph-mon[17078]: pgmap v180: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 9.6 MiB/s wr, 1.31k op/s 2024-04-20T10:39:55.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:55 smithi162 ceph-mon[21854]: pgmap v180: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 9.6 MiB/s wr, 1.31k op/s 2024-04-20T10:39:56.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:55 smithi046 ceph-mon[21574]: pgmap v180: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 9.6 MiB/s wr, 1.31k op/s 2024-04-20T10:39:57.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:57 smithi042 ceph-mon[17078]: pgmap v181: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.8 MiB/s wr, 1.40k op/s 2024-04-20T10:39:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:57 smithi162 ceph-mon[21854]: pgmap v181: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.8 MiB/s wr, 1.40k op/s 2024-04-20T10:39:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:57 smithi046 ceph-mon[21574]: pgmap v181: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 9.8 MiB/s wr, 1.40k op/s 2024-04-20T10:39:59.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:39:59 smithi042 ceph-mon[17078]: pgmap v182: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.6 MiB/s wr, 1.48k op/s 2024-04-20T10:39:59.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:39:59 smithi162 ceph-mon[21854]: pgmap v182: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.6 MiB/s wr, 1.48k op/s 2024-04-20T10:40:00.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:39:59 smithi046 ceph-mon[21574]: pgmap v182: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.6 MiB/s wr, 1.48k op/s 2024-04-20T10:40:00.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:00 smithi162 ceph-mon[21854]: overall HEALTH_OK 2024-04-20T10:40:01.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:00 smithi046 ceph-mon[21574]: overall HEALTH_OK 2024-04-20T10:40:01.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:00 smithi042 ceph-mon[17078]: overall HEALTH_OK 2024-04-20T10:40:01.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:01 smithi162 ceph-mon[21854]: pgmap v183: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.4 MiB/s wr, 1.32k op/s 2024-04-20T10:40:02.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:01 smithi046 ceph-mon[21574]: pgmap v183: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.4 MiB/s wr, 1.32k op/s 2024-04-20T10:40:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:01 smithi042 ceph-mon[17078]: pgmap v183: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.4 MiB/s wr, 1.32k op/s 2024-04-20T10:40:02.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:40:02.873 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:40:03.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.099+0000 7fc46e800640 1 -- 172.21.15.42:0/2047161746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc468103030 msgr2=0x7fc468103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.099+0000 7fc46e800640 1 --2- 172.21.15.42:0/2047161746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc468103030 0x7fc468103410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009900 tx=0x7fc45c02bcc0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.099+0000 7fc46e800640 1 -- 172.21.15.42:0/2047161746 shutdown_connections 2024-04-20T10:40:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.099+0000 7fc46e800640 1 --2- 172.21.15.42:0/2047161746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc46810c450 0x7fc46810e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.099+0000 7fc46e800640 1 --2- 172.21.15.42:0/2047161746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc468103950 0x7fc46810be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.099+0000 7fc46e800640 1 --2- 172.21.15.42:0/2047161746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc468103030 0x7fc468103410 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.099+0000 7fc46e800640 1 -- 172.21.15.42:0/2047161746 >> 172.21.15.42:0/2047161746 conn(0x7fc4680fc870 msgr2=0x7fc4680fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.099+0000 7fc46e800640 1 -- 172.21.15.42:0/2047161746 shutdown_connections 2024-04-20T10:40:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.099+0000 7fc46e800640 1 -- 172.21.15.42:0/2047161746 wait complete. 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.100+0000 7fc46e800640 1 Processor -- start 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.100+0000 7fc46e800640 1 -- start start 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.100+0000 7fc46e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc468103030 0x7fc4681a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.100+0000 7fc46e800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc468103950 0x7fc4681a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc46810c450 0x7fc46819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc468112280 con 0x7fc46810c450 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc468112100 con 0x7fc468103030 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46e800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc468112400 con 0x7fc468103950 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc468103030 0x7fc4681a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc468103030 0x7fc4681a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53206/0 (socket says 172.21.15.42:53206) 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46d400640 1 -- 172.21.15.42:0/2492082042 learned_addr learned my addr 172.21.15.42:0/2492082042 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46ca00640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc468103950 0x7fc4681a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46de00640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc46810c450 0x7fc46819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46ca00640 1 -- 172.21.15.42:0/2492082042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc468103030 msgr2=0x7fc4681a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46ca00640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc468103030 0x7fc4681a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46ca00640 1 -- 172.21.15.42:0/2492082042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc46810c450 msgr2=0x7fc46819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46ca00640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc46810c450 0x7fc46819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46ca00640 1 -- 172.21.15.42:0/2492082042 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46819b020 con 0x7fc468103950 2024-04-20T10:40:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.101+0000 7fc46de00640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc46810c450 0x7fc46819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:40:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.102+0000 7fc46ca00640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc468103950 0x7fc4681a0be0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc45400e7a0 tx=0x7fc45400ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.102+0000 7fc466000640 1 -- 172.21.15.42:0/2492082042 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45400ccc0 con 0x7fc468103950 2024-04-20T10:40:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.102+0000 7fc466000640 1 -- 172.21.15.42:0/2492082042 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc454010910 con 0x7fc468103950 2024-04-20T10:40:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.102+0000 7fc46e800640 1 -- 172.21.15.42:0/2492082042 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46819b310 con 0x7fc468103950 2024-04-20T10:40:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.102+0000 7fc466000640 1 -- 172.21.15.42:0/2492082042 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454018820 con 0x7fc468103950 2024-04-20T10:40:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.102+0000 7fc46e800640 1 -- 172.21.15.42:0/2492082042 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4681a7620 con 0x7fc468103950 2024-04-20T10:40:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.105+0000 7fc472beb740 1 Processor -- start 2024-04-20T10:40:03.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.105+0000 7fc472beb740 1 -- start start 2024-04-20T10:40:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.106+0000 7fc472beb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7fa380 0x56472e7fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.106+0000 7fc472beb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 0x56472e803340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.106+0000 7fc472beb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e803880 0x56472e805c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.106+0000 7fc472beb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56472e760150 con 0x56472e7fa380 2024-04-20T10:40:03.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.106+0000 7fc472beb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56472e706370 con 0x56472e803880 2024-04-20T10:40:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.106+0000 7fc472beb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56472e75c110 con 0x56472e7faca0 2024-04-20T10:40:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.106+0000 7fc46de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7fa380 0x56472e7fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.106+0000 7fc466000640 1 -- 172.21.15.42:0/2492082042 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc454010ac0 con 0x7fc468103950 2024-04-20T10:40:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.106+0000 7fc46de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7fa380 0x56472e7fa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59632/0 (socket says 172.21.15.42:59632) 2024-04-20T10:40:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.106+0000 7fc46de00640 1 -- 172.21.15.42:0/2270976890 learned_addr learned my addr 172.21.15.42:0/2270976890 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.106+0000 7fc46d400640 1 --2- 172.21.15.42:0/2270976890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 0x56472e803340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.107+0000 7fc46d400640 1 -- 172.21.15.42:0/2270976890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e803880 msgr2=0x56472e805c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.107+0000 7fc46d400640 1 --2- 172.21.15.42:0/2270976890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e803880 0x56472e805c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.107+0000 7fc46d400640 1 -- 172.21.15.42:0/2270976890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7fa380 msgr2=0x56472e7fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.107+0000 7fc46d400640 1 --2- 172.21.15.42:0/2270976890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7fa380 0x56472e7fa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.107+0000 7fc46d400640 1 -- 172.21.15.42:0/2270976890 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472e53c7d0 con 0x56472e7faca0 2024-04-20T10:40:03.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.107+0000 7fc46de00640 1 --2- 172.21.15.42:0/2270976890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7fa380 0x56472e7fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:40:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.107+0000 7fc46d400640 1 --2- 172.21.15.42:0/2270976890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 0x56472e803340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc45c02c2d0 tx=0x7fc45c005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8638ec0f6d428034 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.107+0000 7fc45a600640 1 -- 172.21.15.42:0/2270976890 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c004240 con 0x56472e7faca0 2024-04-20T10:40:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.107+0000 7fc45a600640 1 -- 172.21.15.42:0/2270976890 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c02cdf0 con 0x56472e7faca0 2024-04-20T10:40:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.107+0000 7fc45a600640 1 -- 172.21.15.42:0/2270976890 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0046c0 con 0x56472e7faca0 2024-04-20T10:40:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.108+0000 7fc466000640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc4500a3da0 0x7fc4500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.108+0000 7fc466000640 1 -- 172.21.15.42:0/2492082042 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc4540d0ca0 con 0x7fc468103950 2024-04-20T10:40:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.108+0000 7fc46de00640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc4500a3da0 0x7fc4500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.108+0000 7fc472beb740 1 -- 172.21.15.42:0/2270976890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 msgr2=0x56472e803340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.108+0000 7fc472beb740 1 --2- 172.21.15.42:0/2270976890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 0x56472e803340 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc45c02c2d0 tx=0x7fc45c005de0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.108+0000 7fc472beb740 1 -- 172.21.15.42:0/2270976890 shutdown_connections 2024-04-20T10:40:03.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.109+0000 7fc472beb740 1 --2- 172.21.15.42:0/2270976890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e803880 0x56472e805c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.109+0000 7fc472beb740 1 --2- 172.21.15.42:0/2270976890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 0x56472e803340 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.109+0000 7fc472beb740 1 --2- 172.21.15.42:0/2270976890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7fa380 0x56472e7fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.109+0000 7fc472beb740 1 -- 172.21.15.42:0/2270976890 >> 172.21.15.42:0/2270976890 conn(0x56472e7f4ab0 msgr2=0x56472e7f4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.109+0000 7fc472beb740 1 -- 172.21.15.42:0/2270976890 shutdown_connections 2024-04-20T10:40:03.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.109+0000 7fc472beb740 1 -- 172.21.15.42:0/2270976890 wait complete. 2024-04-20T10:40:03.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.109+0000 7fc46de00640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc4500a3da0 0x7fc4500a6260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc460009d90 tx=0x7fc460009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.109+0000 7fc472beb740 1 -- start start 2024-04-20T10:40:03.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.110+0000 7fc472beb740 1 Processor -- start 2024-04-20T10:40:03.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.110+0000 7fc472beb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7fa380 0x56472e8008c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.110+0000 7fc472beb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 0x56472e800e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.110+0000 7fc472beb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e803880 0x56472e801340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.110+0000 7fc472beb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56472e806610 con 0x56472e803880 2024-04-20T10:40:03.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.110+0000 7fc472beb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56472e719740 con 0x56472e7fa380 2024-04-20T10:40:03.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.110+0000 7fc472beb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56472e806790 con 0x56472e7faca0 2024-04-20T10:40:03.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.110+0000 7fc472beb740 10 client.0 _ll_register_callbacks cb 0x56472e714b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:40:03.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.110+0000 7fc46ca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 0x56472e800e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.110+0000 7fc46ca00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 0x56472e800e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:58290/0 (socket says 172.21.15.42:58290) 2024-04-20T10:40:03.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.110+0000 7fc46ca00640 1 -- 172.21.15.42:0/2422207030 learned_addr learned my addr 172.21.15.42:0/2422207030 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:03.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.111+0000 7fc46d400640 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7fa380 0x56472e8008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.111+0000 7fc46de00640 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e803880 0x56472e801340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.111+0000 7fc46ca00640 1 -- 172.21.15.42:0/2422207030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7fa380 msgr2=0x56472e8008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.111+0000 7fc46ca00640 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7fa380 0x56472e8008c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.111+0000 7fc46ca00640 1 -- 172.21.15.42:0/2422207030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e803880 msgr2=0x56472e801340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.111+0000 7fc46ca00640 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e803880 0x56472e801340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.111+0000 7fc46ca00640 1 -- 172.21.15.42:0/2422207030 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472e4723f0 con 0x56472e7faca0 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.111+0000 7fc46ca00640 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 0x56472e800e00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc45409c8e0 tx=0x7fc45409c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ce4dcc5d72278b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.111+0000 7fc44ea00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.111+0000 7fc44ea00640 1 -- 172.21.15.42:0/2422207030 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4540ce810 con 0x56472e7faca0 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.111+0000 7fc44ea00640 1 -- 172.21.15.42:0/2422207030 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc45409c030 con 0x56472e7faca0 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.111+0000 7fc472beb740 1 -- 172.21.15.42:0/2422207030 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc45c02cdf0 con 0x56472e7faca0 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.112+0000 7fc44ea00640 1 -- 172.21.15.42:0/2422207030 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc4540b6120 con 0x56472e7faca0 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.112+0000 7fc472beb740 10 client.26101 fetch_fsmap learned FSMap version 19 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.112+0000 7fc472beb740 1 -- 172.21.15.42:0/2422207030 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56472e4320f0 con 0x56472e7faca0 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.112+0000 7fc44ea00640 1 -- 172.21.15.42:0/2422207030 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c02cdf0 con 0x56472e7faca0 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.112+0000 7fc44ea00640 1 -- 172.21.15.42:0/2422207030 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc4540cec00 con 0x56472e7faca0 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.112+0000 7fc472beb740 10 client.26101 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:40:03.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.112+0000 7fc472beb740 10 client.26101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.112+0000 7fc472beb740 20 client.26101 populate_metadata read hostname 'smithi042' 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.112+0000 7fc472beb740 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56472e813b10 0x56472e815f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.112+0000 7fc472beb740 4 client.26101 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.112+0000 7fc472beb740 1 -- 172.21.15.42:0/2422207030 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56472e76db70 con 0x56472e813b10 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.113+0000 7fc46d400640 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56472e813b10 0x56472e815f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.113+0000 7fc46d400640 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56472e813b10 0x56472e815f30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bdd2bc71b57d788 server_cookie=7a2cf41351e64db4 in_seq=0 out_seq=0 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.113+0000 7fc44ea00640 10 client.26101 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.114+0000 7fc44ea00640 1 -- 172.21.15.42:0/2422207030 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56472e76db70 con 0x56472e813b10 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.114+0000 7fc44ea00640 10 client.26101 handle_command_reply: tid=0 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.114+0000 7fc472beb740 1 client.26101 shutdown 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.115+0000 7fc472beb740 1 -- 172.21.15.42:0/2422207030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 msgr2=0x56472e800e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.115+0000 7fc472beb740 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 0x56472e800e00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc45409c8e0 tx=0x7fc45409c540 comp rx=0 tx=0).stop 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.115+0000 7fc472beb740 1 -- 172.21.15.42:0/2422207030 shutdown_connections 2024-04-20T10:40:03.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.115+0000 7fc472beb740 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56472e813b10 0x56472e815f30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.115+0000 7fc472beb740 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e803880 0x56472e801340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.115+0000 7fc472beb740 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7faca0 0x56472e800e00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.115+0000 7fc472beb740 1 --2- 172.21.15.42:0/2422207030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7fa380 0x56472e8008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.115+0000 7fc472beb740 1 -- 172.21.15.42:0/2422207030 >> 172.21.15.42:0/2422207030 conn(0x56472e7f4ab0 msgr2=0x56472e7f6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.115+0000 7fc44ea00640 0 client.26101 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:03.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.115+0000 7fc472beb740 1 -- 172.21.15.42:0/2422207030 shutdown_connections 2024-04-20T10:40:03.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.115+0000 7fc472beb740 1 -- 172.21.15.42:0/2422207030 wait complete. 2024-04-20T10:40:03.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.115+0000 7fc472beb740 20 client.26101 trim_cache size 0 max 16384 2024-04-20T10:40:03.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.140+0000 7fc472beb740 1 Processor -- start 2024-04-20T10:40:03.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.140+0000 7fc472beb740 1 -- start start 2024-04-20T10:40:03.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.141+0000 7fc472beb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7fa380 0x56472e80f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.141+0000 7fc472beb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7faca0 0x56472e7f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.141+0000 7fc472beb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e803880 0x56472e800f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.141+0000 7fc472beb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681add50 con 0x56472e803880 2024-04-20T10:40:03.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.141+0000 7fc472beb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681b05d0 con 0x56472e7faca0 2024-04-20T10:40:03.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.141+0000 7fc472beb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681ae3c0 con 0x56472e7fa380 2024-04-20T10:40:03.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.142+0000 7fc46d400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7fa380 0x56472e80f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.143+0000 7fc46d400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7fa380 0x56472e80f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:58306/0 (socket says 172.21.15.42:58306) 2024-04-20T10:40:03.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.143+0000 7fc46d400640 1 -- 172.21.15.42:0/127578894 learned_addr learned my addr 172.21.15.42:0/127578894 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:03.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.143+0000 7fc46de00640 1 --2- 172.21.15.42:0/127578894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e803880 0x56472e800f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.143+0000 7fc46ca00640 1 --2- 172.21.15.42:0/127578894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7faca0 0x56472e7f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.144+0000 7fc46de00640 1 -- 172.21.15.42:0/127578894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7fa380 msgr2=0x56472e80f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.144+0000 7fc46de00640 1 --2- 172.21.15.42:0/127578894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7fa380 0x56472e80f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.144+0000 7fc46de00640 1 -- 172.21.15.42:0/127578894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7faca0 msgr2=0x56472e7f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.144+0000 7fc46de00640 1 --2- 172.21.15.42:0/127578894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7faca0 0x56472e7f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.144+0000 7fc46de00640 1 -- 172.21.15.42:0/127578894 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472e7f5840 con 0x56472e803880 2024-04-20T10:40:03.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.145+0000 7fc46de00640 1 --2- 172.21.15.42:0/127578894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e803880 0x56472e800f90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc460005fd0 tx=0x7fc460016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6ccd3d8e7c28e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.145+0000 7fc459c00640 1 -- 172.21.15.42:0/127578894 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460004b20 con 0x56472e803880 2024-04-20T10:40:03.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.146+0000 7fc459c00640 1 -- 172.21.15.42:0/127578894 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc46001fde0 con 0x56472e803880 2024-04-20T10:40:03.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.146+0000 7fc459c00640 1 -- 172.21.15.42:0/127578894 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460005450 con 0x56472e803880 2024-04-20T10:40:03.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.147+0000 7fc472beb740 1 -- 172.21.15.42:0/127578894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e803880 msgr2=0x56472e800f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.147+0000 7fc472beb740 1 --2- 172.21.15.42:0/127578894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e803880 0x56472e800f90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc460005fd0 tx=0x7fc460016000 comp rx=0 tx=0).stop 2024-04-20T10:40:03.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.147+0000 7fc472beb740 1 -- 172.21.15.42:0/127578894 shutdown_connections 2024-04-20T10:40:03.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.147+0000 7fc472beb740 1 --2- 172.21.15.42:0/127578894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e803880 0x56472e800f90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.147+0000 7fc472beb740 1 --2- 172.21.15.42:0/127578894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7faca0 0x56472e7f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.147+0000 7fc472beb740 1 --2- 172.21.15.42:0/127578894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e7fa380 0x56472e80f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.147+0000 7fc472beb740 1 -- 172.21.15.42:0/127578894 >> 172.21.15.42:0/127578894 conn(0x56472e7f4ab0 msgr2=0x56472e800880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.148+0000 7fc472beb740 1 -- 172.21.15.42:0/127578894 shutdown_connections 2024-04-20T10:40:03.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.148+0000 7fc472beb740 1 -- 172.21.15.42:0/127578894 wait complete. 2024-04-20T10:40:03.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.148+0000 7fc472beb740 1 -- start start 2024-04-20T10:40:03.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.149+0000 7fc472beb740 1 Processor -- start 2024-04-20T10:40:03.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.149+0000 7fc472beb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7fa380 0x56472e80b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.149+0000 7fc472beb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7faca0 0x56472e80bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.149+0000 7fc472beb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e803880 0x56472e883fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.149+0000 7fc472beb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681aea90 con 0x56472e7faca0 2024-04-20T10:40:03.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.149+0000 7fc472beb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681aff00 con 0x56472e7fa380 2024-04-20T10:40:03.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.149+0000 7fc472beb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc4681af160 con 0x56472e803880 2024-04-20T10:40:03.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.149+0000 7fc472beb740 10 client.0 _ll_register_callbacks cb 0x7fc4540056f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:40:03.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc46de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e803880 0x56472e883fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc46d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7fa380 0x56472e80b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc46de00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e803880 0x56472e883fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:58316/0 (socket says 172.21.15.42:58316) 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc46de00640 1 -- 172.21.15.42:0/3758317086 learned_addr learned my addr 172.21.15.42:0/3758317086 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc46ca00640 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7faca0 0x56472e80bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc46de00640 1 -- 172.21.15.42:0/3758317086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7fa380 msgr2=0x56472e80b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc46de00640 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7fa380 0x56472e80b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc46de00640 1 -- 172.21.15.42:0/3758317086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7faca0 msgr2=0x56472e80bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc46de00640 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7faca0 0x56472e80bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc46de00640 1 -- 172.21.15.42:0/3758317086 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472e884500 con 0x56472e803880 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc46ca00640 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7faca0 0x56472e80bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc46de00640 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e803880 0x56472e883fc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc460018040 tx=0x7fc460009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee47147b400aa31a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.150+0000 7fc44ea00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:40:03.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.151+0000 7fc44ea00640 1 -- 172.21.15.42:0/3758317086 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4600055e0 con 0x56472e803880 2024-04-20T10:40:03.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.151+0000 7fc44ea00640 1 -- 172.21.15.42:0/3758317086 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fc460025040 con 0x56472e803880 2024-04-20T10:40:03.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.151+0000 7fc472beb740 1 -- 172.21.15.42:0/3758317086 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc46001fde0 con 0x56472e803880 2024-04-20T10:40:03.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.151+0000 7fc44ea00640 1 -- 172.21.15.42:0/3758317086 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc460005780 con 0x56472e803880 2024-04-20T10:40:03.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.151+0000 7fc472beb740 10 client.26107 fetch_fsmap learned FSMap version 19 2024-04-20T10:40:03.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.151+0000 7fc472beb740 1 -- 172.21.15.42:0/3758317086 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56472e8940c0 con 0x56472e803880 2024-04-20T10:40:03.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.151+0000 7fc44ea00640 1 -- 172.21.15.42:0/3758317086 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc46001fde0 con 0x56472e803880 2024-04-20T10:40:03.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.152+0000 7fc44ea00640 1 -- 172.21.15.42:0/3758317086 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fc460031070 con 0x56472e803880 2024-04-20T10:40:03.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.152+0000 7fc472beb740 10 client.26107 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:40:03.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.152+0000 7fc472beb740 10 client.26107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:40:03.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.152+0000 7fc472beb740 20 client.26107 populate_metadata read hostname 'smithi042' 2024-04-20T10:40:03.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.152+0000 7fc472beb740 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56472e8947b0 0x56472e896bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.152+0000 7fc472beb740 4 client.26107 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:40:03.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.152+0000 7fc472beb740 1 -- 172.21.15.42:0/3758317086 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56472e7679e0 con 0x56472e8947b0 2024-04-20T10:40:03.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.152+0000 7fc46d400640 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56472e8947b0 0x56472e896bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.153+0000 7fc46d400640 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56472e8947b0 0x56472e896bd0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad26766cbbba7fa9 server_cookie=dc24b6f0ec3da6f7 in_seq=0 out_seq=0 2024-04-20T10:40:03.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.153+0000 7fc44ea00640 10 client.26107 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc44ea00640 1 -- 172.21.15.42:0/3758317086 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56472e7679e0 con 0x56472e8947b0 2024-04-20T10:40:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc44ea00640 10 client.26107 handle_command_reply: tid=0 2024-04-20T10:40:03.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc472beb740 1 client.26107 shutdown 2024-04-20T10:40:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc472beb740 1 -- 172.21.15.42:0/3758317086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e803880 msgr2=0x56472e883fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc472beb740 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e803880 0x56472e883fc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc460018040 tx=0x7fc460009760 comp rx=0 tx=0).stop 2024-04-20T10:40:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc472beb740 1 -- 172.21.15.42:0/3758317086 shutdown_connections 2024-04-20T10:40:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc472beb740 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56472e8947b0 0x56472e896bd0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc472beb740 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56472e803880 0x56472e883fc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc44ea00640 0 client.26107 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc472beb740 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56472e7faca0 0x56472e80bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc472beb740 1 --2- 172.21.15.42:0/3758317086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56472e7fa380 0x56472e80b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc472beb740 1 -- 172.21.15.42:0/3758317086 >> 172.21.15.42:0/3758317086 conn(0x56472e7f4ab0 msgr2=0x56472e7f3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc472beb740 1 -- 172.21.15.42:0/3758317086 shutdown_connections 2024-04-20T10:40:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc472beb740 1 -- 172.21.15.42:0/3758317086 wait complete. 2024-04-20T10:40:03.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.156+0000 7fc472beb740 20 client.26107 trim_cache size 0 max 16384 2024-04-20T10:40:03.162 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:40:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:40:03.162 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:40:03.163 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:40:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.157+0000 7fc46e800640 1 -- 172.21.15.42:0/2492082042 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc4500a3da0 msgr2=0x7fc4500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.157+0000 7fc46e800640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc4500a3da0 0x7fc4500a6260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc460009d90 tx=0x7fc460009500 comp rx=0 tx=0).stop 2024-04-20T10:40:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.157+0000 7fc46e800640 1 -- 172.21.15.42:0/2492082042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc468103950 msgr2=0x7fc4681a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.157+0000 7fc46e800640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc468103950 0x7fc4681a0be0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc45400e7a0 tx=0x7fc45400ec60 comp rx=0 tx=0).stop 2024-04-20T10:40:03.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.157+0000 7fc46e800640 1 -- 172.21.15.42:0/2492082042 shutdown_connections 2024-04-20T10:40:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.157+0000 7fc46e800640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc4500a3da0 0x7fc4500a6260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.157+0000 7fc46e800640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc46810c450 0x7fc46819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.157+0000 7fc46e800640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc468103950 0x7fc4681a0be0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.157+0000 7fc46e800640 1 --2- 172.21.15.42:0/2492082042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc468103030 0x7fc4681a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.157+0000 7fc46e800640 1 -- 172.21.15.42:0/2492082042 >> 172.21.15.42:0/2492082042 conn(0x7fc4680fc870 msgr2=0x7fc468100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.157+0000 7fc46e800640 1 -- 172.21.15.42:0/2492082042 shutdown_connections 2024-04-20T10:40:03.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.157+0000 7fc46e800640 1 -- 172.21.15.42:0/2492082042 wait complete. 2024-04-20T10:40:03.176 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:40:03.176 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:40:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.361+0000 7f7f72200640 1 -- 172.21.15.42:0/3902078632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f6c108590 msgr2=0x7f7f6c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.361+0000 7f7f72200640 1 --2- 172.21.15.42:0/3902078632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f6c108590 0x7f7f6c108a10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009930 tx=0x7f7f5802bc30 comp rx=0 tx=0).stop 2024-04-20T10:40:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.361+0000 7f7f72200640 1 -- 172.21.15.42:0/3902078632 shutdown_connections 2024-04-20T10:40:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.361+0000 7f7f72200640 1 --2- 172.21.15.42:0/3902078632 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f6c108f50 0x7f7f6c115410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.361+0000 7f7f72200640 1 --2- 172.21.15.42:0/3902078632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f6c108590 0x7f7f6c108a10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.361+0000 7f7f72200640 1 --2- 172.21.15.42:0/3902078632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f6c10e650 0x7f7f6c10ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.361+0000 7f7f72200640 1 -- 172.21.15.42:0/3902078632 >> 172.21.15.42:0/3902078632 conn(0x7f7f6c075760 msgr2=0x7f7f6c106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.361+0000 7f7f72200640 1 -- 172.21.15.42:0/3902078632 shutdown_connections 2024-04-20T10:40:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.361+0000 7f7f72200640 1 -- 172.21.15.42:0/3902078632 wait complete. 2024-04-20T10:40:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.362+0000 7f7f72200640 1 Processor -- start 2024-04-20T10:40:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.363+0000 7f7f72200640 1 -- start start 2024-04-20T10:40:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.363+0000 7f7f72200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f6c108590 0x7f7f6c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.363+0000 7f7f72200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f6c108f50 0x7f7f6c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.363+0000 7f7f72200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f6c10e650 0x7f7f6c1a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.363+0000 7f7f72200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1189e0 con 0x7f7f6c108590 2024-04-20T10:40:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.363+0000 7f7f72200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c118860 con 0x7f7f6c10e650 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.363+0000 7f7f72200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c118b60 con 0x7f7f6c108f50 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.363+0000 7f7f6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f6c10e650 0x7f7f6c1a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.363+0000 7f7f6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f6c10e650 0x7f7f6c1a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53260/0 (socket says 172.21.15.42:53260) 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.363+0000 7f7f6be00640 1 -- 172.21.15.42:0/3593765007 learned_addr learned my addr 172.21.15.42:0/3593765007 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.363+0000 7f7f6aa00640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f6c108f50 0x7f7f6c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.363+0000 7f7f6b400640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f6c108590 0x7f7f6c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.364+0000 7f7f6be00640 1 -- 172.21.15.42:0/3593765007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f6c108f50 msgr2=0x7f7f6c19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.364+0000 7f7f6be00640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f6c108f50 0x7f7f6c19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.364+0000 7f7f6be00640 1 -- 172.21.15.42:0/3593765007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f6c108590 msgr2=0x7f7f6c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.364+0000 7f7f6be00640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f6c108590 0x7f7f6c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.364+0000 7f7f6be00640 1 -- 172.21.15.42:0/3593765007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c1a3e30 con 0x7f7f6c10e650 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.364+0000 7f7f6b400640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f6c108590 0x7f7f6c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:40:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.364+0000 7f7f6be00640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f6c10e650 0x7f7f6c1a3700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400eb10 tx=0x7f7f6400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.364+0000 7f7f63e00640 1 -- 172.21.15.42:0/3593765007 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64004b20 con 0x7f7f6c10e650 2024-04-20T10:40:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.364+0000 7f7f63e00640 1 -- 172.21.15.42:0/3593765007 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f64004cc0 con 0x7f7f6c10e650 2024-04-20T10:40:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.364+0000 7f7f72200640 1 -- 172.21.15.42:0/3593765007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c1a4120 con 0x7f7f6c10e650 2024-04-20T10:40:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.364+0000 7f7f63e00640 1 -- 172.21.15.42:0/3593765007 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64010760 con 0x7f7f6c10e650 2024-04-20T10:40:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.364+0000 7f7f72200640 1 -- 172.21.15.42:0/3593765007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c1ab9b0 con 0x7f7f6c10e650 2024-04-20T10:40:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.365+0000 7f7f72200640 1 -- 172.21.15.42:0/3593765007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6c0691d0 con 0x7f7f6c10e650 2024-04-20T10:40:03.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.367+0000 7f7f63e00640 1 -- 172.21.15.42:0/3593765007 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7f64005020 con 0x7f7f6c10e650 2024-04-20T10:40:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.369+0000 7f7f63e00640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7f500a3da0 0x7f7f500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.369+0000 7f7f63e00640 1 -- 172.21.15.42:0/3593765007 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f7f640d0f40 con 0x7f7f6c10e650 2024-04-20T10:40:03.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.369+0000 7f7f6b400640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7f500a3da0 0x7f7f500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.370+0000 7f7f6b400640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7f500a3da0 0x7f7f500a6260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009c20 tx=0x7f7f5c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.373+0000 7f7f63e00640 1 -- 172.21.15.42:0/3593765007 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7f6409d580 con 0x7f7f6c10e650 2024-04-20T10:40:03.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.638+0000 7f7f72200640 1 -- 172.21.15.42:0/3593765007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f6c02d5d0 con 0x7f7f6c10e650 2024-04-20T10:40:03.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.639+0000 7f7f63e00640 1 -- 172.21.15.42:0/3593765007 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f7f6409ce80 con 0x7f7f6c10e650 2024-04-20T10:40:03.640 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:40:03.640 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:40:03.641 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:40:03.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.642+0000 7f7f72200640 1 -- 172.21.15.42:0/3593765007 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7f500a3da0 msgr2=0x7f7f500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.642+0000 7f7f72200640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7f500a3da0 0x7f7f500a6260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009c20 tx=0x7f7f5c005f90 comp rx=0 tx=0).stop 2024-04-20T10:40:03.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.642+0000 7f7f72200640 1 -- 172.21.15.42:0/3593765007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f6c10e650 msgr2=0x7f7f6c1a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.642+0000 7f7f72200640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f6c10e650 0x7f7f6c1a3700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400eb10 tx=0x7f7f6400c2d0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.642+0000 7f7f72200640 1 -- 172.21.15.42:0/3593765007 shutdown_connections 2024-04-20T10:40:03.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.642+0000 7f7f72200640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7f500a3da0 0x7f7f500a6260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.642+0000 7f7f72200640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f6c10e650 0x7f7f6c1a3700 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.642+0000 7f7f72200640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f6c108f50 0x7f7f6c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.642+0000 7f7f72200640 1 --2- 172.21.15.42:0/3593765007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f6c108590 0x7f7f6c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.642+0000 7f7f72200640 1 -- 172.21.15.42:0/3593765007 >> 172.21.15.42:0/3593765007 conn(0x7f7f6c075760 msgr2=0x7f7f6c076720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.643+0000 7f7f72200640 1 -- 172.21.15.42:0/3593765007 shutdown_connections 2024-04-20T10:40:03.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.643+0000 7f7f72200640 1 -- 172.21.15.42:0/3593765007 wait complete. 2024-04-20T10:40:03.663 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:40:03.854 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:03 smithi046 ceph-mon[21574]: pgmap v184: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.1 MiB/s wr, 1.35k op/s 2024-04-20T10:40:03.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.888+0000 7fa4f5200640 1 -- 172.21.15.42:0/2368535825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f010e6a0 msgr2=0x7fa4f0110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.888+0000 7fa4f5200640 1 --2- 172.21.15.42:0/2368535825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f010e6a0 0x7fa4f0110ab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009be0 tx=0x7fa4e802bf80 comp rx=0 tx=0).stop 2024-04-20T10:40:03.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.888+0000 7fa4f5200640 1 -- 172.21.15.42:0/2368535825 shutdown_connections 2024-04-20T10:40:03.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.888+0000 7fa4f5200640 1 --2- 172.21.15.42:0/2368535825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f010e6a0 0x7fa4f0110ab0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.888+0000 7fa4f5200640 1 --2- 172.21.15.42:0/2368535825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0074c50 0x7fa4f010e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.888+0000 7fa4f5200640 1 --2- 172.21.15.42:0/2368535825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4f0074330 0x7fa4f0074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.888+0000 7fa4f5200640 1 -- 172.21.15.42:0/2368535825 >> 172.21.15.42:0/2368535825 conn(0x7fa4f00fd930 msgr2=0x7fa4f00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.888+0000 7fa4f5200640 1 -- 172.21.15.42:0/2368535825 shutdown_connections 2024-04-20T10:40:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.888+0000 7fa4f5200640 1 -- 172.21.15.42:0/2368535825 wait complete. 2024-04-20T10:40:03.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.890+0000 7fa4f5200640 1 Processor -- start 2024-04-20T10:40:03.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.890+0000 7fa4f5200640 1 -- start start 2024-04-20T10:40:03.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.890+0000 7fa4f5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0074330 0x7fa4f0101dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.890+0000 7fa4f5200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0074c50 0x7fa4f0102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.890+0000 7fa4f5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4f010e6a0 0x7fa4f0102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.890+0000 7fa4f5200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f0114430 con 0x7fa4f010e6a0 2024-04-20T10:40:03.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.890+0000 7fa4f5200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01142b0 con 0x7fa4f0074330 2024-04-20T10:40:03.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.890+0000 7fa4f5200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01145b0 con 0x7fa4f0074c50 2024-04-20T10:40:03.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.891+0000 7fa4ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0074330 0x7fa4f0101dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.891+0000 7fa4ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0074330 0x7fa4f0101dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53282/0 (socket says 172.21.15.42:53282) 2024-04-20T10:40:03.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.891+0000 7fa4ef400640 1 -- 172.21.15.42:0/241248248 learned_addr learned my addr 172.21.15.42:0/241248248 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:03.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.891+0000 7fa4eea00640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0074c50 0x7fa4f0102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.891+0000 7fa4efe00640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4f010e6a0 0x7fa4f0102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.891+0000 7fa4ef400640 1 -- 172.21.15.42:0/241248248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0074c50 msgr2=0x7fa4f0102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.891+0000 7fa4ef400640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0074c50 0x7fa4f0102300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.891+0000 7fa4ef400640 1 -- 172.21.15.42:0/241248248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4f010e6a0 msgr2=0x7fa4f0102840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.891+0000 7fa4ef400640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4f010e6a0 0x7fa4f0102840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.891+0000 7fa4ef400640 1 -- 172.21.15.42:0/241248248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f01ab050 con 0x7fa4f0074330 2024-04-20T10:40:03.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.891+0000 7fa4efe00640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4f010e6a0 0x7fa4f0102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:40:03.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.891+0000 7fa4ef400640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0074330 0x7fa4f0101dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000e9c0 tx=0x7fa4e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.892+0000 7fa4e7e00640 1 -- 172.21.15.42:0/241248248 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e000ccc0 con 0x7fa4f0074330 2024-04-20T10:40:03.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.892+0000 7fa4e7e00640 1 -- 172.21.15.42:0/241248248 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e0005500 con 0x7fa4f0074330 2024-04-20T10:40:03.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.892+0000 7fa4e7e00640 1 -- 172.21.15.42:0/241248248 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0010690 con 0x7fa4f0074330 2024-04-20T10:40:03.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.892+0000 7fa4f5200640 1 -- 172.21.15.42:0/241248248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f01ab340 con 0x7fa4f0074330 2024-04-20T10:40:03.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.892+0000 7fa4f5200640 1 -- 172.21.15.42:0/241248248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f01ab8e0 con 0x7fa4f0074330 2024-04-20T10:40:03.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.895+0000 7fa4f962e740 1 Processor -- start 2024-04-20T10:40:03.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.895+0000 7fa4e7e00640 1 -- 172.21.15.42:0/241248248 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa4e00046c0 con 0x7fa4f0074330 2024-04-20T10:40:03.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.895+0000 7fa4f962e740 1 -- start start 2024-04-20T10:40:03.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.896+0000 7fa4f962e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d48170 0x55e723d48550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.896+0000 7fa4f962e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 0x55e723d51130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.896+0000 7fa4f962e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d51670 0x55e723d53a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.896+0000 7fa4f962e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e723cad050 con 0x55e723d51670 2024-04-20T10:40:03.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.896+0000 7fa4f962e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e723c53490 con 0x55e723d48a90 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.896+0000 7fa4f962e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55e723ca9010 con 0x55e723d48170 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.896+0000 7fa4ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 0x55e723d51130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.896+0000 7fa4eea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d51670 0x55e723d53a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.896+0000 7fa4efe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d48170 0x55e723d48550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.896+0000 7fa4ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 0x55e723d51130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53292/0 (socket says 172.21.15.42:53292) 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.896+0000 7fa4ef400640 1 -- 172.21.15.42:0/2435524230 learned_addr learned my addr 172.21.15.42:0/2435524230 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.896+0000 7fa4efe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d48170 0x55e723d48550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:58362/0 (socket says 172.21.15.42:58362) 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.897+0000 7fa4ef400640 1 -- 172.21.15.42:0/2435524230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d48170 msgr2=0x55e723d48550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.897+0000 7fa4ef400640 1 --2- 172.21.15.42:0/2435524230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d48170 0x55e723d48550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.897+0000 7fa4ef400640 1 -- 172.21.15.42:0/2435524230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d51670 msgr2=0x55e723d53a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.897+0000 7fa4ef400640 1 --2- 172.21.15.42:0/2435524230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d51670 0x55e723d53a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.897+0000 7fa4ef400640 1 -- 172.21.15.42:0/2435524230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e723c5b070 con 0x55e723d48a90 2024-04-20T10:40:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.897+0000 7fa4ef400640 1 --2- 172.21.15.42:0/2435524230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 0x55e723d51130 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00ce0e0 tx=0x7fa4e00d4a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b1b55359281c1e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.897+0000 7fa4e4c00640 1 -- 172.21.15.42:0/2435524230 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00d98b0 con 0x55e723d48a90 2024-04-20T10:40:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.897+0000 7fa4e4c00640 1 -- 172.21.15.42:0/2435524230 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e00d7070 con 0x55e723d48a90 2024-04-20T10:40:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.897+0000 7fa4e4c00640 1 -- 172.21.15.42:0/2435524230 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00dd3c0 con 0x55e723d48a90 2024-04-20T10:40:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.897+0000 7fa4e7e00640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa4d40a3da0 0x7fa4d40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.898+0000 7fa4e7e00640 1 -- 172.21.15.42:0/241248248 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fa4e00d0ab0 con 0x7fa4f0074330 2024-04-20T10:40:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.898+0000 7fa4efe00640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa4d40a3da0 0x7fa4d40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.898+0000 7fa4f962e740 1 -- 172.21.15.42:0/2435524230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 msgr2=0x55e723d51130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.898+0000 7fa4f962e740 1 --2- 172.21.15.42:0/2435524230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 0x55e723d51130 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00ce0e0 tx=0x7fa4e00d4a90 comp rx=0 tx=0).stop 2024-04-20T10:40:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.898+0000 7fa4f962e740 1 -- 172.21.15.42:0/2435524230 shutdown_connections 2024-04-20T10:40:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.898+0000 7fa4f962e740 1 --2- 172.21.15.42:0/2435524230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d51670 0x55e723d53a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.899+0000 7fa4f962e740 1 --2- 172.21.15.42:0/2435524230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 0x55e723d51130 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.899+0000 7fa4f962e740 1 --2- 172.21.15.42:0/2435524230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d48170 0x55e723d48550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.899+0000 7fa4f962e740 1 -- 172.21.15.42:0/2435524230 >> 172.21.15.42:0/2435524230 conn(0x55e723d428a0 msgr2=0x55e723d42c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.899+0000 7fa4efe00640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa4d40a3da0 0x7fa4d40a6260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4e802c4c0 tx=0x7fa4e8009280 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.899+0000 7fa4f962e740 1 -- 172.21.15.42:0/2435524230 shutdown_connections 2024-04-20T10:40:03.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.899+0000 7fa4f962e740 1 -- 172.21.15.42:0/2435524230 wait complete. 2024-04-20T10:40:03.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.899+0000 7fa4f962e740 1 -- start start 2024-04-20T10:40:03.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.899+0000 7fa4f962e740 1 Processor -- start 2024-04-20T10:40:03.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:03 smithi162 ceph-mon[21854]: pgmap v184: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.1 MiB/s wr, 1.35k op/s 2024-04-20T10:40:03.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4f962e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 0x55e723d4e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4f962e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48a90 0x55e723d4ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4f962e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 0x55e723d4f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4f962e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e723d54460 con 0x55e723d48a90 2024-04-20T10:40:03.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4f962e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e723c66860 con 0x55e723d48170 2024-04-20T10:40:03.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4f962e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55e723d545e0 con 0x55e723d51670 2024-04-20T10:40:03.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4f962e740 10 client.0 _ll_register_callbacks cb 0x55e723c34fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:40:03.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 0x55e723d4e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 0x55e723d4e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53302/0 (socket says 172.21.15.42:53302) 2024-04-20T10:40:03.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4ef400640 1 -- 172.21.15.42:0/2172715650 learned_addr learned my addr 172.21.15.42:0/2172715650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:03.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4eea00640 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48a90 0x55e723d4ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4efe00640 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 0x55e723d4f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4ef400640 1 -- 172.21.15.42:0/2172715650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 msgr2=0x55e723d4f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4ef400640 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 0x55e723d4f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4ef400640 1 -- 172.21.15.42:0/2172715650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48a90 msgr2=0x55e723d4ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4ef400640 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48a90 0x55e723d4ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.900+0000 7fa4ef400640 1 -- 172.21.15.42:0/2172715650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e723c6a950 con 0x55e723d48170 2024-04-20T10:40:03.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.901+0000 7fa4ef400640 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 0x55e723d4e6f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00d4f90 tx=0x7fa4e0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5836a19c747f2dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.901+0000 7fa4d9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:40:03.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.901+0000 7fa4d9600640 1 -- 172.21.15.42:0/2172715650 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0098c90 con 0x55e723d48170 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.901+0000 7fa4d9600640 1 -- 172.21.15.42:0/2172715650 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fa4e00de420 con 0x55e723d48170 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.901+0000 7fa4f962e740 1 -- 172.21.15.42:0/2172715650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e00d7070 con 0x55e723d48170 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.901+0000 7fa4d9600640 1 -- 172.21.15.42:0/2172715650 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa4e00056b0 con 0x55e723d48170 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.901+0000 7fa4f962e740 10 client.26111 fetch_fsmap learned FSMap version 19 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.901+0000 7fa4f962e740 1 -- 172.21.15.42:0/2172715650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e723c38820 con 0x55e723d48170 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.901+0000 7fa4d9600640 1 -- 172.21.15.42:0/2172715650 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e00d7070 con 0x55e723d48170 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.902+0000 7fa4d9600640 1 -- 172.21.15.42:0/2172715650 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fa4e009cc50 con 0x55e723d48170 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.902+0000 7fa4f962e740 10 client.26111 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.902+0000 7fa4f962e740 10 client.26111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.902+0000 7fa4f962e740 20 client.26111 populate_metadata read hostname 'smithi042' 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.902+0000 7fa4f962e740 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e723d61570 0x55e723d63960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.902+0000 7fa4f962e740 4 client.26111 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.902+0000 7fa4f962e740 1 -- 172.21.15.42:0/2172715650 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e723cbaa70 con 0x55e723d61570 2024-04-20T10:40:03.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.902+0000 7fa4eea00640 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e723d61570 0x55e723d63960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.903+0000 7fa4eea00640 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e723d61570 0x55e723d63960 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe3c076bb550342 server_cookie=5bcdc5c6d5b1654a in_seq=0 out_seq=0 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.903+0000 7fa4d9600640 10 client.26111 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4d9600640 1 -- 172.21.15.42:0/2172715650 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e723cbaa70 con 0x55e723d61570 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4d9600640 10 client.26111 handle_command_reply: tid=0 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4f962e740 1 client.26111 shutdown 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4f962e740 1 -- 172.21.15.42:0/2172715650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 msgr2=0x55e723d4e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4f962e740 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 0x55e723d4e6f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00d4f90 tx=0x7fa4e0009f90 comp rx=0 tx=0).stop 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4f962e740 1 -- 172.21.15.42:0/2172715650 shutdown_connections 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4f962e740 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e723d61570 0x55e723d63960 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4f962e740 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 0x55e723d4f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4f962e740 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48a90 0x55e723d4ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4f962e740 1 --2- 172.21.15.42:0/2172715650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 0x55e723d4e6f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4d9600640 0 client.26111 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4f962e740 1 -- 172.21.15.42:0/2172715650 >> 172.21.15.42:0/2172715650 conn(0x55e723d428a0 msgr2=0x55e723d445f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4f962e740 1 -- 172.21.15.42:0/2172715650 shutdown_connections 2024-04-20T10:40:03.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.904+0000 7fa4f962e740 1 -- 172.21.15.42:0/2172715650 wait complete. 2024-04-20T10:40:03.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.905+0000 7fa4f962e740 20 client.26111 trim_cache size 0 max 16384 2024-04-20T10:40:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.922+0000 7fa4f962e740 1 Processor -- start 2024-04-20T10:40:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.923+0000 7fa4f962e740 1 -- start start 2024-04-20T10:40:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.923+0000 7fa4f962e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48170 0x55e723d50e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.923+0000 7fa4f962e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 0x55e723d42c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.923+0000 7fa4f962e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 0x55e723d66980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.923+0000 7fa4f962e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01b21c0 con 0x55e723d48170 2024-04-20T10:40:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.923+0000 7fa4f962e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01b4a40 con 0x55e723d48a90 2024-04-20T10:40:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.923+0000 7fa4f962e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01b2830 con 0x55e723d51670 2024-04-20T10:40:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.923+0000 7fa4eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 0x55e723d42c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.923+0000 7fa4ef400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48170 0x55e723d50e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.923+0000 7fa4eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 0x55e723d42c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53306/0 (socket says 172.21.15.42:53306) 2024-04-20T10:40:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.923+0000 7fa4efe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 0x55e723d66980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.923+0000 7fa4eea00640 1 -- 172.21.15.42:0/526628280 learned_addr learned my addr 172.21.15.42:0/526628280 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.924+0000 7fa4eea00640 1 -- 172.21.15.42:0/526628280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 msgr2=0x55e723d66980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.924+0000 7fa4eea00640 1 --2- 172.21.15.42:0/526628280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 0x55e723d66980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.924+0000 7fa4eea00640 1 -- 172.21.15.42:0/526628280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48170 msgr2=0x55e723d50e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.924+0000 7fa4eea00640 1 --2- 172.21.15.42:0/526628280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48170 0x55e723d50e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.924+0000 7fa4eea00640 1 -- 172.21.15.42:0/526628280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e723d4d6a0 con 0x55e723d48a90 2024-04-20T10:40:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.924+0000 7fa4ef400640 1 --2- 172.21.15.42:0/526628280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48170 0x55e723d50e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:40:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.924+0000 7fa4eea00640 1 --2- 172.21.15.42:0/526628280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 0x55e723d42c80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc005d00 tx=0x7fa4dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a58b745e36f1f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.924+0000 7fa4dbe00640 1 -- 172.21.15.42:0/526628280 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc025070 con 0x55e723d48a90 2024-04-20T10:40:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.924+0000 7fa4dbe00640 1 -- 172.21.15.42:0/526628280 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4dc003d20 con 0x55e723d48a90 2024-04-20T10:40:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.924+0000 7fa4dbe00640 1 -- 172.21.15.42:0/526628280 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc010a00 con 0x55e723d48a90 2024-04-20T10:40:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.925+0000 7fa4f962e740 1 -- 172.21.15.42:0/526628280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 msgr2=0x55e723d42c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.925+0000 7fa4f962e740 1 --2- 172.21.15.42:0/526628280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 0x55e723d42c80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc005d00 tx=0x7fa4dc0046c0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.925+0000 7fa4f962e740 1 -- 172.21.15.42:0/526628280 shutdown_connections 2024-04-20T10:40:03.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.925+0000 7fa4f962e740 1 --2- 172.21.15.42:0/526628280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 0x55e723d66980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.925+0000 7fa4f962e740 1 --2- 172.21.15.42:0/526628280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48a90 0x55e723d42c80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.925+0000 7fa4f962e740 1 --2- 172.21.15.42:0/526628280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48170 0x55e723d50e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.925+0000 7fa4f962e740 1 -- 172.21.15.42:0/526628280 >> 172.21.15.42:0/526628280 conn(0x55e723d428a0 msgr2=0x55e723d41960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.925+0000 7fa4f962e740 1 -- 172.21.15.42:0/526628280 shutdown_connections 2024-04-20T10:40:03.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.926+0000 7fa4f962e740 1 -- 172.21.15.42:0/526628280 wait complete. 2024-04-20T10:40:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.926+0000 7fa4f962e740 1 -- start start 2024-04-20T10:40:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.926+0000 7fa4f962e740 1 Processor -- start 2024-04-20T10:40:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.926+0000 7fa4f962e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 0x55e723d646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4f962e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48a90 0x55e723d64be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4f962e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 0x55e723d65140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4f962e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01b2f00 con 0x55e723d48a90 2024-04-20T10:40:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4f962e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01b4370 con 0x55e723d48170 2024-04-20T10:40:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4f962e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01b35d0 con 0x55e723d51670 2024-04-20T10:40:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4f962e740 10 client.0 _ll_register_callbacks cb 0x7fa4e00d4910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:40:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 0x55e723d646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 0x55e723d646a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53320/0 (socket says 172.21.15.42:53320) 2024-04-20T10:40:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4ef400640 1 -- 172.21.15.42:0/877009909 learned_addr learned my addr 172.21.15.42:0/877009909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4eea00640 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48a90 0x55e723d64be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4efe00640 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 0x55e723d65140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4ef400640 1 -- 172.21.15.42:0/877009909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 msgr2=0x55e723d65140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4ef400640 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 0x55e723d65140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4ef400640 1 -- 172.21.15.42:0/877009909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48a90 msgr2=0x55e723d64be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4ef400640 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48a90 0x55e723d64be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4ef400640 1 -- 172.21.15.42:0/877009909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7239bf3f0 con 0x55e723d48170 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4eea00640 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48a90 0x55e723d64be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4ef400640 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 0x55e723d646a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0099370 tx=0x7fa4e00d49c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec598e8125d2f8dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.927+0000 7fa4d9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.928+0000 7fa4d9600640 1 -- 172.21.15.42:0/877009909 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00b7020 con 0x55e723d48170 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.928+0000 7fa4d9600640 1 -- 172.21.15.42:0/877009909 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fa4e00d4250 con 0x55e723d48170 2024-04-20T10:40:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.928+0000 7fa4f962e740 1 -- 172.21.15.42:0/877009909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4dc003d20 con 0x55e723d48170 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.928+0000 7fa4d9600640 1 -- 172.21.15.42:0/877009909 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa4e009c350 con 0x55e723d48170 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.928+0000 7fa4f962e740 10 client.26117 fetch_fsmap learned FSMap version 19 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.928+0000 7fa4f962e740 1 -- 172.21.15.42:0/877009909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e723a897d0 con 0x55e723d48170 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.928+0000 7fa4d9600640 1 -- 172.21.15.42:0/877009909 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4dc003d20 con 0x55e723d48170 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.928+0000 7fa4d9600640 1 -- 172.21.15.42:0/877009909 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fa4e00a51d0 con 0x55e723d48170 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.928+0000 7fa4f962e740 10 client.26117 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.928+0000 7fa4f962e740 10 client.26117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.928+0000 7fa4f962e740 20 client.26117 populate_metadata read hostname 'smithi042' 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.928+0000 7fa4f962e740 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e723dc7d00 0x55e723dca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.929+0000 7fa4f962e740 4 client.26117 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.929+0000 7fa4f962e740 1 -- 172.21.15.42:0/877009909 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e723cb48e0 con 0x55e723dc7d00 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.929+0000 7fa4eea00640 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e723dc7d00 0x55e723dca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.929+0000 7fa4eea00640 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e723dc7d00 0x55e723dca0f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cfcf4b6d259c7e1 server_cookie=274318872ef2d798 in_seq=0 out_seq=0 2024-04-20T10:40:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.929+0000 7fa4d9600640 10 client.26117 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.930+0000 7fa4d9600640 1 -- 172.21.15.42:0/877009909 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e723cb48e0 con 0x55e723dc7d00 2024-04-20T10:40:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.930+0000 7fa4d9600640 10 client.26117 handle_command_reply: tid=0 2024-04-20T10:40:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.930+0000 7fa4f962e740 1 client.26117 shutdown 2024-04-20T10:40:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.930+0000 7fa4f962e740 1 -- 172.21.15.42:0/877009909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 msgr2=0x55e723d646a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.930+0000 7fa4f962e740 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 0x55e723d646a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0099370 tx=0x7fa4e00d49c0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.930+0000 7fa4f962e740 1 -- 172.21.15.42:0/877009909 shutdown_connections 2024-04-20T10:40:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.930+0000 7fa4f962e740 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e723dc7d00 0x55e723dca0f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.930+0000 7fa4f962e740 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e723d51670 0x55e723d65140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.930+0000 7fa4f962e740 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e723d48a90 0x55e723d64be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.930+0000 7fa4f962e740 1 --2- 172.21.15.42:0/877009909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e723d48170 0x55e723d646a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.930+0000 7fa4d9600640 0 client.26117 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.930+0000 7fa4f962e740 1 -- 172.21.15.42:0/877009909 >> 172.21.15.42:0/877009909 conn(0x55e723d428a0 msgr2=0x55e723d41960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.931+0000 7fa4f962e740 1 -- 172.21.15.42:0/877009909 shutdown_connections 2024-04-20T10:40:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.931+0000 7fa4f962e740 1 -- 172.21.15.42:0/877009909 wait complete. 2024-04-20T10:40:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.931+0000 7fa4f962e740 20 client.26117 trim_cache size 0 max 16384 2024-04-20T10:40:03.935 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:40:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.931+0000 7fa4f5200640 1 -- 172.21.15.42:0/241248248 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa4d40a3da0 msgr2=0x7fa4d40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.931+0000 7fa4f5200640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa4d40a3da0 0x7fa4d40a6260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4e802c4c0 tx=0x7fa4e8009280 comp rx=0 tx=0).stop 2024-04-20T10:40:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.931+0000 7fa4f5200640 1 -- 172.21.15.42:0/241248248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0074330 msgr2=0x7fa4f0101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.931+0000 7fa4f5200640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0074330 0x7fa4f0101dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000e9c0 tx=0x7fa4e000ee80 comp rx=0 tx=0).stop 2024-04-20T10:40:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.932+0000 7fa4f5200640 1 -- 172.21.15.42:0/241248248 shutdown_connections 2024-04-20T10:40:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.932+0000 7fa4f5200640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa4d40a3da0 0x7fa4d40a6260 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.932+0000 7fa4f5200640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4f010e6a0 0x7fa4f0102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.932+0000 7fa4f5200640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0074c50 0x7fa4f0102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.932+0000 7fa4f5200640 1 --2- 172.21.15.42:0/241248248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0074330 0x7fa4f0101dc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.932+0000 7fa4f5200640 1 -- 172.21.15.42:0/241248248 >> 172.21.15.42:0/241248248 conn(0x7fa4f00fd930 msgr2=0x7fa4f00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.932+0000 7fa4f5200640 1 -- 172.21.15.42:0/241248248 shutdown_connections 2024-04-20T10:40:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:03.932+0000 7fa4f5200640 1 -- 172.21.15.42:0/241248248 wait complete. 2024-04-20T10:40:03.947 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:03 smithi042 ceph-mon[17078]: pgmap v184: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.1 MiB/s wr, 1.35k op/s 2024-04-20T10:40:04.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:04 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3593765007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:40:04.951 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:40:04.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:40:04.951 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:40:05.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:04 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3593765007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:40:05.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:04 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3593765007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:40:05.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.185+0000 7f2e46600640 1 -- 172.21.15.42:0/1308119016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e40101560 msgr2=0x7f2e4010e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.185+0000 7f2e46600640 1 --2- 172.21.15.42:0/1308119016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e40101560 0x7f2e4010e220 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009900 tx=0x7f2e2c02bc30 comp rx=0 tx=0).stop 2024-04-20T10:40:05.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.185+0000 7f2e46600640 1 -- 172.21.15.42:0/1308119016 shutdown_connections 2024-04-20T10:40:05.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.185+0000 7f2e46600640 1 --2- 172.21.15.42:0/1308119016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e4010e760 0x7f2e40110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.185+0000 7f2e46600640 1 --2- 172.21.15.42:0/1308119016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e40101560 0x7f2e4010e220 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.185+0000 7f2e46600640 1 --2- 172.21.15.42:0/1308119016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e40100bb0 0x7f2e40100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.185+0000 7f2e46600640 1 -- 172.21.15.42:0/1308119016 >> 172.21.15.42:0/1308119016 conn(0x7f2e400fc7d0 msgr2=0x7f2e400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:05.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.186+0000 7f2e46600640 1 -- 172.21.15.42:0/1308119016 shutdown_connections 2024-04-20T10:40:05.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.186+0000 7f2e46600640 1 -- 172.21.15.42:0/1308119016 wait complete. 2024-04-20T10:40:05.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.187+0000 7f2e46600640 1 Processor -- start 2024-04-20T10:40:05.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.187+0000 7f2e46600640 1 -- start start 2024-04-20T10:40:05.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.187+0000 7f2e46600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e40100bb0 0x7f2e401a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e46600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e40101560 0x7f2e401a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e46600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4010e760 0x7f2e4019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e46600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40114430 con 0x7f2e40100bb0 2024-04-20T10:40:05.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e46600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401142b0 con 0x7f2e4010e760 2024-04-20T10:40:05.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e46600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401145b0 con 0x7f2e40101560 2024-04-20T10:40:05.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4010e760 0x7f2e4019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e3fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e40101560 0x7f2e401a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e45200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e40100bb0 0x7f2e401a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4010e760 0x7f2e4019a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53352/0 (socket says 172.21.15.42:53352) 2024-04-20T10:40:05.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e45c00640 1 -- 172.21.15.42:0/1618519616 learned_addr learned my addr 172.21.15.42:0/1618519616 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:05.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e3fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e40101560 0x7f2e401a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:58400/0 (socket says 172.21.15.42:58400) 2024-04-20T10:40:05.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e45c00640 1 -- 172.21.15.42:0/1618519616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e40101560 msgr2=0x7f2e401a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e45c00640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e40101560 0x7f2e401a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e45c00640 1 -- 172.21.15.42:0/1618519616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e40100bb0 msgr2=0x7f2e401a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e45c00640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e40100bb0 0x7f2e401a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e45c00640 1 -- 172.21.15.42:0/1618519616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4019afa0 con 0x7f2e4010e760 2024-04-20T10:40:05.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.188+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e40101560 0x7f2e401a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:40:05.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.189+0000 7f2e45c00640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4010e760 0x7f2e4019a7d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800eb40 tx=0x7f2e3800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:05.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.189+0000 7f2e3d600640 1 -- 172.21.15.42:0/1618519616 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38004b20 con 0x7f2e4010e760 2024-04-20T10:40:05.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.189+0000 7f2e3d600640 1 -- 172.21.15.42:0/1618519616 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e38004cc0 con 0x7f2e4010e760 2024-04-20T10:40:05.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.189+0000 7f2e46600640 1 -- 172.21.15.42:0/1618519616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4019b290 con 0x7f2e4010e760 2024-04-20T10:40:05.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.189+0000 7f2e3d600640 1 -- 172.21.15.42:0/1618519616 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38010850 con 0x7f2e4010e760 2024-04-20T10:40:05.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.189+0000 7f2e46600640 1 -- 172.21.15.42:0/1618519616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e401a74b0 con 0x7f2e4010e760 2024-04-20T10:40:05.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.193+0000 7f2e4a96e740 1 Processor -- start 2024-04-20T10:40:05.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.194+0000 7f2e3d600640 1 -- 172.21.15.42:0/1618519616 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2e38005020 con 0x7f2e4010e760 2024-04-20T10:40:05.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.195+0000 7f2e4a96e740 1 -- start start 2024-04-20T10:40:05.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.195+0000 7f2e4a96e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc2182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.195+0000 7f2e4a96e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc2298c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.195+0000 7f2e4a96e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc229e00 0x55f6dc22c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.195+0000 7f2e4a96e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f6dc17cd20 con 0x55f6dc217f00 2024-04-20T10:40:05.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.195+0000 7f2e4a96e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f6dc1233e0 con 0x55f6dc2188b0 2024-04-20T10:40:05.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.195+0000 7f2e4a96e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f6dc178ce0 con 0x55f6dc229e00 2024-04-20T10:40:05.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.195+0000 7f2e45200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc2298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.195+0000 7f2e45200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc2298c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53364/0 (socket says 172.21.15.42:53364) 2024-04-20T10:40:05.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.195+0000 7f2e45200640 1 -- 172.21.15.42:0/3079026568 learned_addr learned my addr 172.21.15.42:0/3079026568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:05.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.195+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/3079026568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc2182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.196+0000 7f2e45200640 1 -- 172.21.15.42:0/3079026568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc229e00 msgr2=0x55f6dc22c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.196+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/3079026568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc229e00 0x55f6dc22c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.196+0000 7f2e45200640 1 --2- 172.21.15.42:0/3079026568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc229e00 0x55f6dc22c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.196+0000 7f2e45200640 1 -- 172.21.15.42:0/3079026568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 msgr2=0x55f6dc2182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.196+0000 7f2e45200640 1 --2- 172.21.15.42:0/3079026568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc2182e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.196+0000 7f2e45200640 1 -- 172.21.15.42:0/3079026568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6dc142980 con 0x55f6dc2188b0 2024-04-20T10:40:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.196+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/3079026568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc229e00 0x55f6dc22c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:40:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.196+0000 7f2e45200640 1 --2- 172.21.15.42:0/3079026568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc2298c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009d20 tx=0x7f2e34011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b85ce0bdd1dcfb39 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.196+0000 7f2e3d600640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2e280a3d20 0x7f2e280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.197+0000 7f2e32000640 1 -- 172.21.15.42:0/3079026568 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34004230 con 0x55f6dc2188b0 2024-04-20T10:40:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.197+0000 7f2e32000640 1 -- 172.21.15.42:0/3079026568 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e340043d0 con 0x55f6dc2188b0 2024-04-20T10:40:05.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.197+0000 7f2e32000640 1 -- 172.21.15.42:0/3079026568 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34019930 con 0x55f6dc2188b0 2024-04-20T10:40:05.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.197+0000 7f2e3d600640 1 -- 172.21.15.42:0/1618519616 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f2e380d0fe0 con 0x7f2e4010e760 2024-04-20T10:40:05.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.197+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2e280a3d20 0x7f2e280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.198+0000 7f2e4a96e740 1 -- 172.21.15.42:0/3079026568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 msgr2=0x55f6dc2298c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.198+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/3079026568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc2298c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009d20 tx=0x7f2e34011810 comp rx=0 tx=0).stop 2024-04-20T10:40:05.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.198+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2e280a3d20 0x7f2e280a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c000c00 tx=0x7f2e2c005fd0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:05.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.198+0000 7f2e4a96e740 1 -- 172.21.15.42:0/3079026568 shutdown_connections 2024-04-20T10:40:05.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.198+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/3079026568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc229e00 0x55f6dc22c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.198+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/3079026568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc2298c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.198+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/3079026568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc2182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.198+0000 7f2e4a96e740 1 -- 172.21.15.42:0/3079026568 >> 172.21.15.42:0/3079026568 conn(0x55f6dc212820 msgr2=0x55f6dc212c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:05.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.198+0000 7f2e4a96e740 1 -- 172.21.15.42:0/3079026568 shutdown_connections 2024-04-20T10:40:05.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.198+0000 7f2e4a96e740 1 -- 172.21.15.42:0/3079026568 wait complete. 2024-04-20T10:40:05.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.198+0000 7f2e4a96e740 1 -- start start 2024-04-20T10:40:05.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.199+0000 7f2e4a96e740 1 Processor -- start 2024-04-20T10:40:05.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.199+0000 7f2e4a96e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc226e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.199+0000 7f2e4a96e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc2188b0 0x55f6dc227390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.199+0000 7f2e4a96e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc229e00 0x55f6dc2278d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.199+0000 7f2e4a96e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f6dc22cbc0 con 0x55f6dc217f00 2024-04-20T10:40:05.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.199+0000 7f2e4a96e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f6dc1367a0 con 0x55f6dc229e00 2024-04-20T10:40:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.199+0000 7f2e4a96e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f6dc22cd40 con 0x55f6dc2188b0 2024-04-20T10:40:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.199+0000 7f2e4a96e740 10 client.0 _ll_register_callbacks cb 0x55f6dc138b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:40:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc229e00 0x55f6dc2278d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc229e00 0x55f6dc2278d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53372/0 (socket says 172.21.15.42:53372) 2024-04-20T10:40:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e45c00640 1 -- 172.21.15.42:0/2968322496 learned_addr learned my addr 172.21.15.42:0/2968322496 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc2188b0 0x55f6dc227390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e45200640 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc226e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e45c00640 1 -- 172.21.15.42:0/2968322496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc2188b0 msgr2=0x55f6dc227390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e45c00640 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc2188b0 0x55f6dc227390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e45c00640 1 -- 172.21.15.42:0/2968322496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 msgr2=0x55f6dc226e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e45c00640 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc226e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e45c00640 1 -- 172.21.15.42:0/2968322496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6dbf577e0 con 0x55f6dc229e00 2024-04-20T10:40:05.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e45c00640 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc229e00 0x55f6dc2278d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2e380a5660 tx=0x7f2e3809c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46baa14ff27fe046 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:05.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e26a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:40:05.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.200+0000 7f2e26a00640 1 -- 172.21.15.42:0/2968322496 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380b5020 con 0x55f6dc229e00 2024-04-20T10:40:05.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e26a00640 1 -- 172.21.15.42:0/2968322496 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f2e380b6220 con 0x55f6dc229e00 2024-04-20T10:40:05.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2968322496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e340043d0 con 0x55f6dc229e00 2024-04-20T10:40:05.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e26a00640 1 -- 172.21.15.42:0/2968322496 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2e380a12d0 con 0x55f6dc229e00 2024-04-20T10:40:05.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e4a96e740 10 client.26129 fetch_fsmap learned FSMap version 19 2024-04-20T10:40:05.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2968322496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f6dbfb7f70 con 0x55f6dc229e00 2024-04-20T10:40:05.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e26a00640 1 -- 172.21.15.42:0/2968322496 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e340043d0 con 0x55f6dc229e00 2024-04-20T10:40:05.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e26a00640 1 -- 172.21.15.42:0/2968322496 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f2e3809c370 con 0x55f6dc229e00 2024-04-20T10:40:05.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e4a96e740 10 client.26129 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:40:05.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e4a96e740 10 client.26129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:40:05.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e4a96e740 20 client.26129 populate_metadata read hostname 'smithi042' 2024-04-20T10:40:05.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f6dc23a120 0x55f6dc23c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e4a96e740 4 client.26129 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:40:05.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.201+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2968322496 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6dc18a740 con 0x55f6dc23a120 2024-04-20T10:40:05.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.202+0000 7f2e45200640 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f6dc23a120 0x55f6dc23c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.202+0000 7f2e45200640 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f6dc23a120 0x55f6dc23c570 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c01e0c82e9c029 server_cookie=c33dd49993ad59a1 in_seq=0 out_seq=0 2024-04-20T10:40:05.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.202+0000 7f2e26a00640 10 client.26129 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:05.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e26a00640 1 -- 172.21.15.42:0/2968322496 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6dc18a740 con 0x55f6dc23a120 2024-04-20T10:40:05.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e26a00640 10 client.26129 handle_command_reply: tid=0 2024-04-20T10:40:05.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e4a96e740 1 client.26129 shutdown 2024-04-20T10:40:05.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2968322496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc229e00 msgr2=0x55f6dc2278d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc229e00 0x55f6dc2278d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2e380a5660 tx=0x7f2e3809c940 comp rx=0 tx=0).stop 2024-04-20T10:40:05.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2968322496 shutdown_connections 2024-04-20T10:40:05.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f6dc23a120 0x55f6dc23c570 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc229e00 0x55f6dc2278d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc2188b0 0x55f6dc227390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/2968322496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc226e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e26a00640 0 client.26129 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:05.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2968322496 >> 172.21.15.42:0/2968322496 conn(0x55f6dc212820 msgr2=0x55f6dc214570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:05.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2968322496 shutdown_connections 2024-04-20T10:40:05.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.204+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2968322496 wait complete. 2024-04-20T10:40:05.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.205+0000 7f2e4a96e740 20 client.26129 trim_cache size 0 max 16384 2024-04-20T10:40:05.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.226+0000 7f2e4a96e740 1 Processor -- start 2024-04-20T10:40:05.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.226+0000 7f2e4a96e740 1 -- start start 2024-04-20T10:40:05.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.226+0000 7f2e4a96e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc229580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.226+0000 7f2e4a96e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc235ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.226+0000 7f2e4a96e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc229e00 0x55f6dc211880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.226+0000 7f2e4a96e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401a98e0 con 0x55f6dc217f00 2024-04-20T10:40:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.226+0000 7f2e4a96e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401ac160 con 0x55f6dc2188b0 2024-04-20T10:40:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.226+0000 7f2e4a96e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401a9f50 con 0x55f6dc229e00 2024-04-20T10:40:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.226+0000 7f2e3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc235ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.226+0000 7f2e3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc235ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53386/0 (socket says 172.21.15.42:53386) 2024-04-20T10:40:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.226+0000 7f2e3fe00640 1 -- 172.21.15.42:0/2032429943 learned_addr learned my addr 172.21.15.42:0/2032429943 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:05.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.227+0000 7f2e45200640 1 --2- 172.21.15.42:0/2032429943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc229580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.227+0000 7f2e45c00640 1 --2- 172.21.15.42:0/2032429943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc229e00 0x55f6dc211880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.227+0000 7f2e3fe00640 1 -- 172.21.15.42:0/2032429943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc229e00 msgr2=0x55f6dc211880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.227+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/2032429943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc229e00 0x55f6dc211880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.227+0000 7f2e3fe00640 1 -- 172.21.15.42:0/2032429943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 msgr2=0x55f6dc229580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.227+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/2032429943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc229580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.227+0000 7f2e3fe00640 1 -- 172.21.15.42:0/2032429943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6dbe4f170 con 0x55f6dc2188b0 2024-04-20T10:40:05.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.227+0000 7f2e45200640 1 --2- 172.21.15.42:0/2032429943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc229580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:40:05.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.227+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/2032429943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc235ac0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009230 tx=0x7f2e2c042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=633a85b2a156ab20 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:05.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.228+0000 7f2e31600640 1 -- 172.21.15.42:0/2032429943 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c02c940 con 0x55f6dc2188b0 2024-04-20T10:40:05.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.228+0000 7f2e31600640 1 -- 172.21.15.42:0/2032429943 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e2c005400 con 0x55f6dc2188b0 2024-04-20T10:40:05.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.228+0000 7f2e31600640 1 -- 172.21.15.42:0/2032429943 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c003970 con 0x55f6dc2188b0 2024-04-20T10:40:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.228+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2032429943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 msgr2=0x55f6dc235ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.228+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/2032429943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc235ac0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009230 tx=0x7f2e2c042000 comp rx=0 tx=0).stop 2024-04-20T10:40:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.229+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2032429943 shutdown_connections 2024-04-20T10:40:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.229+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/2032429943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc229e00 0x55f6dc211880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.229+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/2032429943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc235ac0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.229+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/2032429943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc217f00 0x55f6dc229580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.229+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2032429943 >> 172.21.15.42:0/2032429943 conn(0x55f6dc212820 msgr2=0x55f6dc211170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.229+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2032429943 shutdown_connections 2024-04-20T10:40:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.229+0000 7f2e4a96e740 1 -- 172.21.15.42:0/2032429943 wait complete. 2024-04-20T10:40:05.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.229+0000 7f2e4a96e740 1 -- start start 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e4a96e740 1 Processor -- start 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e4a96e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc217f00 0x55f6dc223d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e4a96e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc2242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e4a96e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc229e00 0x55f6dc2247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e4a96e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401aa620 con 0x55f6dc229e00 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e4a96e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401aba90 con 0x55f6dc2188b0 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e4a96e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401aacf0 con 0x55f6dc217f00 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e4a96e740 10 client.0 _ll_register_callbacks cb 0x7f2e38005920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc2242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc229e00 0x55f6dc2247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc2242b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53400/0 (socket says 172.21.15.42:53400) 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e45c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc229e00 0x55f6dc2247f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59796/0 (socket says 172.21.15.42:59796) 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e3fe00640 1 -- 172.21.15.42:0/1305003211 learned_addr learned my addr 172.21.15.42:0/1305003211 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:05.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.230+0000 7f2e45200640 1 --2- 172.21.15.42:0/1305003211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc217f00 0x55f6dc223d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.231+0000 7f2e3fe00640 1 -- 172.21.15.42:0/1305003211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc217f00 msgr2=0x55f6dc223d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.231+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/1305003211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc217f00 0x55f6dc223d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.231+0000 7f2e3fe00640 1 -- 172.21.15.42:0/1305003211 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc229e00 msgr2=0x55f6dc2247f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.231+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/1305003211 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc229e00 0x55f6dc2247f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.231+0000 7f2e3fe00640 1 -- 172.21.15.42:0/1305003211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6dc29f8b0 con 0x55f6dc2188b0 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.231+0000 7f2e3fe00640 1 --2- 172.21.15.42:0/1305003211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc2242b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009900 tx=0x7f2e2c02cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f8cb4fb9f4bb40c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.231+0000 7f2e26a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.231+0000 7f2e26a00640 1 -- 172.21.15.42:0/1305003211 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c05b070 con 0x55f6dc2188b0 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.231+0000 7f2e26a00640 1 -- 172.21.15.42:0/1305003211 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f2e2c0035f0 con 0x55f6dc2188b0 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.231+0000 7f2e4a96e740 1 -- 172.21.15.42:0/1305003211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e2c005400 con 0x55f6dc2188b0 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.232+0000 7f2e26a00640 1 -- 172.21.15.42:0/1305003211 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2e2c067050 con 0x55f6dc2188b0 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.232+0000 7f2e4a96e740 10 client.26135 fetch_fsmap learned FSMap version 19 2024-04-20T10:40:05.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.232+0000 7f2e4a96e740 1 -- 172.21.15.42:0/1305003211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f6dc2a0bd0 con 0x55f6dc2188b0 2024-04-20T10:40:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.232+0000 7f2e26a00640 1 -- 172.21.15.42:0/1305003211 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e2c005400 con 0x55f6dc2188b0 2024-04-20T10:40:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.232+0000 7f2e26a00640 1 -- 172.21.15.42:0/1305003211 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f2e2c057720 con 0x55f6dc2188b0 2024-04-20T10:40:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.232+0000 7f2e4a96e740 10 client.26135 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:40:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.232+0000 7f2e4a96e740 10 client.26135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:40:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.232+0000 7f2e4a96e740 20 client.26135 populate_metadata read hostname 'smithi042' 2024-04-20T10:40:05.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.232+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/1305003211 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f6dc2a12a0 0x55f6dc2a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.232+0000 7f2e4a96e740 4 client.26135 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:40:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.232+0000 7f2e4a96e740 1 -- 172.21.15.42:0/1305003211 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f6dc1845b0 con 0x55f6dc2a12a0 2024-04-20T10:40:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.233+0000 7f2e45200640 1 --2- 172.21.15.42:0/1305003211 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f6dc2a12a0 0x55f6dc2a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.233+0000 7f2e45200640 1 --2- 172.21.15.42:0/1305003211 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f6dc2a12a0 0x55f6dc2a36c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=539f83714c606faf server_cookie=4b17c1dbdd12f138 in_seq=0 out_seq=0 2024-04-20T10:40:05.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.233+0000 7f2e26a00640 10 client.26135 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:05.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.234+0000 7f2e26a00640 1 -- 172.21.15.42:0/1305003211 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f6dc1845b0 con 0x55f6dc2a12a0 2024-04-20T10:40:05.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.234+0000 7f2e26a00640 10 client.26135 handle_command_reply: tid=0 2024-04-20T10:40:05.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.234+0000 7f2e4a96e740 1 client.26135 shutdown 2024-04-20T10:40:05.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.235+0000 7f2e4a96e740 1 -- 172.21.15.42:0/1305003211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 msgr2=0x55f6dc2242b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.235+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/1305003211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc2242b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009900 tx=0x7f2e2c02cb70 comp rx=0 tx=0).stop 2024-04-20T10:40:05.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.235+0000 7f2e4a96e740 1 -- 172.21.15.42:0/1305003211 shutdown_connections 2024-04-20T10:40:05.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.235+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/1305003211 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f6dc2a12a0 0x55f6dc2a36c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.235+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/1305003211 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f6dc229e00 0x55f6dc2247f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.235+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/1305003211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f6dc2188b0 0x55f6dc2242b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.235+0000 7f2e4a96e740 1 --2- 172.21.15.42:0/1305003211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f6dc217f00 0x55f6dc223d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.235+0000 7f2e26a00640 0 client.26135 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:05.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.235+0000 7f2e4a96e740 1 -- 172.21.15.42:0/1305003211 >> 172.21.15.42:0/1305003211 conn(0x55f6dc212820 msgr2=0x55f6dc211a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:05.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.235+0000 7f2e4a96e740 1 -- 172.21.15.42:0/1305003211 shutdown_connections 2024-04-20T10:40:05.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.235+0000 7f2e4a96e740 1 -- 172.21.15.42:0/1305003211 wait complete. 2024-04-20T10:40:05.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.235+0000 7f2e4a96e740 20 client.26135 trim_cache size 0 max 16384 2024-04-20T10:40:05.245 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:40:05.245 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:40:05.245 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "47829d3f-d202-4c03-8498-5cd769840bae", 2024-04-20T10:40:05.245 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:40:05.246 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:40:05.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.236+0000 7f2e46600640 1 -- 172.21.15.42:0/1618519616 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2e280a3d20 msgr2=0x7f2e280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.236+0000 7f2e46600640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2e280a3d20 0x7f2e280a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c000c00 tx=0x7f2e2c005fd0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.236+0000 7f2e46600640 1 -- 172.21.15.42:0/1618519616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4010e760 msgr2=0x7f2e4019a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.236+0000 7f2e46600640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4010e760 0x7f2e4019a7d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800eb40 tx=0x7f2e3800c2d0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.236+0000 7f2e46600640 1 -- 172.21.15.42:0/1618519616 shutdown_connections 2024-04-20T10:40:05.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.236+0000 7f2e46600640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2e280a3d20 0x7f2e280a61e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.236+0000 7f2e46600640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e4010e760 0x7f2e4019a7d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.236+0000 7f2e46600640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e40101560 0x7f2e401a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.236+0000 7f2e46600640 1 --2- 172.21.15.42:0/1618519616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e40100bb0 0x7f2e401a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.236+0000 7f2e46600640 1 -- 172.21.15.42:0/1618519616 >> 172.21.15.42:0/1618519616 conn(0x7f2e400fc7d0 msgr2=0x7f2e400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:05.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.236+0000 7f2e46600640 1 -- 172.21.15.42:0/1618519616 shutdown_connections 2024-04-20T10:40:05.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.236+0000 7f2e46600640 1 -- 172.21.15.42:0/1618519616 wait complete. 2024-04-20T10:40:05.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:40:05.259 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:40:05.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.468+0000 7fb660000640 1 -- 172.21.15.42:0/1368325525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb658104410 msgr2=0x7fb6581047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.468+0000 7fb660000640 1 --2- 172.21.15.42:0/1368325525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb658104410 0x7fb6581047f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009900 tx=0x7fb64c02bc40 comp rx=0 tx=0).stop 2024-04-20T10:40:05.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.468+0000 7fb660000640 1 -- 172.21.15.42:0/1368325525 shutdown_connections 2024-04-20T10:40:05.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.468+0000 7fb660000640 1 --2- 172.21.15.42:0/1368325525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6581094e0 0x7fb6581110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.468+0000 7fb660000640 1 --2- 172.21.15.42:0/1368325525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb658104dc0 0x7fb658108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.468+0000 7fb660000640 1 --2- 172.21.15.42:0/1368325525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb658104410 0x7fb6581047f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.468+0000 7fb660000640 1 -- 172.21.15.42:0/1368325525 >> 172.21.15.42:0/1368325525 conn(0x7fb6580ffcf0 msgr2=0x7fb658102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:05.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.468+0000 7fb660000640 1 -- 172.21.15.42:0/1368325525 shutdown_connections 2024-04-20T10:40:05.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.468+0000 7fb660000640 1 -- 172.21.15.42:0/1368325525 wait complete. 2024-04-20T10:40:05.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.469+0000 7fb660000640 1 Processor -- start 2024-04-20T10:40:05.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.469+0000 7fb660000640 1 -- start start 2024-04-20T10:40:05.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.470+0000 7fb660000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb658104410 0x7fb6581a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.470+0000 7fb660000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6581094e0 0x7fb6581a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.470+0000 7fb660000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb65819a920 0x7fb65819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.470+0000 7fb660000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581145e0 con 0x7fb658104410 2024-04-20T10:40:05.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.470+0000 7fb660000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb658114460 con 0x7fb6581094e0 2024-04-20T10:40:05.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.470+0000 7fb660000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb658114760 con 0x7fb65819a920 2024-04-20T10:40:05.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.470+0000 7fb65cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6581094e0 0x7fb6581a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.470+0000 7fb65cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6581094e0 0x7fb6581a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53422/0 (socket says 172.21.15.42:53422) 2024-04-20T10:40:05.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.470+0000 7fb65cc00640 1 -- 172.21.15.42:0/85686650 learned_addr learned my addr 172.21.15.42:0/85686650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:05.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.470+0000 7fb65d600640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb658104410 0x7fb6581a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.470+0000 7fb65e000640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb65819a920 0x7fb65819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.471+0000 7fb65cc00640 1 -- 172.21.15.42:0/85686650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb65819a920 msgr2=0x7fb65819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.471+0000 7fb65cc00640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb65819a920 0x7fb65819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.471+0000 7fb65cc00640 1 -- 172.21.15.42:0/85686650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb658104410 msgr2=0x7fb6581a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.471+0000 7fb65cc00640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb658104410 0x7fb6581a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.471+0000 7fb65cc00640 1 -- 172.21.15.42:0/85686650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65819b440 con 0x7fb6581094e0 2024-04-20T10:40:05.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.471+0000 7fb65cc00640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6581094e0 0x7fb6581a0cb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb64400e9f0 tx=0x7fb64400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:05.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.471+0000 7fb656000640 1 -- 172.21.15.42:0/85686650 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64400ccc0 con 0x7fb6581094e0 2024-04-20T10:40:05.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.471+0000 7fb656000640 1 -- 172.21.15.42:0/85686650 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb644005500 con 0x7fb6581094e0 2024-04-20T10:40:05.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.471+0000 7fb660000640 1 -- 172.21.15.42:0/85686650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65819b730 con 0x7fb6581094e0 2024-04-20T10:40:05.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.472+0000 7fb656000640 1 -- 172.21.15.42:0/85686650 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644010690 con 0x7fb6581094e0 2024-04-20T10:40:05.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.472+0000 7fb660000640 1 -- 172.21.15.42:0/85686650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6581a7930 con 0x7fb6581094e0 2024-04-20T10:40:05.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.475+0000 7fb662de2740 1 Processor -- start 2024-04-20T10:40:05.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.475+0000 7fb662de2740 1 -- start start 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.475+0000 7fb662de2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16a71a0 0x55f7b16a7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.475+0000 7fb662de2740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a7ac0 0x55f7b16b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.475+0000 7fb662de2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16b06a0 0x55f7b16b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.475+0000 7fb662de2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f7b160d150 con 0x55f7b16a71a0 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.475+0000 7fb662de2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f7b15b3370 con 0x55f7b16b06a0 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.475+0000 7fb662de2740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f7b1609110 con 0x55f7b16a7ac0 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.476+0000 7fb65e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16a71a0 0x55f7b16a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.476+0000 7fb65e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16a71a0 0x55f7b16a7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59816/0 (socket says 172.21.15.42:59816) 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.476+0000 7fb65e000640 1 -- 172.21.15.42:0/1167411102 learned_addr learned my addr 172.21.15.42:0/1167411102 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.476+0000 7fb65d600640 1 --2- 172.21.15.42:0/1167411102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a7ac0 0x55f7b16b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.476+0000 7fb65e000640 1 -- 172.21.15.42:0/1167411102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a7ac0 msgr2=0x55f7b16b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.476+0000 7fb65e000640 1 --2- 172.21.15.42:0/1167411102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a7ac0 0x55f7b16b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.476+0000 7fb65e000640 1 -- 172.21.15.42:0/1167411102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16b06a0 msgr2=0x55f7b16b2a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:40:05.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.476+0000 7fb65e000640 1 --2- 172.21.15.42:0/1167411102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16b06a0 0x55f7b16b2a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.476+0000 7fb65e000640 1 -- 172.21.15.42:0/1167411102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b13e97d0 con 0x55f7b16a71a0 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.477+0000 7fb65e000640 1 --2- 172.21.15.42:0/1167411102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16a71a0 0x55f7b16a7580 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb650009f50 tx=0x7fb6500118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3727a6763ba57da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.477+0000 7fb64a600640 1 -- 172.21.15.42:0/1167411102 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650004d20 con 0x55f7b16a71a0 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.477+0000 7fb64a600640 1 -- 172.21.15.42:0/1167411102 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb650004050 con 0x55f7b16a71a0 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.477+0000 7fb656000640 1 -- 172.21.15.42:0/85686650 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb6440046c0 con 0x7fb6581094e0 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.477+0000 7fb64a600640 1 -- 172.21.15.42:0/1167411102 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650005220 con 0x55f7b16a71a0 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.478+0000 7fb662de2740 1 -- 172.21.15.42:0/1167411102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16a71a0 msgr2=0x55f7b16a7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.478+0000 7fb662de2740 1 --2- 172.21.15.42:0/1167411102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16a71a0 0x55f7b16a7580 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb650009f50 tx=0x7fb6500118d0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.478+0000 7fb662de2740 1 -- 172.21.15.42:0/1167411102 shutdown_connections 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.478+0000 7fb662de2740 1 --2- 172.21.15.42:0/1167411102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16b06a0 0x55f7b16b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.478+0000 7fb662de2740 1 --2- 172.21.15.42:0/1167411102 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a7ac0 0x55f7b16b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.478+0000 7fb662de2740 1 --2- 172.21.15.42:0/1167411102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16a71a0 0x55f7b16a7580 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.478+0000 7fb662de2740 1 -- 172.21.15.42:0/1167411102 >> 172.21.15.42:0/1167411102 conn(0x55f7b16a18d0 msgr2=0x55f7b16a1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.479+0000 7fb662de2740 1 -- 172.21.15.42:0/1167411102 shutdown_connections 2024-04-20T10:40:05.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.479+0000 7fb662de2740 1 -- 172.21.15.42:0/1167411102 wait complete. 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.479+0000 7fb656000640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb6400a3c30 0x7fb6400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.479+0000 7fb662de2740 1 -- start start 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.480+0000 7fb656000640 1 -- 172.21.15.42:0/85686650 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb6440d0bd0 con 0x7fb6581094e0 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.480+0000 7fb65e000640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb6400a3c30 0x7fb6400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.480+0000 7fb662de2740 1 Processor -- start 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.480+0000 7fb662de2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a71a0 0x55f7b16ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.480+0000 7fb65e000640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb6400a3c30 0x7fb6400a60f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb65001b040 tx=0x7fb650000f40 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.480+0000 7fb662de2740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a7ac0 0x55f7b16adc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb662de2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16b06a0 0x55f7b16ae160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb662de2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f7b16b34e0 con 0x55f7b16b06a0 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb662de2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f7b15c6740 con 0x55f7b16a71a0 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb662de2740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f7b16b3660 con 0x55f7b16a7ac0 2024-04-20T10:40:05.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb662de2740 10 client.0 _ll_register_callbacks cb 0x55f7b15c1b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb65d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a71a0 0x55f7b16ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb65d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a71a0 0x55f7b16ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53434/0 (socket says 172.21.15.42:53434) 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb65d600640 1 -- 172.21.15.42:0/2801855372 learned_addr learned my addr 172.21.15.42:0/2801855372 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb65e000640 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16b06a0 0x55f7b16ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb65cc00640 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a7ac0 0x55f7b16adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb65d600640 1 -- 172.21.15.42:0/2801855372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a7ac0 msgr2=0x55f7b16adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb65d600640 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a7ac0 0x55f7b16adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb65d600640 1 -- 172.21.15.42:0/2801855372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16b06a0 msgr2=0x55f7b16ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb65d600640 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16b06a0 0x55f7b16ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb65d600640 1 -- 172.21.15.42:0/2801855372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b131f3f0 con 0x55f7b16a71a0 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.481+0000 7fb65e000640 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16b06a0 0x55f7b16ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.482+0000 7fb65d600640 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a71a0 0x55f7b16ad6e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009230 tx=0x7fb64c0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=364b017b93715b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.482+0000 7fb63ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.482+0000 7fb63ea00640 1 -- 172.21.15.42:0/2801855372 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c02cca0 con 0x55f7b16a71a0 2024-04-20T10:40:05.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.482+0000 7fb662de2740 1 -- 172.21.15.42:0/2801855372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb650004050 con 0x55f7b16a71a0 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.482+0000 7fb63ea00640 1 -- 172.21.15.42:0/2801855372 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb64c0046c0 con 0x55f7b16a71a0 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.482+0000 7fb63ea00640 1 -- 172.21.15.42:0/2801855372 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb64c0048c0 con 0x55f7b16a71a0 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.482+0000 7fb662de2740 10 client.26144 fetch_fsmap learned FSMap version 19 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.482+0000 7fb662de2740 1 -- 172.21.15.42:0/2801855372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f7b12df0f0 con 0x55f7b16a71a0 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.483+0000 7fb63ea00640 1 -- 172.21.15.42:0/2801855372 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb650004050 con 0x55f7b16a71a0 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.483+0000 7fb63ea00640 1 -- 172.21.15.42:0/2801855372 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fb64c046070 con 0x55f7b16a71a0 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.483+0000 7fb662de2740 10 client.26144 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.483+0000 7fb662de2740 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.483+0000 7fb662de2740 20 client.26144 populate_metadata read hostname 'smithi042' 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.483+0000 7fb662de2740 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f7b16c09f0 0x55f7b16c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.483+0000 7fb662de2740 4 client.26144 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.483+0000 7fb662de2740 1 -- 172.21.15.42:0/2801855372 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7b161ab70 con 0x55f7b16c09f0 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.483+0000 7fb65cc00640 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f7b16c09f0 0x55f7b16c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.484+0000 7fb65cc00640 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f7b16c09f0 0x55f7b16c2de0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c90014b4a4a1b857 server_cookie=2bd40ff22f762235 in_seq=0 out_seq=0 2024-04-20T10:40:05.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.484+0000 7fb63ea00640 10 client.26144 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.491+0000 7fb63ea00640 1 -- 172.21.15.42:0/2801855372 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7b161ab70 con 0x55f7b16c09f0 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.491+0000 7fb63ea00640 10 client.26144 handle_command_reply: tid=0 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.491+0000 7fb662de2740 1 client.26144 shutdown 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.491+0000 7fb662de2740 1 -- 172.21.15.42:0/2801855372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a71a0 msgr2=0x55f7b16ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.491+0000 7fb662de2740 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a71a0 0x55f7b16ad6e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009230 tx=0x7fb64c0045f0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.491+0000 7fb662de2740 1 -- 172.21.15.42:0/2801855372 shutdown_connections 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.491+0000 7fb662de2740 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f7b16c09f0 0x55f7b16c2de0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.491+0000 7fb662de2740 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16b06a0 0x55f7b16ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.491+0000 7fb662de2740 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a7ac0 0x55f7b16adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.491+0000 7fb662de2740 1 --2- 172.21.15.42:0/2801855372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a71a0 0x55f7b16ad6e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.491+0000 7fb63ea00640 0 client.26144 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.491+0000 7fb662de2740 1 -- 172.21.15.42:0/2801855372 >> 172.21.15.42:0/2801855372 conn(0x55f7b16a18d0 msgr2=0x55f7b16a3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.492+0000 7fb662de2740 1 -- 172.21.15.42:0/2801855372 shutdown_connections 2024-04-20T10:40:05.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.492+0000 7fb662de2740 1 -- 172.21.15.42:0/2801855372 wait complete. 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.492+0000 7fb662de2740 20 client.26144 trim_cache size 0 max 16384 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.514+0000 7fb662de2740 1 Processor -- start 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.514+0000 7fb662de2740 1 -- start start 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.514+0000 7fb662de2740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a71a0 0x55f7b16afe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.514+0000 7fb662de2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16a7ac0 0x55f7b16a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.514+0000 7fb662de2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16b06a0 0x55f7b16c8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.514+0000 7fb662de2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581adf50 con 0x55f7b16a7ac0 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.514+0000 7fb662de2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581b07d0 con 0x55f7b16b06a0 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.514+0000 7fb662de2740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581ae5c0 con 0x55f7b16a71a0 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.515+0000 7fb65e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16b06a0 0x55f7b16c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.515+0000 7fb65e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16b06a0 0x55f7b16c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53444/0 (socket says 172.21.15.42:53444) 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.515+0000 7fb65e000640 1 -- 172.21.15.42:0/3897054528 learned_addr learned my addr 172.21.15.42:0/3897054528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:05.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.515+0000 7fb65cc00640 1 --2- 172.21.15.42:0/3897054528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16a7ac0 0x55f7b16a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.515+0000 7fb65d600640 1 --2- 172.21.15.42:0/3897054528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a71a0 0x55f7b16afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.515+0000 7fb65d600640 1 -- 172.21.15.42:0/3897054528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16b06a0 msgr2=0x55f7b16c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.515+0000 7fb65d600640 1 --2- 172.21.15.42:0/3897054528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16b06a0 0x55f7b16c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.515+0000 7fb65d600640 1 -- 172.21.15.42:0/3897054528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16a7ac0 msgr2=0x55f7b16a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.515+0000 7fb65d600640 1 --2- 172.21.15.42:0/3897054528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16a7ac0 0x55f7b16a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.515+0000 7fb65d600640 1 -- 172.21.15.42:0/3897054528 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b16a78f0 con 0x55f7b16a71a0 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.515+0000 7fb65e000640 1 --2- 172.21.15.42:0/3897054528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16b06a0 0x55f7b16c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.515+0000 7fb65d600640 1 --2- 172.21.15.42:0/3897054528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a71a0 0x55f7b16afe10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb64c02c180 tx=0x7fb64c02bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcab7ee04b8363b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.516+0000 7fb649c00640 1 -- 172.21.15.42:0/3897054528 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c000e60 con 0x55f7b16a71a0 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.516+0000 7fb649c00640 1 -- 172.21.15.42:0/3897054528 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb64c009520 con 0x55f7b16a71a0 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.516+0000 7fb649c00640 1 -- 172.21.15.42:0/3897054528 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c004f80 con 0x55f7b16a71a0 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.517+0000 7fb662de2740 1 -- 172.21.15.42:0/3897054528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a71a0 msgr2=0x55f7b16afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.517+0000 7fb662de2740 1 --2- 172.21.15.42:0/3897054528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a71a0 0x55f7b16afe10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb64c02c180 tx=0x7fb64c02bc40 comp rx=0 tx=0).stop 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.517+0000 7fb662de2740 1 -- 172.21.15.42:0/3897054528 shutdown_connections 2024-04-20T10:40:05.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.517+0000 7fb662de2740 1 --2- 172.21.15.42:0/3897054528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16b06a0 0x55f7b16c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.517+0000 7fb662de2740 1 --2- 172.21.15.42:0/3897054528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16a7ac0 0x55f7b16a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.517+0000 7fb662de2740 1 --2- 172.21.15.42:0/3897054528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a71a0 0x55f7b16afe10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.517+0000 7fb662de2740 1 -- 172.21.15.42:0/3897054528 >> 172.21.15.42:0/3897054528 conn(0x55f7b16a18d0 msgr2=0x55f7b16a2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.517+0000 7fb662de2740 1 -- 172.21.15.42:0/3897054528 shutdown_connections 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.517+0000 7fb662de2740 1 -- 172.21.15.42:0/3897054528 wait complete. 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.517+0000 7fb662de2740 1 -- start start 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.517+0000 7fb662de2740 1 Processor -- start 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.518+0000 7fb662de2740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a71a0 0x55f7b16b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.518+0000 7fb662de2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a7ac0 0x55f7b16b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.518+0000 7fb662de2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16b06a0 0x55f7b1725f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.518+0000 7fb662de2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581aec90 con 0x55f7b16b06a0 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.518+0000 7fb662de2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581b0100 con 0x55f7b16a7ac0 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.518+0000 7fb662de2740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581af360 con 0x55f7b16a71a0 2024-04-20T10:40:05.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.518+0000 7fb662de2740 10 client.0 _ll_register_callbacks cb 0x7fb64c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.518+0000 7fb65cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a7ac0 0x55f7b16b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.518+0000 7fb65cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a7ac0 0x55f7b16b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:53458/0 (socket says 172.21.15.42:53458) 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.518+0000 7fb65cc00640 1 -- 172.21.15.42:0/4143343341 learned_addr learned my addr 172.21.15.42:0/4143343341 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.518+0000 7fb65e000640 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16b06a0 0x55f7b1725f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.518+0000 7fb65d600640 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a71a0 0x55f7b16b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.519+0000 7fb65cc00640 1 -- 172.21.15.42:0/4143343341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a71a0 msgr2=0x55f7b16b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.519+0000 7fb65cc00640 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a71a0 0x55f7b16b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.519+0000 7fb65cc00640 1 -- 172.21.15.42:0/4143343341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16b06a0 msgr2=0x55f7b1725f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.519+0000 7fb65cc00640 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16b06a0 0x55f7b1725f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.519+0000 7fb65cc00640 1 -- 172.21.15.42:0/4143343341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b16c3150 con 0x55f7b16a7ac0 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.519+0000 7fb65cc00640 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a7ac0 0x55f7b16b4b40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb6440ce320 tx=0x7fb6440bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6471e955d355e27d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.519+0000 7fb63ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.519+0000 7fb63ea00640 1 -- 172.21.15.42:0/4143343341 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64409cb10 con 0x55f7b16a7ac0 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.519+0000 7fb63ea00640 1 -- 172.21.15.42:0/4143343341 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb644099180 con 0x55f7b16a7ac0 2024-04-20T10:40:05.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.519+0000 7fb662de2740 1 -- 172.21.15.42:0/4143343341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c009520 con 0x55f7b16a7ac0 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.520+0000 7fb63ea00640 1 -- 172.21.15.42:0/4143343341 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb6440a0ec0 con 0x55f7b16a7ac0 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.520+0000 7fb662de2740 10 client.26150 fetch_fsmap learned FSMap version 19 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.520+0000 7fb662de2740 1 -- 172.21.15.42:0/4143343341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f7b17273c0 con 0x55f7b16a7ac0 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.520+0000 7fb63ea00640 1 -- 172.21.15.42:0/4143343341 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb64c009520 con 0x55f7b16a7ac0 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.520+0000 7fb63ea00640 1 -- 172.21.15.42:0/4143343341 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fb64409bf60 con 0x55f7b16a7ac0 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.520+0000 7fb662de2740 10 client.26150 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.520+0000 7fb662de2740 10 client.26150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.520+0000 7fb662de2740 20 client.26150 populate_metadata read hostname 'smithi042' 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.520+0000 7fb662de2740 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f7b1727aa0 0x55f7b1729ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.520+0000 7fb662de2740 4 client.26150 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.520+0000 7fb662de2740 1 -- 172.21.15.42:0/4143343341 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f7b16149e0 con 0x55f7b1727aa0 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.521+0000 7fb65d600640 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f7b1727aa0 0x55f7b1729ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.521+0000 7fb63ea00640 10 client.26150 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:05.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.521+0000 7fb65d600640 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f7b1727aa0 0x55f7b1729ec0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c20b5953e723f4 server_cookie=494bb94cefb7490a in_seq=0 out_seq=0 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb63ea00640 1 -- 172.21.15.42:0/4143343341 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55f7b16149e0 con 0x55f7b1727aa0 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb63ea00640 10 client.26150 handle_command_reply: tid=0 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb662de2740 1 client.26150 shutdown 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb662de2740 1 -- 172.21.15.42:0/4143343341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a7ac0 msgr2=0x55f7b16b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb662de2740 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a7ac0 0x55f7b16b4b40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb6440ce320 tx=0x7fb6440bd040 comp rx=0 tx=0).stop 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb662de2740 1 -- 172.21.15.42:0/4143343341 shutdown_connections 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb662de2740 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f7b1727aa0 0x55f7b1729ec0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb662de2740 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f7b16b06a0 0x55f7b1725f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb662de2740 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f7b16a7ac0 0x55f7b16b4b40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb662de2740 1 --2- 172.21.15.42:0/4143343341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f7b16a71a0 0x55f7b16b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb662de2740 1 -- 172.21.15.42:0/4143343341 >> 172.21.15.42:0/4143343341 conn(0x55f7b16a18d0 msgr2=0x55f7b16a0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb63ea00640 0 client.26150 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb662de2740 1 -- 172.21.15.42:0/4143343341 shutdown_connections 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb662de2740 1 -- 172.21.15.42:0/4143343341 wait complete. 2024-04-20T10:40:05.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.524+0000 7fb662de2740 20 client.26150 trim_cache size 0 max 16384 2024-04-20T10:40:05.716 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:40:05.716 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (600 inodes in the stack)", 2024-04-20T10:40:05.716 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:40:05.716 INFO:teuthology.orchestra.run.smithi042.stdout: "47829d3f-d202-4c03-8498-5cd769840bae": { 2024-04-20T10:40:05.716 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:40:05.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "47829d3f-d202-4c03-8498-5cd769840bae", 2024-04-20T10:40:05.716 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:40:05.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:40:05.717 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:40:05.717 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:40:05.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.525+0000 7fb660000640 1 -- 172.21.15.42:0/85686650 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb6400a3c30 msgr2=0x7fb6400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.525+0000 7fb660000640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb6400a3c30 0x7fb6400a60f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb65001b040 tx=0x7fb650000f40 comp rx=0 tx=0).stop 2024-04-20T10:40:05.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.525+0000 7fb660000640 1 -- 172.21.15.42:0/85686650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6581094e0 msgr2=0x7fb6581a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:05.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.525+0000 7fb660000640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6581094e0 0x7fb6581a0cb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb64400e9f0 tx=0x7fb64400eeb0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.525+0000 7fb660000640 1 -- 172.21.15.42:0/85686650 shutdown_connections 2024-04-20T10:40:05.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.525+0000 7fb660000640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb6400a3c30 0x7fb6400a60f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.525+0000 7fb660000640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb65819a920 0x7fb65819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.525+0000 7fb660000640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6581094e0 0x7fb6581a0cb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.525+0000 7fb660000640 1 --2- 172.21.15.42:0/85686650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb658104410 0x7fb6581a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:05.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.525+0000 7fb660000640 1 -- 172.21.15.42:0/85686650 >> 172.21.15.42:0/85686650 conn(0x7fb6580ffcf0 msgr2=0x7fb658101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:05.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.525+0000 7fb660000640 1 -- 172.21.15.42:0/85686650 shutdown_connections 2024-04-20T10:40:05.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:05.525+0000 7fb660000640 1 -- 172.21.15.42:0/85686650 wait complete. 2024-04-20T10:40:05.719 INFO:tasks.cephfs.filesystem:scrub status for tag:47829d3f-d202-4c03-8498-5cd769840bae - {'path': '/', 'tag': '47829d3f-d202-4c03-8498-5cd769840bae', 'options': 'recursive,force'} 2024-04-20T10:40:05.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:05 smithi162 ceph-mon[21854]: pgmap v185: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.1 MiB/s wr, 212 op/s 2024-04-20T10:40:05.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:05 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:40:05.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:05 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:40:05.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:05 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:40:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:05 smithi046 ceph-mon[21574]: pgmap v185: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.1 MiB/s wr, 212 op/s 2024-04-20T10:40:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:05 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:40:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:05 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:40:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:05 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:40:06.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:05 smithi042 ceph-mon[17078]: pgmap v185: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.1 MiB/s wr, 212 op/s 2024-04-20T10:40:06.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:05 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:40:06.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:05 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:40:06.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:05 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:40:07.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:07 smithi162 ceph-mon[21854]: pgmap v186: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-04-20T10:40:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:07 smithi046 ceph-mon[21574]: pgmap v186: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-04-20T10:40:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:07 smithi042 ceph-mon[17078]: pgmap v186: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-04-20T10:40:09.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:09 smithi162 ceph-mon[21854]: pgmap v187: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-04-20T10:40:10.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:09 smithi046 ceph-mon[21574]: pgmap v187: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-04-20T10:40:10.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:09 smithi042 ceph-mon[17078]: pgmap v187: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 270 op/s 2024-04-20T10:40:11.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:11 smithi162 ceph-mon[21854]: pgmap v188: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 168 op/s 2024-04-20T10:40:12.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:11 smithi046 ceph-mon[21574]: pgmap v188: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 168 op/s 2024-04-20T10:40:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:11 smithi042 ceph-mon[17078]: pgmap v188: 97 pgs: 97 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 168 op/s 2024-04-20T10:40:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:13 smithi046 ceph-mon[21574]: pgmap v189: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-04-20T10:40:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:13 smithi042 ceph-mon[17078]: pgmap v189: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-04-20T10:40:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:13 smithi162 ceph-mon[21854]: pgmap v189: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-04-20T10:40:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:15 smithi046 ceph-mon[21574]: pgmap v190: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 402 op/s 2024-04-20T10:40:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:15 smithi042 ceph-mon[17078]: pgmap v190: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 402 op/s 2024-04-20T10:40:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:15 smithi162 ceph-mon[21854]: pgmap v190: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 402 op/s 2024-04-20T10:40:18.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:17 smithi046 ceph-mon[21574]: pgmap v191: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 590 op/s 2024-04-20T10:40:18.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:17 smithi042 ceph-mon[17078]: pgmap v191: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 590 op/s 2024-04-20T10:40:18.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:17 smithi162 ceph-mon[21854]: pgmap v191: 97 pgs: 97 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 590 op/s 2024-04-20T10:40:19.050 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:18 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:40:19.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:18 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:40:19.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:18 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:40:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:19 smithi046 ceph-mon[21574]: pgmap v192: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.9 MiB/s wr, 1.20k op/s 2024-04-20T10:40:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:19 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:40:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:19 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:40:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:19 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:40:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:19 smithi042 ceph-mon[17078]: pgmap v192: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.9 MiB/s wr, 1.20k op/s 2024-04-20T10:40:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:19 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:40:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:19 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:40:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:19 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:40:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:19 smithi162 ceph-mon[21854]: pgmap v192: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.9 MiB/s wr, 1.20k op/s 2024-04-20T10:40:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:19 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:40:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:19 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:40:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:19 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:40:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:21 smithi046 ceph-mon[21574]: pgmap v193: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.2 MiB/s wr, 1.13k op/s 2024-04-20T10:40:22.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:21 smithi042 ceph-mon[17078]: pgmap v193: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.2 MiB/s wr, 1.13k op/s 2024-04-20T10:40:22.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:21 smithi162 ceph-mon[21854]: pgmap v193: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.2 MiB/s wr, 1.13k op/s 2024-04-20T10:40:24.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:23 smithi046 ceph-mon[21574]: pgmap v194: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.8 MiB/s wr, 1.97k op/s 2024-04-20T10:40:24.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:23 smithi042 ceph-mon[17078]: pgmap v194: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.8 MiB/s wr, 1.97k op/s 2024-04-20T10:40:24.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:23 smithi162 ceph-mon[21854]: pgmap v194: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.8 MiB/s wr, 1.97k op/s 2024-04-20T10:40:26.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:25 smithi046 ceph-mon[21574]: pgmap v195: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 1.71k op/s 2024-04-20T10:40:26.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:25 smithi042 ceph-mon[17078]: pgmap v195: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 1.71k op/s 2024-04-20T10:40:26.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:25 smithi162 ceph-mon[21854]: pgmap v195: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 1.71k op/s 2024-04-20T10:40:28.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:27 smithi046 ceph-mon[21574]: pgmap v196: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 1.94k op/s 2024-04-20T10:40:28.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:27 smithi042 ceph-mon[17078]: pgmap v196: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 1.94k op/s 2024-04-20T10:40:28.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:27 smithi162 ceph-mon[21854]: pgmap v196: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 1.94k op/s 2024-04-20T10:40:30.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:29 smithi046 ceph-mon[21574]: pgmap v197: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 2.21k op/s 2024-04-20T10:40:30.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:29 smithi042 ceph-mon[17078]: pgmap v197: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 2.21k op/s 2024-04-20T10:40:30.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:29 smithi162 ceph-mon[21854]: pgmap v197: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 2.21k op/s 2024-04-20T10:40:32.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:31 smithi046 ceph-mon[21574]: pgmap v198: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 1.52k op/s 2024-04-20T10:40:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:31 smithi042 ceph-mon[17078]: pgmap v198: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 1.52k op/s 2024-04-20T10:40:32.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:31 smithi162 ceph-mon[21854]: pgmap v198: 97 pgs: 97 active+clean; 1.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 1.52k op/s 2024-04-20T10:40:34.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:33 smithi046 ceph-mon[21574]: pgmap v199: 97 pgs: 97 active+clean; 1.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 2.10k op/s 2024-04-20T10:40:34.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:33 smithi042 ceph-mon[17078]: pgmap v199: 97 pgs: 97 active+clean; 1.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 2.10k op/s 2024-04-20T10:40:34.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:33 smithi162 ceph-mon[21854]: pgmap v199: 97 pgs: 97 active+clean; 1.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 2.10k op/s 2024-04-20T10:40:35.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:40:35.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:40:35.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.946+0000 7fd33fe00640 1 -- 172.21.15.42:0/781252542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd340074c50 msgr2=0x7fd34010e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:35.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.946+0000 7fd33fe00640 1 --2- 172.21.15.42:0/781252542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd340074c50 0x7fd34010e180 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd328009900 tx=0x7fd32802bcd0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.946+0000 7fd33fe00640 1 -- 172.21.15.42:0/781252542 shutdown_connections 2024-04-20T10:40:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.946+0000 7fd33fe00640 1 --2- 172.21.15.42:0/781252542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd34010e6c0 0x7fd340110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.946+0000 7fd33fe00640 1 --2- 172.21.15.42:0/781252542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd340074c50 0x7fd34010e180 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.946+0000 7fd33fe00640 1 --2- 172.21.15.42:0/781252542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd340074330 0x7fd340074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.946+0000 7fd33fe00640 1 -- 172.21.15.42:0/781252542 >> 172.21.15.42:0/781252542 conn(0x7fd3400fd930 msgr2=0x7fd3400ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.946+0000 7fd33fe00640 1 -- 172.21.15.42:0/781252542 shutdown_connections 2024-04-20T10:40:35.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.947+0000 7fd33fe00640 1 -- 172.21.15.42:0/781252542 wait complete. 2024-04-20T10:40:35.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd33fe00640 1 Processor -- start 2024-04-20T10:40:35.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd33fe00640 1 -- start start 2024-04-20T10:40:35.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd340074330 0x7fd3401a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd340074c50 0x7fd3401a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd34010e6c0 0x7fd34019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd33fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd340114440 con 0x7fd34010e6c0 2024-04-20T10:40:35.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd33fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3401142c0 con 0x7fd340074330 2024-04-20T10:40:35.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd33fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd3401145c0 con 0x7fd340074c50 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd336000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd340074c50 0x7fd3401a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd33ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd340074330 0x7fd3401a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd336000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd340074c50 0x7fd3401a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:41918/0 (socket says 172.21.15.42:41918) 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd336000640 1 -- 172.21.15.42:0/2009125826 learned_addr learned my addr 172.21.15.42:0/2009125826 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.948+0000 7fd33f400640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd34010e6c0 0x7fd34019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.949+0000 7fd33ea00640 1 -- 172.21.15.42:0/2009125826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd340074c50 msgr2=0x7fd3401a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.949+0000 7fd33ea00640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd340074c50 0x7fd3401a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.949+0000 7fd33ea00640 1 -- 172.21.15.42:0/2009125826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd34010e6c0 msgr2=0x7fd34019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.949+0000 7fd33ea00640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd34010e6c0 0x7fd34019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.949+0000 7fd33ea00640 1 -- 172.21.15.42:0/2009125826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34019af90 con 0x7fd340074330 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.949+0000 7fd33ea00640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd340074330 0x7fd3401a0650 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd33000e9f0 tx=0x7fd33000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.949+0000 7fd344a00640 1 -- 172.21.15.42:0/2009125826 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33000ccc0 con 0x7fd340074330 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.949+0000 7fd344a00640 1 -- 172.21.15.42:0/2009125826 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd330010910 con 0x7fd340074330 2024-04-20T10:40:35.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.949+0000 7fd33fe00640 1 -- 172.21.15.42:0/2009125826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34019b280 con 0x7fd340074330 2024-04-20T10:40:35.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.949+0000 7fd344a00640 1 -- 172.21.15.42:0/2009125826 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330018960 con 0x7fd340074330 2024-04-20T10:40:35.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.950+0000 7fd33fe00640 1 -- 172.21.15.42:0/2009125826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3401a74a0 con 0x7fd340074330 2024-04-20T10:40:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.953+0000 7fd344a00640 1 -- 172.21.15.42:0/2009125826 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd330010ac0 con 0x7fd340074330 2024-04-20T10:40:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.953+0000 7fd348b8f740 1 Processor -- start 2024-04-20T10:40:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.953+0000 7fd348b8f740 1 -- start start 2024-04-20T10:40:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.953+0000 7fd348b8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 0x564233e44760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.953+0000 7fd348b8f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44ca0 0x564233e4d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.953+0000 7fd348b8f740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e4d880 0x564233e4fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.953+0000 7fd348b8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564233daa150 con 0x564233e44380 2024-04-20T10:40:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.953+0000 7fd348b8f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564233d50370 con 0x564233e44ca0 2024-04-20T10:40:35.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.953+0000 7fd348b8f740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x564233da6110 con 0x564233e4d880 2024-04-20T10:40:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.953+0000 7fd33ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44ca0 0x564233e4d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.953+0000 7fd33f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 0x564233e44760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd33f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 0x564233e44760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38418/0 (socket says 172.21.15.42:38418) 2024-04-20T10:40:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd33f400640 1 -- 172.21.15.42:0/1042769368 learned_addr learned my addr 172.21.15.42:0/1042769368 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.953+0000 7fd33ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44ca0 0x564233e4d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:38342/0 (socket says 172.21.15.42:38342) 2024-04-20T10:40:35.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd336000640 1 --2- 172.21.15.42:0/1042769368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e4d880 0x564233e4fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd33f400640 1 -- 172.21.15.42:0/1042769368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e4d880 msgr2=0x564233e4fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd33f400640 1 --2- 172.21.15.42:0/1042769368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e4d880 0x564233e4fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd33f400640 1 -- 172.21.15.42:0/1042769368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44ca0 msgr2=0x564233e4d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd33f400640 1 --2- 172.21.15.42:0/1042769368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44ca0 0x564233e4d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd33f400640 1 -- 172.21.15.42:0/1042769368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564233b867d0 con 0x564233e44380 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd33ea00640 1 --2- 172.21.15.42:0/1042769368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44ca0 0x564233e4d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd344a00640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd3200a3da0 0x7fd3200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd33f400640 1 --2- 172.21.15.42:0/1042769368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 0x564233e44760 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd338009f30 tx=0x7fd3380118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67dd3a693ad85022 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd344a00640 1 -- 172.21.15.42:0/2009125826 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd3300d0b70 con 0x7fd340074330 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.954+0000 7fd32fe00640 1 -- 172.21.15.42:0/1042769368 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338004d20 con 0x564233e44380 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.955+0000 7fd32fe00640 1 -- 172.21.15.42:0/1042769368 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd338004050 con 0x564233e44380 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.955+0000 7fd32fe00640 1 -- 172.21.15.42:0/1042769368 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338005260 con 0x564233e44380 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.955+0000 7fd33ea00640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd3200a3da0 0x7fd3200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.955+0000 7fd348b8f740 1 -- 172.21.15.42:0/1042769368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 msgr2=0x564233e44760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.955+0000 7fd348b8f740 1 --2- 172.21.15.42:0/1042769368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 0x564233e44760 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd338009f30 tx=0x7fd3380118d0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.955+0000 7fd33ea00640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd3200a3da0 0x7fd3200a6260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x564233e43e50 tx=0x7fd3300d4620 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.955+0000 7fd348b8f740 1 -- 172.21.15.42:0/1042769368 shutdown_connections 2024-04-20T10:40:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.955+0000 7fd348b8f740 1 --2- 172.21.15.42:0/1042769368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e4d880 0x564233e4fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.955+0000 7fd348b8f740 1 --2- 172.21.15.42:0/1042769368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44ca0 0x564233e4d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.955+0000 7fd348b8f740 1 --2- 172.21.15.42:0/1042769368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 0x564233e44760 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.955+0000 7fd348b8f740 1 -- 172.21.15.42:0/1042769368 >> 172.21.15.42:0/1042769368 conn(0x564233e3eab0 msgr2=0x564233e3ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.956+0000 7fd348b8f740 1 -- 172.21.15.42:0/1042769368 shutdown_connections 2024-04-20T10:40:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.956+0000 7fd348b8f740 1 -- 172.21.15.42:0/1042769368 wait complete. 2024-04-20T10:40:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.956+0000 7fd348b8f740 1 -- start start 2024-04-20T10:40:35.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.956+0000 7fd348b8f740 1 Processor -- start 2024-04-20T10:40:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd348b8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 0x564233e4a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd348b8f740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e44ca0 0x564233e4ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd348b8f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e4d880 0x564233e4b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd348b8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564233e50670 con 0x564233e44380 2024-04-20T10:40:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd348b8f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564233d63740 con 0x564233e4d880 2024-04-20T10:40:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd348b8f740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x564233e507f0 con 0x564233e44ca0 2024-04-20T10:40:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd348b8f740 10 client.0 _ll_register_callbacks cb 0x564233d5eb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:40:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd33f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 0x564233e4a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd33f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 0x564233e4a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38430/0 (socket says 172.21.15.42:38430) 2024-04-20T10:40:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd33f400640 1 -- 172.21.15.42:0/1119740000 learned_addr learned my addr 172.21.15.42:0/1119740000 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd33f400640 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e4d880 0x564233e4b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd336000640 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e44ca0 0x564233e4ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd33f400640 1 -- 172.21.15.42:0/1119740000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e44ca0 msgr2=0x564233e4ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:35.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd33f400640 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e44ca0 0x564233e4ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd33f400640 1 -- 172.21.15.42:0/1119740000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e4d880 msgr2=0x564233e4b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:35.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.957+0000 7fd33f400640 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e4d880 0x564233e4b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.958+0000 7fd33f400640 1 -- 172.21.15.42:0/1119740000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564233abc3f0 con 0x564233e44380 2024-04-20T10:40:35.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.958+0000 7fd336000640 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e44ca0 0x564233e4ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:40:35.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.958+0000 7fd33f400640 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 0x564233e4a940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd338005b40 tx=0x7fd338005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d73bbf15ca2a0d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.958+0000 7fd32cc00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.958+0000 7fd32cc00640 1 -- 172.21.15.42:0/1119740000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33802c020 con 0x564233e44380 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.958+0000 7fd32cc00640 1 -- 172.21.15.42:0/1119740000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd338028450 con 0x564233e44380 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.958+0000 7fd348b8f740 1 -- 172.21.15.42:0/1119740000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd338004050 con 0x564233e44380 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.958+0000 7fd32cc00640 1 -- 172.21.15.42:0/1119740000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd338037050 con 0x564233e44380 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.958+0000 7fd348b8f740 10 client.16047 fetch_fsmap learned FSMap version 19 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.958+0000 7fd348b8f740 1 -- 172.21.15.42:0/1119740000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x564233a7c0f0 con 0x564233e44380 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.959+0000 7fd32cc00640 1 -- 172.21.15.42:0/1119740000 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd338004050 con 0x564233e44380 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.959+0000 7fd32cc00640 1 -- 172.21.15.42:0/1119740000 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fd338027560 con 0x564233e44380 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.959+0000 7fd348b8f740 10 client.16047 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.959+0000 7fd348b8f740 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.959+0000 7fd348b8f740 20 client.16047 populate_metadata read hostname 'smithi042' 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.959+0000 7fd348b8f740 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564233e5dc30 0x564233e60020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.959+0000 7fd348b8f740 4 client.16047 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:40:35.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.959+0000 7fd348b8f740 1 -- 172.21.15.42:0/1119740000 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564233db7b70 con 0x564233e5dc30 2024-04-20T10:40:35.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.959+0000 7fd336000640 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564233e5dc30 0x564233e60020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.960+0000 7fd336000640 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564233e5dc30 0x564233e60020 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbffa7e0cad1ca22 server_cookie=108c202efcf9ad36 in_seq=0 out_seq=0 2024-04-20T10:40:35.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.960+0000 7fd32cc00640 10 client.16047 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:35.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd32cc00640 1 -- 172.21.15.42:0/1119740000 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564233db7b70 con 0x564233e5dc30 2024-04-20T10:40:35.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd32cc00640 10 client.16047 handle_command_reply: tid=0 2024-04-20T10:40:35.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd348b8f740 1 client.16047 shutdown 2024-04-20T10:40:35.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd348b8f740 1 -- 172.21.15.42:0/1119740000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 msgr2=0x564233e4a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:35.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd348b8f740 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 0x564233e4a940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd338005b40 tx=0x7fd338005a70 comp rx=0 tx=0).stop 2024-04-20T10:40:35.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd348b8f740 1 -- 172.21.15.42:0/1119740000 shutdown_connections 2024-04-20T10:40:35.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd348b8f740 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564233e5dc30 0x564233e60020 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd348b8f740 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e4d880 0x564233e4b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd348b8f740 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e44ca0 0x564233e4ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd32cc00640 0 client.16047 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:35.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd348b8f740 1 --2- 172.21.15.42:0/1119740000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44380 0x564233e4a940 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd348b8f740 1 -- 172.21.15.42:0/1119740000 >> 172.21.15.42:0/1119740000 conn(0x564233e3eab0 msgr2=0x564233e40800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:35.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd348b8f740 1 -- 172.21.15.42:0/1119740000 shutdown_connections 2024-04-20T10:40:35.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.962+0000 7fd348b8f740 1 -- 172.21.15.42:0/1119740000 wait complete. 2024-04-20T10:40:35.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.963+0000 7fd348b8f740 20 client.16047 trim_cache size 0 max 16384 2024-04-20T10:40:35.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.981+0000 7fd348b8f740 1 Processor -- start 2024-04-20T10:40:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.981+0000 7fd348b8f740 1 -- start start 2024-04-20T10:40:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.981+0000 7fd348b8f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44380 0x564233e59530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.981+0000 7fd348b8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44ca0 0x564233e3d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.981+0000 7fd348b8f740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e4d880 0x564233e4a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.981+0000 7fd348b8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd3401ade10 con 0x564233e44ca0 2024-04-20T10:40:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.981+0000 7fd348b8f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3401b0690 con 0x564233e44380 2024-04-20T10:40:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.981+0000 7fd348b8f740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd3401ae480 con 0x564233e4d880 2024-04-20T10:40:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.982+0000 7fd33f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44380 0x564233e59530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.982+0000 7fd33f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44380 0x564233e59530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:38360/0 (socket says 172.21.15.42:38360) 2024-04-20T10:40:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.982+0000 7fd33f400640 1 -- 172.21.15.42:0/2020856358 learned_addr learned my addr 172.21.15.42:0/2020856358 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.982+0000 7fd336000640 1 --2- 172.21.15.42:0/2020856358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44ca0 0x564233e3d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.982+0000 7fd33f400640 1 --2- 172.21.15.42:0/2020856358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e4d880 0x564233e4a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.982+0000 7fd33f400640 1 -- 172.21.15.42:0/2020856358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e4d880 msgr2=0x564233e4a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.982+0000 7fd33f400640 1 --2- 172.21.15.42:0/2020856358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e4d880 0x564233e4a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.982+0000 7fd33f400640 1 -- 172.21.15.42:0/2020856358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44ca0 msgr2=0x564233e3d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.982+0000 7fd33f400640 1 --2- 172.21.15.42:0/2020856358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44ca0 0x564233e3d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.982+0000 7fd33f400640 1 -- 172.21.15.42:0/2020856358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564233e44ad0 con 0x564233e44380 2024-04-20T10:40:35.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.982+0000 7fd33f400640 1 --2- 172.21.15.42:0/2020856358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44380 0x564233e59530 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd338000f80 tx=0x7fd338009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4027cc3c690505bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.982+0000 7fd32f400640 1 -- 172.21.15.42:0/2020856358 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33802c040 con 0x564233e44380 2024-04-20T10:40:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.983+0000 7fd32f400640 1 -- 172.21.15.42:0/2020856358 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd338003f10 con 0x564233e44380 2024-04-20T10:40:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.983+0000 7fd32f400640 1 -- 172.21.15.42:0/2020856358 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338004c20 con 0x564233e44380 2024-04-20T10:40:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.983+0000 7fd348b8f740 1 -- 172.21.15.42:0/2020856358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44380 msgr2=0x564233e59530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.983+0000 7fd348b8f740 1 --2- 172.21.15.42:0/2020856358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44380 0x564233e59530 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd338000f80 tx=0x7fd338009500 comp rx=0 tx=0).stop 2024-04-20T10:40:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.983+0000 7fd348b8f740 1 -- 172.21.15.42:0/2020856358 shutdown_connections 2024-04-20T10:40:35.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.983+0000 7fd348b8f740 1 --2- 172.21.15.42:0/2020856358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e4d880 0x564233e4a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.983+0000 7fd348b8f740 1 --2- 172.21.15.42:0/2020856358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e44ca0 0x564233e3d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.983+0000 7fd348b8f740 1 --2- 172.21.15.42:0/2020856358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44380 0x564233e59530 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.983+0000 7fd348b8f740 1 -- 172.21.15.42:0/2020856358 >> 172.21.15.42:0/2020856358 conn(0x564233e3eab0 msgr2=0x564233e4b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.983+0000 7fd348b8f740 1 -- 172.21.15.42:0/2020856358 shutdown_connections 2024-04-20T10:40:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.984+0000 7fd348b8f740 1 -- 172.21.15.42:0/2020856358 wait complete. 2024-04-20T10:40:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.984+0000 7fd348b8f740 1 -- start start 2024-04-20T10:40:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.984+0000 7fd348b8f740 1 Processor -- start 2024-04-20T10:40:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd348b8f740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44380 0x564233e55a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd348b8f740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e44ca0 0x564233e55f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd348b8f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e4d880 0x564233e46bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd348b8f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd3401aeb50 con 0x564233e4d880 2024-04-20T10:40:35.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd348b8f740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3401affc0 con 0x564233e44380 2024-04-20T10:40:35.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd348b8f740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd3401af220 con 0x564233e44ca0 2024-04-20T10:40:35.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd348b8f740 10 client.0 _ll_register_callbacks cb 0x7fd3300d44f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:40:35.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd33f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44380 0x564233e55a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd33f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e4d880 0x564233e46bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd33f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e4d880 0x564233e46bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38454/0 (socket says 172.21.15.42:38454) 2024-04-20T10:40:35.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd33f400640 1 -- 172.21.15.42:0/4263315052 learned_addr learned my addr 172.21.15.42:0/4263315052 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:40:35.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd336000640 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e44ca0 0x564233e55f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd33f400640 1 -- 172.21.15.42:0/4263315052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e44ca0 msgr2=0x564233e55f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:35.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd33f400640 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e44ca0 0x564233e55f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd33f400640 1 -- 172.21.15.42:0/4263315052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44380 msgr2=0x564233e55a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:40:35.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd33f400640 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44380 0x564233e55a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd33f400640 1 -- 172.21.15.42:0/4263315052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564233e47170 con 0x564233e4d880 2024-04-20T10:40:35.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.985+0000 7fd336000640 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e44ca0 0x564233e55f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:40:35.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.986+0000 7fd33f400640 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e4d880 0x564233e46bb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd338006010 tx=0x7fd3380118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd4a44ee58db420 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:40:35.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.986+0000 7fd32cc00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:40:35.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.986+0000 7fd32cc00640 1 -- 172.21.15.42:0/4263315052 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33802c040 con 0x564233e4d880 2024-04-20T10:40:35.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.986+0000 7fd32cc00640 1 -- 172.21.15.42:0/4263315052 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd338027440 con 0x564233e4d880 2024-04-20T10:40:35.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.986+0000 7fd348b8f740 1 -- 172.21.15.42:0/4263315052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd338003f10 con 0x564233e4d880 2024-04-20T10:40:35.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.986+0000 7fd32cc00640 1 -- 172.21.15.42:0/4263315052 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd338027640 con 0x564233e4d880 2024-04-20T10:40:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.986+0000 7fd348b8f740 10 client.16053 fetch_fsmap learned FSMap version 19 2024-04-20T10:40:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.986+0000 7fd348b8f740 1 -- 172.21.15.42:0/4263315052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x564233ec38d0 con 0x564233e4d880 2024-04-20T10:40:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.987+0000 7fd32cc00640 1 -- 172.21.15.42:0/4263315052 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd338003f10 con 0x564233e4d880 2024-04-20T10:40:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.987+0000 7fd32cc00640 1 -- 172.21.15.42:0/4263315052 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fd33800fc80 con 0x564233e4d880 2024-04-20T10:40:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.987+0000 7fd348b8f740 10 client.16053 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:40:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.987+0000 7fd348b8f740 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:40:35.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.987+0000 7fd348b8f740 20 client.16053 populate_metadata read hostname 'smithi042' 2024-04-20T10:40:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.987+0000 7fd348b8f740 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564233ec3ee0 0x564233ec62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:40:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.987+0000 7fd348b8f740 4 client.16053 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:40:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.987+0000 7fd348b8f740 1 -- 172.21.15.42:0/4263315052 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564233db19e0 con 0x564233ec3ee0 2024-04-20T10:40:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.987+0000 7fd336000640 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564233ec3ee0 0x564233ec62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:40:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.989+0000 7fd336000640 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564233ec3ee0 0x564233ec62d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54f0d5aa4af56083 server_cookie=6212ab12e6f197b0 in_seq=0 out_seq=0 2024-04-20T10:40:35.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.989+0000 7fd32cc00640 10 client.16053 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:35.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.990+0000 7fd32cc00640 1 -- 172.21.15.42:0/4263315052 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x564233db19e0 con 0x564233ec3ee0 2024-04-20T10:40:35.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.990+0000 7fd32cc00640 10 client.16053 handle_command_reply: tid=0 2024-04-20T10:40:35.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd348b8f740 1 client.16053 shutdown 2024-04-20T10:40:35.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd348b8f740 1 -- 172.21.15.42:0/4263315052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e4d880 msgr2=0x564233e46bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:35.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd348b8f740 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e4d880 0x564233e46bb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd338006010 tx=0x7fd3380118d0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd348b8f740 1 -- 172.21.15.42:0/4263315052 shutdown_connections 2024-04-20T10:40:35.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd348b8f740 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564233ec3ee0 0x564233ec62d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd348b8f740 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564233e4d880 0x564233e46bb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd348b8f740 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564233e44ca0 0x564233e55f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd32cc00640 0 client.16053 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:40:35.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd348b8f740 1 --2- 172.21.15.42:0/4263315052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564233e44380 0x564233e55a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:35.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd348b8f740 1 -- 172.21.15.42:0/4263315052 >> 172.21.15.42:0/4263315052 conn(0x564233e3eab0 msgr2=0x564233e3fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:35.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd348b8f740 1 -- 172.21.15.42:0/4263315052 shutdown_connections 2024-04-20T10:40:35.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd348b8f740 1 -- 172.21.15.42:0/4263315052 wait complete. 2024-04-20T10:40:35.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.991+0000 7fd348b8f740 20 client.16053 trim_cache size 0 max 16384 2024-04-20T10:40:35.999 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:40:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (3914 inodes in the stack)", 2024-04-20T10:40:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:40:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "47829d3f-d202-4c03-8498-5cd769840bae": { 2024-04-20T10:40:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:40:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "47829d3f-d202-4c03-8498-5cd769840bae", 2024-04-20T10:40:35.999 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:40:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:40:36.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:40:36.000 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:40:36.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.992+0000 7fd33fe00640 1 -- 172.21.15.42:0/2009125826 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd3200a3da0 msgr2=0x7fd3200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:36.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.992+0000 7fd33fe00640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd3200a3da0 0x7fd3200a6260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x564233e43e50 tx=0x7fd3300d4620 comp rx=0 tx=0).stop 2024-04-20T10:40:36.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.992+0000 7fd33fe00640 1 -- 172.21.15.42:0/2009125826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd340074330 msgr2=0x7fd3401a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:40:36.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.992+0000 7fd33fe00640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd340074330 0x7fd3401a0650 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd33000e9f0 tx=0x7fd33000eeb0 comp rx=0 tx=0).stop 2024-04-20T10:40:36.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.992+0000 7fd33fe00640 1 -- 172.21.15.42:0/2009125826 shutdown_connections 2024-04-20T10:40:36.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.992+0000 7fd33fe00640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd3200a3da0 0x7fd3200a6260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:36.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.992+0000 7fd33fe00640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd34010e6c0 0x7fd34019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:36.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.992+0000 7fd33fe00640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd340074c50 0x7fd3401a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:36.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.992+0000 7fd33fe00640 1 --2- 172.21.15.42:0/2009125826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd340074330 0x7fd3401a0650 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:40:36.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.992+0000 7fd33fe00640 1 -- 172.21.15.42:0/2009125826 >> 172.21.15.42:0/2009125826 conn(0x7fd3400fd930 msgr2=0x7fd3400ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:40:36.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.992+0000 7fd33fe00640 1 -- 172.21.15.42:0/2009125826 shutdown_connections 2024-04-20T10:40:36.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:40:35.992+0000 7fd33fe00640 1 -- 172.21.15.42:0/2009125826 wait complete. 2024-04-20T10:40:36.006 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:35 smithi042 ceph-mon[17078]: pgmap v200: 97 pgs: 97 active+clean; 1.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 1.27k op/s 2024-04-20T10:40:36.008 INFO:tasks.cephfs.filesystem:scrub status for tag:47829d3f-d202-4c03-8498-5cd769840bae - {'path': '/', 'tag': '47829d3f-d202-4c03-8498-5cd769840bae', 'options': 'recursive,force'} 2024-04-20T10:40:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:35 smithi046 ceph-mon[21574]: pgmap v200: 97 pgs: 97 active+clean; 1.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 1.27k op/s 2024-04-20T10:40:36.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:35 smithi162 ceph-mon[21854]: pgmap v200: 97 pgs: 97 active+clean; 1.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 1.27k op/s 2024-04-20T10:40:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:37 smithi046 ceph-mon[21574]: pgmap v201: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 1.67k op/s 2024-04-20T10:40:38.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:37 smithi042 ceph-mon[17078]: pgmap v201: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 1.67k op/s 2024-04-20T10:40:38.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:37 smithi162 ceph-mon[21854]: pgmap v201: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 1.67k op/s 2024-04-20T10:40:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:39 smithi046 ceph-mon[21574]: pgmap v202: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.6 MiB/s wr, 2.13k op/s 2024-04-20T10:40:40.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:39 smithi042 ceph-mon[17078]: pgmap v202: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.6 MiB/s wr, 2.13k op/s 2024-04-20T10:40:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:39 smithi162 ceph-mon[21854]: pgmap v202: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.6 MiB/s wr, 2.13k op/s 2024-04-20T10:40:42.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:41 smithi046 ceph-mon[21574]: pgmap v203: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.6 MiB/s wr, 1.68k op/s 2024-04-20T10:40:42.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:41 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:40:42.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:41 smithi042 ceph-mon[17078]: pgmap v203: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.6 MiB/s wr, 1.68k op/s 2024-04-20T10:40:42.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:41 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:40:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:41 smithi162 ceph-mon[21854]: pgmap v203: 97 pgs: 97 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.6 MiB/s wr, 1.68k op/s 2024-04-20T10:40:42.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:41 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:40:44.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:43 smithi046 ceph-mon[21574]: pgmap v204: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.2 MiB/s wr, 1.86k op/s 2024-04-20T10:40:44.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:43 smithi042 ceph-mon[17078]: pgmap v204: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.2 MiB/s wr, 1.86k op/s 2024-04-20T10:40:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:43 smithi162 ceph-mon[21854]: pgmap v204: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.2 MiB/s wr, 1.86k op/s 2024-04-20T10:40:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:45 smithi046 ceph-mon[21574]: pgmap v205: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 1.28k op/s 2024-04-20T10:40:46.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:45 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:40:46.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:45 smithi042 ceph-mon[17078]: pgmap v205: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 1.28k op/s 2024-04-20T10:40:46.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:45 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:40:46.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:45 smithi162 ceph-mon[21854]: pgmap v205: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 1.28k op/s 2024-04-20T10:40:46.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:45 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:40:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:47 smithi046 ceph-mon[21574]: pgmap v206: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.8 MiB/s wr, 1.29k op/s 2024-04-20T10:40:48.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:47 smithi042 ceph-mon[17078]: pgmap v206: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.8 MiB/s wr, 1.29k op/s 2024-04-20T10:40:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:47 smithi162 ceph-mon[21854]: pgmap v206: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.8 MiB/s wr, 1.29k op/s 2024-04-20T10:40:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:49 smithi046 ceph-mon[21574]: pgmap v207: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 886 op/s 2024-04-20T10:40:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:49 smithi042 ceph-mon[17078]: pgmap v207: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 886 op/s 2024-04-20T10:40:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:49 smithi162 ceph-mon[21854]: pgmap v207: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 886 op/s 2024-04-20T10:40:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:51 smithi046 ceph-mon[21574]: pgmap v208: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 753 KiB/s wr, 188 op/s 2024-04-20T10:40:52.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:51 smithi042 ceph-mon[17078]: pgmap v208: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 753 KiB/s wr, 188 op/s 2024-04-20T10:40:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:51 smithi162 ceph-mon[21854]: pgmap v208: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 753 KiB/s wr, 188 op/s 2024-04-20T10:40:54.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:53 smithi046 ceph-mon[21574]: pgmap v209: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 856 KiB/s wr, 194 op/s 2024-04-20T10:40:54.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:53 smithi042 ceph-mon[17078]: pgmap v209: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 856 KiB/s wr, 194 op/s 2024-04-20T10:40:54.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:53 smithi162 ceph-mon[21854]: pgmap v209: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 856 KiB/s wr, 194 op/s 2024-04-20T10:40:55.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:54 smithi046 ceph-mon[21574]: pgmap v210: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 211 KiB/s wr, 10 op/s 2024-04-20T10:40:55.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:54 smithi042 ceph-mon[17078]: pgmap v210: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 211 KiB/s wr, 10 op/s 2024-04-20T10:40:55.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:54 smithi162 ceph-mon[21854]: pgmap v210: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 211 KiB/s wr, 10 op/s 2024-04-20T10:40:57.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:57 smithi042 ceph-mon[17078]: pgmap v211: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 555 KiB/s wr, 22 op/s 2024-04-20T10:40:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:57 smithi162 ceph-mon[21854]: pgmap v211: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 555 KiB/s wr, 22 op/s 2024-04-20T10:40:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:57 smithi046 ceph-mon[21574]: pgmap v211: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 555 KiB/s wr, 22 op/s 2024-04-20T10:40:59.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:40:59 smithi042 ceph-mon[17078]: pgmap v212: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 491 KiB/s wr, 91 op/s 2024-04-20T10:40:59.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:40:59 smithi162 ceph-mon[21854]: pgmap v212: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 491 KiB/s wr, 91 op/s 2024-04-20T10:41:00.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:40:59 smithi046 ceph-mon[21574]: pgmap v212: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 491 KiB/s wr, 91 op/s 2024-04-20T10:41:01.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:01 smithi042 ceph-mon[17078]: pgmap v213: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 485 KiB/s wr, 89 op/s 2024-04-20T10:41:01.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:01 smithi162 ceph-mon[21854]: pgmap v213: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 485 KiB/s wr, 89 op/s 2024-04-20T10:41:02.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:01 smithi046 ceph-mon[21574]: pgmap v213: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 485 KiB/s wr, 89 op/s 2024-04-20T10:41:03.855 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:03 smithi046 ceph-mon[21574]: pgmap v214: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.4 MiB/s wr, 582 op/s 2024-04-20T10:41:03.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:03 smithi162 ceph-mon[21854]: pgmap v214: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.4 MiB/s wr, 582 op/s 2024-04-20T10:41:04.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:03 smithi042 ceph-mon[17078]: pgmap v214: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.4 MiB/s wr, 582 op/s 2024-04-20T10:41:05.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:05 smithi162 ceph-mon[21854]: pgmap v215: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.3 MiB/s wr, 576 op/s 2024-04-20T10:41:05.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:05 smithi042 ceph-mon[17078]: pgmap v215: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.3 MiB/s wr, 576 op/s 2024-04-20T10:41:06.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:41:06.010 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:41:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:05 smithi046 ceph-mon[21574]: pgmap v215: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.3 MiB/s wr, 576 op/s 2024-04-20T10:41:06.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.255+0000 7fd60d600640 1 -- 172.21.15.42:0/3740614243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd608107150 msgr2=0x7fd60810ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.255+0000 7fd60d600640 1 --2- 172.21.15.42:0/3740614243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd608107150 0x7fd60810ed20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd600009ca0 tx=0x7fd60002c100 comp rx=0 tx=0).stop 2024-04-20T10:41:06.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.255+0000 7fd60d600640 1 -- 172.21.15.42:0/3740614243 shutdown_connections 2024-04-20T10:41:06.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.255+0000 7fd60d600640 1 --2- 172.21.15.42:0/3740614243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd608107150 0x7fd60810ed20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.255+0000 7fd60d600640 1 --2- 172.21.15.42:0/3740614243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd608102a30 0x7fd608106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.255+0000 7fd60d600640 1 --2- 172.21.15.42:0/3740614243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd608102080 0x7fd608102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.255+0000 7fd60d600640 1 -- 172.21.15.42:0/3740614243 >> 172.21.15.42:0/3740614243 conn(0x7fd6080fd900 msgr2=0x7fd6080ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:06.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.255+0000 7fd60d600640 1 -- 172.21.15.42:0/3740614243 shutdown_connections 2024-04-20T10:41:06.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.255+0000 7fd60d600640 1 -- 172.21.15.42:0/3740614243 wait complete. 2024-04-20T10:41:06.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd60d600640 1 Processor -- start 2024-04-20T10:41:06.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd60d600640 1 -- start start 2024-04-20T10:41:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd60d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd608102080 0x7fd6081a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd60d600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd608107150 0x7fd6081a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd60d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd60819a730 0x7fd60819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd60d600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081121f0 con 0x7fd60819a730 2024-04-20T10:41:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd60d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd608112070 con 0x7fd608102080 2024-04-20T10:41:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd60d600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd608112370 con 0x7fd608107150 2024-04-20T10:41:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd607e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd608102080 0x7fd6081a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd607400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd608107150 0x7fd6081a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd60cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd60819a730 0x7fd60819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd607e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd608102080 0x7fd6081a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:44444/0 (socket says 172.21.15.42:44444) 2024-04-20T10:41:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.257+0000 7fd607e00640 1 -- 172.21.15.42:0/2051417004 learned_addr learned my addr 172.21.15.42:0/2051417004 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:06.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.258+0000 7fd607e00640 1 -- 172.21.15.42:0/2051417004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd608107150 msgr2=0x7fd6081a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.258+0000 7fd607e00640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd608107150 0x7fd6081a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.258+0000 7fd607e00640 1 -- 172.21.15.42:0/2051417004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd60819a730 msgr2=0x7fd60819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.258+0000 7fd607e00640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd60819a730 0x7fd60819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.258+0000 7fd607e00640 1 -- 172.21.15.42:0/2051417004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60819b330 con 0x7fd608102080 2024-04-20T10:41:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.258+0000 7fd607400640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd608107150 0x7fd6081a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:41:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.258+0000 7fd60cc00640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd60819a730 0x7fd60819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:41:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.258+0000 7fd607e00640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd608102080 0x7fd6081a0670 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800e9f0 tx=0x7fd5f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.258+0000 7fd604c00640 1 -- 172.21.15.42:0/2051417004 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800cd80 con 0x7fd608102080 2024-04-20T10:41:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.258+0000 7fd604c00640 1 -- 172.21.15.42:0/2051417004 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f8010910 con 0x7fd608102080 2024-04-20T10:41:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.258+0000 7fd604c00640 1 -- 172.21.15.42:0/2051417004 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8018990 con 0x7fd608102080 2024-04-20T10:41:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.259+0000 7fd60d600640 1 -- 172.21.15.42:0/2051417004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6081a7410 con 0x7fd608102080 2024-04-20T10:41:06.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.259+0000 7fd60d600640 1 -- 172.21.15.42:0/2051417004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6081a7870 con 0x7fd608102080 2024-04-20T10:41:06.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.262+0000 7fd611882740 1 Processor -- start 2024-04-20T10:41:06.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.262+0000 7fd611882740 1 -- start start 2024-04-20T10:41:06.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.262+0000 7fd611882740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.262+0000 7fd611882740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030a7ca0 0x5638030b0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.262+0000 7fd604c00640 1 -- 172.21.15.42:0/2051417004 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd5f8010ac0 con 0x7fd608102080 2024-04-20T10:41:06.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.262+0000 7fd611882740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030b0880 0x5638030b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.262+0000 7fd611882740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56380300d150 con 0x5638030a7ca0 2024-04-20T10:41:06.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.262+0000 7fd611882740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563802fb3370 con 0x5638030b0880 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.262+0000 7fd611882740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563803009110 con 0x5638030a7380 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.262+0000 7fd60cc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.262+0000 7fd60cc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030a7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:59994/0 (socket says 172.21.15.42:59994) 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.262+0000 7fd60cc00640 1 -- 172.21.15.42:0/2169558062 learned_addr learned my addr 172.21.15.42:0/2169558062 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.263+0000 7fd607e00640 1 --2- 172.21.15.42:0/2169558062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030a7ca0 0x5638030b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.263+0000 7fd607400640 1 --2- 172.21.15.42:0/2169558062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030b0880 0x5638030b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.263+0000 7fd60cc00640 1 -- 172.21.15.42:0/2169558062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030b0880 msgr2=0x5638030b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.263+0000 7fd60cc00640 1 --2- 172.21.15.42:0/2169558062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030b0880 0x5638030b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.263+0000 7fd60cc00640 1 -- 172.21.15.42:0/2169558062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030a7ca0 msgr2=0x5638030b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.263+0000 7fd60cc00640 1 --2- 172.21.15.42:0/2169558062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030a7ca0 0x5638030b0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.263+0000 7fd60cc00640 1 -- 172.21.15.42:0/2169558062 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563802de97d0 con 0x5638030a7380 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.263+0000 7fd60cc00640 1 --2- 172.21.15.42:0/2169558062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030a7760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd600009ca0 tx=0x7fd600009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bab03e2626e61c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.264+0000 7fd5fd200640 1 -- 172.21.15.42:0/2169558062 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600043070 con 0x5638030a7380 2024-04-20T10:41:06.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.264+0000 7fd5fd200640 1 -- 172.21.15.42:0/2169558062 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd600003970 con 0x5638030a7380 2024-04-20T10:41:06.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.264+0000 7fd5fd200640 1 -- 172.21.15.42:0/2169558062 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600039040 con 0x5638030a7380 2024-04-20T10:41:06.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.264+0000 7fd611882740 1 -- 172.21.15.42:0/2169558062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 msgr2=0x5638030a7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.264+0000 7fd611882740 1 --2- 172.21.15.42:0/2169558062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030a7760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd600009ca0 tx=0x7fd600009900 comp rx=0 tx=0).stop 2024-04-20T10:41:06.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.264+0000 7fd611882740 1 -- 172.21.15.42:0/2169558062 shutdown_connections 2024-04-20T10:41:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.264+0000 7fd611882740 1 --2- 172.21.15.42:0/2169558062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030b0880 0x5638030b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.264+0000 7fd604c00640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd5ec0a3da0 0x7fd5ec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.264+0000 7fd611882740 1 --2- 172.21.15.42:0/2169558062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030a7ca0 0x5638030b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.264+0000 7fd611882740 1 --2- 172.21.15.42:0/2169558062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030a7760 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.264+0000 7fd611882740 1 -- 172.21.15.42:0/2169558062 >> 172.21.15.42:0/2169558062 conn(0x5638030a1ab0 msgr2=0x5638030a1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.265+0000 7fd604c00640 1 -- 172.21.15.42:0/2051417004 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7fd5f80d0ae0 con 0x7fd608102080 2024-04-20T10:41:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.265+0000 7fd611882740 1 -- 172.21.15.42:0/2169558062 shutdown_connections 2024-04-20T10:41:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.265+0000 7fd60cc00640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd5ec0a3da0 0x7fd5ec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.265+0000 7fd611882740 1 -- 172.21.15.42:0/2169558062 wait complete. 2024-04-20T10:41:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.265+0000 7fd611882740 1 -- start start 2024-04-20T10:41:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.265+0000 7fd60cc00640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd5ec0a3da0 0x7fd5ec0a6260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd60002c640 tx=0x7fd60002c100 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.265+0000 7fd611882740 1 Processor -- start 2024-04-20T10:41:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.266+0000 7fd611882740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030a7380 0x5638030ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.266+0000 7fd611882740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7ca0 0x5638030ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.266+0000 7fd611882740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030b0880 0x5638030ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.266+0000 7fd611882740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5638030b3610 con 0x5638030a7380 2024-04-20T10:41:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.266+0000 7fd611882740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563802fc6740 con 0x5638030b0880 2024-04-20T10:41:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.266+0000 7fd611882740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5638030b3790 con 0x5638030a7ca0 2024-04-20T10:41:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.266+0000 7fd611882740 10 client.0 _ll_register_callbacks cb 0x563802fc1b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:41:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.266+0000 7fd607400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7ca0 0x5638030ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.266+0000 7fd607400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7ca0 0x5638030ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:60010/0 (socket says 172.21.15.42:60010) 2024-04-20T10:41:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.266+0000 7fd607400640 1 -- 172.21.15.42:0/3700564588 learned_addr learned my addr 172.21.15.42:0/3700564588 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.266+0000 7fd60cc00640 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030b0880 0x5638030ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.266+0000 7fd607e00640 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030a7380 0x5638030ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.267+0000 7fd607400640 1 -- 172.21.15.42:0/3700564588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030b0880 msgr2=0x5638030ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.267+0000 7fd607400640 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030b0880 0x5638030ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.267+0000 7fd607400640 1 -- 172.21.15.42:0/3700564588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030a7380 msgr2=0x5638030ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.267+0000 7fd607400640 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030a7380 0x5638030ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.267+0000 7fd607400640 1 -- 172.21.15.42:0/3700564588 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563802d1f3f0 con 0x5638030a7ca0 2024-04-20T10:41:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.267+0000 7fd607400640 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7ca0 0x5638030ade30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4012d50 tx=0x7fd5f400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f50970e529839ead server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.267+0000 7fd5f1600640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:41:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.267+0000 7fd5f1600640 1 -- 172.21.15.42:0/3700564588 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4012110 con 0x5638030a7ca0 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.267+0000 7fd611882740 1 -- 172.21.15.42:0/3700564588 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd600003970 con 0x5638030a7ca0 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.267+0000 7fd5f1600640 1 -- 172.21.15.42:0/3700564588 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7fd5f4004050 con 0x5638030a7ca0 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.267+0000 7fd5f1600640 1 -- 172.21.15.42:0/3700564588 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd5f40122b0 con 0x5638030a7ca0 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.268+0000 7fd611882740 10 client.26194 fetch_fsmap learned FSMap version 19 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.268+0000 7fd611882740 1 -- 172.21.15.42:0/3700564588 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563802cdf0f0 con 0x5638030a7ca0 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.268+0000 7fd5f1600640 1 -- 172.21.15.42:0/3700564588 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd600003970 con 0x5638030a7ca0 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.268+0000 7fd5f1600640 1 -- 172.21.15.42:0/3700564588 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fd5f4010430 con 0x5638030a7ca0 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.268+0000 7fd611882740 10 client.26194 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.268+0000 7fd611882740 10 client.26194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.268+0000 7fd611882740 20 client.26194 populate_metadata read hostname 'smithi042' 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.268+0000 7fd611882740 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5638030c0cf0 0x5638030c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.268+0000 7fd611882740 4 client.26194 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.268+0000 7fd611882740 1 -- 172.21.15.42:0/3700564588 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56380301ab70 con 0x5638030c0cf0 2024-04-20T10:41:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.269+0000 7fd607e00640 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5638030c0cf0 0x5638030c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.269+0000 7fd607e00640 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5638030c0cf0 0x5638030c3170 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6623051071b89948 server_cookie=e2e2dc362c470479 in_seq=0 out_seq=0 2024-04-20T10:41:06.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.269+0000 7fd5f1600640 10 client.26194 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:06.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd5f1600640 1 -- 172.21.15.42:0/3700564588 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56380301ab70 con 0x5638030c0cf0 2024-04-20T10:41:06.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd5f1600640 10 client.26194 handle_command_reply: tid=0 2024-04-20T10:41:06.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd611882740 1 client.26194 shutdown 2024-04-20T10:41:06.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd611882740 1 -- 172.21.15.42:0/3700564588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7ca0 msgr2=0x5638030ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd611882740 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7ca0 0x5638030ade30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4012d50 tx=0x7fd5f400f790 comp rx=0 tx=0).stop 2024-04-20T10:41:06.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd611882740 1 -- 172.21.15.42:0/3700564588 shutdown_connections 2024-04-20T10:41:06.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd611882740 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5638030c0cf0 0x5638030c3170 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd611882740 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030b0880 0x5638030ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd611882740 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7ca0 0x5638030ade30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd611882740 1 --2- 172.21.15.42:0/3700564588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030a7380 0x5638030ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd611882740 1 -- 172.21.15.42:0/3700564588 >> 172.21.15.42:0/3700564588 conn(0x5638030a1ab0 msgr2=0x5638030a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:06.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd5f1600640 0 client.26194 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:06.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd611882740 1 -- 172.21.15.42:0/3700564588 shutdown_connections 2024-04-20T10:41:06.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.271+0000 7fd611882740 1 -- 172.21.15.42:0/3700564588 wait complete. 2024-04-20T10:41:06.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.272+0000 7fd611882740 20 client.26194 trim_cache size 0 max 16384 2024-04-20T10:41:06.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.291+0000 7fd611882740 1 Processor -- start 2024-04-20T10:41:06.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.291+0000 7fd611882740 1 -- start start 2024-04-20T10:41:06.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.291+0000 7fd611882740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030b0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.291+0000 7fd611882740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030a7ca0 0x5638030bc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.291+0000 7fd611882740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030b0880 0x5638030adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.291+0000 7fd611882740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081ade00 con 0x5638030b0880 2024-04-20T10:41:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.291+0000 7fd611882740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081b0660 con 0x5638030a7ca0 2024-04-20T10:41:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.291+0000 7fd611882740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081ae450 con 0x5638030a7380 2024-04-20T10:41:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd607e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd607e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:60020/0 (socket says 172.21.15.42:60020) 2024-04-20T10:41:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd607e00640 1 -- 172.21.15.42:0/1711220416 learned_addr learned my addr 172.21.15.42:0/1711220416 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd607400640 1 --2- 172.21.15.42:0/1711220416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030a7ca0 0x5638030bc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd60cc00640 1 --2- 172.21.15.42:0/1711220416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030b0880 0x5638030adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd607e00640 1 -- 172.21.15.42:0/1711220416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030a7ca0 msgr2=0x5638030bc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd607e00640 1 --2- 172.21.15.42:0/1711220416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030a7ca0 0x5638030bc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd607e00640 1 -- 172.21.15.42:0/1711220416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030b0880 msgr2=0x5638030adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd607e00640 1 --2- 172.21.15.42:0/1711220416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030b0880 0x5638030adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd607e00640 1 -- 172.21.15.42:0/1711220416 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638030ac8c0 con 0x5638030a7380 2024-04-20T10:41:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd60cc00640 1 --2- 172.21.15.42:0/1711220416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030b0880 0x5638030adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:41:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd607400640 1 --2- 172.21.15.42:0/1711220416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030a7ca0 0x5638030bc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:41:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd607e00640 1 --2- 172.21.15.42:0/1711220416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030b0020 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80ce230 tx=0x7fd5f80d0f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a67d4626bb1eb298 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd5f3e00640 1 -- 172.21.15.42:0/1711220416 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80a0dd0 con 0x5638030a7380 2024-04-20T10:41:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd5f3e00640 1 -- 172.21.15.42:0/1711220416 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f80a5690 con 0x5638030a7380 2024-04-20T10:41:06.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.292+0000 7fd5f3e00640 1 -- 172.21.15.42:0/1711220416 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80a5200 con 0x5638030a7380 2024-04-20T10:41:06.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.293+0000 7fd611882740 1 -- 172.21.15.42:0/1711220416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 msgr2=0x5638030b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.293+0000 7fd611882740 1 --2- 172.21.15.42:0/1711220416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030b0020 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80ce230 tx=0x7fd5f80d0f90 comp rx=0 tx=0).stop 2024-04-20T10:41:06.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.293+0000 7fd611882740 1 -- 172.21.15.42:0/1711220416 shutdown_connections 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.293+0000 7fd611882740 1 --2- 172.21.15.42:0/1711220416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030b0880 0x5638030adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.293+0000 7fd611882740 1 --2- 172.21.15.42:0/1711220416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030a7ca0 0x5638030bc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.293+0000 7fd611882740 1 --2- 172.21.15.42:0/1711220416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030b0020 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.293+0000 7fd611882740 1 -- 172.21.15.42:0/1711220416 >> 172.21.15.42:0/1711220416 conn(0x5638030a1ab0 msgr2=0x5638030ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.293+0000 7fd611882740 1 -- 172.21.15.42:0/1711220416 shutdown_connections 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.293+0000 7fd611882740 1 -- 172.21.15.42:0/1711220416 wait complete. 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.294+0000 7fd611882740 1 -- start start 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.294+0000 7fd611882740 1 Processor -- start 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.294+0000 7fd611882740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.294+0000 7fd611882740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030a7ca0 0x5638030b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.294+0000 7fd611882740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030b0880 0x5638030b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.294+0000 7fd611882740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081aeb20 con 0x5638030b0880 2024-04-20T10:41:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.294+0000 7fd611882740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081aff90 con 0x5638030a7ca0 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.294+0000 7fd611882740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081af1f0 con 0x5638030a7380 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.294+0000 7fd611882740 10 client.0 _ll_register_callbacks cb 0x7fd5f4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.294+0000 7fd607e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.294+0000 7fd607400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030a7ca0 0x5638030b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd607e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030b89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:60024/0 (socket says 172.21.15.42:60024) 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd607e00640 1 -- 172.21.15.42:0/573782183 learned_addr learned my addr 172.21.15.42:0/573782183 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd60cc00640 1 --2- 172.21.15.42:0/573782183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030b0880 0x5638030b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd607e00640 1 -- 172.21.15.42:0/573782183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030a7ca0 msgr2=0x5638030b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd607e00640 1 --2- 172.21.15.42:0/573782183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030a7ca0 0x5638030b8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd607e00640 1 -- 172.21.15.42:0/573782183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030b0880 msgr2=0x5638030b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd607e00640 1 --2- 172.21.15.42:0/573782183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030b0880 0x5638030b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd607e00640 1 -- 172.21.15.42:0/573782183 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638030a7ad0 con 0x5638030a7380 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd607e00640 1 --2- 172.21.15.42:0/573782183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030b89e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800cfd0 tx=0x7fd5f809cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef1253f6fc4771c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:06.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd5f1600640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd5f1600640 1 -- 172.21.15.42:0/573782183 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80a5200 con 0x5638030a7380 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd611882740 1 -- 172.21.15.42:0/573782183 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5f80a5690 con 0x5638030a7380 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.295+0000 7fd5f1600640 1 -- 172.21.15.42:0/573782183 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7fd5f80a53a0 con 0x5638030a7380 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.296+0000 7fd5f1600640 1 -- 172.21.15.42:0/573782183 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd5f8018b30 con 0x5638030a7380 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.296+0000 7fd611882740 10 client.26200 fetch_fsmap learned FSMap version 19 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.296+0000 7fd611882740 1 -- 172.21.15.42:0/573782183 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563803127a40 con 0x5638030a7380 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.296+0000 7fd5f1600640 1 -- 172.21.15.42:0/573782183 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f80a5690 con 0x5638030a7380 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.296+0000 7fd5f1600640 1 -- 172.21.15.42:0/573782183 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fd5f8098820 con 0x5638030a7380 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.296+0000 7fd611882740 10 client.26200 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.296+0000 7fd611882740 10 client.26200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.296+0000 7fd611882740 20 client.26200 populate_metadata read hostname 'smithi042' 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.296+0000 7fd611882740 1 --2- 172.21.15.42:0/573782183 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563803127f60 0x56380312a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.296+0000 7fd611882740 4 client.26200 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:41:06.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.296+0000 7fd611882740 1 -- 172.21.15.42:0/573782183 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5638030149e0 con 0x563803127f60 2024-04-20T10:41:06.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.297+0000 7fd607400640 1 --2- 172.21.15.42:0/573782183 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563803127f60 0x56380312a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.297+0000 7fd607400640 1 --2- 172.21.15.42:0/573782183 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563803127f60 0x56380312a380 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787626b747f8db5a server_cookie=e83ce04ec864c28b in_seq=0 out_seq=0 2024-04-20T10:41:06.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.297+0000 7fd5f1600640 10 client.26200 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:06.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd5f1600640 1 -- 172.21.15.42:0/573782183 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5638030149e0 con 0x563803127f60 2024-04-20T10:41:06.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd5f1600640 10 client.26200 handle_command_reply: tid=0 2024-04-20T10:41:06.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd611882740 1 client.26200 shutdown 2024-04-20T10:41:06.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd611882740 1 -- 172.21.15.42:0/573782183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 msgr2=0x5638030b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd611882740 1 --2- 172.21.15.42:0/573782183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030b89e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800cfd0 tx=0x7fd5f809cc50 comp rx=0 tx=0).stop 2024-04-20T10:41:06.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd611882740 1 -- 172.21.15.42:0/573782183 shutdown_connections 2024-04-20T10:41:06.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd611882740 1 --2- 172.21.15.42:0/573782183 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563803127f60 0x56380312a380 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd611882740 1 --2- 172.21.15.42:0/573782183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5638030b0880 0x5638030b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd611882740 1 --2- 172.21.15.42:0/573782183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638030a7ca0 0x5638030b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd5f1600640 0 client.26200 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:06.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd611882740 1 --2- 172.21.15.42:0/573782183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5638030a7380 0x5638030b89e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd611882740 1 -- 172.21.15.42:0/573782183 >> 172.21.15.42:0/573782183 conn(0x5638030a1ab0 msgr2=0x5638030a3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:06.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd611882740 1 -- 172.21.15.42:0/573782183 shutdown_connections 2024-04-20T10:41:06.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd611882740 1 -- 172.21.15.42:0/573782183 wait complete. 2024-04-20T10:41:06.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.299+0000 7fd611882740 20 client.26200 trim_cache size 0 max 16384 2024-04-20T10:41:06.304 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:41:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:41:06.304 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:41:06.304 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:41:06.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.300+0000 7fd60d600640 1 -- 172.21.15.42:0/2051417004 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd5ec0a3da0 msgr2=0x7fd5ec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.300+0000 7fd60d600640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd5ec0a3da0 0x7fd5ec0a6260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd60002c640 tx=0x7fd60002c100 comp rx=0 tx=0).stop 2024-04-20T10:41:06.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.300+0000 7fd60d600640 1 -- 172.21.15.42:0/2051417004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd608102080 msgr2=0x7fd6081a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.300+0000 7fd60d600640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd608102080 0x7fd6081a0670 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800e9f0 tx=0x7fd5f800eeb0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.300+0000 7fd60d600640 1 -- 172.21.15.42:0/2051417004 shutdown_connections 2024-04-20T10:41:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.300+0000 7fd60d600640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd5ec0a3da0 0x7fd5ec0a6260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.300+0000 7fd60d600640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd60819a730 0x7fd60819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.300+0000 7fd60d600640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd608107150 0x7fd6081a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.300+0000 7fd60d600640 1 --2- 172.21.15.42:0/2051417004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd608102080 0x7fd6081a0670 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.300+0000 7fd60d600640 1 -- 172.21.15.42:0/2051417004 >> 172.21.15.42:0/2051417004 conn(0x7fd6080fd900 msgr2=0x7fd6080fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.300+0000 7fd60d600640 1 -- 172.21.15.42:0/2051417004 shutdown_connections 2024-04-20T10:41:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.300+0000 7fd60d600640 1 -- 172.21.15.42:0/2051417004 wait complete. 2024-04-20T10:41:06.319 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:41:06.319 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:41:06.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.536+0000 7f5e49200640 1 -- 172.21.15.42:0/513003281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e44108da0 msgr2=0x7f5e44115210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.536+0000 7f5e49200640 1 --2- 172.21.15.42:0/513003281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e44108da0 0x7f5e44115210 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009900 tx=0x7f5e3c02bcd0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.536+0000 7f5e49200640 1 -- 172.21.15.42:0/513003281 shutdown_connections 2024-04-20T10:41:06.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.536+0000 7f5e49200640 1 --2- 172.21.15.42:0/513003281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e44108da0 0x7f5e44115210 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.536+0000 7f5e49200640 1 --2- 172.21.15.42:0/513003281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e44108400 0x7f5e44108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.536+0000 7f5e49200640 1 --2- 172.21.15.42:0/513003281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e4410e470 0x7f5e4410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.536+0000 7f5e49200640 1 -- 172.21.15.42:0/513003281 >> 172.21.15.42:0/513003281 conn(0x7f5e44077ae0 msgr2=0x7f5e44079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.536+0000 7f5e49200640 1 -- 172.21.15.42:0/513003281 shutdown_connections 2024-04-20T10:41:06.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.536+0000 7f5e49200640 1 -- 172.21.15.42:0/513003281 wait complete. 2024-04-20T10:41:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.537+0000 7f5e49200640 1 Processor -- start 2024-04-20T10:41:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.537+0000 7f5e49200640 1 -- start start 2024-04-20T10:41:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e49200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e44108400 0x7f5e441a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e49200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e44108da0 0x7f5e441a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e49200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e4410e470 0x7f5e4419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e49200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e44118730 con 0x7f5e44108400 2024-04-20T10:41:06.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e49200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e441185b0 con 0x7f5e44108da0 2024-04-20T10:41:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e49200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5e441188b0 con 0x7f5e4410e470 2024-04-20T10:41:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e42a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e44108da0 0x7f5e441a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e42a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e44108da0 0x7f5e441a4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:44514/0 (socket says 172.21.15.42:44514) 2024-04-20T10:41:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e42a00640 1 -- 172.21.15.42:0/296165003 learned_addr learned my addr 172.21.15.42:0/296165003 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e43400640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e44108400 0x7f5e441a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e43e00640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e4410e470 0x7f5e4419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e42a00640 1 -- 172.21.15.42:0/296165003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e4410e470 msgr2=0x7f5e4419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e42a00640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e4410e470 0x7f5e4419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e42a00640 1 -- 172.21.15.42:0/296165003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e44108400 msgr2=0x7f5e441a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e42a00640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e44108400 0x7f5e441a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e42a00640 1 -- 172.21.15.42:0/296165003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4419f0d0 con 0x7f5e44108da0 2024-04-20T10:41:06.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.538+0000 7f5e43e00640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e4410e470 0x7f5e4419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:41:06.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.539+0000 7f5e42a00640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e44108da0 0x7f5e441a4ce0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00ea60 tx=0x7f5e2c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:06.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.539+0000 7f5e3be00640 1 -- 172.21.15.42:0/296165003 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00cde0 con 0x7f5e44108da0 2024-04-20T10:41:06.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.539+0000 7f5e49200640 1 -- 172.21.15.42:0/296165003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4419f3c0 con 0x7f5e44108da0 2024-04-20T10:41:06.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.539+0000 7f5e3be00640 1 -- 172.21.15.42:0/296165003 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e2c0035f0 con 0x7f5e44108da0 2024-04-20T10:41:06.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.539+0000 7f5e3be00640 1 -- 172.21.15.42:0/296165003 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c010670 con 0x7f5e44108da0 2024-04-20T10:41:06.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.539+0000 7f5e49200640 1 -- 172.21.15.42:0/296165003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e441ab660 con 0x7f5e44108da0 2024-04-20T10:41:06.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.540+0000 7f5e49200640 1 -- 172.21.15.42:0/296165003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e440690f0 con 0x7f5e44108da0 2024-04-20T10:41:06.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.542+0000 7f5e3be00640 1 -- 172.21.15.42:0/296165003 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5e2c0037a0 con 0x7f5e44108da0 2024-04-20T10:41:06.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.544+0000 7f5e3be00640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5e280a3d20 0x7f5e280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:06.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.544+0000 7f5e3be00640 1 -- 172.21.15.42:0/296165003 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f5e2c0d0c00 con 0x7f5e44108da0 2024-04-20T10:41:06.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.544+0000 7f5e43400640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5e280a3d20 0x7f5e280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:06.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.545+0000 7f5e43400640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5e280a3d20 0x7f5e280a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5e4419fa90 tx=0x7f5e34009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:06.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.553+0000 7f5e3be00640 1 -- 172.21.15.42:0/296165003 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5e2c09d1e0 con 0x7f5e44108da0 2024-04-20T10:41:06.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.823+0000 7f5e49200640 1 -- 172.21.15.42:0/296165003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e4419fe80 con 0x7f5e44108da0 2024-04-20T10:41:06.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.824+0000 7f5e3be00640 1 -- 172.21.15.42:0/296165003 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f5e2c09cae0 con 0x7f5e44108da0 2024-04-20T10:41:06.824 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:41:06.824 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:41:06.825 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:41:06.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.827+0000 7f5e49200640 1 -- 172.21.15.42:0/296165003 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5e280a3d20 msgr2=0x7f5e280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.827+0000 7f5e49200640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5e280a3d20 0x7f5e280a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5e4419fa90 tx=0x7f5e34009500 comp rx=0 tx=0).stop 2024-04-20T10:41:06.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.827+0000 7f5e49200640 1 -- 172.21.15.42:0/296165003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e44108da0 msgr2=0x7f5e441a4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:06.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.827+0000 7f5e49200640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e44108da0 0x7f5e441a4ce0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00ea60 tx=0x7f5e2c00ef20 comp rx=0 tx=0).stop 2024-04-20T10:41:06.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.828+0000 7f5e49200640 1 -- 172.21.15.42:0/296165003 shutdown_connections 2024-04-20T10:41:06.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.828+0000 7f5e49200640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5e280a3d20 0x7f5e280a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.828+0000 7f5e49200640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e4410e470 0x7f5e4419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.828+0000 7f5e49200640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e44108da0 0x7f5e441a4ce0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.828+0000 7f5e49200640 1 --2- 172.21.15.42:0/296165003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e44108400 0x7f5e441a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:06.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.828+0000 7f5e49200640 1 -- 172.21.15.42:0/296165003 >> 172.21.15.42:0/296165003 conn(0x7f5e44077ae0 msgr2=0x7f5e44079280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:06.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.828+0000 7f5e49200640 1 -- 172.21.15.42:0/296165003 shutdown_connections 2024-04-20T10:41:06.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:06.828+0000 7f5e49200640 1 -- 172.21.15.42:0/296165003 wait complete. 2024-04-20T10:41:06.851 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:41:06.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:06 smithi162 ceph-mon[21854]: osdmap e89: 12 total, 12 up, 12 in 2024-04-20T10:41:06.925 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:06 smithi042 ceph-mon[17078]: osdmap e89: 12 total, 12 up, 12 in 2024-04-20T10:41:07.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:06 smithi046 ceph-mon[21574]: osdmap e89: 12 total, 12 up, 12 in 2024-04-20T10:41:07.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.078+0000 7fa96fc00640 1 -- 172.21.15.42:0/2120790051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9681072c0 msgr2=0x7fa96810ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:07.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.078+0000 7fa96fc00640 1 --2- 172.21.15.42:0/2120790051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9681072c0 0x7fa96810ee90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa960009ca0 tx=0x7fa96002c100 comp rx=0 tx=0).stop 2024-04-20T10:41:07.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.079+0000 7fa96fc00640 1 -- 172.21.15.42:0/2120790051 shutdown_connections 2024-04-20T10:41:07.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.079+0000 7fa96fc00640 1 --2- 172.21.15.42:0/2120790051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9681072c0 0x7fa96810ee90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.079+0000 7fa96fc00640 1 --2- 172.21.15.42:0/2120790051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa968102ba0 0x7fa968106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.079+0000 7fa96fc00640 1 --2- 172.21.15.42:0/2120790051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9681021f0 0x7fa9681025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.079+0000 7fa96fc00640 1 -- 172.21.15.42:0/2120790051 >> 172.21.15.42:0/2120790051 conn(0x7fa9680fda70 msgr2=0x7fa9680ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:07.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.079+0000 7fa96fc00640 1 -- 172.21.15.42:0/2120790051 shutdown_connections 2024-04-20T10:41:07.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.079+0000 7fa96fc00640 1 -- 172.21.15.42:0/2120790051 wait complete. 2024-04-20T10:41:07.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.080+0000 7fa96fc00640 1 Processor -- start 2024-04-20T10:41:07.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.080+0000 7fa96fc00640 1 -- start start 2024-04-20T10:41:07.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96fc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9681021f0 0x7fa9681a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96fc00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9681072c0 0x7fa9681a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96fc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96819a8a0 0x7fa96819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96fc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa968112360 con 0x7fa9681021f0 2024-04-20T10:41:07.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96fc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681121e0 con 0x7fa96819a8a0 2024-04-20T10:41:07.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96fc00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681124e0 con 0x7fa9681072c0 2024-04-20T10:41:07.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9681021f0 0x7fa9681a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9681021f0 0x7fa9681a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45378/0 (socket says 172.21.15.42:45378) 2024-04-20T10:41:07.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96d200640 1 -- 172.21.15.42:0/2156463909 learned_addr learned my addr 172.21.15.42:0/2156463909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:07.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96819a8a0 0x7fa96819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa967e00640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9681072c0 0x7fa9681a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96dc00640 1 -- 172.21.15.42:0/2156463909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9681072c0 msgr2=0x7fa9681a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:07.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96dc00640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9681072c0 0x7fa9681a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96dc00640 1 -- 172.21.15.42:0/2156463909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9681021f0 msgr2=0x7fa9681a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:07.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96dc00640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9681021f0 0x7fa9681a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.081+0000 7fa96dc00640 1 -- 172.21.15.42:0/2156463909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96819b3c0 con 0x7fa96819a8a0 2024-04-20T10:41:07.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.082+0000 7fa96dc00640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96819a8a0 0x7fa96819ad50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa960009bf0 tx=0x7fa960035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:07.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.082+0000 7fa965600640 1 -- 172.21.15.42:0/2156463909 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960039040 con 0x7fa96819a8a0 2024-04-20T10:41:07.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.082+0000 7fa965600640 1 -- 172.21.15.42:0/2156463909 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa960044070 con 0x7fa96819a8a0 2024-04-20T10:41:07.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.082+0000 7fa96fc00640 1 -- 172.21.15.42:0/2156463909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96819b650 con 0x7fa96819a8a0 2024-04-20T10:41:07.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.082+0000 7fa965600640 1 -- 172.21.15.42:0/2156463909 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600043c0 con 0x7fa96819a8a0 2024-04-20T10:41:07.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.082+0000 7fa96fc00640 1 -- 172.21.15.42:0/2156463909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9681a78a0 con 0x7fa96819a8a0 2024-04-20T10:41:07.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.087+0000 7fa97285d740 1 Processor -- start 2024-04-20T10:41:07.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.087+0000 7fa965600640 1 -- 172.21.15.42:0/2156463909 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa960004560 con 0x7fa96819a8a0 2024-04-20T10:41:07.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.087+0000 7fa97285d740 1 -- start start 2024-04-20T10:41:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.087+0000 7fa97285d740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.088+0000 7fa97285d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad08d0 0x560f55ad8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.088+0000 7fa97285d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad94b0 0x560f55adb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.088+0000 7fa97285d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560f55a35050 con 0x560f55ad08d0 2024-04-20T10:41:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.088+0000 7fa97285d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560f559db490 con 0x560f55ad94b0 2024-04-20T10:41:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.088+0000 7fa97285d740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x560f55a31010 con 0x560f55acffb0 2024-04-20T10:41:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.088+0000 7fa967e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.088+0000 7fa967e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:60056/0 (socket says 172.21.15.42:60056) 2024-04-20T10:41:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.088+0000 7fa96d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad08d0 0x560f55ad8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.088+0000 7fa967e00640 1 -- 172.21.15.42:0/1948423482 learned_addr learned my addr 172.21.15.42:0/1948423482 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.088+0000 7fa967e00640 1 --2- 172.21.15.42:0/1948423482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad94b0 0x560f55adb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.089+0000 7fa967e00640 1 -- 172.21.15.42:0/1948423482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad94b0 msgr2=0x560f55adb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.089+0000 7fa967e00640 1 --2- 172.21.15.42:0/1948423482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad94b0 0x560f55adb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.089+0000 7fa967e00640 1 -- 172.21.15.42:0/1948423482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad08d0 msgr2=0x560f55ad8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.089+0000 7fa967e00640 1 --2- 172.21.15.42:0/1948423482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad08d0 0x560f55ad8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.089+0000 7fa967e00640 1 -- 172.21.15.42:0/1948423482 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f559e3070 con 0x560f55acffb0 2024-04-20T10:41:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.089+0000 7fa967e00640 1 --2- 172.21.15.42:0/1948423482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad0390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa954011700 tx=0x7fa954004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f7844b4a3eeea71 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.089+0000 7fa965600640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa9500a3da0 0x7fa9500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.089+0000 7fa95a000640 1 -- 172.21.15.42:0/1948423482 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95400f7c0 con 0x560f55acffb0 2024-04-20T10:41:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.089+0000 7fa95a000640 1 -- 172.21.15.42:0/1948423482 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95400fe40 con 0x560f55acffb0 2024-04-20T10:41:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.089+0000 7fa95a000640 1 -- 172.21.15.42:0/1948423482 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95401fe20 con 0x560f55acffb0 2024-04-20T10:41:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.089+0000 7fa965600640 1 -- 172.21.15.42:0/2156463909 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7fa9600f2bd0 con 0x7fa96819a8a0 2024-04-20T10:41:07.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.090+0000 7fa96d200640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa9500a3da0 0x7fa9500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.090+0000 7fa97285d740 1 -- 172.21.15.42:0/1948423482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 msgr2=0x560f55ad0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:07.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.090+0000 7fa97285d740 1 --2- 172.21.15.42:0/1948423482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad0390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa954011700 tx=0x7fa954004730 comp rx=0 tx=0).stop 2024-04-20T10:41:07.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.090+0000 7fa97285d740 1 -- 172.21.15.42:0/1948423482 shutdown_connections 2024-04-20T10:41:07.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.090+0000 7fa97285d740 1 --2- 172.21.15.42:0/1948423482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad94b0 0x560f55adb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.090+0000 7fa97285d740 1 --2- 172.21.15.42:0/1948423482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad08d0 0x560f55ad8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.090+0000 7fa97285d740 1 --2- 172.21.15.42:0/1948423482 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad0390 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.090+0000 7fa97285d740 1 -- 172.21.15.42:0/1948423482 >> 172.21.15.42:0/1948423482 conn(0x560f55aca6e0 msgr2=0x560f55acaac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.090+0000 7fa97285d740 1 -- 172.21.15.42:0/1948423482 shutdown_connections 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.090+0000 7fa97285d740 1 -- 172.21.15.42:0/1948423482 wait complete. 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.091+0000 7fa96d200640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa9500a3da0 0x7fa9500a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x560f55acfa80 tx=0x7fa95c00b480 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.091+0000 7fa97285d740 1 -- start start 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.091+0000 7fa97285d740 1 Processor -- start 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa97285d740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa97285d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 0x560f55ad6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa97285d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad94b0 0x560f55ad6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa97285d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560f55adc2a0 con 0x560f55ad94b0 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa97285d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560f559ee860 con 0x560f55ad08d0 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa97285d740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x560f55adc420 con 0x560f55acffb0 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa97285d740 10 client.0 _ll_register_callbacks cb 0x560f559bcfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:41:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa967e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 0x560f55ad6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa967e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 0x560f55ad6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:44554/0 (socket says 172.21.15.42:44554) 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa967e00640 1 -- 172.21.15.42:0/1044680035 learned_addr learned my addr 172.21.15.42:0/1044680035 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa96dc00640 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad94b0 0x560f55ad6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa96d200640 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa96dc00640 1 -- 172.21.15.42:0/1044680035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 msgr2=0x560f55ad6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa96dc00640 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa96dc00640 1 -- 172.21.15.42:0/1044680035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 msgr2=0x560f55ad6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa96dc00640 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 0x560f55ad6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.092+0000 7fa96dc00640 1 -- 172.21.15.42:0/1044680035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f559f2950 con 0x560f55ad94b0 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.093+0000 7fa96dc00640 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad94b0 0x560f55ad6f90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa9600f0370 tx=0x7fa9600be520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154ad668f35a0cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.093+0000 7fa94ea00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.093+0000 7fa94ea00640 1 -- 172.21.15.42:0/1044680035 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600bebc0 con 0x560f55ad94b0 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.093+0000 7fa94ea00640 1 -- 172.21.15.42:0/1044680035 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7fa9600df0c0 con 0x560f55ad94b0 2024-04-20T10:41:07.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.093+0000 7fa97285d740 1 -- 172.21.15.42:0/1044680035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa95400fe40 con 0x560f55ad94b0 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.093+0000 7fa94ea00640 1 -- 172.21.15.42:0/1044680035 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa9600c2f70 con 0x560f55ad94b0 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.093+0000 7fa97285d740 10 client.16089 fetch_fsmap learned FSMap version 19 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.093+0000 7fa97285d740 1 -- 172.21.15.42:0/1044680035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560f559c0820 con 0x560f55ad94b0 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.094+0000 7fa94ea00640 1 -- 172.21.15.42:0/1044680035 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95400fe40 con 0x560f55ad94b0 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.094+0000 7fa94ea00640 1 -- 172.21.15.42:0/1044680035 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fa96003c030 con 0x560f55ad94b0 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.094+0000 7fa97285d740 10 client.16089 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.094+0000 7fa97285d740 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.094+0000 7fa97285d740 20 client.16089 populate_metadata read hostname 'smithi042' 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.094+0000 7fa97285d740 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560f55af1c60 0x560f55af40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.094+0000 7fa97285d740 4 client.16089 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.094+0000 7fa97285d740 1 -- 172.21.15.42:0/1044680035 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f55a42a70 con 0x560f55af1c60 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.094+0000 7fa96d200640 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560f55af1c60 0x560f55af40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.095+0000 7fa96d200640 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560f55af1c60 0x560f55af40a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47193707f4744fd7 server_cookie=4dd98015cd6b9ed0 in_seq=0 out_seq=0 2024-04-20T10:41:07.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.095+0000 7fa94ea00640 10 client.16089 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:07.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa94ea00640 1 -- 172.21.15.42:0/1044680035 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f55a42a70 con 0x560f55af1c60 2024-04-20T10:41:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa94ea00640 10 client.16089 handle_command_reply: tid=0 2024-04-20T10:41:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa97285d740 1 client.16089 shutdown 2024-04-20T10:41:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa97285d740 1 -- 172.21.15.42:0/1044680035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad94b0 msgr2=0x560f55ad6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa97285d740 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad94b0 0x560f55ad6f90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa9600f0370 tx=0x7fa9600be520 comp rx=0 tx=0).stop 2024-04-20T10:41:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa97285d740 1 -- 172.21.15.42:0/1044680035 shutdown_connections 2024-04-20T10:41:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa97285d740 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560f55af1c60 0x560f55af40a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa97285d740 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad94b0 0x560f55ad6f90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa97285d740 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 0x560f55ad6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa97285d740 1 --2- 172.21.15.42:0/1044680035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa94ea00640 0 client.16089 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa97285d740 1 -- 172.21.15.42:0/1044680035 >> 172.21.15.42:0/1044680035 conn(0x560f55aca6e0 msgr2=0x560f55acc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:07.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa97285d740 1 -- 172.21.15.42:0/1044680035 shutdown_connections 2024-04-20T10:41:07.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa97285d740 1 -- 172.21.15.42:0/1044680035 wait complete. 2024-04-20T10:41:07.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.096+0000 7fa97285d740 20 client.16089 trim_cache size 0 max 16384 2024-04-20T10:41:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.111+0000 7fa97285d740 1 Processor -- start 2024-04-20T10:41:07.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.112+0000 7fa97285d740 1 -- start start 2024-04-20T10:41:07.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.112+0000 7fa97285d740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.112+0000 7fa97285d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 0x560f55aeda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.112+0000 7fa97285d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad94b0 0x560f55af99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.112+0000 7fa97285d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681a9bb0 con 0x560f55ad94b0 2024-04-20T10:41:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.112+0000 7fa97285d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681ac3d0 con 0x560f55ad08d0 2024-04-20T10:41:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.112+0000 7fa97285d740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681aa200 con 0x560f55acffb0 2024-04-20T10:41:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.112+0000 7fa967e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 0x560f55aeda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.112+0000 7fa967e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 0x560f55aeda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:44568/0 (socket says 172.21.15.42:44568) 2024-04-20T10:41:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.112+0000 7fa967e00640 1 -- 172.21.15.42:0/3562389718 learned_addr learned my addr 172.21.15.42:0/3562389718 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.112+0000 7fa96dc00640 1 --2- 172.21.15.42:0/3562389718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad94b0 0x560f55af99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.112+0000 7fa96d200640 1 --2- 172.21.15.42:0/3562389718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.113+0000 7fa967e00640 1 -- 172.21.15.42:0/3562389718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 msgr2=0x560f55ad8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.113+0000 7fa967e00640 1 --2- 172.21.15.42:0/3562389718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.113+0000 7fa967e00640 1 -- 172.21.15.42:0/3562389718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad94b0 msgr2=0x560f55af99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:07.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.113+0000 7fa967e00640 1 --2- 172.21.15.42:0/3562389718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad94b0 0x560f55af99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.113+0000 7fa967e00640 1 -- 172.21.15.42:0/3562389718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f557473f0 con 0x560f55ad08d0 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.113+0000 7fa967e00640 1 --2- 172.21.15.42:0/3562389718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 0x560f55aeda20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa9540099a0 tx=0x7fa95401fa60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738d23271fcadd0b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.113+0000 7fa959600640 1 -- 172.21.15.42:0/3562389718 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95401fb00 con 0x560f55ad08d0 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.113+0000 7fa959600640 1 -- 172.21.15.42:0/3562389718 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95401fca0 con 0x560f55ad08d0 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.113+0000 7fa959600640 1 -- 172.21.15.42:0/3562389718 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954004410 con 0x560f55ad08d0 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.114+0000 7fa97285d740 1 -- 172.21.15.42:0/3562389718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 msgr2=0x560f55aeda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.114+0000 7fa97285d740 1 --2- 172.21.15.42:0/3562389718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 0x560f55aeda20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa9540099a0 tx=0x7fa95401fa60 comp rx=0 tx=0).stop 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.114+0000 7fa97285d740 1 -- 172.21.15.42:0/3562389718 shutdown_connections 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.114+0000 7fa97285d740 1 --2- 172.21.15.42:0/3562389718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad94b0 0x560f55af99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.114+0000 7fa97285d740 1 --2- 172.21.15.42:0/3562389718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad08d0 0x560f55aeda20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.114+0000 7fa97285d740 1 --2- 172.21.15.42:0/3562389718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.114+0000 7fa97285d740 1 -- 172.21.15.42:0/3562389718 >> 172.21.15.42:0/3562389718 conn(0x560f55aca6e0 msgr2=0x560f55acc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.114+0000 7fa97285d740 1 -- 172.21.15.42:0/3562389718 shutdown_connections 2024-04-20T10:41:07.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.114+0000 7fa97285d740 1 -- 172.21.15.42:0/3562389718 wait complete. 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.115+0000 7fa97285d740 1 -- start start 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.115+0000 7fa97285d740 1 Processor -- start 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.115+0000 7fa97285d740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.115+0000 7fa97285d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad08d0 0x560f55ad3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.115+0000 7fa97285d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad94b0 0x560f55b57220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.115+0000 7fa97285d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681aa890 con 0x560f55ad08d0 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.115+0000 7fa97285d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681abd00 con 0x560f55ad94b0 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.115+0000 7fa97285d740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681aaf60 con 0x560f55acffb0 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.115+0000 7fa97285d740 10 client.0 _ll_register_callbacks cb 0x7fa9600be990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.115+0000 7fa96dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad94b0 0x560f55b57220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa967e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad08d0 0x560f55ad3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa96dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad94b0 0x560f55b57220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:44582/0 (socket says 172.21.15.42:44582) 2024-04-20T10:41:07.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa96dc00640 1 -- 172.21.15.42:0/2881933830 learned_addr learned my addr 172.21.15.42:0/2881933830 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa96d200640 1 --2- 172.21.15.42:0/2881933830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa96dc00640 1 -- 172.21.15.42:0/2881933830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 msgr2=0x560f55ad3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa96dc00640 1 --2- 172.21.15.42:0/2881933830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa96dc00640 1 -- 172.21.15.42:0/2881933830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad08d0 msgr2=0x560f55ad3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa96dc00640 1 --2- 172.21.15.42:0/2881933830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad08d0 0x560f55ad3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa96dc00640 1 -- 172.21.15.42:0/2881933830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f558117d0 con 0x560f55ad94b0 2024-04-20T10:41:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa96dc00640 1 --2- 172.21.15.42:0/2881933830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad94b0 0x560f55b57220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa9600d5040 tx=0x7fa9600bd0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=558eb81843771ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa94ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:41:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa94ea00640 1 -- 172.21.15.42:0/2881933830 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600bebc0 con 0x560f55ad94b0 2024-04-20T10:41:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa94ea00640 1 -- 172.21.15.42:0/2881933830 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7fa960004d40 con 0x560f55ad94b0 2024-04-20T10:41:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.116+0000 7fa97285d740 1 -- 172.21.15.42:0/2881933830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa95401fca0 con 0x560f55ad94b0 2024-04-20T10:41:07.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.117+0000 7fa94ea00640 1 -- 172.21.15.42:0/2881933830 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa9600bae60 con 0x560f55ad94b0 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.117+0000 7fa97285d740 10 client.26210 fetch_fsmap learned FSMap version 19 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.117+0000 7fa97285d740 1 -- 172.21.15.42:0/2881933830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560f55b58800 con 0x560f55ad94b0 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.117+0000 7fa94ea00640 1 -- 172.21.15.42:0/2881933830 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95401fca0 con 0x560f55ad94b0 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.117+0000 7fa94ea00640 1 -- 172.21.15.42:0/2881933830 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fa9600be010 con 0x560f55ad94b0 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.117+0000 7fa97285d740 10 client.26210 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.117+0000 7fa97285d740 10 client.26210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.117+0000 7fa97285d740 20 client.26210 populate_metadata read hostname 'smithi042' 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.117+0000 7fa97285d740 1 --2- 172.21.15.42:0/2881933830 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560f55b58d90 0x560f55b5b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.117+0000 7fa97285d740 4 client.26210 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.117+0000 7fa97285d740 1 -- 172.21.15.42:0/2881933830 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f55a3c8e0 con 0x560f55b58d90 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.117+0000 7fa96d200640 1 --2- 172.21.15.42:0/2881933830 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560f55b58d90 0x560f55b5b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.118+0000 7fa96d200640 1 --2- 172.21.15.42:0/2881933830 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560f55b58d90 0x560f55b5b180 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc6d2128cfbf8ef server_cookie=eb688a788b1773f0 in_seq=0 out_seq=0 2024-04-20T10:41:07.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.118+0000 7fa94ea00640 10 client.26210 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa94ea00640 1 -- 172.21.15.42:0/2881933830 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f55a3c8e0 con 0x560f55b58d90 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa94ea00640 10 client.26210 handle_command_reply: tid=0 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa97285d740 1 client.26210 shutdown 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa97285d740 1 -- 172.21.15.42:0/2881933830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad94b0 msgr2=0x560f55b57220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa97285d740 1 --2- 172.21.15.42:0/2881933830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad94b0 0x560f55b57220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa9600d5040 tx=0x7fa9600bd0d0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa97285d740 1 -- 172.21.15.42:0/2881933830 shutdown_connections 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa97285d740 1 --2- 172.21.15.42:0/2881933830 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x560f55b58d90 0x560f55b5b180 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa97285d740 1 --2- 172.21.15.42:0/2881933830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560f55ad94b0 0x560f55b57220 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa97285d740 1 --2- 172.21.15.42:0/2881933830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560f55ad08d0 0x560f55ad3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa97285d740 1 --2- 172.21.15.42:0/2881933830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560f55acffb0 0x560f55ad3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa97285d740 1 -- 172.21.15.42:0/2881933830 >> 172.21.15.42:0/2881933830 conn(0x560f55aca6e0 msgr2=0x560f55ac8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa94ea00640 0 client.26210 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa97285d740 1 -- 172.21.15.42:0/2881933830 shutdown_connections 2024-04-20T10:41:07.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa97285d740 1 -- 172.21.15.42:0/2881933830 wait complete. 2024-04-20T10:41:07.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.119+0000 7fa97285d740 20 client.26210 trim_cache size 0 max 16384 2024-04-20T10:41:07.190 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:41:07.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.120+0000 7fa96fc00640 1 -- 172.21.15.42:0/2156463909 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa9500a3da0 msgr2=0x7fa9500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:07.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.120+0000 7fa96fc00640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa9500a3da0 0x7fa9500a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x560f55acfa80 tx=0x7fa95c00b480 comp rx=0 tx=0).stop 2024-04-20T10:41:07.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.120+0000 7fa96fc00640 1 -- 172.21.15.42:0/2156463909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96819a8a0 msgr2=0x7fa96819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:07.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.120+0000 7fa96fc00640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96819a8a0 0x7fa96819ad50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa960009bf0 tx=0x7fa960035d10 comp rx=0 tx=0).stop 2024-04-20T10:41:07.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.120+0000 7fa96fc00640 1 -- 172.21.15.42:0/2156463909 shutdown_connections 2024-04-20T10:41:07.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.120+0000 7fa96fc00640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa9500a3da0 0x7fa9500a6260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.120+0000 7fa96fc00640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa96819a8a0 0x7fa96819ad50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.120+0000 7fa96fc00640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9681072c0 0x7fa9681a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.120+0000 7fa96fc00640 1 --2- 172.21.15.42:0/2156463909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9681021f0 0x7fa9681a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:07.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.120+0000 7fa96fc00640 1 -- 172.21.15.42:0/2156463909 >> 172.21.15.42:0/2156463909 conn(0x7fa9680fda70 msgr2=0x7fa9680ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:07.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.120+0000 7fa96fc00640 1 -- 172.21.15.42:0/2156463909 shutdown_connections 2024-04-20T10:41:07.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:07.120+0000 7fa96fc00640 1 -- 172.21.15.42:0/2156463909 wait complete. 2024-04-20T10:41:07.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:07 smithi162 ceph-mon[21854]: pgmap v217: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 883 op/s 2024-04-20T10:41:07.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:07 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/296165003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:41:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:07 smithi046 ceph-mon[21574]: pgmap v217: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 883 op/s 2024-04-20T10:41:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:07 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/296165003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:41:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:07 smithi042 ceph-mon[17078]: pgmap v217: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 883 op/s 2024-04-20T10:41:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:07 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/296165003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:41:08.193 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:41:08.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:41:08.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:41:08.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.415+0000 7fe5bd800640 1 -- 172.21.15.42:0/3305558902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5b8074c20 msgr2=0x7fe5b81128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:08.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.415+0000 7fe5bd800640 1 --2- 172.21.15.42:0/3305558902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5b8074c20 0x7fe5b81128d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009900 tx=0x7fe5a402bcc0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.415+0000 7fe5bd800640 1 -- 172.21.15.42:0/3305558902 shutdown_connections 2024-04-20T10:41:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.415+0000 7fe5bd800640 1 --2- 172.21.15.42:0/3305558902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b8112e10 0x7fe5b8115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.415+0000 7fe5bd800640 1 --2- 172.21.15.42:0/3305558902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5b8074c20 0x7fe5b81128d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.415+0000 7fe5bd800640 1 --2- 172.21.15.42:0/3305558902 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5b8074300 0x7fe5b80746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.415+0000 7fe5bd800640 1 -- 172.21.15.42:0/3305558902 >> 172.21.15.42:0/3305558902 conn(0x7fe5b80fd930 msgr2=0x7fe5b80ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.416+0000 7fe5bd800640 1 -- 172.21.15.42:0/3305558902 shutdown_connections 2024-04-20T10:41:08.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.416+0000 7fe5bd800640 1 -- 172.21.15.42:0/3305558902 wait complete. 2024-04-20T10:41:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.417+0000 7fe5bd800640 1 Processor -- start 2024-04-20T10:41:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.417+0000 7fe5bd800640 1 -- start start 2024-04-20T10:41:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.417+0000 7fe5bd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5b8074300 0x7fe5b81a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.417+0000 7fe5bd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b8074c20 0x7fe5b81a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.417+0000 7fe5bd800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5b8112e10 0x7fe5b819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.417+0000 7fe5bd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b8118850 con 0x7fe5b8074c20 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.417+0000 7fe5bd800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81186d0 con 0x7fe5b8074300 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.417+0000 7fe5bd800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81189d0 con 0x7fe5b8112e10 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.418+0000 7fe5bce00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5b8112e10 0x7fe5b819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.418+0000 7fe5b7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b8074c20 0x7fe5b81a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.418+0000 7fe5b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5b8074300 0x7fe5b81a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.418+0000 7fe5b7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b8074c20 0x7fe5b81a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45434/0 (socket says 172.21.15.42:45434) 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.418+0000 7fe5b7400640 1 -- 172.21.15.42:0/2856657591 learned_addr learned my addr 172.21.15.42:0/2856657591 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.418+0000 7fe5b7400640 1 -- 172.21.15.42:0/2856657591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5b8112e10 msgr2=0x7fe5b819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.418+0000 7fe5b7400640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5b8112e10 0x7fe5b819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.418+0000 7fe5b7400640 1 -- 172.21.15.42:0/2856657591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5b8074300 msgr2=0x7fe5b81a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.418+0000 7fe5b7400640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5b8074300 0x7fe5b81a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.418+0000 7fe5b7400640 1 -- 172.21.15.42:0/2856657591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b819f3f0 con 0x7fe5b8074c20 2024-04-20T10:41:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.418+0000 7fe5bce00640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5b8112e10 0x7fe5b819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:41:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.419+0000 7fe5b7400640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b8074c20 0x7fe5b81a4fb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4000c00 tx=0x7fe5a4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.419+0000 7fe5b4c00640 1 -- 172.21.15.42:0/2856657591 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4042020 con 0x7fe5b8074c20 2024-04-20T10:41:08.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.420+0000 7fe5bd800640 1 -- 172.21.15.42:0/2856657591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b819f680 con 0x7fe5b8074c20 2024-04-20T10:41:08.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.420+0000 7fe5bd800640 1 -- 172.21.15.42:0/2856657591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b81ab9f0 con 0x7fe5b8074c20 2024-04-20T10:41:08.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.420+0000 7fe5b4c00640 1 -- 172.21.15.42:0/2856657591 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a40053e0 con 0x7fe5b8074c20 2024-04-20T10:41:08.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.420+0000 7fe5b4c00640 1 -- 172.21.15.42:0/2856657591 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a403e450 con 0x7fe5b8074c20 2024-04-20T10:41:08.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.424+0000 7fe5c1b96740 1 Processor -- start 2024-04-20T10:41:08.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.424+0000 7fe5c1b96740 1 -- start start 2024-04-20T10:41:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.424+0000 7fe5c1b96740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb2e0f0 0x55f98eb2e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5c1b96740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2ea10 0x55f98eb370b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5c1b96740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb375f0 0x55f98eb399e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5c1b96740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f98ea92d20 con 0x55f98eb375f0 2024-04-20T10:41:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5c1b96740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f98ea393e0 con 0x55f98eb2e0f0 2024-04-20T10:41:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5c1b96740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f98ea8ece0 con 0x55f98eb2ea10 2024-04-20T10:41:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5bce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb2e0f0 0x55f98eb2e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5bce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb2e0f0 0x55f98eb2e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:44612/0 (socket says 172.21.15.42:44612) 2024-04-20T10:41:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5bce00640 1 -- 172.21.15.42:0/1024567422 learned_addr learned my addr 172.21.15.42:0/1024567422 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:08.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5b4c00640 1 -- 172.21.15.42:0/2856657591 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe5a4045020 con 0x7fe5b8074c20 2024-04-20T10:41:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5b7e00640 1 --2- 172.21.15.42:0/1024567422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2ea10 0x55f98eb370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5bce00640 1 -- 172.21.15.42:0/1024567422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2ea10 msgr2=0x55f98eb370b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5bce00640 1 --2- 172.21.15.42:0/1024567422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2ea10 0x55f98eb370b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5b7400640 1 --2- 172.21.15.42:0/1024567422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb375f0 0x55f98eb399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.425+0000 7fe5bce00640 1 -- 172.21.15.42:0/1024567422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb375f0 msgr2=0x55f98eb399e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.426+0000 7fe5bce00640 1 --2- 172.21.15.42:0/1024567422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb375f0 0x55f98eb399e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.426+0000 7fe5bce00640 1 -- 172.21.15.42:0/1024567422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f98ea58980 con 0x55f98eb2e0f0 2024-04-20T10:41:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.426+0000 7fe5b7400640 1 --2- 172.21.15.42:0/1024567422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb375f0 0x55f98eb399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:41:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.426+0000 7fe5bce00640 1 --2- 172.21.15.42:0/1024567422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb2e0f0 0x55f98eb2e4d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000f5d0 tx=0x7fe5b000fa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=365a200e54756abe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.426+0000 7fe5ad600640 1 -- 172.21.15.42:0/1024567422 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0004d20 con 0x55f98eb2e0f0 2024-04-20T10:41:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.426+0000 7fe5ad600640 1 -- 172.21.15.42:0/1024567422 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b000dd30 con 0x55f98eb2e0f0 2024-04-20T10:41:08.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.426+0000 7fe5ad600640 1 -- 172.21.15.42:0/1024567422 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b00058e0 con 0x55f98eb2e0f0 2024-04-20T10:41:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.427+0000 7fe5b4c00640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe59c0a3da0 0x7fe59c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.427+0000 7fe5b4c00640 1 -- 172.21.15.42:0/2856657591 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7fe5a40f1dd0 con 0x7fe5b8074c20 2024-04-20T10:41:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.427+0000 7fe5b7e00640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe59c0a3da0 0x7fe59c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.427+0000 7fe5c1b96740 1 -- 172.21.15.42:0/1024567422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb2e0f0 msgr2=0x55f98eb2e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.427+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/1024567422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb2e0f0 0x55f98eb2e4d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe5b000f5d0 tx=0x7fe5b000fa90 comp rx=0 tx=0).stop 2024-04-20T10:41:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.428+0000 7fe5c1b96740 1 -- 172.21.15.42:0/1024567422 shutdown_connections 2024-04-20T10:41:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.428+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/1024567422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb375f0 0x55f98eb399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.428+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/1024567422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2ea10 0x55f98eb370b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.428+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/1024567422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb2e0f0 0x55f98eb2e4d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.428+0000 7fe5c1b96740 1 -- 172.21.15.42:0/1024567422 >> 172.21.15.42:0/1024567422 conn(0x55f98eb28820 msgr2=0x55f98eb28c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.428+0000 7fe5c1b96740 1 -- 172.21.15.42:0/1024567422 shutdown_connections 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.428+0000 7fe5c1b96740 1 -- 172.21.15.42:0/1024567422 wait complete. 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.428+0000 7fe5b7e00640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe59c0a3da0 0x7fe59c0a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55f98eb2dbc0 tx=0x7fe5a800b390 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.428+0000 7fe5c1b96740 1 -- start start 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.429+0000 7fe5c1b96740 1 Processor -- start 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.429+0000 7fe5c1b96740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2e0f0 0x55f98eb34690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.429+0000 7fe5c1b96740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2ea10 0x55f98eb34bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.429+0000 7fe5c1b96740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb375f0 0x55f98eb35110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.429+0000 7fe5c1b96740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f98eb3a3e0 con 0x55f98eb2ea10 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.429+0000 7fe5c1b96740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f98ea4c7a0 con 0x55f98eb375f0 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.429+0000 7fe5c1b96740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55f98eb3a560 con 0x55f98eb2e0f0 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.429+0000 7fe5c1b96740 10 client.0 _ll_register_callbacks cb 0x55f98ea4eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5bce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb375f0 0x55f98eb35110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5bce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb375f0 0x55f98eb35110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:44618/0 (socket says 172.21.15.42:44618) 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5bce00640 1 -- 172.21.15.42:0/3124140650 learned_addr learned my addr 172.21.15.42:0/3124140650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5b7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2e0f0 0x55f98eb34690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5b7400640 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2ea10 0x55f98eb34bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5bce00640 1 -- 172.21.15.42:0/3124140650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2e0f0 msgr2=0x55f98eb34690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5bce00640 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2e0f0 0x55f98eb34690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5bce00640 1 -- 172.21.15.42:0/3124140650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2ea10 msgr2=0x55f98eb34bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5bce00640 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2ea10 0x55f98eb34bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5bce00640 1 -- 172.21.15.42:0/3124140650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f98e86d7e0 con 0x55f98eb375f0 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5b7400640 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2ea10 0x55f98eb34bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5bce00640 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb375f0 0x55f98eb35110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0005460 tx=0x7fe5b0005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f9df39e377fa91 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.430+0000 7fe5a2000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.431+0000 7fe5a2000640 1 -- 172.21.15.42:0/3124140650 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b001d040 con 0x55f98eb375f0 2024-04-20T10:41:08.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.431+0000 7fe5a2000640 1 -- 172.21.15.42:0/3124140650 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7fe5b0003b70 con 0x55f98eb375f0 2024-04-20T10:41:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.431+0000 7fe5c1b96740 1 -- 172.21.15.42:0/3124140650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b000dd30 con 0x55f98eb375f0 2024-04-20T10:41:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.431+0000 7fe5a2000640 1 -- 172.21.15.42:0/3124140650 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe5b0030050 con 0x55f98eb375f0 2024-04-20T10:41:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.431+0000 7fe5c1b96740 10 client.26222 fetch_fsmap learned FSMap version 19 2024-04-20T10:41:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.431+0000 7fe5c1b96740 1 -- 172.21.15.42:0/3124140650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f98e8cdf70 con 0x55f98eb375f0 2024-04-20T10:41:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.431+0000 7fe5a2000640 1 -- 172.21.15.42:0/3124140650 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b000dd30 con 0x55f98eb375f0 2024-04-20T10:41:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.432+0000 7fe5a2000640 1 -- 172.21.15.42:0/3124140650 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fe5b0004150 con 0x55f98eb375f0 2024-04-20T10:41:08.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.432+0000 7fe5c1b96740 10 client.26222 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:41:08.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.432+0000 7fe5c1b96740 10 client.26222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:41:08.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.432+0000 7fe5c1b96740 20 client.26222 populate_metadata read hostname 'smithi042' 2024-04-20T10:41:08.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.432+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f98eb478e0 0x55f98eb49cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.432+0000 7fe5c1b96740 4 client.26222 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:41:08.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.432+0000 7fe5c1b96740 1 -- 172.21.15.42:0/3124140650 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f98eaa0740 con 0x55f98eb478e0 2024-04-20T10:41:08.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.432+0000 7fe5b7e00640 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f98eb478e0 0x55f98eb49cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.433+0000 7fe5b7e00640 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f98eb478e0 0x55f98eb49cd0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c7e89c75d1745d3 server_cookie=7a9c4203e6f4e72b in_seq=0 out_seq=0 2024-04-20T10:41:08.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.433+0000 7fe5a2000640 10 client.26222 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:08.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5a2000640 1 -- 172.21.15.42:0/3124140650 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f98eaa0740 con 0x55f98eb478e0 2024-04-20T10:41:08.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5a2000640 10 client.26222 handle_command_reply: tid=0 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5c1b96740 1 client.26222 shutdown 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5c1b96740 1 -- 172.21.15.42:0/3124140650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb375f0 msgr2=0x55f98eb35110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb375f0 0x55f98eb35110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0005460 tx=0x7fe5b0005910 comp rx=0 tx=0).stop 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5c1b96740 1 -- 172.21.15.42:0/3124140650 shutdown_connections 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f98eb478e0 0x55f98eb49cd0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb375f0 0x55f98eb35110 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2ea10 0x55f98eb34bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5a2000640 0 client.26222 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/3124140650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2e0f0 0x55f98eb34690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5c1b96740 1 -- 172.21.15.42:0/3124140650 >> 172.21.15.42:0/3124140650 conn(0x55f98eb28820 msgr2=0x55f98eb2a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5c1b96740 1 -- 172.21.15.42:0/3124140650 shutdown_connections 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.434+0000 7fe5c1b96740 1 -- 172.21.15.42:0/3124140650 wait complete. 2024-04-20T10:41:08.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.435+0000 7fe5c1b96740 20 client.26222 trim_cache size 0 max 16384 2024-04-20T10:41:08.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.470+0000 7fe5c1b96740 1 Processor -- start 2024-04-20T10:41:08.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.470+0000 7fe5c1b96740 1 -- start start 2024-04-20T10:41:08.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.470+0000 7fe5c1b96740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb28820 0x55f98eb432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.470+0000 7fe5c1b96740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2e0f0 0x55f98eb33e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.470+0000 7fe5c1b96740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb2ea10 0x55f98eb364c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.470+0000 7fe5c1b96740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81b2210 con 0x55f98eb2e0f0 2024-04-20T10:41:08.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.470+0000 7fe5c1b96740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81b4a70 con 0x55f98eb2ea10 2024-04-20T10:41:08.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.470+0000 7fe5c1b96740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81b2860 con 0x55f98eb28820 2024-04-20T10:41:08.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.470+0000 7fe5b7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb28820 0x55f98eb432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.470+0000 7fe5b7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2e0f0 0x55f98eb33e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.470+0000 7fe5b7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb28820 0x55f98eb432a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:60152/0 (socket says 172.21.15.42:60152) 2024-04-20T10:41:08.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.470+0000 7fe5b7e00640 1 -- 172.21.15.42:0/4015359165 learned_addr learned my addr 172.21.15.42:0/4015359165 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:08.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.471+0000 7fe5bce00640 1 --2- 172.21.15.42:0/4015359165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb2ea10 0x55f98eb364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.471+0000 7fe5b7e00640 1 -- 172.21.15.42:0/4015359165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb2ea10 msgr2=0x55f98eb364c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:08.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.471+0000 7fe5b7e00640 1 --2- 172.21.15.42:0/4015359165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb2ea10 0x55f98eb364c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.471+0000 7fe5b7e00640 1 -- 172.21.15.42:0/4015359165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2e0f0 msgr2=0x55f98eb33e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:08.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.471+0000 7fe5b7e00640 1 --2- 172.21.15.42:0/4015359165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2e0f0 0x55f98eb33e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.471+0000 7fe5b7e00640 1 -- 172.21.15.42:0/4015359165 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f98eb2e840 con 0x55f98eb28820 2024-04-20T10:41:08.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.471+0000 7fe5b7e00640 1 --2- 172.21.15.42:0/4015359165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb28820 0x55f98eb432a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55f98eb4a2d0 tx=0x7fe5a8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91b7b8c9a2ace7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:08.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.471+0000 7fe5acc00640 1 -- 172.21.15.42:0/4015359165 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a802a530 con 0x55f98eb28820 2024-04-20T10:41:08.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.472+0000 7fe5acc00640 1 -- 172.21.15.42:0/4015359165 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a802f070 con 0x55f98eb28820 2024-04-20T10:41:08.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.472+0000 7fe5acc00640 1 -- 172.21.15.42:0/4015359165 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8011850 con 0x55f98eb28820 2024-04-20T10:41:08.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.472+0000 7fe5c1b96740 1 -- 172.21.15.42:0/4015359165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb28820 msgr2=0x55f98eb432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.472+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/4015359165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb28820 0x55f98eb432a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55f98eb4a2d0 tx=0x7fe5a8019000 comp rx=0 tx=0).stop 2024-04-20T10:41:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.472+0000 7fe5c1b96740 1 -- 172.21.15.42:0/4015359165 shutdown_connections 2024-04-20T10:41:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.472+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/4015359165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb2ea10 0x55f98eb364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.472+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/4015359165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2e0f0 0x55f98eb33e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.472+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/4015359165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb28820 0x55f98eb432a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.472+0000 7fe5c1b96740 1 -- 172.21.15.42:0/4015359165 >> 172.21.15.42:0/4015359165 conn(0x55f98eb23c00 msgr2=0x55f98eb35db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.472+0000 7fe5c1b96740 1 -- 172.21.15.42:0/4015359165 shutdown_connections 2024-04-20T10:41:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.473+0000 7fe5c1b96740 1 -- 172.21.15.42:0/4015359165 wait complete. 2024-04-20T10:41:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.473+0000 7fe5c1b96740 1 -- start start 2024-04-20T10:41:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.473+0000 7fe5c1b96740 1 Processor -- start 2024-04-20T10:41:08.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5c1b96740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb28820 0x55f98eb3b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5c1b96740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2e0f0 0x55f98eb3b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5c1b96740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2ea10 0x55f98eb3bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5c1b96740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81b2f30 con 0x55f98eb2e0f0 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5c1b96740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81b43a0 con 0x55f98eb28820 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5c1b96740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81b3600 con 0x55f98eb2ea10 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5c1b96740 10 client.0 _ll_register_callbacks cb 0x7fe5a8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb28820 0x55f98eb3b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb28820 0x55f98eb3b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:44638/0 (socket says 172.21.15.42:44638) 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5b7e00640 1 -- 172.21.15.42:0/2212785980 learned_addr learned my addr 172.21.15.42:0/2212785980 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5b7400640 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2e0f0 0x55f98eb3b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5bce00640 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2ea10 0x55f98eb3bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.474+0000 7fe5b7e00640 1 -- 172.21.15.42:0/2212785980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2ea10 msgr2=0x55f98eb3bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.475+0000 7fe5b7e00640 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2ea10 0x55f98eb3bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.475+0000 7fe5b7e00640 1 -- 172.21.15.42:0/2212785980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2e0f0 msgr2=0x55f98eb3b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:08.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.475+0000 7fe5b7e00640 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2e0f0 0x55f98eb3b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.475+0000 7fe5b7e00640 1 -- 172.21.15.42:0/2212785980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f98e765170 con 0x55f98eb28820 2024-04-20T10:41:08.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.475+0000 7fe5b7e00640 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb28820 0x55f98eb3b150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8018040 tx=0x7fe5a800bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a15bea02ad5984e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:08.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.475+0000 7fe5a2000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:41:08.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.475+0000 7fe5a2000640 1 -- 172.21.15.42:0/2212785980 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8039070 con 0x55f98eb28820 2024-04-20T10:41:08.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.475+0000 7fe5a2000640 1 -- 172.21.15.42:0/2212785980 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7fe5a802a730 con 0x55f98eb28820 2024-04-20T10:41:08.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.475+0000 7fe5c1b96740 1 -- 172.21.15.42:0/2212785980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a802f070 con 0x55f98eb28820 2024-04-20T10:41:08.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.476+0000 7fe5a2000640 1 -- 172.21.15.42:0/2212785980 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe5a8045050 con 0x55f98eb28820 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.476+0000 7fe5c1b96740 10 client.26228 fetch_fsmap learned FSMap version 19 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.476+0000 7fe5c1b96740 1 -- 172.21.15.42:0/2212785980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f98ebadc70 con 0x55f98eb28820 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.476+0000 7fe5a2000640 1 -- 172.21.15.42:0/2212785980 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a802f070 con 0x55f98eb28820 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.476+0000 7fe5a2000640 1 -- 172.21.15.42:0/2212785980 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fe5a8011b50 con 0x55f98eb28820 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.476+0000 7fe5c1b96740 10 client.26228 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.476+0000 7fe5c1b96740 10 client.26228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.476+0000 7fe5c1b96740 20 client.26228 populate_metadata read hostname 'smithi042' 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.476+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f98eb375f0 0x55f98ebb03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.476+0000 7fe5c1b96740 4 client.26228 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.476+0000 7fe5c1b96740 1 -- 172.21.15.42:0/2212785980 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f98ea9a5b0 con 0x55f98eb375f0 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.477+0000 7fe5b7400640 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f98eb375f0 0x55f98ebb03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.477+0000 7fe5b7400640 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f98eb375f0 0x55f98ebb03a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a492d76d36b1c282 server_cookie=551d4f779a247cf2 in_seq=0 out_seq=0 2024-04-20T10:41:08.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.477+0000 7fe5a2000640 10 client.26228 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:08.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.877+0000 7fe5a2000640 1 -- 172.21.15.42:0/2212785980 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f98ea9a5b0 con 0x55f98eb375f0 2024-04-20T10:41:08.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.877+0000 7fe5a2000640 10 client.26228 handle_command_reply: tid=0 2024-04-20T10:41:08.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.877+0000 7fe5c1b96740 1 client.26228 shutdown 2024-04-20T10:41:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.877+0000 7fe5c1b96740 1 -- 172.21.15.42:0/2212785980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb28820 msgr2=0x55f98eb3b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.877+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb28820 0x55f98eb3b150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8018040 tx=0x7fe5a800bf50 comp rx=0 tx=0).stop 2024-04-20T10:41:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.878+0000 7fe5c1b96740 1 -- 172.21.15.42:0/2212785980 shutdown_connections 2024-04-20T10:41:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.878+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55f98eb375f0 0x55f98ebb03a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.878+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f98eb2ea10 0x55f98eb3bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.878+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f98eb2e0f0 0x55f98eb3b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.878+0000 7fe5c1b96740 1 --2- 172.21.15.42:0/2212785980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f98eb28820 0x55f98eb3b150 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.878+0000 7fe5c1b96740 1 -- 172.21.15.42:0/2212785980 >> 172.21.15.42:0/2212785980 conn(0x55f98eb23c00 msgr2=0x55f98eb365d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:08.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.878+0000 7fe5a2000640 0 client.26228 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:08.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.879+0000 7fe5c1b96740 1 -- 172.21.15.42:0/2212785980 shutdown_connections 2024-04-20T10:41:08.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.879+0000 7fe5c1b96740 1 -- 172.21.15.42:0/2212785980 wait complete. 2024-04-20T10:41:08.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.881+0000 7fe5c1b96740 20 client.26228 trim_cache size 0 max 16384 2024-04-20T10:41:08.883 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:41:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:41:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "fcb92670-a42c-431a-8e9d-6ccb4a432966", 2024-04-20T10:41:08.883 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:41:08.883 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:41:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.884+0000 7fe5bd800640 1 -- 172.21.15.42:0/2856657591 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe59c0a3da0 msgr2=0x7fe59c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.884+0000 7fe5bd800640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe59c0a3da0 0x7fe59c0a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55f98eb2dbc0 tx=0x7fe5a800b390 comp rx=0 tx=0).stop 2024-04-20T10:41:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.884+0000 7fe5bd800640 1 -- 172.21.15.42:0/2856657591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b8074c20 msgr2=0x7fe5b81a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:08.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.884+0000 7fe5bd800640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b8074c20 0x7fe5b81a4fb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4000c00 tx=0x7fe5a4004430 comp rx=0 tx=0).stop 2024-04-20T10:41:08.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.885+0000 7fe5bd800640 1 -- 172.21.15.42:0/2856657591 shutdown_connections 2024-04-20T10:41:08.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.885+0000 7fe5bd800640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe59c0a3da0 0x7fe59c0a6260 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.886+0000 7fe5bd800640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5b8112e10 0x7fe5b819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.886+0000 7fe5bd800640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b8074c20 0x7fe5b81a4fb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.886+0000 7fe5bd800640 1 --2- 172.21.15.42:0/2856657591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5b8074300 0x7fe5b81a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:08.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.886+0000 7fe5bd800640 1 -- 172.21.15.42:0/2856657591 >> 172.21.15.42:0/2856657591 conn(0x7fe5b80fd930 msgr2=0x7fe5b80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.889+0000 7fe5bd800640 1 -- 172.21.15.42:0/2856657591 shutdown_connections 2024-04-20T10:41:08.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:08.889+0000 7fe5bd800640 1 -- 172.21.15.42:0/2856657591 wait complete. 2024-04-20T10:41:08.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:41:08.922 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:41:09.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.123+0000 7fefe3c00640 1 -- 172.21.15.42:0/2325416754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc109370 msgr2=0x7fefdc110f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:09.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.123+0000 7fefe3c00640 1 --2- 172.21.15.42:0/2325416754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc109370 0x7fefdc110f40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009ca0 tx=0x7fefd402c100 comp rx=0 tx=0).stop 2024-04-20T10:41:09.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.123+0000 7fefe3c00640 1 -- 172.21.15.42:0/2325416754 shutdown_connections 2024-04-20T10:41:09.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.123+0000 7fefe3c00640 1 --2- 172.21.15.42:0/2325416754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc109370 0x7fefdc110f40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.123+0000 7fefe3c00640 1 --2- 172.21.15.42:0/2325416754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefdc104c50 0x7fefdc108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.123+0000 7fefe3c00640 1 --2- 172.21.15.42:0/2325416754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc1042a0 0x7fefdc104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.123+0000 7fefe3c00640 1 -- 172.21.15.42:0/2325416754 >> 172.21.15.42:0/2325416754 conn(0x7fefdc0ffae0 msgr2=0x7fefdc101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:09.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.123+0000 7fefe3c00640 1 -- 172.21.15.42:0/2325416754 shutdown_connections 2024-04-20T10:41:09.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.123+0000 7fefe3c00640 1 -- 172.21.15.42:0/2325416754 wait complete. 2024-04-20T10:41:09.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.125+0000 7fefe3c00640 1 Processor -- start 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.125+0000 7fefe3c00640 1 -- start start 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.125+0000 7fefe3c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc1042a0 0x7fefdc1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.125+0000 7fefe3c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefdc109370 0x7fefdc1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.125+0000 7fefe3c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc19a7d0 0x7fefdc19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.125+0000 7fefe3c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc114410 con 0x7fefdc109370 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.125+0000 7fefe3c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc114290 con 0x7fefdc19a7d0 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.125+0000 7fefe3c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc114590 con 0x7fefdc1042a0 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.126+0000 7fefd3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefdc109370 0x7fefdc1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.126+0000 7fefe1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc19a7d0 0x7fefdc19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.126+0000 7fefe1200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc1042a0 0x7fefdc1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.126+0000 7fefe1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc19a7d0 0x7fefdc19ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:57836/0 (socket says 172.21.15.42:57836) 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.126+0000 7fefd3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefdc109370 0x7fefdc1a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45490/0 (socket says 172.21.15.42:45490) 2024-04-20T10:41:09.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.126+0000 7fefe1c00640 1 -- 172.21.15.42:0/57020255 learned_addr learned my addr 172.21.15.42:0/57020255 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:09.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.126+0000 7fefe1c00640 1 -- 172.21.15.42:0/57020255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc1042a0 msgr2=0x7fefdc1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:09.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.126+0000 7fefe1c00640 1 --2- 172.21.15.42:0/57020255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc1042a0 0x7fefdc1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.126+0000 7fefe1c00640 1 -- 172.21.15.42:0/57020255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefdc109370 msgr2=0x7fefdc1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:09.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.126+0000 7fefe1c00640 1 --2- 172.21.15.42:0/57020255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefdc109370 0x7fefdc1a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.126+0000 7fefe1c00640 1 -- 172.21.15.42:0/57020255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc19b360 con 0x7fefdc19a7d0 2024-04-20T10:41:09.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.126+0000 7fefe1c00640 1 --2- 172.21.15.42:0/57020255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc19a7d0 0x7fefdc19ac80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fefd4037040 tx=0x7fefd4032f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:09.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.127+0000 7fefda000640 1 -- 172.21.15.42:0/57020255 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4039040 con 0x7fefdc19a7d0 2024-04-20T10:41:09.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.127+0000 7fefe3c00640 1 -- 172.21.15.42:0/57020255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefdc19b5f0 con 0x7fefdc19a7d0 2024-04-20T10:41:09.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.127+0000 7fefda000640 1 -- 172.21.15.42:0/57020255 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd4044070 con 0x7fefdc19a7d0 2024-04-20T10:41:09.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.127+0000 7fefda000640 1 -- 172.21.15.42:0/57020255 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd40042a0 con 0x7fefdc19a7d0 2024-04-20T10:41:09.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.127+0000 7fefe3c00640 1 -- 172.21.15.42:0/57020255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefdc1a7780 con 0x7fefdc19a7d0 2024-04-20T10:41:09.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.131+0000 7fefe6837740 1 Processor -- start 2024-04-20T10:41:09.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.132+0000 7fefda000640 1 -- 172.21.15.42:0/57020255 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fefd4003970 con 0x7fefdc19a7d0 2024-04-20T10:41:09.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.132+0000 7fefe6837740 1 -- start start 2024-04-20T10:41:09.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.133+0000 7fefe6837740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 0x5585f32ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.133+0000 7fefe6837740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.133+0000 7fefe6837740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32d3880 0x5585f32d5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.133+0000 7fefe6837740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5585f3230150 con 0x5585f32caca0 2024-04-20T10:41:09.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.133+0000 7fefe6837740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5585f31d6370 con 0x5585f32d3880 2024-04-20T10:41:09.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.133+0000 7fefe6837740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5585f322c110 con 0x5585f32ca380 2024-04-20T10:41:09.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.133+0000 7fefd3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 0x5585f32ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.133+0000 7fefd3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 0x5585f32ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:45292/0 (socket says 172.21.15.42:45292) 2024-04-20T10:41:09.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.133+0000 7fefd3e00640 1 -- 172.21.15.42:0/2637038318 learned_addr learned my addr 172.21.15.42:0/2637038318 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:09.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.133+0000 7fefe1200640 1 --2- 172.21.15.42:0/2637038318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.133+0000 7fefd3e00640 1 --2- 172.21.15.42:0/2637038318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32d3880 0x5585f32d5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.134+0000 7fefe1200640 1 -- 172.21.15.42:0/2637038318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 msgr2=0x5585f32ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:09.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.134+0000 7fefe1200640 1 --2- 172.21.15.42:0/2637038318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 0x5585f32ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.134+0000 7fefe1200640 1 -- 172.21.15.42:0/2637038318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32d3880 msgr2=0x5585f32d5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:09.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.134+0000 7fefe1200640 1 --2- 172.21.15.42:0/2637038318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32d3880 0x5585f32d5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.134+0000 7fefe1200640 1 -- 172.21.15.42:0/2637038318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f300c7d0 con 0x5585f32caca0 2024-04-20T10:41:09.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.134+0000 7fefd3e00640 1 --2- 172.21.15.42:0/2637038318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 0x5585f32ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:41:09.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.134+0000 7fefda000640 1 --2- 172.21.15.42:0/57020255 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fefc00a3da0 0x7fefc00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.134+0000 7fefe1200640 1 --2- 172.21.15.42:0/2637038318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d3340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009c20 tx=0x7fefcc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad56aef976b2d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.134+0000 7fefda000640 1 -- 172.21.15.42:0/57020255 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fefd40f2af0 con 0x7fefdc19a7d0 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.135+0000 7fefd3e00640 1 --2- 172.21.15.42:0/57020255 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fefc00a3da0 0x7fefc00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.135+0000 7fefd2000640 1 -- 172.21.15.42:0/2637038318 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc004050 con 0x5585f32caca0 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.135+0000 7fefd2000640 1 -- 172.21.15.42:0/2637038318 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefcc005500 con 0x5585f32caca0 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.135+0000 7fefd2000640 1 -- 172.21.15.42:0/2637038318 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc004db0 con 0x5585f32caca0 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.136+0000 7fefe6837740 1 -- 172.21.15.42:0/2637038318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 msgr2=0x5585f32d3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.136+0000 7fefd3e00640 1 --2- 172.21.15.42:0/57020255 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fefc00a3da0 0x7fefc00a6260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x5585f32c5c40 tx=0x7fefc4005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.136+0000 7fefe6837740 1 --2- 172.21.15.42:0/2637038318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d3340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009c20 tx=0x7fefcc011810 comp rx=0 tx=0).stop 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.136+0000 7fefe6837740 1 -- 172.21.15.42:0/2637038318 shutdown_connections 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.136+0000 7fefe6837740 1 --2- 172.21.15.42:0/2637038318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32d3880 0x5585f32d5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.136+0000 7fefe6837740 1 --2- 172.21.15.42:0/2637038318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d3340 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.136+0000 7fefe6837740 1 --2- 172.21.15.42:0/2637038318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 0x5585f32ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.136+0000 7fefe6837740 1 -- 172.21.15.42:0/2637038318 >> 172.21.15.42:0/2637038318 conn(0x5585f32c4ab0 msgr2=0x5585f32c4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:09.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.136+0000 7fefe6837740 1 -- 172.21.15.42:0/2637038318 shutdown_connections 2024-04-20T10:41:09.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.136+0000 7fefe6837740 1 -- 172.21.15.42:0/2637038318 wait complete. 2024-04-20T10:41:09.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.137+0000 7fefe6837740 1 -- start start 2024-04-20T10:41:09.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.137+0000 7fefe6837740 1 Processor -- start 2024-04-20T10:41:09.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.137+0000 7fefe6837740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 0x5585f32d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.137+0000 7fefe6837740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.137+0000 7fefe6837740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32d3880 0x5585f32d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.137+0000 7fefe6837740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5585f32d6670 con 0x5585f32caca0 2024-04-20T10:41:09.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe6837740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5585f31e9740 con 0x5585f32d3880 2024-04-20T10:41:09.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe6837740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5585f32d67f0 con 0x5585f32ca380 2024-04-20T10:41:09.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe6837740 10 client.0 _ll_register_callbacks cb 0x5585f31e4b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:41:09.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe1200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 0x5585f32d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe1200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 0x5585f32d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:45294/0 (socket says 172.21.15.42:45294) 2024-04-20T10:41:09.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32d3880 0x5585f32d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe1200640 1 -- 172.21.15.42:0/3390787908 learned_addr learned my addr 172.21.15.42:0/3390787908 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefd3e00640 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe1200640 1 -- 172.21.15.42:0/3390787908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32d3880 msgr2=0x5585f32d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe1200640 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32d3880 0x5585f32d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe1200640 1 -- 172.21.15.42:0/3390787908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 msgr2=0x5585f32d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe1200640 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe1200640 1 -- 172.21.15.42:0/3390787908 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585f2f423f0 con 0x5585f32ca380 2024-04-20T10:41:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.138+0000 7fefe1c00640 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32d3880 0x5585f32d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:41:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.139+0000 7fefe1200640 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 0x5585f32d08f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fefcc004480 tx=0x7fefcc005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4de091006d21545 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.139+0000 7fefcaa00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:41:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.139+0000 7fefcaa00640 1 -- 172.21.15.42:0/3390787908 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc029070 con 0x5585f32ca380 2024-04-20T10:41:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.139+0000 7fefcaa00640 1 -- 172.21.15.42:0/3390787908 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fefcc004f00 con 0x5585f32ca380 2024-04-20T10:41:09.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.139+0000 7fefe6837740 1 -- 172.21.15.42:0/3390787908 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefcc005500 con 0x5585f32ca380 2024-04-20T10:41:09.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.140+0000 7fefcaa00640 1 -- 172.21.15.42:0/3390787908 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fefcc033050 con 0x5585f32ca380 2024-04-20T10:41:09.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.140+0000 7fefe6837740 10 client.26254 fetch_fsmap learned FSMap version 19 2024-04-20T10:41:09.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.140+0000 7fefe6837740 1 -- 172.21.15.42:0/3390787908 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5585f2f020f0 con 0x5585f32ca380 2024-04-20T10:41:09.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.140+0000 7fefcaa00640 1 -- 172.21.15.42:0/3390787908 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefcc005500 con 0x5585f32ca380 2024-04-20T10:41:09.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.140+0000 7fefcaa00640 1 -- 172.21.15.42:0/3390787908 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fefcc02e440 con 0x5585f32ca380 2024-04-20T10:41:09.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.141+0000 7fefe6837740 10 client.26254 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:41:09.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.141+0000 7fefe6837740 10 client.26254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:41:09.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.141+0000 7fefe6837740 20 client.26254 populate_metadata read hostname 'smithi042' 2024-04-20T10:41:09.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.141+0000 7fefe6837740 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5585f32e3c20 0x5585f32e6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.141+0000 7fefe6837740 4 client.26254 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:41:09.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.141+0000 7fefe6837740 1 -- 172.21.15.42:0/3390787908 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585f323db70 con 0x5585f32e3c20 2024-04-20T10:41:09.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.141+0000 7fefd3e00640 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5585f32e3c20 0x5585f32e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.142+0000 7fefd3e00640 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5585f32e3c20 0x5585f32e6010 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b7f36b7cbec6be server_cookie=77689e6d0d51553c in_seq=0 out_seq=0 2024-04-20T10:41:09.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.142+0000 7fefcaa00640 10 client.26254 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:09.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.143+0000 7fefcaa00640 1 -- 172.21.15.42:0/3390787908 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585f323db70 con 0x5585f32e3c20 2024-04-20T10:41:09.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.143+0000 7fefcaa00640 10 client.26254 handle_command_reply: tid=0 2024-04-20T10:41:09.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.143+0000 7fefe6837740 1 client.26254 shutdown 2024-04-20T10:41:09.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.143+0000 7fefe6837740 1 -- 172.21.15.42:0/3390787908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 msgr2=0x5585f32d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:09.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.143+0000 7fefe6837740 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 0x5585f32d08f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fefcc004480 tx=0x7fefcc005d90 comp rx=0 tx=0).stop 2024-04-20T10:41:09.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.144+0000 7fefe6837740 1 -- 172.21.15.42:0/3390787908 shutdown_connections 2024-04-20T10:41:09.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.144+0000 7fefe6837740 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5585f32e3c20 0x5585f32e6010 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.144+0000 7fefe6837740 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32d3880 0x5585f32d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.144+0000 7fefe6837740 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.144+0000 7fefe6837740 1 --2- 172.21.15.42:0/3390787908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32ca380 0x5585f32d08f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.144+0000 7fefcaa00640 0 client.26254 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:09.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.144+0000 7fefe6837740 1 -- 172.21.15.42:0/3390787908 >> 172.21.15.42:0/3390787908 conn(0x5585f32c4ab0 msgr2=0x5585f32c6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:09.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.144+0000 7fefe6837740 1 -- 172.21.15.42:0/3390787908 shutdown_connections 2024-04-20T10:41:09.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.144+0000 7fefe6837740 1 -- 172.21.15.42:0/3390787908 wait complete. 2024-04-20T10:41:09.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.144+0000 7fefe6837740 20 client.26254 trim_cache size 0 max 16384 2024-04-20T10:41:09.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.167+0000 7fefe6837740 1 Processor -- start 2024-04-20T10:41:09.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.167+0000 7fefe6837740 1 -- start start 2024-04-20T10:41:09.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe6837740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 0x5585f32d3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe6837740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe6837740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 0x5585f32df560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe6837740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1add80 con 0x5585f32caca0 2024-04-20T10:41:09.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe6837740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1b05e0 con 0x5585f32ca380 2024-04-20T10:41:09.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe6837740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1ae3d0 con 0x5585f32d3880 2024-04-20T10:41:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 0x5585f32d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe1c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 0x5585f32df560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe1c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 0x5585f32df560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:45300/0 (socket says 172.21.15.42:45300) 2024-04-20T10:41:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe1c00640 1 -- 172.21.15.42:0/2450038702 learned_addr learned my addr 172.21.15.42:0/2450038702 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe1200640 1 --2- 172.21.15.42:0/2450038702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 0x5585f32d3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:57864/0 (socket says 172.21.15.42:57864) 2024-04-20T10:41:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefd3e00640 1 --2- 172.21.15.42:0/2450038702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe1c00640 1 -- 172.21.15.42:0/2450038702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 msgr2=0x5585f32d3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe1c00640 1 --2- 172.21.15.42:0/2450038702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 0x5585f32d3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.168+0000 7fefe1c00640 1 -- 172.21.15.42:0/2450038702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 msgr2=0x5585f32cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.169+0000 7fefe1c00640 1 --2- 172.21.15.42:0/2450038702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.169+0000 7fefe1c00640 1 -- 172.21.15.42:0/2450038702 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f32caad0 con 0x5585f32d3880 2024-04-20T10:41:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.169+0000 7fefe1c00640 1 --2- 172.21.15.42:0/2450038702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 0x5585f32df560 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fefd40bb570 tx=0x7fefd40095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3078521994cb7ce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.169+0000 7fefd1600640 1 -- 172.21.15.42:0/2450038702 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd40beba0 con 0x5585f32d3880 2024-04-20T10:41:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.169+0000 7fefd1600640 1 -- 172.21.15.42:0/2450038702 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd40c2f50 con 0x5585f32d3880 2024-04-20T10:41:09.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.169+0000 7fefd1600640 1 -- 172.21.15.42:0/2450038702 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd40bf060 con 0x5585f32d3880 2024-04-20T10:41:09.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.170+0000 7fefe6837740 1 -- 172.21.15.42:0/2450038702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 msgr2=0x5585f32df560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:09.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.170+0000 7fefe6837740 1 --2- 172.21.15.42:0/2450038702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 0x5585f32df560 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fefd40bb570 tx=0x7fefd40095c0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.170+0000 7fefe6837740 1 -- 172.21.15.42:0/2450038702 shutdown_connections 2024-04-20T10:41:09.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.170+0000 7fefe6837740 1 --2- 172.21.15.42:0/2450038702 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 0x5585f32df560 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.170+0000 7fefe6837740 1 --2- 172.21.15.42:0/2450038702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.170+0000 7fefe6837740 1 --2- 172.21.15.42:0/2450038702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 0x5585f32d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.170+0000 7fefe6837740 1 -- 172.21.15.42:0/2450038702 >> 172.21.15.42:0/2450038702 conn(0x5585f32c4ab0 msgr2=0x5585f32d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:09.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.170+0000 7fefe6837740 1 -- 172.21.15.42:0/2450038702 shutdown_connections 2024-04-20T10:41:09.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.170+0000 7fefe6837740 1 -- 172.21.15.42:0/2450038702 wait complete. 2024-04-20T10:41:09.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.170+0000 7fefe6837740 1 -- start start 2024-04-20T10:41:09.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.171+0000 7fefe6837740 1 Processor -- start 2024-04-20T10:41:09.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.171+0000 7fefe6837740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 0x5585f32dbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.171+0000 7fefe6837740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.171+0000 7fefe6837740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 0x5585f32d76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.171+0000 7fefe6837740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1aeaa0 con 0x5585f32caca0 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.171+0000 7fefe6837740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1aff10 con 0x5585f32ca380 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.171+0000 7fefe6837740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1af170 con 0x5585f32d3880 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.171+0000 7fefe6837740 10 client.0 _ll_register_callbacks cb 0x7fefc4009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefe1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 0x5585f32dbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefe1c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 0x5585f32d76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefe1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 0x5585f32dbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:57872/0 (socket says 172.21.15.42:57872) 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefe1200640 1 -- 172.21.15.42:0/604256956 learned_addr learned my addr 172.21.15.42:0/604256956 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefd3e00640 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefe1200640 1 -- 172.21.15.42:0/604256956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 msgr2=0x5585f32d76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefe1200640 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 0x5585f32d76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefe1200640 1 -- 172.21.15.42:0/604256956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 msgr2=0x5585f32d71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefe1200640 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefe1200640 1 -- 172.21.15.42:0/604256956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585f32d7ec0 con 0x5585f32ca380 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefe1c00640 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 0x5585f32d76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:41:09.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefd3e00640 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:41:09.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefe1200640 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 0x5585f32dbe30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fefcc02e890 tx=0x7fefcc009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb000933f2bc0838 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:09.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.172+0000 7fefcaa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:41:09.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.173+0000 7fefcaa00640 1 -- 172.21.15.42:0/604256956 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc03c020 con 0x5585f32ca380 2024-04-20T10:41:09.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.173+0000 7fefcaa00640 1 -- 172.21.15.42:0/604256956 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fefcc0045c0 con 0x5585f32ca380 2024-04-20T10:41:09.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.173+0000 7fefe6837740 1 -- 172.21.15.42:0/604256956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefd40c2f50 con 0x5585f32ca380 2024-04-20T10:41:09.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.173+0000 7fefcaa00640 1 -- 172.21.15.42:0/604256956 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fefcc029050 con 0x5585f32ca380 2024-04-20T10:41:09.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.173+0000 7fefe6837740 10 client.26246 fetch_fsmap learned FSMap version 19 2024-04-20T10:41:09.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.173+0000 7fefe6837740 1 -- 172.21.15.42:0/604256956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5585f334a090 con 0x5585f32ca380 2024-04-20T10:41:09.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.173+0000 7fefcaa00640 1 -- 172.21.15.42:0/604256956 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd40c2f50 con 0x5585f32ca380 2024-04-20T10:41:09.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.174+0000 7fefcaa00640 1 -- 172.21.15.42:0/604256956 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fefcc0035f0 con 0x5585f32ca380 2024-04-20T10:41:09.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.174+0000 7fefe6837740 10 client.26246 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:41:09.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.174+0000 7fefe6837740 10 client.26246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:41:09.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.174+0000 7fefe6837740 20 client.26246 populate_metadata read hostname 'smithi042' 2024-04-20T10:41:09.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.174+0000 7fefe6837740 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5585f334a740 0x5585f334cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:09.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.174+0000 7fefe6837740 4 client.26246 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:41:09.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.174+0000 7fefe6837740 1 -- 172.21.15.42:0/604256956 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585f32379e0 con 0x5585f334a740 2024-04-20T10:41:09.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.174+0000 7fefd3e00640 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5585f334a740 0x5585f334cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:09.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.175+0000 7fefd3e00640 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5585f334a740 0x5585f334cb30 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9789988f36dc7f8 server_cookie=6e0f86264223f21f in_seq=0 out_seq=0 2024-04-20T10:41:09.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.175+0000 7fefcaa00640 10 client.26246 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:09.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefcaa00640 1 -- 172.21.15.42:0/604256956 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5585f32379e0 con 0x5585f334a740 2024-04-20T10:41:09.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefcaa00640 10 client.26246 handle_command_reply: tid=0 2024-04-20T10:41:09.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefe6837740 1 client.26246 shutdown 2024-04-20T10:41:09.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefe6837740 1 -- 172.21.15.42:0/604256956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 msgr2=0x5585f32dbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:09.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefe6837740 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 0x5585f32dbe30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fefcc02e890 tx=0x7fefcc009a70 comp rx=0 tx=0).stop 2024-04-20T10:41:09.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefe6837740 1 -- 172.21.15.42:0/604256956 shutdown_connections 2024-04-20T10:41:09.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefe6837740 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5585f334a740 0x5585f334cb30 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefe6837740 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585f32d3880 0x5585f32d76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefe6837740 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585f32caca0 0x5585f32d71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefcaa00640 0 client.26246 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:09.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefe6837740 1 --2- 172.21.15.42:0/604256956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585f32ca380 0x5585f32dbe30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefe6837740 1 -- 172.21.15.42:0/604256956 >> 172.21.15.42:0/604256956 conn(0x5585f32c4ab0 msgr2=0x5585f32c3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:09.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefe6837740 1 -- 172.21.15.42:0/604256956 shutdown_connections 2024-04-20T10:41:09.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.176+0000 7fefe6837740 1 -- 172.21.15.42:0/604256956 wait complete. 2024-04-20T10:41:09.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.177+0000 7fefe6837740 20 client.26246 trim_cache size 0 max 16384 2024-04-20T10:41:09.179 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:41:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (3032 inodes in the stack)", 2024-04-20T10:41:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:41:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "fcb92670-a42c-431a-8e9d-6ccb4a432966": { 2024-04-20T10:41:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:41:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "fcb92670-a42c-431a-8e9d-6ccb4a432966", 2024-04-20T10:41:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:41:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:41:09.180 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:41:09.180 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:41:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.177+0000 7fefe3c00640 1 -- 172.21.15.42:0/57020255 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fefc00a3da0 msgr2=0x7fefc00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.177+0000 7fefe3c00640 1 --2- 172.21.15.42:0/57020255 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fefc00a3da0 0x7fefc00a6260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x5585f32c5c40 tx=0x7fefc4005f90 comp rx=0 tx=0).stop 2024-04-20T10:41:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.177+0000 7fefe3c00640 1 -- 172.21.15.42:0/57020255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc19a7d0 msgr2=0x7fefdc19ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.177+0000 7fefe3c00640 1 --2- 172.21.15.42:0/57020255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc19a7d0 0x7fefdc19ac80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fefd4037040 tx=0x7fefd4032f30 comp rx=0 tx=0).stop 2024-04-20T10:41:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.177+0000 7fefe3c00640 1 -- 172.21.15.42:0/57020255 shutdown_connections 2024-04-20T10:41:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.177+0000 7fefe3c00640 1 --2- 172.21.15.42:0/57020255 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fefc00a3da0 0x7fefc00a6260 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.177+0000 7fefe3c00640 1 --2- 172.21.15.42:0/57020255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc19a7d0 0x7fefdc19ac80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.177+0000 7fefe3c00640 1 --2- 172.21.15.42:0/57020255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefdc109370 0x7fefdc1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.177+0000 7fefe3c00640 1 --2- 172.21.15.42:0/57020255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc1042a0 0x7fefdc1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.177+0000 7fefe3c00640 1 -- 172.21.15.42:0/57020255 >> 172.21.15.42:0/57020255 conn(0x7fefdc0ffae0 msgr2=0x7fefdc101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.178+0000 7fefe3c00640 1 -- 172.21.15.42:0/57020255 shutdown_connections 2024-04-20T10:41:09.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:09.178+0000 7fefe3c00640 1 -- 172.21.15.42:0/57020255 wait complete. 2024-04-20T10:41:09.197 INFO:tasks.cephfs.filesystem:scrub status for tag:fcb92670-a42c-431a-8e9d-6ccb4a432966 - {'path': '/', 'tag': 'fcb92670-a42c-431a-8e9d-6ccb4a432966', 'options': 'recursive,force'} 2024-04-20T10:41:09.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:09 smithi162 ceph-mon[21854]: pgmap v218: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 1.08k op/s 2024-04-20T10:41:09.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:09 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:41:09.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:09 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:41:09.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:09 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:41:09.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:09 smithi162 ceph-mon[21854]: osdmap e90: 12 total, 12 up, 12 in 2024-04-20T10:41:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:09 smithi046 ceph-mon[21574]: pgmap v218: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 1.08k op/s 2024-04-20T10:41:10.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:09 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:41:10.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:09 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:41:10.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:09 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:41:10.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:09 smithi046 ceph-mon[21574]: osdmap e90: 12 total, 12 up, 12 in 2024-04-20T10:41:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:09 smithi042 ceph-mon[17078]: pgmap v218: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 1.08k op/s 2024-04-20T10:41:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:09 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:41:10.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:09 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:41:10.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:09 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:41:10.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:09 smithi042 ceph-mon[17078]: osdmap e90: 12 total, 12 up, 12 in 2024-04-20T10:41:12.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:11 smithi046 ceph-mon[21574]: pgmap v220: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 613 op/s 2024-04-20T10:41:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:11 smithi042 ceph-mon[17078]: pgmap v220: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 613 op/s 2024-04-20T10:41:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:11 smithi162 ceph-mon[21854]: pgmap v220: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 613 op/s 2024-04-20T10:41:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:13 smithi046 ceph-mon[21574]: pgmap v221: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.8 MiB/s wr, 996 op/s 2024-04-20T10:41:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:13 smithi042 ceph-mon[17078]: pgmap v221: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.8 MiB/s wr, 996 op/s 2024-04-20T10:41:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:13 smithi162 ceph-mon[21854]: pgmap v221: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.8 MiB/s wr, 996 op/s 2024-04-20T10:41:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:15 smithi046 ceph-mon[21574]: pgmap v222: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 893 op/s 2024-04-20T10:41:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:15 smithi042 ceph-mon[17078]: pgmap v222: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 893 op/s 2024-04-20T10:41:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:15 smithi162 ceph-mon[21854]: pgmap v222: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 893 op/s 2024-04-20T10:41:18.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:17 smithi046 ceph-mon[21574]: pgmap v223: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 940 op/s 2024-04-20T10:41:18.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:17 smithi042 ceph-mon[17078]: pgmap v223: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 940 op/s 2024-04-20T10:41:18.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:17 smithi162 ceph-mon[21854]: pgmap v223: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 940 op/s 2024-04-20T10:41:19.951 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:19 smithi162 ceph-mon[21854]: pgmap v224: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 1.23k op/s 2024-04-20T10:41:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:19 smithi046 ceph-mon[21574]: pgmap v224: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 1.23k op/s 2024-04-20T10:41:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:19 smithi042 ceph-mon[17078]: pgmap v224: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 1.23k op/s 2024-04-20T10:41:21.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:20 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:41:21.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:20 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:41:21.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:20 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:41:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:21 smithi046 ceph-mon[21574]: pgmap v225: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 1.06k op/s 2024-04-20T10:41:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:21 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:41:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:21 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:41:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:21 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:41:22.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:21 smithi042 ceph-mon[17078]: pgmap v225: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 1.06k op/s 2024-04-20T10:41:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:21 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:41:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:21 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:41:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:21 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:41:22.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:21 smithi162 ceph-mon[21854]: pgmap v225: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 1.06k op/s 2024-04-20T10:41:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:21 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:41:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:21 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:41:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:21 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:41:24.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:23 smithi046 ceph-mon[21574]: pgmap v226: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 1.40k op/s 2024-04-20T10:41:24.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:23 smithi042 ceph-mon[17078]: pgmap v226: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 1.40k op/s 2024-04-20T10:41:24.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:23 smithi162 ceph-mon[21854]: pgmap v226: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 1.40k op/s 2024-04-20T10:41:26.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:25 smithi046 ceph-mon[21574]: pgmap v227: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 1.15k op/s 2024-04-20T10:41:26.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:25 smithi042 ceph-mon[17078]: pgmap v227: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 1.15k op/s 2024-04-20T10:41:26.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:25 smithi162 ceph-mon[21854]: pgmap v227: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 1.15k op/s 2024-04-20T10:41:28.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:27 smithi046 ceph-mon[21574]: pgmap v228: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 1.45k op/s 2024-04-20T10:41:28.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:27 smithi042 ceph-mon[17078]: pgmap v228: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 1.45k op/s 2024-04-20T10:41:28.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:27 smithi162 ceph-mon[21854]: pgmap v228: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 1.45k op/s 2024-04-20T10:41:30.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:29 smithi046 ceph-mon[21574]: pgmap v229: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 1.68k op/s 2024-04-20T10:41:30.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:29 smithi042 ceph-mon[17078]: pgmap v229: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 1.68k op/s 2024-04-20T10:41:30.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:29 smithi162 ceph-mon[21854]: pgmap v229: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 1.68k op/s 2024-04-20T10:41:32.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:31 smithi046 ceph-mon[21574]: pgmap v230: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 1.21k op/s 2024-04-20T10:41:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:31 smithi042 ceph-mon[17078]: pgmap v230: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 1.21k op/s 2024-04-20T10:41:32.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:31 smithi162 ceph-mon[21854]: pgmap v230: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 1.21k op/s 2024-04-20T10:41:34.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:33 smithi046 ceph-mon[21574]: pgmap v231: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 2.23k op/s 2024-04-20T10:41:34.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:33 smithi042 ceph-mon[17078]: pgmap v231: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 2.23k op/s 2024-04-20T10:41:34.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:33 smithi162 ceph-mon[21854]: pgmap v231: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 2.23k op/s 2024-04-20T10:41:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:35 smithi046 ceph-mon[21574]: pgmap v232: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 1.85k op/s 2024-04-20T10:41:36.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:35 smithi042 ceph-mon[17078]: pgmap v232: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 1.85k op/s 2024-04-20T10:41:36.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:35 smithi162 ceph-mon[21854]: pgmap v232: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 1.85k op/s 2024-04-20T10:41:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:37 smithi046 ceph-mon[21574]: pgmap v233: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 2.10k op/s 2024-04-20T10:41:38.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:37 smithi042 ceph-mon[17078]: pgmap v233: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 2.10k op/s 2024-04-20T10:41:38.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:37 smithi162 ceph-mon[21854]: pgmap v233: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 2.10k op/s 2024-04-20T10:41:39.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:41:39.199 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:41:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.425+0000 7fb9df200640 1 -- 172.21.15.42:0/3285930223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e0100dd0 msgr2=0x7fb9e01011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.425+0000 7fb9df200640 1 --2- 172.21.15.42:0/3285930223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e0100dd0 0x7fb9e01011b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009900 tx=0x7fb9d002bd10 comp rx=0 tx=0).stop 2024-04-20T10:41:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.426+0000 7fb9df200640 1 -- 172.21.15.42:0/3285930223 shutdown_connections 2024-04-20T10:41:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.426+0000 7fb9df200640 1 --2- 172.21.15.42:0/3285930223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e01061d0 0x7fb9e0106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.426+0000 7fb9df200640 1 --2- 172.21.15.42:0/3285930223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e01016f0 0x7fb9e0105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.426+0000 7fb9df200640 1 --2- 172.21.15.42:0/3285930223 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e0100dd0 0x7fb9e01011b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.426+0000 7fb9df200640 1 -- 172.21.15.42:0/3285930223 >> 172.21.15.42:0/3285930223 conn(0x7fb9e00fc7e0 msgr2=0x7fb9e00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.426+0000 7fb9df200640 1 -- 172.21.15.42:0/3285930223 shutdown_connections 2024-04-20T10:41:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.426+0000 7fb9df200640 1 -- 172.21.15.42:0/3285930223 wait complete. 2024-04-20T10:41:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.427+0000 7fb9df200640 1 Processor -- start 2024-04-20T10:41:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.427+0000 7fb9df200640 1 -- start start 2024-04-20T10:41:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.427+0000 7fb9df200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e0100dd0 0x7fb9e019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9df200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e01016f0 0x7fb9e019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9df200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e01061d0 0x7fb9e01a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9df200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0118840 con 0x7fb9e01016f0 2024-04-20T10:41:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9df200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01186c0 con 0x7fb9e0100dd0 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9df200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01189c0 con 0x7fb9e01061d0 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e0100dd0 0x7fb9e019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e01016f0 0x7fb9e019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e0100dd0 0x7fb9e019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:43982/0 (socket says 172.21.15.42:43982) 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9dde00640 1 -- 172.21.15.42:0/1130075796 learned_addr learned my addr 172.21.15.42:0/1130075796 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9de800640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e01061d0 0x7fb9e01a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9de800640 1 -- 172.21.15.42:0/1130075796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e0100dd0 msgr2=0x7fb9e019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9de800640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e0100dd0 0x7fb9e019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9de800640 1 -- 172.21.15.42:0/1130075796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e01016f0 msgr2=0x7fb9e019f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9de800640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e01016f0 0x7fb9e019f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9de800640 1 -- 172.21.15.42:0/1130075796 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e01a3d30 con 0x7fb9e01061d0 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.428+0000 7fb9dde00640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e0100dd0 0x7fb9e019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:41:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.429+0000 7fb9de800640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e01061d0 0x7fb9e01a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800ec50 tx=0x7fb9d800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:39.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.429+0000 7fb9d6a00640 1 -- 172.21.15.42:0/1130075796 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8003b70 con 0x7fb9e01061d0 2024-04-20T10:41:39.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.429+0000 7fb9d6a00640 1 -- 172.21.15.42:0/1130075796 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d8003d10 con 0x7fb9e01061d0 2024-04-20T10:41:39.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.429+0000 7fb9d6a00640 1 -- 172.21.15.42:0/1130075796 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8010890 con 0x7fb9e01061d0 2024-04-20T10:41:39.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.429+0000 7fb9df200640 1 -- 172.21.15.42:0/1130075796 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e01a4020 con 0x7fb9e01061d0 2024-04-20T10:41:39.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.429+0000 7fb9df200640 1 -- 172.21.15.42:0/1130075796 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e01ab8b0 con 0x7fb9e01061d0 2024-04-20T10:41:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.433+0000 7fb9d6a00640 1 -- 172.21.15.42:0/1130075796 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb9d8003fc0 con 0x7fb9e01061d0 2024-04-20T10:41:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.433+0000 7fb9e77af740 1 Processor -- start 2024-04-20T10:41:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.434+0000 7fb9e77af740 1 -- start start 2024-04-20T10:41:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.434+0000 7fb9e77af740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b009760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.434+0000 7fb9e77af740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 0x55b57b012340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.434+0000 7fb9e77af740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 0x55b57b014c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.434+0000 7fb9e77af740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b57af6f150 con 0x55b57b009ca0 2024-04-20T10:41:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.434+0000 7fb9e77af740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b57af15370 con 0x55b57b012880 2024-04-20T10:41:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.434+0000 7fb9e77af740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b57af6b110 con 0x55b57b009380 2024-04-20T10:41:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9dde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 0x55b57b012340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 0x55b57b014c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 0x55b57b014c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:43984/0 (socket says 172.21.15.42:43984) 2024-04-20T10:41:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9dd400640 1 -- 172.21.15.42:0/3805277943 learned_addr learned my addr 172.21.15.42:0/3805277943 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9dd400640 1 --2- 172.21.15.42:0/3805277943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b009760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9d6a00640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb9c40a3da0 0x7fb9c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9dde00640 1 -- 172.21.15.42:0/3805277943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 msgr2=0x55b57b009760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9dde00640 1 --2- 172.21.15.42:0/3805277943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b009760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9dde00640 1 -- 172.21.15.42:0/3805277943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 msgr2=0x55b57b014c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9dde00640 1 --2- 172.21.15.42:0/3805277943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 0x55b57b014c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9dde00640 1 -- 172.21.15.42:0/3805277943 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b57ad4b7d0 con 0x55b57b009ca0 2024-04-20T10:41:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9dd400640 1 --2- 172.21.15.42:0/3805277943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 0x55b57b014c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:41:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.435+0000 7fb9d6a00640 1 -- 172.21.15.42:0/1130075796 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fb9d80d0d20 con 0x7fb9e01061d0 2024-04-20T10:41:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.436+0000 7fb9de800640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb9c40a3da0 0x7fb9c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.436+0000 7fb9dde00640 1 --2- 172.21.15.42:0/3805277943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 0x55b57b012340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005ab0 tx=0x7fb9d0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b2a3838f52a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.436+0000 7fb9cb000640 1 -- 172.21.15.42:0/3805277943 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0043070 con 0x55b57b009ca0 2024-04-20T10:41:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.436+0000 7fb9cb000640 1 -- 172.21.15.42:0/3805277943 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d002c6d0 con 0x55b57b009ca0 2024-04-20T10:41:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.436+0000 7fb9cb000640 1 -- 172.21.15.42:0/3805277943 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0004f40 con 0x55b57b009ca0 2024-04-20T10:41:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.436+0000 7fb9de800640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb9c40a3da0 0x7fb9c40a6260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80ce440 tx=0x7fb9d800ef50 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.437+0000 7fb9e77af740 1 -- 172.21.15.42:0/3805277943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 msgr2=0x55b57b012340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.437+0000 7fb9e77af740 1 --2- 172.21.15.42:0/3805277943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 0x55b57b012340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005ab0 tx=0x7fb9d0009560 comp rx=0 tx=0).stop 2024-04-20T10:41:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.437+0000 7fb9e77af740 1 -- 172.21.15.42:0/3805277943 shutdown_connections 2024-04-20T10:41:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.437+0000 7fb9e77af740 1 --2- 172.21.15.42:0/3805277943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 0x55b57b014c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.437+0000 7fb9e77af740 1 --2- 172.21.15.42:0/3805277943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 0x55b57b012340 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.437+0000 7fb9e77af740 1 --2- 172.21.15.42:0/3805277943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b009760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.437+0000 7fb9e77af740 1 -- 172.21.15.42:0/3805277943 >> 172.21.15.42:0/3805277943 conn(0x55b57b003ab0 msgr2=0x55b57b003e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.438+0000 7fb9e77af740 1 -- 172.21.15.42:0/3805277943 shutdown_connections 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.438+0000 7fb9e77af740 1 -- 172.21.15.42:0/3805277943 wait complete. 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.438+0000 7fb9e77af740 1 -- start start 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.438+0000 7fb9e77af740 1 Processor -- start 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.439+0000 7fb9e77af740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009380 0x55b57b00f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.439+0000 7fb9e77af740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 0x55b57b00fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.439+0000 7fb9e77af740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b012880 0x55b57b010340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.439+0000 7fb9e77af740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b57b015610 con 0x55b57b009380 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.439+0000 7fb9e77af740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b57af28740 con 0x55b57b009ca0 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.439+0000 7fb9e77af740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b57b015790 con 0x55b57b012880 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.439+0000 7fb9e77af740 10 client.0 _ll_register_callbacks cb 0x55b57af23b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.439+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009380 0x55b57b00f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.439+0000 7fb9dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 0x55b57b00fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.439+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009380 0x55b57b00f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51276/0 (socket says 172.21.15.42:51276) 2024-04-20T10:41:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.439+0000 7fb9dd400640 1 -- 172.21.15.42:0/3973939411 learned_addr learned my addr 172.21.15.42:0/3973939411 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.439+0000 7fb9dd400640 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b012880 0x55b57b010340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.440+0000 7fb9dde00640 1 -- 172.21.15.42:0/3973939411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b012880 msgr2=0x55b57b010340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.440+0000 7fb9dde00640 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b012880 0x55b57b010340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.440+0000 7fb9dde00640 1 -- 172.21.15.42:0/3973939411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009380 msgr2=0x55b57b00f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.440+0000 7fb9dde00640 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009380 0x55b57b00f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.440+0000 7fb9dde00640 1 -- 172.21.15.42:0/3973939411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b57ac813f0 con 0x55b57b009ca0 2024-04-20T10:41:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.440+0000 7fb9dd400640 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009380 0x55b57b00f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:41:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.440+0000 7fb9dde00640 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 0x55b57b00fe00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb9d002c8a0 tx=0x7fb9d0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7734b3c16d39bcf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.440+0000 7fb9bf400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:41:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.440+0000 7fb9bf400640 1 -- 172.21.15.42:0/3973939411 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0043040 con 0x55b57b009ca0 2024-04-20T10:41:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.441+0000 7fb9e77af740 1 -- 172.21.15.42:0/3973939411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9d002c6d0 con 0x55b57b009ca0 2024-04-20T10:41:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.441+0000 7fb9bf400640 1 -- 172.21.15.42:0/3973939411 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fb9d0005510 con 0x55b57b009ca0 2024-04-20T10:41:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.441+0000 7fb9bf400640 1 -- 172.21.15.42:0/3973939411 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb9d0059050 con 0x55b57b009ca0 2024-04-20T10:41:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.441+0000 7fb9e77af740 10 client.26258 fetch_fsmap learned FSMap version 19 2024-04-20T10:41:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.441+0000 7fb9e77af740 1 -- 172.21.15.42:0/3973939411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b57ac410f0 con 0x55b57b009ca0 2024-04-20T10:41:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.441+0000 7fb9bf400640 1 -- 172.21.15.42:0/3973939411 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d002c6d0 con 0x55b57b009ca0 2024-04-20T10:41:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.441+0000 7fb9bf400640 1 -- 172.21.15.42:0/3973939411 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fb9d003f440 con 0x55b57b009ca0 2024-04-20T10:41:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.441+0000 7fb9e77af740 10 client.26258 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:41:39.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.441+0000 7fb9e77af740 10 client.26258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:41:39.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.442+0000 7fb9e77af740 20 client.26258 populate_metadata read hostname 'smithi042' 2024-04-20T10:41:39.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.442+0000 7fb9e77af740 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b57b022a50 0x55b57b024e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.442+0000 7fb9e77af740 4 client.26258 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:41:39.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.442+0000 7fb9e77af740 1 -- 172.21.15.42:0/3973939411 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b57af7cb70 con 0x55b57b022a50 2024-04-20T10:41:39.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.442+0000 7fb9dd400640 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b57b022a50 0x55b57b024e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.442+0000 7fb9dd400640 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b57b022a50 0x55b57b024e40 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12e5af06ff3248c server_cookie=a36901bf784234aa in_seq=0 out_seq=0 2024-04-20T10:41:39.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.442+0000 7fb9bf400640 10 client.26258 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:39.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.444+0000 7fb9bf400640 1 -- 172.21.15.42:0/3973939411 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b57af7cb70 con 0x55b57b022a50 2024-04-20T10:41:39.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.444+0000 7fb9bf400640 10 client.26258 handle_command_reply: tid=0 2024-04-20T10:41:39.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.444+0000 7fb9e77af740 1 client.26258 shutdown 2024-04-20T10:41:39.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.444+0000 7fb9e77af740 1 -- 172.21.15.42:0/3973939411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 msgr2=0x55b57b00fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:39.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.444+0000 7fb9e77af740 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 0x55b57b00fe00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb9d002c8a0 tx=0x7fb9d0009260 comp rx=0 tx=0).stop 2024-04-20T10:41:39.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.444+0000 7fb9e77af740 1 -- 172.21.15.42:0/3973939411 shutdown_connections 2024-04-20T10:41:39.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.444+0000 7fb9e77af740 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b57b022a50 0x55b57b024e40 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.444+0000 7fb9e77af740 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b012880 0x55b57b010340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.444+0000 7fb9e77af740 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 0x55b57b00fe00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.444+0000 7fb9bf400640 0 client.26258 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:39.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.444+0000 7fb9e77af740 1 --2- 172.21.15.42:0/3973939411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009380 0x55b57b00f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.444+0000 7fb9e77af740 1 -- 172.21.15.42:0/3973939411 >> 172.21.15.42:0/3973939411 conn(0x55b57b003ab0 msgr2=0x55b57b005800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:39.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.445+0000 7fb9e77af740 1 -- 172.21.15.42:0/3973939411 shutdown_connections 2024-04-20T10:41:39.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.445+0000 7fb9e77af740 1 -- 172.21.15.42:0/3973939411 wait complete. 2024-04-20T10:41:39.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.445+0000 7fb9e77af740 20 client.26258 trim_cache size 0 max 16384 2024-04-20T10:41:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.467+0000 7fb9e77af740 1 Processor -- start 2024-04-20T10:41:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.467+0000 7fb9e77af740 1 -- start start 2024-04-20T10:41:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9e77af740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b011ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9e77af740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 0x55b57b01e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9e77af740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 0x55b57b02a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9e77af740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01b21b0 con 0x55b57b009ca0 2024-04-20T10:41:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9e77af740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01b4a10 con 0x55b57b012880 2024-04-20T10:41:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9e77af740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01b2800 con 0x55b57b009380 2024-04-20T10:41:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b011ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9dde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 0x55b57b01e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9dde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 0x55b57b01e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51278/0 (socket says 172.21.15.42:51278) 2024-04-20T10:41:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9dd400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b011ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:47888/0 (socket says 172.21.15.42:47888) 2024-04-20T10:41:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9dde00640 1 -- 172.21.15.42:0/1595490845 learned_addr learned my addr 172.21.15.42:0/1595490845 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9dd400640 1 --2- 172.21.15.42:0/1595490845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 0x55b57b02a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9dde00640 1 -- 172.21.15.42:0/1595490845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 msgr2=0x55b57b011ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9dde00640 1 --2- 172.21.15.42:0/1595490845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b011ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9dde00640 1 -- 172.21.15.42:0/1595490845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 msgr2=0x55b57b02a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9dde00640 1 --2- 172.21.15.42:0/1595490845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 0x55b57b02a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.468+0000 7fb9dde00640 1 -- 172.21.15.42:0/1595490845 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b57b0251b0 con 0x55b57b009ca0 2024-04-20T10:41:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.469+0000 7fb9dd400640 1 --2- 172.21.15.42:0/1595490845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 0x55b57b02a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:41:39.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.469+0000 7fb9dde00640 1 --2- 172.21.15.42:0/1595490845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 0x55b57b01e4d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0036040 tx=0x7fb9d00097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a4e095e2992654 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:39.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.469+0000 7fb9ca600640 1 -- 172.21.15.42:0/1595490845 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0062070 con 0x55b57b009ca0 2024-04-20T10:41:39.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.469+0000 7fb9ca600640 1 -- 172.21.15.42:0/1595490845 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d003fe50 con 0x55b57b009ca0 2024-04-20T10:41:39.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.469+0000 7fb9ca600640 1 -- 172.21.15.42:0/1595490845 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0043070 con 0x55b57b009ca0 2024-04-20T10:41:39.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.470+0000 7fb9e77af740 1 -- 172.21.15.42:0/1595490845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 msgr2=0x55b57b01e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:39.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.470+0000 7fb9e77af740 1 --2- 172.21.15.42:0/1595490845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 0x55b57b01e4d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0036040 tx=0x7fb9d00097a0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.470+0000 7fb9e77af740 1 -- 172.21.15.42:0/1595490845 shutdown_connections 2024-04-20T10:41:39.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.470+0000 7fb9e77af740 1 --2- 172.21.15.42:0/1595490845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b012880 0x55b57b02a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.470+0000 7fb9e77af740 1 --2- 172.21.15.42:0/1595490845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b009ca0 0x55b57b01e4d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.470+0000 7fb9e77af740 1 --2- 172.21.15.42:0/1595490845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b011ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.470+0000 7fb9e77af740 1 -- 172.21.15.42:0/1595490845 >> 172.21.15.42:0/1595490845 conn(0x55b57b003ab0 msgr2=0x55b57b003e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:39.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.470+0000 7fb9e77af740 1 -- 172.21.15.42:0/1595490845 shutdown_connections 2024-04-20T10:41:39.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.471+0000 7fb9e77af740 1 -- 172.21.15.42:0/1595490845 wait complete. 2024-04-20T10:41:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.471+0000 7fb9e77af740 1 -- start start 2024-04-20T10:41:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.471+0000 7fb9e77af740 1 Processor -- start 2024-04-20T10:41:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.472+0000 7fb9e77af740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b0159a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.472+0000 7fb9e77af740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 0x55b57b015ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.472+0000 7fb9e77af740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b012880 0x55b57b086c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.472+0000 7fb9e77af740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01b2ed0 con 0x55b57b012880 2024-04-20T10:41:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.472+0000 7fb9e77af740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01b4340 con 0x55b57b009ca0 2024-04-20T10:41:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.472+0000 7fb9e77af740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e01b35a0 con 0x55b57b009380 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.472+0000 7fb9e77af740 10 client.0 _ll_register_callbacks cb 0x7fb9cc009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.472+0000 7fb9dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 0x55b57b015ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.472+0000 7fb9dde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 0x55b57b015ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:43998/0 (socket says 172.21.15.42:43998) 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.472+0000 7fb9dde00640 1 -- 172.21.15.42:0/2895486918 learned_addr learned my addr 172.21.15.42:0/2895486918 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.472+0000 7fb9dd400640 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b0159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.472+0000 7fb9dd400640 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b012880 0x55b57b086c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.473+0000 7fb9dde00640 1 -- 172.21.15.42:0/2895486918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 msgr2=0x55b57b0159a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.473+0000 7fb9dde00640 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b0159a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.473+0000 7fb9dde00640 1 -- 172.21.15.42:0/2895486918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b012880 msgr2=0x55b57b086c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.473+0000 7fb9dde00640 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b012880 0x55b57b086c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.473+0000 7fb9dde00640 1 -- 172.21.15.42:0/2895486918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b57b016780 con 0x55b57b009ca0 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.473+0000 7fb9dde00640 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 0x55b57b015ee0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005ab0 tx=0x7fb9d0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6155ab54f9717002 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.473+0000 7fb9bf400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:41:39.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.473+0000 7fb9bf400640 1 -- 172.21.15.42:0/2895486918 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0062030 con 0x55b57b009ca0 2024-04-20T10:41:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.473+0000 7fb9bf400640 1 -- 172.21.15.42:0/2895486918 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fb9d003f440 con 0x55b57b009ca0 2024-04-20T10:41:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.473+0000 7fb9e77af740 1 -- 172.21.15.42:0/2895486918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9d003fe50 con 0x55b57b009ca0 2024-04-20T10:41:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.474+0000 7fb9bf400640 1 -- 172.21.15.42:0/2895486918 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb9d0043040 con 0x55b57b009ca0 2024-04-20T10:41:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.474+0000 7fb9e77af740 10 client.26264 fetch_fsmap learned FSMap version 19 2024-04-20T10:41:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.474+0000 7fb9e77af740 1 -- 172.21.15.42:0/2895486918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b57b088290 con 0x55b57b009ca0 2024-04-20T10:41:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.474+0000 7fb9bf400640 1 -- 172.21.15.42:0/2895486918 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d003fe50 con 0x55b57b009ca0 2024-04-20T10:41:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.474+0000 7fb9bf400640 1 -- 172.21.15.42:0/2895486918 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fb9d006f070 con 0x55b57b009ca0 2024-04-20T10:41:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.474+0000 7fb9e77af740 10 client.26264 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:41:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.474+0000 7fb9e77af740 10 client.26264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:41:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.474+0000 7fb9e77af740 20 client.26264 populate_metadata read hostname 'smithi042' 2024-04-20T10:41:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.474+0000 7fb9e77af740 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b57b088950 0x55b57b08ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:41:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.474+0000 7fb9e77af740 4 client.26264 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:41:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.474+0000 7fb9e77af740 1 -- 172.21.15.42:0/2895486918 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b57af769e0 con 0x55b57b088950 2024-04-20T10:41:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.475+0000 7fb9dd400640 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b57b088950 0x55b57b08ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:41:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.475+0000 7fb9bf400640 10 client.26264 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.475+0000 7fb9dd400640 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b57b088950 0x55b57b08ad70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a43f9a79067147a5 server_cookie=df6b96cea348fa80 in_seq=0 out_seq=0 2024-04-20T10:41:39.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.477+0000 7fb9bf400640 1 -- 172.21.15.42:0/2895486918 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55b57af769e0 con 0x55b57b088950 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.477+0000 7fb9bf400640 10 client.26264 handle_command_reply: tid=0 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.477+0000 7fb9e77af740 1 client.26264 shutdown 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.478+0000 7fb9e77af740 1 -- 172.21.15.42:0/2895486918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 msgr2=0x55b57b015ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.478+0000 7fb9e77af740 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 0x55b57b015ee0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005ab0 tx=0x7fb9d0009260 comp rx=0 tx=0).stop 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.478+0000 7fb9e77af740 1 -- 172.21.15.42:0/2895486918 shutdown_connections 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.478+0000 7fb9e77af740 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b57b088950 0x55b57b08ad70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.478+0000 7fb9e77af740 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b57b012880 0x55b57b086c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.478+0000 7fb9e77af740 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b57b009ca0 0x55b57b015ee0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.478+0000 7fb9e77af740 1 --2- 172.21.15.42:0/2895486918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b57b009380 0x55b57b0159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.478+0000 7fb9e77af740 1 -- 172.21.15.42:0/2895486918 >> 172.21.15.42:0/2895486918 conn(0x55b57b003ab0 msgr2=0x55b57b00f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.478+0000 7fb9bf400640 0 client.26264 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.478+0000 7fb9e77af740 1 -- 172.21.15.42:0/2895486918 shutdown_connections 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.478+0000 7fb9e77af740 1 -- 172.21.15.42:0/2895486918 wait complete. 2024-04-20T10:41:39.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.478+0000 7fb9e77af740 20 client.26264 trim_cache size 0 max 16384 2024-04-20T10:41:39.480 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:41:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (14070 inodes in the stack)", 2024-04-20T10:41:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:41:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "fcb92670-a42c-431a-8e9d-6ccb4a432966": { 2024-04-20T10:41:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:41:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "fcb92670-a42c-431a-8e9d-6ccb4a432966", 2024-04-20T10:41:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:41:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:41:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:41:39.481 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:41:39.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.479+0000 7fb9df200640 1 -- 172.21.15.42:0/1130075796 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb9c40a3da0 msgr2=0x7fb9c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:39.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.479+0000 7fb9df200640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb9c40a3da0 0x7fb9c40a6260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80ce440 tx=0x7fb9d800ef50 comp rx=0 tx=0).stop 2024-04-20T10:41:39.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.479+0000 7fb9df200640 1 -- 172.21.15.42:0/1130075796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e01061d0 msgr2=0x7fb9e01a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:41:39.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.479+0000 7fb9df200640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e01061d0 0x7fb9e01a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800ec50 tx=0x7fb9d800c6a0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.479+0000 7fb9df200640 1 -- 172.21.15.42:0/1130075796 shutdown_connections 2024-04-20T10:41:39.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.479+0000 7fb9df200640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb9c40a3da0 0x7fb9c40a6260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.479+0000 7fb9df200640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9e01061d0 0x7fb9e01a3600 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.479+0000 7fb9df200640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e01016f0 0x7fb9e019f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.479+0000 7fb9df200640 1 --2- 172.21.15.42:0/1130075796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e0100dd0 0x7fb9e019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:41:39.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.479+0000 7fb9df200640 1 -- 172.21.15.42:0/1130075796 >> 172.21.15.42:0/1130075796 conn(0x7fb9e00fc7e0 msgr2=0x7fb9e00fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:41:39.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.480+0000 7fb9df200640 1 -- 172.21.15.42:0/1130075796 shutdown_connections 2024-04-20T10:41:39.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:41:39.480+0000 7fb9df200640 1 -- 172.21.15.42:0/1130075796 wait complete. 2024-04-20T10:41:39.498 INFO:tasks.cephfs.filesystem:scrub status for tag:fcb92670-a42c-431a-8e9d-6ccb4a432966 - {'path': '/', 'tag': 'fcb92670-a42c-431a-8e9d-6ccb4a432966', 'options': 'recursive,force'} 2024-04-20T10:41:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:39 smithi046 ceph-mon[21574]: pgmap v234: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 2.23k op/s 2024-04-20T10:41:40.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:39 smithi042 ceph-mon[17078]: pgmap v234: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 2.23k op/s 2024-04-20T10:41:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:39 smithi162 ceph-mon[21854]: pgmap v234: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 2.23k op/s 2024-04-20T10:41:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:41 smithi046 ceph-mon[21574]: pgmap v235: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 1.71k op/s 2024-04-20T10:41:42.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:41 smithi042 ceph-mon[17078]: pgmap v235: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 1.71k op/s 2024-04-20T10:41:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:41 smithi162 ceph-mon[21854]: pgmap v235: 97 pgs: 97 active+clean; 1.6 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 1.71k op/s 2024-04-20T10:41:44.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:43 smithi046 ceph-mon[21574]: pgmap v236: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.0 MiB/s wr, 2.79k op/s 2024-04-20T10:41:44.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:43 smithi042 ceph-mon[17078]: pgmap v236: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.0 MiB/s wr, 2.79k op/s 2024-04-20T10:41:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:43 smithi162 ceph-mon[21854]: pgmap v236: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.0 MiB/s wr, 2.79k op/s 2024-04-20T10:41:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:45 smithi046 ceph-mon[21574]: pgmap v237: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 1.76k op/s 2024-04-20T10:41:46.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:45 smithi042 ceph-mon[17078]: pgmap v237: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 1.76k op/s 2024-04-20T10:41:46.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:45 smithi162 ceph-mon[21854]: pgmap v237: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 1.76k op/s 2024-04-20T10:41:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:47 smithi046 ceph-mon[21574]: pgmap v238: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 2.17k op/s 2024-04-20T10:41:48.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:47 smithi042 ceph-mon[17078]: pgmap v238: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 2.17k op/s 2024-04-20T10:41:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:47 smithi162 ceph-mon[21854]: pgmap v238: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 2.17k op/s 2024-04-20T10:41:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:49 smithi046 ceph-mon[21574]: pgmap v239: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 2.69k op/s 2024-04-20T10:41:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:49 smithi042 ceph-mon[17078]: pgmap v239: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 2.69k op/s 2024-04-20T10:41:50.112 INFO:tasks.workunit.client.0.smithi042.stderr:+ cd linux-6.5.11 2024-04-20T10:41:50.112 INFO:tasks.workunit.client.0.smithi042.stderr:+ make defconfig 2024-04-20T10:41:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:49 smithi162 ceph-mon[21854]: pgmap v239: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 2.69k op/s 2024-04-20T10:41:51.362 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/basic/fixdep 2024-04-20T10:41:51.718 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/kconfig/conf.o 2024-04-20T10:41:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:51 smithi046 ceph-mon[21574]: pgmap v240: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 2.26k op/s 2024-04-20T10:41:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:51 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:41:52.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:51 smithi042 ceph-mon[17078]: pgmap v240: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 2.26k op/s 2024-04-20T10:41:52.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:51 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:41:52.149 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/kconfig/confdata.o 2024-04-20T10:41:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:51 smithi162 ceph-mon[21854]: pgmap v240: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 2.26k op/s 2024-04-20T10:41:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:51 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:41:52.503 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/kconfig/expr.o 2024-04-20T10:41:52.792 INFO:tasks.workunit.client.0.smithi042.stdout: LEX scripts/kconfig/lexer.lex.c 2024-04-20T10:41:52.822 INFO:tasks.workunit.client.0.smithi042.stdout: YACC scripts/kconfig/parser.tab.[ch] 2024-04-20T10:41:52.923 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/kconfig/lexer.lex.o 2024-04-20T10:41:54.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:53 smithi046 ceph-mon[21574]: pgmap v241: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 2.86k op/s 2024-04-20T10:41:54.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:53 smithi042 ceph-mon[17078]: pgmap v241: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 2.86k op/s 2024-04-20T10:41:54.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:53 smithi162 ceph-mon[21854]: pgmap v241: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 2.86k op/s 2024-04-20T10:41:54.825 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/kconfig/menu.o 2024-04-20T10:41:55.050 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/kconfig/parser.tab.o 2024-04-20T10:41:55.379 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/kconfig/preprocess.o 2024-04-20T10:41:55.544 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/kconfig/symbol.o 2024-04-20T10:41:55.928 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/kconfig/util.o 2024-04-20T10:41:56.030 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTLD scripts/kconfig/conf 2024-04-20T10:41:56.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:55 smithi046 ceph-mon[21574]: pgmap v242: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 1.84k op/s 2024-04-20T10:41:56.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:55 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:41:56.082 INFO:tasks.workunit.client.0.smithi042.stdout:*** Default configuration is based on 'x86_64_defconfig' 2024-04-20T10:41:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:55 smithi042 ceph-mon[17078]: pgmap v242: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 1.84k op/s 2024-04-20T10:41:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:55 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:41:56.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:55 smithi162 ceph-mon[21854]: pgmap v242: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 1.84k op/s 2024-04-20T10:41:56.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:55 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:41:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:57 smithi046 ceph-mon[21574]: pgmap v243: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 1.96k op/s 2024-04-20T10:41:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:57 smithi042 ceph-mon[17078]: pgmap v243: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 1.96k op/s 2024-04-20T10:41:58.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:57 smithi162 ceph-mon[21854]: pgmap v243: 97 pgs: 97 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 1.96k op/s 2024-04-20T10:42:00.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:41:59 smithi046 ceph-mon[21574]: pgmap v244: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 1.78k op/s 2024-04-20T10:42:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:41:59 smithi042 ceph-mon[17078]: pgmap v244: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 1.78k op/s 2024-04-20T10:42:00.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:41:59 smithi162 ceph-mon[21854]: pgmap v244: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 1.78k op/s 2024-04-20T10:42:02.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:01 smithi046 ceph-mon[21574]: pgmap v245: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 1.00k op/s 2024-04-20T10:42:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:01 smithi042 ceph-mon[17078]: pgmap v245: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 1.00k op/s 2024-04-20T10:42:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:01 smithi162 ceph-mon[21854]: pgmap v245: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 1.00k op/s 2024-04-20T10:42:04.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:03 smithi042 ceph-mon[17078]: pgmap v246: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.7 MiB/s wr, 1.50k op/s 2024-04-20T10:42:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:03 smithi162 ceph-mon[21854]: pgmap v246: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.7 MiB/s wr, 1.50k op/s 2024-04-20T10:42:04.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:03 smithi046 ceph-mon[21574]: pgmap v246: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.7 MiB/s wr, 1.50k op/s 2024-04-20T10:42:06.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:05 smithi042 ceph-mon[17078]: pgmap v247: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 4.5 MiB/s wr, 971 op/s 2024-04-20T10:42:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:05 smithi162 ceph-mon[21854]: pgmap v247: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 4.5 MiB/s wr, 971 op/s 2024-04-20T10:42:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:05 smithi046 ceph-mon[21574]: pgmap v247: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 4.5 MiB/s wr, 971 op/s 2024-04-20T10:42:07.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:06 smithi042 ceph-mon[17078]: osdmap e91: 12 total, 12 up, 12 in 2024-04-20T10:42:07.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:06 smithi162 ceph-mon[21854]: osdmap e91: 12 total, 12 up, 12 in 2024-04-20T10:42:07.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:06 smithi046 ceph-mon[21574]: osdmap e91: 12 total, 12 up, 12 in 2024-04-20T10:42:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:07 smithi042 ceph-mon[17078]: pgmap v249: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 1.14k op/s 2024-04-20T10:42:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:07 smithi162 ceph-mon[21854]: pgmap v249: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 1.14k op/s 2024-04-20T10:42:08.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:07 smithi046 ceph-mon[21574]: pgmap v249: 97 pgs: 97 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 1.14k op/s 2024-04-20T10:42:09.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:42:09.500 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:42:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.699+0000 7fe477200640 1 -- 172.21.15.42:0/3626026683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe478101580 msgr2=0x7fe47810e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.699+0000 7fe477200640 1 --2- 172.21.15.42:0/3626026683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe478101580 0x7fe47810e1b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe464009900 tx=0x7fe46402bc50 comp rx=0 tx=0).stop 2024-04-20T10:42:09.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.699+0000 7fe477200640 1 -- 172.21.15.42:0/3626026683 shutdown_connections 2024-04-20T10:42:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.699+0000 7fe477200640 1 --2- 172.21.15.42:0/3626026683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe47810e780 0x7fe478110b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.699+0000 7fe477200640 1 --2- 172.21.15.42:0/3626026683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe478101580 0x7fe47810e1b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.699+0000 7fe477200640 1 --2- 172.21.15.42:0/3626026683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe478100bd0 0x7fe478100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.699+0000 7fe477200640 1 -- 172.21.15.42:0/3626026683 >> 172.21.15.42:0/3626026683 conn(0x7fe4780fc7e0 msgr2=0x7fe4780fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.699+0000 7fe477200640 1 -- 172.21.15.42:0/3626026683 shutdown_connections 2024-04-20T10:42:09.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.699+0000 7fe477200640 1 -- 172.21.15.42:0/3626026683 wait complete. 2024-04-20T10:42:09.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.700+0000 7fe477200640 1 Processor -- start 2024-04-20T10:42:09.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.700+0000 7fe477200640 1 -- start start 2024-04-20T10:42:09.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.701+0000 7fe477200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe478100bd0 0x7fe4781a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.701+0000 7fe477200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe478101580 0x7fe4781a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.701+0000 7fe477200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe47810e780 0x7fe47819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.701+0000 7fe477200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe478114460 con 0x7fe478101580 2024-04-20T10:42:09.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.701+0000 7fe477200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe4781142e0 con 0x7fe478100bd0 2024-04-20T10:42:09.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.701+0000 7fe477200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe4781145e0 con 0x7fe47810e780 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.701+0000 7fe475e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe478100bd0 0x7fe4781a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.701+0000 7fe475e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe478100bd0 0x7fe4781a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59118/0 (socket says 172.21.15.42:59118) 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.701+0000 7fe475e00640 1 -- 172.21.15.42:0/1111383135 learned_addr learned my addr 172.21.15.42:0/1111383135 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.701+0000 7fe475400640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe478101580 0x7fe4781a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.701+0000 7fe476800640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe47810e780 0x7fe47819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.701+0000 7fe475e00640 1 -- 172.21.15.42:0/1111383135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe47810e780 msgr2=0x7fe47819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.702+0000 7fe475e00640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe47810e780 0x7fe47819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.702+0000 7fe475e00640 1 -- 172.21.15.42:0/1111383135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe478101580 msgr2=0x7fe4781a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.702+0000 7fe475e00640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe478101580 0x7fe4781a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.702+0000 7fe475e00640 1 -- 172.21.15.42:0/1111383135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47819b000 con 0x7fe478100bd0 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.702+0000 7fe475e00640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe478100bd0 0x7fe4781a0680 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe46800e9f0 tx=0x7fe46800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.702+0000 7fe46ea00640 1 -- 172.21.15.42:0/1111383135 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46800cd80 con 0x7fe478100bd0 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.702+0000 7fe46ea00640 1 -- 172.21.15.42:0/1111383135 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe468010910 con 0x7fe478100bd0 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.702+0000 7fe46ea00640 1 -- 172.21.15.42:0/1111383135 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468018990 con 0x7fe478100bd0 2024-04-20T10:42:09.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.702+0000 7fe477200640 1 -- 172.21.15.42:0/1111383135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe47819b2f0 con 0x7fe478100bd0 2024-04-20T10:42:09.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.702+0000 7fe477200640 1 -- 172.21.15.42:0/1111383135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4781a7600 con 0x7fe478100bd0 2024-04-20T10:42:09.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.706+0000 7fe47fd98740 1 Processor -- start 2024-04-20T10:42:09.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.706+0000 7fe47fd98740 1 -- start start 2024-04-20T10:42:09.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.707+0000 7fe47fd98740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d17380 0x564fb9d17760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.707+0000 7fe47fd98740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17ca0 0x564fb9d20340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.707+0000 7fe47fd98740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d20880 0x564fb9d22c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.707+0000 7fe47fd98740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564fb9c7d150 con 0x564fb9d17380 2024-04-20T10:42:09.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.707+0000 7fe47fd98740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564fb9c23370 con 0x564fb9d17ca0 2024-04-20T10:42:09.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.707+0000 7fe47fd98740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x564fb9c79110 con 0x564fb9d20880 2024-04-20T10:42:09.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.707+0000 7fe476800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d17380 0x564fb9d17760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.707+0000 7fe476800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d17380 0x564fb9d17760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36432/0 (socket says 172.21.15.42:36432) 2024-04-20T10:42:09.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.707+0000 7fe476800640 1 -- 172.21.15.42:0/3636392060 learned_addr learned my addr 172.21.15.42:0/3636392060 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:09.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.707+0000 7fe46ea00640 1 -- 172.21.15.42:0/1111383135 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe468010ac0 con 0x7fe478100bd0 2024-04-20T10:42:09.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.707+0000 7fe475400640 1 --2- 172.21.15.42:0/3636392060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d20880 0x564fb9d22c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.708+0000 7fe476800640 1 -- 172.21.15.42:0/3636392060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d20880 msgr2=0x564fb9d22c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:09.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.708+0000 7fe476800640 1 --2- 172.21.15.42:0/3636392060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d20880 0x564fb9d22c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.708+0000 7fe476800640 1 -- 172.21.15.42:0/3636392060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17ca0 msgr2=0x564fb9d20340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:09.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.708+0000 7fe476800640 1 --2- 172.21.15.42:0/3636392060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17ca0 0x564fb9d20340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.708+0000 7fe476800640 1 -- 172.21.15.42:0/3636392060 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb9a597d0 con 0x564fb9d17380 2024-04-20T10:42:09.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.708+0000 7fe475400640 1 --2- 172.21.15.42:0/3636392060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d20880 0x564fb9d22c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:42:09.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.708+0000 7fe476800640 1 --2- 172.21.15.42:0/3636392060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d17380 0x564fb9d17760 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe470009f30 tx=0x7fe4700118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28884a4d75a3f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:09.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.708+0000 7fe463400640 1 -- 172.21.15.42:0/3636392060 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470004d20 con 0x564fb9d17380 2024-04-20T10:42:09.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.708+0000 7fe463400640 1 -- 172.21.15.42:0/3636392060 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe470004050 con 0x564fb9d17380 2024-04-20T10:42:09.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.708+0000 7fe463400640 1 -- 172.21.15.42:0/3636392060 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470005260 con 0x564fb9d17380 2024-04-20T10:42:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.709+0000 7fe46ea00640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe45c0a3d20 0x7fe45c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.709+0000 7fe47fd98740 1 -- 172.21.15.42:0/3636392060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d17380 msgr2=0x564fb9d17760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.709+0000 7fe47fd98740 1 --2- 172.21.15.42:0/3636392060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d17380 0x564fb9d17760 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe470009f30 tx=0x7fe4700118d0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.709+0000 7fe47fd98740 1 -- 172.21.15.42:0/3636392060 shutdown_connections 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.709+0000 7fe47fd98740 1 --2- 172.21.15.42:0/3636392060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d20880 0x564fb9d22c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.709+0000 7fe47fd98740 1 --2- 172.21.15.42:0/3636392060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17ca0 0x564fb9d20340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.709+0000 7fe47fd98740 1 --2- 172.21.15.42:0/3636392060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d17380 0x564fb9d17760 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.709+0000 7fe47fd98740 1 -- 172.21.15.42:0/3636392060 >> 172.21.15.42:0/3636392060 conn(0x564fb9d11ab0 msgr2=0x564fb9d11e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.709+0000 7fe46ea00640 1 -- 172.21.15.42:0/1111383135 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fe4680d0b50 con 0x7fe478100bd0 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.709+0000 7fe475e00640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe45c0a3d20 0x7fe45c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.709+0000 7fe47fd98740 1 -- 172.21.15.42:0/3636392060 shutdown_connections 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.709+0000 7fe47fd98740 1 -- 172.21.15.42:0/3636392060 wait complete. 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.710+0000 7fe47fd98740 1 -- start start 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.710+0000 7fe475e00640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe45c0a3d20 0x7fe45c0a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x564fb9d16e50 tx=0x7fe468009700 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.710+0000 7fe47fd98740 1 Processor -- start 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.710+0000 7fe47fd98740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d1d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.710+0000 7fe47fd98740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d17ca0 0x564fb9d1de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.710+0000 7fe47fd98740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d20880 0x564fb9d1e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.710+0000 7fe47fd98740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564fb9d23670 con 0x564fb9d17ca0 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.710+0000 7fe47fd98740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564fb9c36740 con 0x564fb9d17380 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.710+0000 7fe47fd98740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x564fb9d237f0 con 0x564fb9d20880 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe47fd98740 10 client.0 _ll_register_callbacks cb 0x564fb9c31b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe476800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d1d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe476800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d1d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59134/0 (socket says 172.21.15.42:59134) 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe476800640 1 -- 172.21.15.42:0/2365108161 learned_addr learned my addr 172.21.15.42:0/2365108161 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe475400640 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d17ca0 0x564fb9d1de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe476800640 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d20880 0x564fb9d1e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe476800640 1 -- 172.21.15.42:0/2365108161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d20880 msgr2=0x564fb9d1e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe476800640 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d20880 0x564fb9d1e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe476800640 1 -- 172.21.15.42:0/2365108161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d17ca0 msgr2=0x564fb9d1de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe476800640 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d17ca0 0x564fb9d1de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe476800640 1 -- 172.21.15.42:0/2365108161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb998f3f0 con 0x564fb9d17380 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe476800640 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d1d910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe470011dd0 tx=0x7fe470009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd560a6a42375848 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.711+0000 7fe457e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.712+0000 7fe457e00640 1 -- 172.21.15.42:0/2365108161 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470028480 con 0x564fb9d17380 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.712+0000 7fe457e00640 1 -- 172.21.15.42:0/2365108161 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fe470028620 con 0x564fb9d17380 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.712+0000 7fe47fd98740 1 -- 172.21.15.42:0/2365108161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe470004050 con 0x564fb9d17380 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.712+0000 7fe457e00640 1 -- 172.21.15.42:0/2365108161 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe470036050 con 0x564fb9d17380 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.712+0000 7fe47fd98740 10 client.26273 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.712+0000 7fe47fd98740 1 -- 172.21.15.42:0/2365108161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x564fb994f0f0 con 0x564fb9d17380 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.712+0000 7fe457e00640 1 -- 172.21.15.42:0/2365108161 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe470004050 con 0x564fb9d17380 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.712+0000 7fe457e00640 1 -- 172.21.15.42:0/2365108161 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fe470025040 con 0x564fb9d17380 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.712+0000 7fe47fd98740 10 client.26273 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.712+0000 7fe47fd98740 10 client.26273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.712+0000 7fe47fd98740 20 client.26273 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.713+0000 7fe47fd98740 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564fb9d30ba0 0x564fb9d32f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.713+0000 7fe47fd98740 4 client.26273 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:42:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.713+0000 7fe47fd98740 1 -- 172.21.15.42:0/2365108161 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fb9c8ab70 con 0x564fb9d30ba0 2024-04-20T10:42:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.713+0000 7fe475400640 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564fb9d30ba0 0x564fb9d32f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.713+0000 7fe475400640 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564fb9d30ba0 0x564fb9d32f90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaedd28a9b7e780e server_cookie=e5e69451f29093a9 in_seq=0 out_seq=0 2024-04-20T10:42:09.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.713+0000 7fe457e00640 10 client.26273 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:09.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.716+0000 7fe457e00640 1 -- 172.21.15.42:0/2365108161 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fb9c8ab70 con 0x564fb9d30ba0 2024-04-20T10:42:09.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.716+0000 7fe457e00640 10 client.26273 handle_command_reply: tid=0 2024-04-20T10:42:09.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.716+0000 7fe47fd98740 1 client.26273 shutdown 2024-04-20T10:42:09.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.716+0000 7fe47fd98740 1 -- 172.21.15.42:0/2365108161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 msgr2=0x564fb9d1d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:09.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.716+0000 7fe47fd98740 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d1d910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe470011dd0 tx=0x7fe470009c00 comp rx=0 tx=0).stop 2024-04-20T10:42:09.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.716+0000 7fe47fd98740 1 -- 172.21.15.42:0/2365108161 shutdown_connections 2024-04-20T10:42:09.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.716+0000 7fe47fd98740 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564fb9d30ba0 0x564fb9d32f90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.717+0000 7fe47fd98740 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d20880 0x564fb9d1e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.717+0000 7fe47fd98740 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d17ca0 0x564fb9d1de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.717+0000 7fe47fd98740 1 --2- 172.21.15.42:0/2365108161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d1d910 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.717+0000 7fe47fd98740 1 -- 172.21.15.42:0/2365108161 >> 172.21.15.42:0/2365108161 conn(0x564fb9d11ab0 msgr2=0x564fb9d13800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.717+0000 7fe457e00640 0 client.26273 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.717+0000 7fe47fd98740 1 -- 172.21.15.42:0/2365108161 shutdown_connections 2024-04-20T10:42:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.717+0000 7fe47fd98740 1 -- 172.21.15.42:0/2365108161 wait complete. 2024-04-20T10:42:09.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.717+0000 7fe47fd98740 20 client.26273 trim_cache size 0 max 16384 2024-04-20T10:42:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.735+0000 7fe47fd98740 1 Processor -- start 2024-04-20T10:42:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.735+0000 7fe47fd98740 1 -- start start 2024-04-20T10:42:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.735+0000 7fe47fd98740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d20040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.735+0000 7fe47fd98740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d17ca0 0x564fb9d18f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.735+0000 7fe47fd98740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d20880 0x564fb9d2c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.735+0000 7fe47fd98740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4781ade40 con 0x564fb9d20880 2024-04-20T10:42:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.735+0000 7fe47fd98740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe4781b06a0 con 0x564fb9d17380 2024-04-20T10:42:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.735+0000 7fe47fd98740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe4781ae490 con 0x564fb9d17ca0 2024-04-20T10:42:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.736+0000 7fe476800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d20880 0x564fb9d2c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.736+0000 7fe475400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d17ca0 0x564fb9d18f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.736+0000 7fe476800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d20880 0x564fb9d2c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36456/0 (socket says 172.21.15.42:36456) 2024-04-20T10:42:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.736+0000 7fe476800640 1 -- 172.21.15.42:0/3761728966 learned_addr learned my addr 172.21.15.42:0/3761728966 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.736+0000 7fe476800640 1 --2- 172.21.15.42:0/3761728966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d20040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.736+0000 7fe476800640 1 -- 172.21.15.42:0/3761728966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d17ca0 msgr2=0x564fb9d18f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.736+0000 7fe476800640 1 --2- 172.21.15.42:0/3761728966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d17ca0 0x564fb9d18f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.736+0000 7fe476800640 1 -- 172.21.15.42:0/3761728966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 msgr2=0x564fb9d20040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.736+0000 7fe476800640 1 --2- 172.21.15.42:0/3761728966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d20040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.736+0000 7fe476800640 1 -- 172.21.15.42:0/3761728966 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb9d22fe0 con 0x564fb9d20880 2024-04-20T10:42:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.737+0000 7fe476800640 1 --2- 172.21.15.42:0/3761728966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d20880 0x564fb9d2c530 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x564fb9d03260 tx=0x7fe470028e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e633eacc9c2c42e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:09.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.737+0000 7fe462a00640 1 -- 172.21.15.42:0/3761728966 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470004320 con 0x564fb9d20880 2024-04-20T10:42:09.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.737+0000 7fe462a00640 1 -- 172.21.15.42:0/3761728966 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe47002d070 con 0x564fb9d20880 2024-04-20T10:42:09.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.737+0000 7fe462a00640 1 -- 172.21.15.42:0/3761728966 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470004110 con 0x564fb9d20880 2024-04-20T10:42:09.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.737+0000 7fe47fd98740 1 -- 172.21.15.42:0/3761728966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d20880 msgr2=0x564fb9d2c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:09.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.737+0000 7fe47fd98740 1 --2- 172.21.15.42:0/3761728966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d20880 0x564fb9d2c530 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x564fb9d03260 tx=0x7fe470028e20 comp rx=0 tx=0).stop 2024-04-20T10:42:09.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.738+0000 7fe47fd98740 1 -- 172.21.15.42:0/3761728966 shutdown_connections 2024-04-20T10:42:09.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.738+0000 7fe47fd98740 1 --2- 172.21.15.42:0/3761728966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d20880 0x564fb9d2c530 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.738+0000 7fe47fd98740 1 --2- 172.21.15.42:0/3761728966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d17ca0 0x564fb9d18f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.738+0000 7fe47fd98740 1 --2- 172.21.15.42:0/3761728966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d20040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.738+0000 7fe47fd98740 1 -- 172.21.15.42:0/3761728966 >> 172.21.15.42:0/3761728966 conn(0x564fb9d11ab0 msgr2=0x564fb9d1d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:09.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.738+0000 7fe47fd98740 1 -- 172.21.15.42:0/3761728966 shutdown_connections 2024-04-20T10:42:09.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.738+0000 7fe47fd98740 1 -- 172.21.15.42:0/3761728966 wait complete. 2024-04-20T10:42:09.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.738+0000 7fe47fd98740 1 -- start start 2024-04-20T10:42:09.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.738+0000 7fe47fd98740 1 Processor -- start 2024-04-20T10:42:09.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe47fd98740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d28a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe47fd98740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d17ca0 0x564fb9d28f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe47fd98740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d20880 0x564fb9d1a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe47fd98740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4781aeb60 con 0x564fb9d20880 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe47fd98740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe4781affd0 con 0x564fb9d17380 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe47fd98740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe4781af230 con 0x564fb9d17ca0 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe47fd98740 10 client.0 _ll_register_callbacks cb 0x7fe470009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe476800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d28a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe476800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d28a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59152/0 (socket says 172.21.15.42:59152) 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe476800640 1 -- 172.21.15.42:0/1833110407 learned_addr learned my addr 172.21.15.42:0/1833110407 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe476800640 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d20880 0x564fb9d1a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe475400640 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d17ca0 0x564fb9d28f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe476800640 1 -- 172.21.15.42:0/1833110407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d17ca0 msgr2=0x564fb9d28f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe476800640 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d17ca0 0x564fb9d28f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe476800640 1 -- 172.21.15.42:0/1833110407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d20880 msgr2=0x564fb9d1a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:09.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe476800640 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d20880 0x564fb9d1a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.739+0000 7fe476800640 1 -- 172.21.15.42:0/1833110407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb9d334f0 con 0x564fb9d17380 2024-04-20T10:42:09.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.740+0000 7fe476800640 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d28a40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe470009560 tx=0x7fe470009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70e52af9673dd4ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:09.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.740+0000 7fe457e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:42:09.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.740+0000 7fe457e00640 1 -- 172.21.15.42:0/1833110407 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470004320 con 0x564fb9d17380 2024-04-20T10:42:09.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.740+0000 7fe47fd98740 1 -- 172.21.15.42:0/1833110407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe47002d070 con 0x564fb9d17380 2024-04-20T10:42:09.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.740+0000 7fe457e00640 1 -- 172.21.15.42:0/1833110407 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fe47000f480 con 0x564fb9d17380 2024-04-20T10:42:09.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.740+0000 7fe457e00640 1 -- 172.21.15.42:0/1833110407 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe47000fb00 con 0x564fb9d17380 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.740+0000 7fe47fd98740 10 client.26279 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.740+0000 7fe47fd98740 1 -- 172.21.15.42:0/1833110407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x564fb9d96980 con 0x564fb9d17380 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.740+0000 7fe457e00640 1 -- 172.21.15.42:0/1833110407 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe47002d070 con 0x564fb9d17380 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.741+0000 7fe457e00640 1 -- 172.21.15.42:0/1833110407 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fe470028a80 con 0x564fb9d17380 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.741+0000 7fe47fd98740 10 client.26279 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.741+0000 7fe47fd98740 10 client.26279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.741+0000 7fe47fd98740 20 client.26279 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.741+0000 7fe47fd98740 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564fb9d97050 0x564fb9d99470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.741+0000 7fe47fd98740 4 client.26279 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.741+0000 7fe47fd98740 1 -- 172.21.15.42:0/1833110407 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564fb9c849e0 con 0x564fb9d97050 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.741+0000 7fe475400640 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564fb9d97050 0x564fb9d99470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.742+0000 7fe475400640 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564fb9d97050 0x564fb9d99470 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79de05add717738c server_cookie=1461dd8978a7bcce in_seq=0 out_seq=0 2024-04-20T10:42:09.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.742+0000 7fe457e00640 10 client.26279 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:09.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe457e00640 1 -- 172.21.15.42:0/1833110407 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564fb9c849e0 con 0x564fb9d97050 2024-04-20T10:42:09.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe457e00640 10 client.26279 handle_command_reply: tid=0 2024-04-20T10:42:09.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe47fd98740 1 client.26279 shutdown 2024-04-20T10:42:09.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe47fd98740 1 -- 172.21.15.42:0/1833110407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 msgr2=0x564fb9d28a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:09.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe47fd98740 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d28a40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe470009560 tx=0x7fe470009230 comp rx=0 tx=0).stop 2024-04-20T10:42:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe47fd98740 1 -- 172.21.15.42:0/1833110407 shutdown_connections 2024-04-20T10:42:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe47fd98740 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x564fb9d97050 0x564fb9d99470 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe47fd98740 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564fb9d20880 0x564fb9d1a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe457e00640 0 client.26279 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe47fd98740 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564fb9d17ca0 0x564fb9d28f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe47fd98740 1 --2- 172.21.15.42:0/1833110407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564fb9d17380 0x564fb9d28a40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe47fd98740 1 -- 172.21.15.42:0/1833110407 >> 172.21.15.42:0/1833110407 conn(0x564fb9d11ab0 msgr2=0x564fb9d1e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe47fd98740 1 -- 172.21.15.42:0/1833110407 shutdown_connections 2024-04-20T10:42:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe47fd98740 1 -- 172.21.15.42:0/1833110407 wait complete. 2024-04-20T10:42:09.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.745+0000 7fe47fd98740 20 client.26279 trim_cache size 0 max 16384 2024-04-20T10:42:09.748 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:42:09.748 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:42:09.749 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:42:09.749 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:42:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.746+0000 7fe477200640 1 -- 172.21.15.42:0/1111383135 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe45c0a3d20 msgr2=0x7fe45c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.746+0000 7fe477200640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe45c0a3d20 0x7fe45c0a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x564fb9d16e50 tx=0x7fe468009700 comp rx=0 tx=0).stop 2024-04-20T10:42:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.746+0000 7fe477200640 1 -- 172.21.15.42:0/1111383135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe478100bd0 msgr2=0x7fe4781a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.746+0000 7fe477200640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe478100bd0 0x7fe4781a0680 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe46800e9f0 tx=0x7fe46800eeb0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.746+0000 7fe477200640 1 -- 172.21.15.42:0/1111383135 shutdown_connections 2024-04-20T10:42:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.746+0000 7fe477200640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe45c0a3d20 0x7fe45c0a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.746+0000 7fe477200640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe47810e780 0x7fe47819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.746+0000 7fe477200640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe478101580 0x7fe4781a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.746+0000 7fe477200640 1 --2- 172.21.15.42:0/1111383135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe478100bd0 0x7fe4781a0680 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.746+0000 7fe477200640 1 -- 172.21.15.42:0/1111383135 >> 172.21.15.42:0/1111383135 conn(0x7fe4780fc7e0 msgr2=0x7fe47810fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.746+0000 7fe477200640 1 -- 172.21.15.42:0/1111383135 shutdown_connections 2024-04-20T10:42:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.746+0000 7fe477200640 1 -- 172.21.15.42:0/1111383135 wait complete. 2024-04-20T10:42:09.762 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:42:09.762 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:42:09.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.968+0000 7f7f1a000640 1 -- 172.21.15.42:0/53468045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f14102190 msgr2=0x7f7f14102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:09.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.968+0000 7f7f1a000640 1 --2- 172.21.15.42:0/53468045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f14102190 0x7f7f14102570 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009900 tx=0x7f7f0402bc40 comp rx=0 tx=0).stop 2024-04-20T10:42:09.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.969+0000 7f7f1a000640 1 -- 172.21.15.42:0/53468045 shutdown_connections 2024-04-20T10:42:09.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.969+0000 7f7f1a000640 1 --2- 172.21.15.42:0/53468045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f14107260 0x7f7f1410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.969+0000 7f7f1a000640 1 --2- 172.21.15.42:0/53468045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f14102b40 0x7f7f14106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.969+0000 7f7f1a000640 1 --2- 172.21.15.42:0/53468045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f14102190 0x7f7f14102570 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.969+0000 7f7f1a000640 1 -- 172.21.15.42:0/53468045 >> 172.21.15.42:0/53468045 conn(0x7f7f14079360 msgr2=0x7f7f14079770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:09.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.969+0000 7f7f1a000640 1 -- 172.21.15.42:0/53468045 shutdown_connections 2024-04-20T10:42:09.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.969+0000 7f7f1a000640 1 -- 172.21.15.42:0/53468045 wait complete. 2024-04-20T10:42:09.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.970+0000 7f7f1a000640 1 Processor -- start 2024-04-20T10:42:09.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.970+0000 7f7f1a000640 1 -- start start 2024-04-20T10:42:09.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.970+0000 7f7f1a000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f14102190 0x7f7f1419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.970+0000 7f7f1a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f14107260 0x7f7f1419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.970+0000 7f7f1a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f14198670 0x7f7f14198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.970+0000 7f7f1a000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7f14112370 con 0x7f7f14198670 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.970+0000 7f7f1a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f141121f0 con 0x7f7f14107260 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.970+0000 7f7f1a000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7f141124f0 con 0x7f7f14102190 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.971+0000 7f7f12a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f14107260 0x7f7f1419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.971+0000 7f7f12a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f14107260 0x7f7f1419e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59166/0 (socket says 172.21.15.42:59166) 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.971+0000 7f7f12a00640 1 -- 172.21.15.42:0/1221737038 learned_addr learned my addr 172.21.15.42:0/1221737038 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.971+0000 7f7f13e00640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f14198670 0x7f7f14198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.971+0000 7f7f13400640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f14102190 0x7f7f1419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.971+0000 7f7f12a00640 1 -- 172.21.15.42:0/1221737038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f14102190 msgr2=0x7f7f1419e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.971+0000 7f7f12a00640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f14102190 0x7f7f1419e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.971+0000 7f7f12a00640 1 -- 172.21.15.42:0/1221737038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f14198670 msgr2=0x7f7f14198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.971+0000 7f7f12a00640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f14198670 0x7f7f14198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:09.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.971+0000 7f7f12a00640 1 -- 172.21.15.42:0/1221737038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f14199190 con 0x7f7f14107260 2024-04-20T10:42:09.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.971+0000 7f7f12a00640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f14107260 0x7f7f1419e9b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000e9c0 tx=0x7f7f0000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:09.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.972+0000 7f7f0be00640 1 -- 172.21.15.42:0/1221737038 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000cd80 con 0x7f7f14107260 2024-04-20T10:42:09.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.972+0000 7f7f0be00640 1 -- 172.21.15.42:0/1221737038 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f00005500 con 0x7f7f14107260 2024-04-20T10:42:09.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.972+0000 7f7f1a000640 1 -- 172.21.15.42:0/1221737038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f14199480 con 0x7f7f14107260 2024-04-20T10:42:09.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.972+0000 7f7f0be00640 1 -- 172.21.15.42:0/1221737038 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00010690 con 0x7f7f14107260 2024-04-20T10:42:09.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.972+0000 7f7f1a000640 1 -- 172.21.15.42:0/1221737038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f141a56d0 con 0x7f7f14107260 2024-04-20T10:42:09.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.973+0000 7f7f1a000640 1 -- 172.21.15.42:0/1221737038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f140691d0 con 0x7f7f14107260 2024-04-20T10:42:09.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.976+0000 7f7f0be00640 1 -- 172.21.15.42:0/1221737038 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7f000046c0 con 0x7f7f14107260 2024-04-20T10:42:09.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.977+0000 7f7f0be00640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7ef80a3da0 0x7f7ef80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:09.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.977+0000 7f7f0be00640 1 -- 172.21.15.42:0/1221737038 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f7f000d0b80 con 0x7f7f14107260 2024-04-20T10:42:09.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.978+0000 7f7f13400640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7ef80a3da0 0x7f7ef80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:09.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.978+0000 7f7f13400640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7ef80a3da0 0x7f7ef80a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f0402efd0 tx=0x7f7f0402cf90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:09.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:09.983+0000 7f7f0be00640 1 -- 172.21.15.42:0/1221737038 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7f0009d190 con 0x7f7f14107260 2024-04-20T10:42:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:09 smithi162 ceph-mon[21854]: pgmap v250: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.8 MiB/s wr, 1.09k op/s 2024-04-20T10:42:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:09 smithi162 ceph-mon[21854]: osdmap e92: 12 total, 12 up, 12 in 2024-04-20T10:42:10.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.236+0000 7f7f1a000640 1 -- 172.21.15.42:0/1221737038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f14102b40 con 0x7f7f14107260 2024-04-20T10:42:10.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.237+0000 7f7f0be00640 1 -- 172.21.15.42:0/1221737038 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f7f000bd0d0 con 0x7f7f14107260 2024-04-20T10:42:10.238 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:42:10.238 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:42:10.238 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:42:10.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.240+0000 7f7f1a000640 1 -- 172.21.15.42:0/1221737038 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7ef80a3da0 msgr2=0x7f7ef80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:10.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.240+0000 7f7f1a000640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7ef80a3da0 0x7f7ef80a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f0402efd0 tx=0x7f7f0402cf90 comp rx=0 tx=0).stop 2024-04-20T10:42:10.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.240+0000 7f7f1a000640 1 -- 172.21.15.42:0/1221737038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f14107260 msgr2=0x7f7f1419e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:10.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.240+0000 7f7f1a000640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f14107260 0x7f7f1419e9b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000e9c0 tx=0x7f7f0000ee80 comp rx=0 tx=0).stop 2024-04-20T10:42:10.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.240+0000 7f7f1a000640 1 -- 172.21.15.42:0/1221737038 shutdown_connections 2024-04-20T10:42:10.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.240+0000 7f7f1a000640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7ef80a3da0 0x7f7ef80a6260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.240+0000 7f7f1a000640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f14198670 0x7f7f14198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.240+0000 7f7f1a000640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f14107260 0x7f7f1419e9b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.240+0000 7f7f1a000640 1 --2- 172.21.15.42:0/1221737038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f14102190 0x7f7f1419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.240+0000 7f7f1a000640 1 -- 172.21.15.42:0/1221737038 >> 172.21.15.42:0/1221737038 conn(0x7f7f14079360 msgr2=0x7f7f14100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:10.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.240+0000 7f7f1a000640 1 -- 172.21.15.42:0/1221737038 shutdown_connections 2024-04-20T10:42:10.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.240+0000 7f7f1a000640 1 -- 172.21.15.42:0/1221737038 wait complete. 2024-04-20T10:42:10.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:09 smithi042 ceph-mon[17078]: pgmap v250: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.8 MiB/s wr, 1.09k op/s 2024-04-20T10:42:10.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:09 smithi042 ceph-mon[17078]: osdmap e92: 12 total, 12 up, 12 in 2024-04-20T10:42:10.260 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:42:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:09 smithi046 ceph-mon[21574]: pgmap v250: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.8 MiB/s wr, 1.09k op/s 2024-04-20T10:42:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:09 smithi046 ceph-mon[21574]: osdmap e92: 12 total, 12 up, 12 in 2024-04-20T10:42:10.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.472+0000 7f97eaa00640 1 -- 172.21.15.42:0/3668018721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97e4102080 msgr2=0x7f97e4102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:10.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.472+0000 7f97eaa00640 1 --2- 172.21.15.42:0/3668018721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97e4102080 0x7f97e4102460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009900 tx=0x7f97d802bc30 comp rx=0 tx=0).stop 2024-04-20T10:42:10.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.473+0000 7f97eaa00640 1 -- 172.21.15.42:0/3668018721 shutdown_connections 2024-04-20T10:42:10.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.473+0000 7f97eaa00640 1 --2- 172.21.15.42:0/3668018721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e4107150 0x7f97e410ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.473+0000 7f97eaa00640 1 --2- 172.21.15.42:0/3668018721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e4102a30 0x7f97e4106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.473+0000 7f97eaa00640 1 --2- 172.21.15.42:0/3668018721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97e4102080 0x7f97e4102460 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.473+0000 7f97eaa00640 1 -- 172.21.15.42:0/3668018721 >> 172.21.15.42:0/3668018721 conn(0x7f97e40fd900 msgr2=0x7f97e40ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:10.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.473+0000 7f97eaa00640 1 -- 172.21.15.42:0/3668018721 shutdown_connections 2024-04-20T10:42:10.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.473+0000 7f97eaa00640 1 -- 172.21.15.42:0/3668018721 wait complete. 2024-04-20T10:42:10.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.474+0000 7f97eaa00640 1 Processor -- start 2024-04-20T10:42:10.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.475+0000 7f97eaa00640 1 -- start start 2024-04-20T10:42:10.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.475+0000 7f97eaa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97e4102080 0x7f97e41a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.475+0000 7f97eaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e4107150 0x7f97e41a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.475+0000 7f97eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e419a760 0x7f97e419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.475+0000 7f97eaa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f97e4112200 con 0x7f97e4107150 2024-04-20T10:42:10.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.475+0000 7f97eaa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97e4112080 con 0x7f97e419a760 2024-04-20T10:42:10.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.475+0000 7f97eaa00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f97e4112380 con 0x7f97e4102080 2024-04-20T10:42:10.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.475+0000 7f97ea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e419a760 0x7f97e419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.475+0000 7f97ea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e419a760 0x7f97e419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59184/0 (socket says 172.21.15.42:59184) 2024-04-20T10:42:10.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.476+0000 7f97ea000640 1 -- 172.21.15.42:0/1216946494 learned_addr learned my addr 172.21.15.42:0/1216946494 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:10.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.476+0000 7f97e8c00640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e4107150 0x7f97e41a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.476+0000 7f97e9600640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97e4102080 0x7f97e41a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.476+0000 7f97ea000640 1 -- 172.21.15.42:0/1216946494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97e4102080 msgr2=0x7f97e41a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:10.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.476+0000 7f97ea000640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97e4102080 0x7f97e41a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.476+0000 7f97ea000640 1 -- 172.21.15.42:0/1216946494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e4107150 msgr2=0x7f97e41a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:10.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.476+0000 7f97ea000640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e4107150 0x7f97e41a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.476+0000 7f97ea000640 1 -- 172.21.15.42:0/1216946494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e419b360 con 0x7f97e419a760 2024-04-20T10:42:10.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.477+0000 7f97ea000640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e419a760 0x7f97e419abe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00eb10 tx=0x7f97dc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:10.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.477+0000 7f97e2000640 1 -- 172.21.15.42:0/1216946494 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc004b20 con 0x7f97e419a760 2024-04-20T10:42:10.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.477+0000 7f97e2000640 1 -- 172.21.15.42:0/1216946494 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97dc004cc0 con 0x7f97e419a760 2024-04-20T10:42:10.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.477+0000 7f97e2000640 1 -- 172.21.15.42:0/1216946494 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc010760 con 0x7f97e419a760 2024-04-20T10:42:10.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.477+0000 7f97eaa00640 1 -- 172.21.15.42:0/1216946494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e41a7440 con 0x7f97e419a760 2024-04-20T10:42:10.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.477+0000 7f97eaa00640 1 -- 172.21.15.42:0/1216946494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e41a77f0 con 0x7f97e419a760 2024-04-20T10:42:10.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.481+0000 7f97eebfc740 1 Processor -- start 2024-04-20T10:42:10.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.481+0000 7f97e2000640 1 -- 172.21.15.42:0/1216946494 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f97dc005020 con 0x7f97e419a760 2024-04-20T10:42:10.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.481+0000 7f97eebfc740 1 -- start start 2024-04-20T10:42:10.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.482+0000 7f97eebfc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ede550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.482+0000 7f97eebfc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3edea90 0x55abc3ee7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.482+0000 7f97eebfc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3ee7670 0x55abc3ee9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.482+0000 7f97eebfc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55abc3e43050 con 0x55abc3ee7670 2024-04-20T10:42:10.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.482+0000 7f97eebfc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55abc3de9490 con 0x55abc3ede170 2024-04-20T10:42:10.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.482+0000 7f97eebfc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55abc3e3f010 con 0x55abc3edea90 2024-04-20T10:42:10.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.482+0000 7f97e9600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3edea90 0x55abc3ee7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.482+0000 7f97e8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ede550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.482+0000 7f97e9600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3edea90 0x55abc3ee7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:37998/0 (socket says 172.21.15.42:37998) 2024-04-20T10:42:10.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.482+0000 7f97e9600640 1 -- 172.21.15.42:0/1197130167 learned_addr learned my addr 172.21.15.42:0/1197130167 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:10.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.482+0000 7f97e8c00640 1 --2- 172.21.15.42:0/1197130167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3ee7670 0x55abc3ee9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97e9600640 1 -- 172.21.15.42:0/1197130167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 msgr2=0x55abc3ede550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:10.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97e9600640 1 --2- 172.21.15.42:0/1197130167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ede550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97e9600640 1 -- 172.21.15.42:0/1197130167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3ee7670 msgr2=0x55abc3ee9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:10.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97e9600640 1 --2- 172.21.15.42:0/1197130167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3ee7670 0x55abc3ee9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97e9600640 1 -- 172.21.15.42:0/1197130167 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc3df1070 con 0x55abc3edea90 2024-04-20T10:42:10.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97e8c00640 1 --2- 172.21.15.42:0/1197130167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ede550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:10.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97e9600640 1 --2- 172.21.15.42:0/1197130167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3edea90 0x55abc3ee7130 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f97d8006010 tx=0x7f97d8005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92bf367144243361 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:10.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97e2000640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f97cc0a3da0 0x7f97cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97e2000640 1 -- 172.21.15.42:0/1216946494 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f97dc0d0df0 con 0x7f97e419a760 2024-04-20T10:42:10.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97d6600640 1 -- 172.21.15.42:0/1197130167 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d802ca80 con 0x55abc3edea90 2024-04-20T10:42:10.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97d6600640 1 -- 172.21.15.42:0/1197130167 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d802cc20 con 0x55abc3edea90 2024-04-20T10:42:10.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97d6600640 1 -- 172.21.15.42:0/1197130167 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8004920 con 0x55abc3edea90 2024-04-20T10:42:10.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.483+0000 7f97e8c00640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f97cc0a3da0 0x7f97cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.484+0000 7f97eebfc740 1 -- 172.21.15.42:0/1197130167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3edea90 msgr2=0x55abc3ee7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:10.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.484+0000 7f97eebfc740 1 --2- 172.21.15.42:0/1197130167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3edea90 0x55abc3ee7130 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f97d8006010 tx=0x7f97d8005dd0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.484+0000 7f97e8c00640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f97cc0a3da0 0x7f97cc0a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55abc3ed9a30 tx=0x7f97d0004730 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:10.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.484+0000 7f97eebfc740 1 -- 172.21.15.42:0/1197130167 shutdown_connections 2024-04-20T10:42:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.484+0000 7f97eebfc740 1 --2- 172.21.15.42:0/1197130167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3ee7670 0x55abc3ee9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.484+0000 7f97eebfc740 1 --2- 172.21.15.42:0/1197130167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3edea90 0x55abc3ee7130 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.484+0000 7f97eebfc740 1 --2- 172.21.15.42:0/1197130167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ede550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.484+0000 7f97eebfc740 1 -- 172.21.15.42:0/1197130167 >> 172.21.15.42:0/1197130167 conn(0x55abc3ed88a0 msgr2=0x55abc3ed8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.485+0000 7f97eebfc740 1 -- 172.21.15.42:0/1197130167 shutdown_connections 2024-04-20T10:42:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.485+0000 7f97eebfc740 1 -- 172.21.15.42:0/1197130167 wait complete. 2024-04-20T10:42:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.485+0000 7f97eebfc740 1 -- start start 2024-04-20T10:42:10.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.485+0000 7f97eebfc740 1 Processor -- start 2024-04-20T10:42:10.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97eebfc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ee4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97eebfc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3edea90 0x55abc3ee4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97eebfc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3ee7670 0x55abc3ee50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97eebfc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55abc3eea400 con 0x55abc3ee7670 2024-04-20T10:42:10.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97eebfc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55abc3dfc860 con 0x55abc3ede170 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97eebfc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55abc3eea580 con 0x55abc3edea90 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97eebfc740 10 client.0 _ll_register_callbacks cb 0x55abc3dcafe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97e9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ee4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97e9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ee4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59196/0 (socket says 172.21.15.42:59196) 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97e9600640 1 -- 172.21.15.42:0/637925220 learned_addr learned my addr 172.21.15.42:0/637925220 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97e8c00640 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3edea90 0x55abc3ee4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97ea000640 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3ee7670 0x55abc3ee50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97e9600640 1 -- 172.21.15.42:0/637925220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3edea90 msgr2=0x55abc3ee4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.486+0000 7f97e9600640 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3edea90 0x55abc3ee4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.487+0000 7f97e9600640 1 -- 172.21.15.42:0/637925220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3ee7670 msgr2=0x55abc3ee50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.487+0000 7f97e9600640 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3ee7670 0x55abc3ee50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.487+0000 7f97e9600640 1 -- 172.21.15.42:0/637925220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc3e00950 con 0x55abc3ede170 2024-04-20T10:42:10.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.487+0000 7f97e9600640 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ee4640 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f97d80096c0 tx=0x7f97d802c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca1dd1a3aeee898 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.487+0000 7f97caa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.487+0000 7f97caa00640 1 -- 172.21.15.42:0/637925220 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d804d060 con 0x55abc3ede170 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.487+0000 7f97eebfc740 1 -- 172.21.15.42:0/637925220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d802cc20 con 0x55abc3ede170 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.487+0000 7f97caa00640 1 -- 172.21.15.42:0/637925220 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f97d80035f0 con 0x55abc3ede170 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.488+0000 7f97caa00640 1 -- 172.21.15.42:0/637925220 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f97d80037f0 con 0x55abc3ede170 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.488+0000 7f97eebfc740 10 client.26297 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.488+0000 7f97eebfc740 1 -- 172.21.15.42:0/637925220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55abc3dce820 con 0x55abc3ede170 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.488+0000 7f97caa00640 1 -- 172.21.15.42:0/637925220 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d802cc20 con 0x55abc3ede170 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.488+0000 7f97caa00640 1 -- 172.21.15.42:0/637925220 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f97d8034b40 con 0x55abc3ede170 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.488+0000 7f97eebfc740 10 client.26297 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.488+0000 7f97eebfc740 10 client.26297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.488+0000 7f97eebfc740 20 client.26297 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.488+0000 7f97eebfc740 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55abc3ef7490 0x55abc3ef9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.488+0000 7f97eebfc740 4 client.26297 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:42:10.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.488+0000 7f97eebfc740 1 -- 172.21.15.42:0/637925220 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abc3e50a70 con 0x55abc3ef7490 2024-04-20T10:42:10.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.489+0000 7f97e8c00640 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55abc3ef7490 0x55abc3ef9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.489+0000 7f97e8c00640 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55abc3ef7490 0x55abc3ef9880 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=405eb04a8f965da9 server_cookie=f4da278ae94f5c79 in_seq=0 out_seq=0 2024-04-20T10:42:10.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.489+0000 7f97caa00640 10 client.26297 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.491+0000 7f97caa00640 1 -- 172.21.15.42:0/637925220 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55abc3e50a70 con 0x55abc3ef7490 2024-04-20T10:42:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.491+0000 7f97caa00640 10 client.26297 handle_command_reply: tid=0 2024-04-20T10:42:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.491+0000 7f97eebfc740 1 client.26297 shutdown 2024-04-20T10:42:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.491+0000 7f97eebfc740 1 -- 172.21.15.42:0/637925220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 msgr2=0x55abc3ee4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.491+0000 7f97eebfc740 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ee4640 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f97d80096c0 tx=0x7f97d802c330 comp rx=0 tx=0).stop 2024-04-20T10:42:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.491+0000 7f97eebfc740 1 -- 172.21.15.42:0/637925220 shutdown_connections 2024-04-20T10:42:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.491+0000 7f97eebfc740 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55abc3ef7490 0x55abc3ef9880 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.491+0000 7f97eebfc740 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3ee7670 0x55abc3ee50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.491+0000 7f97eebfc740 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3edea90 0x55abc3ee4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.491+0000 7f97caa00640 0 client.26297 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.491+0000 7f97eebfc740 1 --2- 172.21.15.42:0/637925220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ee4640 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.491+0000 7f97eebfc740 1 -- 172.21.15.42:0/637925220 >> 172.21.15.42:0/637925220 conn(0x55abc3ed88a0 msgr2=0x55abc3eda5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.492+0000 7f97eebfc740 1 -- 172.21.15.42:0/637925220 shutdown_connections 2024-04-20T10:42:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.492+0000 7f97eebfc740 1 -- 172.21.15.42:0/637925220 wait complete. 2024-04-20T10:42:10.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.492+0000 7f97eebfc740 20 client.26297 trim_cache size 0 max 16384 2024-04-20T10:42:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.506+0000 7f97eebfc740 1 Processor -- start 2024-04-20T10:42:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.507+0000 7f97eebfc740 1 -- start start 2024-04-20T10:42:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.507+0000 7f97eebfc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ee6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.507+0000 7f97eebfc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3edea90 0x55abc3ef3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.507+0000 7f97eebfc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3ee7670 0x55abc3ee2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.507+0000 7f97eebfc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f97e41adde0 con 0x55abc3edea90 2024-04-20T10:42:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.507+0000 7f97eebfc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97e41b0640 con 0x55abc3ede170 2024-04-20T10:42:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.507+0000 7f97eebfc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f97e41ae430 con 0x55abc3ee7670 2024-04-20T10:42:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.507+0000 7f97e9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ee6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.507+0000 7f97e9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ee6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59206/0 (socket says 172.21.15.42:59206) 2024-04-20T10:42:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.507+0000 7f97e9600640 1 -- 172.21.15.42:0/3947900083 learned_addr learned my addr 172.21.15.42:0/3947900083 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:10.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.507+0000 7f97ea000640 1 --2- 172.21.15.42:0/3947900083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3ee7670 0x55abc3ee2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.507+0000 7f97e8c00640 1 --2- 172.21.15.42:0/3947900083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3edea90 0x55abc3ef3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.508+0000 7f97e9600640 1 -- 172.21.15.42:0/3947900083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3ee7670 msgr2=0x55abc3ee2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.508+0000 7f97e9600640 1 --2- 172.21.15.42:0/3947900083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3ee7670 0x55abc3ee2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.508+0000 7f97e9600640 1 -- 172.21.15.42:0/3947900083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3edea90 msgr2=0x55abc3ef3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.508+0000 7f97e9600640 1 --2- 172.21.15.42:0/3947900083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3edea90 0x55abc3ef3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.508+0000 7f97e9600640 1 -- 172.21.15.42:0/3947900083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc3b553f0 con 0x55abc3ede170 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.508+0000 7f97ea000640 1 --2- 172.21.15.42:0/3947900083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3ee7670 0x55abc3ee2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.508+0000 7f97e9600640 1 --2- 172.21.15.42:0/3947900083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ee6d70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f97d802bc70 tx=0x7f97d8005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d2f162a282f92f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.508+0000 7f97d5c00640 1 -- 172.21.15.42:0/3947900083 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d804d040 con 0x55abc3ede170 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.508+0000 7f97d5c00640 1 -- 172.21.15.42:0/3947900083 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d802cc00 con 0x55abc3ede170 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.508+0000 7f97d5c00640 1 -- 172.21.15.42:0/3947900083 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d803da60 con 0x55abc3ede170 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.509+0000 7f97eebfc740 1 -- 172.21.15.42:0/3947900083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 msgr2=0x55abc3ee6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.509+0000 7f97eebfc740 1 --2- 172.21.15.42:0/3947900083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ee6d70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f97d802bc70 tx=0x7f97d8005dd0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.509+0000 7f97eebfc740 1 -- 172.21.15.42:0/3947900083 shutdown_connections 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.509+0000 7f97eebfc740 1 --2- 172.21.15.42:0/3947900083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3ee7670 0x55abc3ee2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.509+0000 7f97eebfc740 1 --2- 172.21.15.42:0/3947900083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3edea90 0x55abc3ef3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.509+0000 7f97eebfc740 1 --2- 172.21.15.42:0/3947900083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3ee6d70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.509+0000 7f97eebfc740 1 -- 172.21.15.42:0/3947900083 >> 172.21.15.42:0/3947900083 conn(0x55abc3ed88a0 msgr2=0x55abc3ed7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.509+0000 7f97eebfc740 1 -- 172.21.15.42:0/3947900083 shutdown_connections 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.509+0000 7f97eebfc740 1 -- 172.21.15.42:0/3947900083 wait complete. 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.509+0000 7f97eebfc740 1 -- start start 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97eebfc740 1 Processor -- start 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97eebfc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3eeb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97eebfc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3edea90 0x55abc3eebe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97eebfc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3ee7670 0x55abc3f5c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97eebfc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f97e41aeb00 con 0x55abc3edea90 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97eebfc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97e41aff70 con 0x55abc3ede170 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97eebfc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f97e41af1d0 con 0x55abc3ee7670 2024-04-20T10:42:10.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97eebfc740 10 client.0 _ll_register_callbacks cb 0x7f97d0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:10.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97e9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3eeb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97e8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3edea90 0x55abc3eebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97e9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3eeb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59208/0 (socket says 172.21.15.42:59208) 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97e9600640 1 -- 172.21.15.42:0/4122687931 learned_addr learned my addr 172.21.15.42:0/4122687931 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.510+0000 7f97ea000640 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3ee7670 0x55abc3f5c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.511+0000 7f97e9600640 1 -- 172.21.15.42:0/4122687931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3ee7670 msgr2=0x55abc3f5c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.511+0000 7f97e9600640 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3ee7670 0x55abc3f5c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.511+0000 7f97e9600640 1 -- 172.21.15.42:0/4122687931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3edea90 msgr2=0x55abc3eebe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.511+0000 7f97e9600640 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3edea90 0x55abc3eebe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.511+0000 7f97e9600640 1 -- 172.21.15.42:0/4122687931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc3c1f7d0 con 0x55abc3ede170 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.511+0000 7f97e8c00640 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3edea90 0x55abc3eebe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.511+0000 7f97ea000640 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3ee7670 0x55abc3f5c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.511+0000 7f97e9600640 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3eeb8d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f97d803d450 tx=0x7f97d8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3fa4ef65cc660ae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.511+0000 7f97caa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:42:10.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.511+0000 7f97caa00640 1 -- 172.21.15.42:0/4122687931 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d804d040 con 0x55abc3ede170 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.511+0000 7f97caa00640 1 -- 172.21.15.42:0/4122687931 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f97d8003b90 con 0x55abc3ede170 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.511+0000 7f97eebfc740 1 -- 172.21.15.42:0/4122687931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d802cc00 con 0x55abc3ede170 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.512+0000 7f97caa00640 1 -- 172.21.15.42:0/4122687931 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f97d8003d90 con 0x55abc3ede170 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.512+0000 7f97eebfc740 10 client.26303 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.512+0000 7f97eebfc740 1 -- 172.21.15.42:0/4122687931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55abc3f5dc60 con 0x55abc3ede170 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.512+0000 7f97caa00640 1 -- 172.21.15.42:0/4122687931 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d802cc00 con 0x55abc3ede170 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.512+0000 7f97caa00640 1 -- 172.21.15.42:0/4122687931 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f97d8034750 con 0x55abc3ede170 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.512+0000 7f97eebfc740 10 client.26303 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.512+0000 7f97eebfc740 10 client.26303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.512+0000 7f97eebfc740 20 client.26303 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.512+0000 7f97eebfc740 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55abc3f5e260 0x55abc3f60670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.512+0000 7f97eebfc740 4 client.26303 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.512+0000 7f97eebfc740 1 -- 172.21.15.42:0/4122687931 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abc3e4a8e0 con 0x55abc3f5e260 2024-04-20T10:42:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.513+0000 7f97e8c00640 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55abc3f5e260 0x55abc3f60670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.513+0000 7f97e8c00640 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55abc3f5e260 0x55abc3f60670 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a8b5e9cddcb114 server_cookie=d0cf17ac54fd840a in_seq=0 out_seq=0 2024-04-20T10:42:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.513+0000 7f97caa00640 10 client.26303 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.515+0000 7f97caa00640 1 -- 172.21.15.42:0/4122687931 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abc3e4a8e0 con 0x55abc3f5e260 2024-04-20T10:42:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.515+0000 7f97caa00640 10 client.26303 handle_command_reply: tid=0 2024-04-20T10:42:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.515+0000 7f97eebfc740 1 client.26303 shutdown 2024-04-20T10:42:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.516+0000 7f97eebfc740 1 -- 172.21.15.42:0/4122687931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 msgr2=0x55abc3eeb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.516+0000 7f97eebfc740 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3eeb8d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f97d803d450 tx=0x7f97d8005fd0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.516+0000 7f97eebfc740 1 -- 172.21.15.42:0/4122687931 shutdown_connections 2024-04-20T10:42:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.516+0000 7f97eebfc740 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55abc3f5e260 0x55abc3f60670 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.516+0000 7f97eebfc740 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55abc3ee7670 0x55abc3f5c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.516+0000 7f97eebfc740 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc3edea90 0x55abc3eebe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.516+0000 7f97caa00640 0 client.26303 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:10.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.516+0000 7f97eebfc740 1 --2- 172.21.15.42:0/4122687931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abc3ede170 0x55abc3eeb8d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.516+0000 7f97eebfc740 1 -- 172.21.15.42:0/4122687931 >> 172.21.15.42:0/4122687931 conn(0x55abc3ed88a0 msgr2=0x55abc3ee45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:10.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.516+0000 7f97eebfc740 1 -- 172.21.15.42:0/4122687931 shutdown_connections 2024-04-20T10:42:10.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.516+0000 7f97eebfc740 1 -- 172.21.15.42:0/4122687931 wait complete. 2024-04-20T10:42:10.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.516+0000 7f97eebfc740 20 client.26303 trim_cache size 0 max 16384 2024-04-20T10:42:10.519 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:42:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.517+0000 7f97eaa00640 1 -- 172.21.15.42:0/1216946494 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f97cc0a3da0 msgr2=0x7f97cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.517+0000 7f97eaa00640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f97cc0a3da0 0x7f97cc0a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55abc3ed9a30 tx=0x7f97d0004730 comp rx=0 tx=0).stop 2024-04-20T10:42:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.517+0000 7f97eaa00640 1 -- 172.21.15.42:0/1216946494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e419a760 msgr2=0x7f97e419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.517+0000 7f97eaa00640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e419a760 0x7f97e419abe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00eb10 tx=0x7f97dc00c2d0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.517+0000 7f97eaa00640 1 -- 172.21.15.42:0/1216946494 shutdown_connections 2024-04-20T10:42:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.517+0000 7f97eaa00640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f97cc0a3da0 0x7f97cc0a6260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.517+0000 7f97eaa00640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e419a760 0x7f97e419abe0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.517+0000 7f97eaa00640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e4107150 0x7f97e41a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.517+0000 7f97eaa00640 1 --2- 172.21.15.42:0/1216946494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97e4102080 0x7f97e41a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.517+0000 7f97eaa00640 1 -- 172.21.15.42:0/1216946494 >> 172.21.15.42:0/1216946494 conn(0x7f97e40fd900 msgr2=0x7f97e40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.517+0000 7f97eaa00640 1 -- 172.21.15.42:0/1216946494 shutdown_connections 2024-04-20T10:42:10.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:10.517+0000 7f97eaa00640 1 -- 172.21.15.42:0/1216946494 wait complete. 2024-04-20T10:42:11.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:10 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1221737038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:42:11.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:10 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1221737038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:42:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:10 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1221737038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:42:11.536 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:42:11.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:42:11.536 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:42:11.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.767+0000 7f7704c00640 1 -- 172.21.15.42:0/1890871775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77001005d0 msgr2=0x7f77001009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:11.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.767+0000 7f7704c00640 1 --2- 172.21.15.42:0/1890871775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77001005d0 0x7f77001009b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009900 tx=0x7f76f002bc30 comp rx=0 tx=0).stop 2024-04-20T10:42:11.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.767+0000 7f7704c00640 1 -- 172.21.15.42:0/1890871775 shutdown_connections 2024-04-20T10:42:11.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.767+0000 7f7704c00640 1 --2- 172.21.15.42:0/1890871775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7700107190 0x7f770010ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.767+0000 7f7704c00640 1 --2- 172.21.15.42:0/1890871775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7700100f80 0x7f7700106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.767+0000 7f7704c00640 1 --2- 172.21.15.42:0/1890871775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77001005d0 0x7f77001009b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.767+0000 7f7704c00640 1 -- 172.21.15.42:0/1890871775 >> 172.21.15.42:0/1890871775 conn(0x7f77000772f0 msgr2=0x7f77000fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:11.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.767+0000 7f7704c00640 1 -- 172.21.15.42:0/1890871775 shutdown_connections 2024-04-20T10:42:11.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.767+0000 7f7704c00640 1 -- 172.21.15.42:0/1890871775 wait complete. 2024-04-20T10:42:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.768+0000 7f7704c00640 1 Processor -- start 2024-04-20T10:42:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.768+0000 7f7704c00640 1 -- start start 2024-04-20T10:42:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f7704c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77001005d0 0x7f77001a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f7704c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7700107190 0x7f77001a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f7704c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f770019a710 0x7f770019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f7704c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7700112200 con 0x7f7700107190 2024-04-20T10:42:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f7704c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7700112080 con 0x7f770019a710 2024-04-20T10:42:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f7704c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7700112380 con 0x7f77001005d0 2024-04-20T10:42:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f76ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f770019a710 0x7f770019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f76ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f770019a710 0x7f770019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59224/0 (socket says 172.21.15.42:59224) 2024-04-20T10:42:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f76ffe00640 1 -- 172.21.15.42:0/3149737529 learned_addr learned my addr 172.21.15.42:0/3149737529 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f76fea00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7700107190 0x7f77001a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f76ff400640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77001005d0 0x7f77001a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f76ffe00640 1 -- 172.21.15.42:0/3149737529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77001005d0 msgr2=0x7f77001a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f76ffe00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77001005d0 0x7f77001a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f76ffe00640 1 -- 172.21.15.42:0/3149737529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7700107190 msgr2=0x7f77001a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f76ffe00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7700107190 0x7f77001a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.769+0000 7f76ffe00640 1 -- 172.21.15.42:0/3149737529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770019b310 con 0x7f770019a710 2024-04-20T10:42:11.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.770+0000 7f76ffe00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f770019a710 0x7f770019ab90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009fd0 tx=0x7f76f800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:11.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.770+0000 7f76f7e00640 1 -- 172.21.15.42:0/3149737529 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8017070 con 0x7f770019a710 2024-04-20T10:42:11.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.770+0000 7f76f7e00640 1 -- 172.21.15.42:0/3149737529 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f8003e50 con 0x7f770019a710 2024-04-20T10:42:11.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.770+0000 7f76f7e00640 1 -- 172.21.15.42:0/3149737529 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f80106c0 con 0x7f770019a710 2024-04-20T10:42:11.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.770+0000 7f7704c00640 1 -- 172.21.15.42:0/3149737529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77001a73f0 con 0x7f770019a710 2024-04-20T10:42:11.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.770+0000 7f7704c00640 1 -- 172.21.15.42:0/3149737529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77001a77a0 con 0x7f770019a710 2024-04-20T10:42:11.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.773+0000 7f76f7e00640 1 -- 172.21.15.42:0/3149737529 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f76f8003970 con 0x7f770019a710 2024-04-20T10:42:11.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.774+0000 7f7708e55740 1 Processor -- start 2024-04-20T10:42:11.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.774+0000 7f7708e55740 1 -- start start 2024-04-20T10:42:11.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.775+0000 7f7708e55740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d6ff00 0x562a02d702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.775+0000 7f7708e55740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d708b0 0x562a02d818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.775+0000 7f7708e55740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d81e00 0x562a02d841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.775+0000 7f7708e55740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562a02cd4d20 con 0x562a02d81e00 2024-04-20T10:42:11.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.775+0000 7f7708e55740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562a02c7b3e0 con 0x562a02d708b0 2024-04-20T10:42:11.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.775+0000 7f7708e55740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562a02cd0ce0 con 0x562a02d6ff00 2024-04-20T10:42:11.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.775+0000 7f76ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d708b0 0x562a02d818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.775+0000 7f76ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d708b0 0x562a02d818c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59234/0 (socket says 172.21.15.42:59234) 2024-04-20T10:42:11.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.775+0000 7f76ff400640 1 -- 172.21.15.42:0/917661343 learned_addr learned my addr 172.21.15.42:0/917661343 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:11.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.775+0000 7f76fea00640 1 --2- 172.21.15.42:0/917661343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d6ff00 0x562a02d702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.775+0000 7f76fea00640 1 --2- 172.21.15.42:0/917661343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d81e00 0x562a02d841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.776+0000 7f76ff400640 1 -- 172.21.15.42:0/917661343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d6ff00 msgr2=0x562a02d702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.776+0000 7f76ff400640 1 --2- 172.21.15.42:0/917661343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d6ff00 0x562a02d702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.776+0000 7f76ff400640 1 -- 172.21.15.42:0/917661343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d81e00 msgr2=0x562a02d841f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.776+0000 7f76ff400640 1 --2- 172.21.15.42:0/917661343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d81e00 0x562a02d841f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.776+0000 7f76ff400640 1 -- 172.21.15.42:0/917661343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a02c9a980 con 0x562a02d708b0 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.776+0000 7f76f7e00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f76e40a3d20 0x7f76e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.776+0000 7f76ff400640 1 --2- 172.21.15.42:0/917661343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d708b0 0x562a02d818c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f76f0035000 tx=0x7f76f0009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a968d50f9d573ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.776+0000 7f76f7e00640 1 -- 172.21.15.42:0/3149737529 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f76f80d0500 con 0x7f770019a710 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.776+0000 7f76fea00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f76e40a3d20 0x7f76e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.776+0000 7f76ebe00640 1 -- 172.21.15.42:0/917661343 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0045070 con 0x562a02d708b0 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.777+0000 7f76ebe00640 1 -- 172.21.15.42:0/917661343 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f002c880 con 0x562a02d708b0 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.777+0000 7f76ebe00640 1 -- 172.21.15.42:0/917661343 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0004f20 con 0x562a02d708b0 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.777+0000 7f7708e55740 1 -- 172.21.15.42:0/917661343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d708b0 msgr2=0x562a02d818c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.777+0000 7f7708e55740 1 --2- 172.21.15.42:0/917661343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d708b0 0x562a02d818c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f76f0035000 tx=0x7f76f0009610 comp rx=0 tx=0).stop 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.777+0000 7f76fea00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f76e40a3d20 0x7f76e40a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x562a02d6bc40 tx=0x7f76ec013040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.778+0000 7f7708e55740 1 -- 172.21.15.42:0/917661343 shutdown_connections 2024-04-20T10:42:11.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.778+0000 7f7708e55740 1 --2- 172.21.15.42:0/917661343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d81e00 0x562a02d841f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.778+0000 7f7708e55740 1 --2- 172.21.15.42:0/917661343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d708b0 0x562a02d818c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.778+0000 7f7708e55740 1 --2- 172.21.15.42:0/917661343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d6ff00 0x562a02d702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.778+0000 7f7708e55740 1 -- 172.21.15.42:0/917661343 >> 172.21.15.42:0/917661343 conn(0x562a02d6a820 msgr2=0x562a02d6ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.778+0000 7f7708e55740 1 -- 172.21.15.42:0/917661343 shutdown_connections 2024-04-20T10:42:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.778+0000 7f7708e55740 1 -- 172.21.15.42:0/917661343 wait complete. 2024-04-20T10:42:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.778+0000 7f7708e55740 1 -- start start 2024-04-20T10:42:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.778+0000 7f7708e55740 1 Processor -- start 2024-04-20T10:42:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.779+0000 7f7708e55740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d7ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.779+0000 7f7708e55740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d708b0 0x562a02d7f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.779+0000 7f7708e55740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d81e00 0x562a02d7f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.779+0000 7f7708e55740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562a02d84b60 con 0x562a02d708b0 2024-04-20T10:42:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.779+0000 7f7708e55740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562a02c8e7a0 con 0x562a02d6ff00 2024-04-20T10:42:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.779+0000 7f7708e55740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562a02d84ce0 con 0x562a02d81e00 2024-04-20T10:42:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.779+0000 7f7708e55740 10 client.0 _ll_register_callbacks cb 0x562a02c90b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.779+0000 7f76ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d7ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.779+0000 7f76ffe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d81e00 0x562a02d7f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.779+0000 7f76fea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d708b0 0x562a02d7f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.779+0000 7f76ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d7ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59248/0 (socket says 172.21.15.42:59248) 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.779+0000 7f76ff400640 1 -- 172.21.15.42:0/4124398504 learned_addr learned my addr 172.21.15.42:0/4124398504 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f76ffe00640 1 -- 172.21.15.42:0/4124398504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 msgr2=0x562a02d7ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f76ffe00640 1 --2- 172.21.15.42:0/4124398504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d7ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f76ffe00640 1 -- 172.21.15.42:0/4124398504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d708b0 msgr2=0x562a02d7f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f76ffe00640 1 --2- 172.21.15.42:0/4124398504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d708b0 0x562a02d7f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f76ffe00640 1 -- 172.21.15.42:0/4124398504 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a02aaf7e0 con 0x562a02d81e00 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f76ff400640 1 --2- 172.21.15.42:0/4124398504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d7ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f76ffe00640 1 --2- 172.21.15.42:0/4124398504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d81e00 0x562a02d7f910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f76f80cdbc0 tx=0x7f76f809be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f68483379177238b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f76e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f76e8c00640 1 -- 172.21.15.42:0/4124398504 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8020040 con 0x562a02d81e00 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f76e8c00640 1 -- 172.21.15.42:0/4124398504 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f76f8098b30 con 0x562a02d81e00 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f7708e55740 1 -- 172.21.15.42:0/4124398504 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76f002c880 con 0x562a02d81e00 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f76e8c00640 1 -- 172.21.15.42:0/4124398504 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f76f80ba050 con 0x562a02d81e00 2024-04-20T10:42:11.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f7708e55740 10 client.26329 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.780+0000 7f7708e55740 1 -- 172.21.15.42:0/4124398504 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562a02b0ff70 con 0x562a02d81e00 2024-04-20T10:42:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.781+0000 7f76e8c00640 1 -- 172.21.15.42:0/4124398504 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f002c880 con 0x562a02d81e00 2024-04-20T10:42:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.781+0000 7f76e8c00640 1 -- 172.21.15.42:0/4124398504 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f76f809b910 con 0x562a02d81e00 2024-04-20T10:42:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.781+0000 7f7708e55740 10 client.26329 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.781+0000 7f7708e55740 10 client.26329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.781+0000 7f7708e55740 20 client.26329 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.781+0000 7f7708e55740 1 --2- 172.21.15.42:0/4124398504 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562a02d92210 0x562a02d94660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.781+0000 7f7708e55740 4 client.26329 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:42:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.781+0000 7f7708e55740 1 -- 172.21.15.42:0/4124398504 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a02ce2740 con 0x562a02d92210 2024-04-20T10:42:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.781+0000 7f76ff400640 1 --2- 172.21.15.42:0/4124398504 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562a02d92210 0x562a02d94660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.782+0000 7f76ff400640 1 --2- 172.21.15.42:0/4124398504 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562a02d92210 0x562a02d94660 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e61350e6646fea server_cookie=11fcb4798369bf9 in_seq=0 out_seq=0 2024-04-20T10:42:11.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.782+0000 7f76e8c00640 10 client.26329 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.784+0000 7f76e8c00640 1 -- 172.21.15.42:0/4124398504 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a02ce2740 con 0x562a02d92210 2024-04-20T10:42:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.784+0000 7f76e8c00640 10 client.26329 handle_command_reply: tid=0 2024-04-20T10:42:11.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.784+0000 7f7708e55740 1 client.26329 shutdown 2024-04-20T10:42:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.784+0000 7f7708e55740 1 -- 172.21.15.42:0/4124398504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d81e00 msgr2=0x562a02d7f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.784+0000 7f7708e55740 1 --2- 172.21.15.42:0/4124398504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d81e00 0x562a02d7f910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f76f80cdbc0 tx=0x7f76f809be20 comp rx=0 tx=0).stop 2024-04-20T10:42:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.784+0000 7f7708e55740 1 -- 172.21.15.42:0/4124398504 shutdown_connections 2024-04-20T10:42:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.784+0000 7f7708e55740 1 --2- 172.21.15.42:0/4124398504 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562a02d92210 0x562a02d94660 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.784+0000 7f7708e55740 1 --2- 172.21.15.42:0/4124398504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d81e00 0x562a02d7f910 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.784+0000 7f7708e55740 1 --2- 172.21.15.42:0/4124398504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d708b0 0x562a02d7f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.784+0000 7f7708e55740 1 --2- 172.21.15.42:0/4124398504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d7ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.784+0000 7f7708e55740 1 -- 172.21.15.42:0/4124398504 >> 172.21.15.42:0/4124398504 conn(0x562a02d6a820 msgr2=0x562a02d6c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.784+0000 7f76e8c00640 0 client.26329 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.785+0000 7f7708e55740 1 -- 172.21.15.42:0/4124398504 shutdown_connections 2024-04-20T10:42:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.785+0000 7f7708e55740 1 -- 172.21.15.42:0/4124398504 wait complete. 2024-04-20T10:42:11.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.786+0000 7f7708e55740 20 client.26329 trim_cache size 0 max 16384 2024-04-20T10:42:11.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.808+0000 7f7708e55740 1 Processor -- start 2024-04-20T10:42:11.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.808+0000 7f7708e55740 1 -- start start 2024-04-20T10:42:11.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.808+0000 7f7708e55740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.808+0000 7f7708e55740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d708b0 0x562a02d8da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.808+0000 7f7708e55740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d81e00 0x562a02d7a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.808+0000 7f7708e55740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f77001a9950 con 0x562a02d708b0 2024-04-20T10:42:11.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.808+0000 7f7708e55740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77001ac130 con 0x562a02d6ff00 2024-04-20T10:42:11.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.808+0000 7f7708e55740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f77001a9fa0 con 0x562a02d81e00 2024-04-20T10:42:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.809+0000 7f76ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.809+0000 7f76ffe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d81e00 0x562a02d7a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.809+0000 7f76ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d815c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59256/0 (socket says 172.21.15.42:59256) 2024-04-20T10:42:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.809+0000 7f76ff400640 1 -- 172.21.15.42:0/2451466847 learned_addr learned my addr 172.21.15.42:0/2451466847 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.809+0000 7f76fea00640 1 --2- 172.21.15.42:0/2451466847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d708b0 0x562a02d8da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.809+0000 7f76ff400640 1 -- 172.21.15.42:0/2451466847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d81e00 msgr2=0x562a02d7a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.809+0000 7f76ff400640 1 --2- 172.21.15.42:0/2451466847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d81e00 0x562a02d7a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.809+0000 7f76ff400640 1 -- 172.21.15.42:0/2451466847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d708b0 msgr2=0x562a02d8da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.809+0000 7f76ff400640 1 --2- 172.21.15.42:0/2451466847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d708b0 0x562a02d8da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.809+0000 7f76ff400640 1 -- 172.21.15.42:0/2451466847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a029a7170 con 0x562a02d6ff00 2024-04-20T10:42:11.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.809+0000 7f76ffe00640 1 --2- 172.21.15.42:0/2451466847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d81e00 0x562a02d7a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:11.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.809+0000 7f76ff400640 1 --2- 172.21.15.42:0/2451466847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d815c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009900 tx=0x7f76f0004890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a62fccbaff9796b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:11.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.810+0000 7f76eb400640 1 -- 172.21.15.42:0/2451466847 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0056070 con 0x562a02d6ff00 2024-04-20T10:42:11.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.810+0000 7f76eb400640 1 -- 172.21.15.42:0/2451466847 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f002cc70 con 0x562a02d6ff00 2024-04-20T10:42:11.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.810+0000 7f76eb400640 1 -- 172.21.15.42:0/2451466847 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0045040 con 0x562a02d6ff00 2024-04-20T10:42:11.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.810+0000 7f7708e55740 1 -- 172.21.15.42:0/2451466847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 msgr2=0x562a02d815c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:11.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.810+0000 7f7708e55740 1 --2- 172.21.15.42:0/2451466847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d815c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009900 tx=0x7f76f0004890 comp rx=0 tx=0).stop 2024-04-20T10:42:11.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.811+0000 7f7708e55740 1 -- 172.21.15.42:0/2451466847 shutdown_connections 2024-04-20T10:42:11.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.811+0000 7f7708e55740 1 --2- 172.21.15.42:0/2451466847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d81e00 0x562a02d7a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.811+0000 7f7708e55740 1 --2- 172.21.15.42:0/2451466847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d708b0 0x562a02d8da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.811+0000 7f7708e55740 1 --2- 172.21.15.42:0/2451466847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d815c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.811+0000 7f7708e55740 1 -- 172.21.15.42:0/2451466847 >> 172.21.15.42:0/2451466847 conn(0x562a02d6a820 msgr2=0x562a02d69170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.811+0000 7f7708e55740 1 -- 172.21.15.42:0/2451466847 shutdown_connections 2024-04-20T10:42:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.811+0000 7f7708e55740 1 -- 172.21.15.42:0/2451466847 wait complete. 2024-04-20T10:42:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.811+0000 7f7708e55740 1 -- start start 2024-04-20T10:42:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.811+0000 7f7708e55740 1 Processor -- start 2024-04-20T10:42:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f7708e55740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d8a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f7708e55740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d708b0 0x562a02d85500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f7708e55740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d81e00 0x562a02d85a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f7708e55740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f77001aa5f0 con 0x562a02d81e00 2024-04-20T10:42:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f7708e55740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77001aba60 con 0x562a02d6ff00 2024-04-20T10:42:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f7708e55740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f77001aacc0 con 0x562a02d708b0 2024-04-20T10:42:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f7708e55740 10 client.0 _ll_register_callbacks cb 0x7f76f80092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f76ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d8a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f76ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d8a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59270/0 (socket says 172.21.15.42:59270) 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f76ff400640 1 -- 172.21.15.42:0/2671996430 learned_addr learned my addr 172.21.15.42:0/2671996430 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f76ffe00640 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d81e00 0x562a02d85a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f76fea00640 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d708b0 0x562a02d85500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f76ff400640 1 -- 172.21.15.42:0/2671996430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d708b0 msgr2=0x562a02d85500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f76ff400640 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d708b0 0x562a02d85500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f76ff400640 1 -- 172.21.15.42:0/2671996430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d81e00 msgr2=0x562a02d85a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f76ff400640 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d81e00 0x562a02d85a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f76ff400640 1 -- 172.21.15.42:0/2671996430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a02d85f80 con 0x562a02d6ff00 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.812+0000 7f76fea00640 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d708b0 0x562a02d85500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.813+0000 7f76ff400640 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d8a070 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f76f0000c00 tx=0x7f76f0005690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cacf9d20dd7afe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.813+0000 7f76e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.813+0000 7f76e8c00640 1 -- 172.21.15.42:0/2671996430 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0056030 con 0x562a02d6ff00 2024-04-20T10:42:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.813+0000 7f76e8c00640 1 -- 172.21.15.42:0/2671996430 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f76f0004030 con 0x562a02d6ff00 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.813+0000 7f7708e55740 1 -- 172.21.15.42:0/2671996430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76f002cc70 con 0x562a02d6ff00 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.813+0000 7f76e8c00640 1 -- 172.21.15.42:0/2671996430 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f76f0060050 con 0x562a02d6ff00 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.813+0000 7f7708e55740 10 client.26321 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.813+0000 7f7708e55740 1 -- 172.21.15.42:0/2671996430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562a02df8510 con 0x562a02d6ff00 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.814+0000 7f76e8c00640 1 -- 172.21.15.42:0/2671996430 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f002cc70 con 0x562a02d6ff00 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.814+0000 7f76e8c00640 1 -- 172.21.15.42:0/2671996430 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f76f00036f0 con 0x562a02d6ff00 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.814+0000 7f7708e55740 10 client.26321 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.814+0000 7f7708e55740 10 client.26321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.814+0000 7f7708e55740 20 client.26321 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.814+0000 7f7708e55740 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562a02df8780 0x562a02dfab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.814+0000 7f7708e55740 4 client.26321 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.814+0000 7f7708e55740 1 -- 172.21.15.42:0/2671996430 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562a02cdc5b0 con 0x562a02df8780 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.814+0000 7f76fea00640 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562a02df8780 0x562a02dfab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.815+0000 7f76fea00640 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562a02df8780 0x562a02dfab70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7783fe35173e5d28 server_cookie=e1d9257066f0dc2c in_seq=0 out_seq=0 2024-04-20T10:42:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.815+0000 7f76e8c00640 10 client.26321 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f76e8c00640 1 -- 172.21.15.42:0/2671996430 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562a02cdc5b0 con 0x562a02df8780 2024-04-20T10:42:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f76e8c00640 10 client.26321 handle_command_reply: tid=0 2024-04-20T10:42:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f7708e55740 1 client.26321 shutdown 2024-04-20T10:42:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f7708e55740 1 -- 172.21.15.42:0/2671996430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 msgr2=0x562a02d8a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f7708e55740 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d8a070 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f76f0000c00 tx=0x7f76f0005690 comp rx=0 tx=0).stop 2024-04-20T10:42:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f7708e55740 1 -- 172.21.15.42:0/2671996430 shutdown_connections 2024-04-20T10:42:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f7708e55740 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x562a02df8780 0x562a02dfab70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f7708e55740 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562a02d81e00 0x562a02d85a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f7708e55740 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a02d708b0 0x562a02d85500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f76e8c00640 0 client.26321 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f7708e55740 1 --2- 172.21.15.42:0/2671996430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a02d6ff00 0x562a02d8a070 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f7708e55740 1 -- 172.21.15.42:0/2671996430 >> 172.21.15.42:0/2671996430 conn(0x562a02d6a820 msgr2=0x562a02d6c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:11.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f7708e55740 1 -- 172.21.15.42:0/2671996430 shutdown_connections 2024-04-20T10:42:11.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f7708e55740 1 -- 172.21.15.42:0/2671996430 wait complete. 2024-04-20T10:42:11.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.817+0000 7f7708e55740 20 client.26321 trim_cache size 0 max 16384 2024-04-20T10:42:11.820 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:42:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:42:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "9e68e6f8-78c9-4034-92eb-4562c94f03a2", 2024-04-20T10:42:11.820 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:42:11.820 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:42:11.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.818+0000 7f7704c00640 1 -- 172.21.15.42:0/3149737529 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f76e40a3d20 msgr2=0x7f76e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:11.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.818+0000 7f7704c00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f76e40a3d20 0x7f76e40a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x562a02d6bc40 tx=0x7f76ec013040 comp rx=0 tx=0).stop 2024-04-20T10:42:11.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.818+0000 7f7704c00640 1 -- 172.21.15.42:0/3149737529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f770019a710 msgr2=0x7f770019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:11.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.818+0000 7f7704c00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f770019a710 0x7f770019ab90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009fd0 tx=0x7f76f800b260 comp rx=0 tx=0).stop 2024-04-20T10:42:11.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.818+0000 7f7704c00640 1 -- 172.21.15.42:0/3149737529 shutdown_connections 2024-04-20T10:42:11.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.818+0000 7f7704c00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f76e40a3d20 0x7f76e40a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.818+0000 7f7704c00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f770019a710 0x7f770019ab90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.818+0000 7f7704c00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7700107190 0x7f77001a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.818+0000 7f7704c00640 1 --2- 172.21.15.42:0/3149737529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77001005d0 0x7f77001a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:11.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.818+0000 7f7704c00640 1 -- 172.21.15.42:0/3149737529 >> 172.21.15.42:0/3149737529 conn(0x7f77000772f0 msgr2=0x7f7700105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:11.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.818+0000 7f7704c00640 1 -- 172.21.15.42:0/3149737529 shutdown_connections 2024-04-20T10:42:11.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:11.818+0000 7f7704c00640 1 -- 172.21.15.42:0/3149737529 wait complete. 2024-04-20T10:42:11.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:42:11.836 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:42:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.034+0000 7f640b200640 1 -- 172.21.15.42:0/3750436146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c074c50 msgr2=0x7f640c10e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.034+0000 7f640b200640 1 --2- 172.21.15.42:0/3750436146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c074c50 0x7f640c10e160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009860 tx=0x7f63f402bc20 comp rx=0 tx=0).stop 2024-04-20T10:42:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.034+0000 7f640b200640 1 -- 172.21.15.42:0/3750436146 shutdown_connections 2024-04-20T10:42:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.034+0000 7f640b200640 1 --2- 172.21.15.42:0/3750436146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f640c10e6a0 0x7f640c110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.034+0000 7f640b200640 1 --2- 172.21.15.42:0/3750436146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c074c50 0x7f640c10e160 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.034+0000 7f640b200640 1 --2- 172.21.15.42:0/3750436146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c074330 0x7f640c074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.034+0000 7f640b200640 1 -- 172.21.15.42:0/3750436146 >> 172.21.15.42:0/3750436146 conn(0x7f640c0fd930 msgr2=0x7f640c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.034+0000 7f640b200640 1 -- 172.21.15.42:0/3750436146 shutdown_connections 2024-04-20T10:42:12.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.034+0000 7f640b200640 1 -- 172.21.15.42:0/3750436146 wait complete. 2024-04-20T10:42:12.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.035+0000 7f640b200640 1 Processor -- start 2024-04-20T10:42:12.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.035+0000 7f640b200640 1 -- start start 2024-04-20T10:42:12.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f640b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c074330 0x7f640c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f640b200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c074c50 0x7f640c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f640b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f640c10e6a0 0x7f640c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f640b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f640c114400 con 0x7f640c10e6a0 2024-04-20T10:42:12.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f640b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f640c114280 con 0x7f640c074330 2024-04-20T10:42:12.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f640b200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f640c114580 con 0x7f640c074c50 2024-04-20T10:42:12.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f6409e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c074330 0x7f640c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f6401400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c074c50 0x7f640c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f6409e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c074330 0x7f640c1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59290/0 (socket says 172.21.15.42:59290) 2024-04-20T10:42:12.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f6409e00640 1 -- 172.21.15.42:0/2299712638 learned_addr learned my addr 172.21.15.42:0/2299712638 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:12.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f640a800640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f640c10e6a0 0x7f640c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f6401400640 1 -- 172.21.15.42:0/2299712638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c074330 msgr2=0x7f640c1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:12.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f6401400640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c074330 0x7f640c1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f6401400640 1 -- 172.21.15.42:0/2299712638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f640c10e6a0 msgr2=0x7f640c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:12.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.036+0000 7f6401400640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f640c10e6a0 0x7f640c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.037+0000 7f6401400640 1 -- 172.21.15.42:0/2299712638 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c19b000 con 0x7f640c074c50 2024-04-20T10:42:12.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.037+0000 7f6409e00640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c074330 0x7f640c1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:12.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.037+0000 7f6401400640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c074c50 0x7f640c1a0bc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f63f4037040 tx=0x7f63f40344c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:12.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.037+0000 7f6403400640 1 -- 172.21.15.42:0/2299712638 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4041020 con 0x7f640c074c50 2024-04-20T10:42:12.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.037+0000 7f6403400640 1 -- 172.21.15.42:0/2299712638 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f402ce50 con 0x7f640c074c50 2024-04-20T10:42:12.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.037+0000 7f6403400640 1 -- 172.21.15.42:0/2299712638 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4039040 con 0x7f640c074c50 2024-04-20T10:42:12.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.037+0000 7f640b200640 1 -- 172.21.15.42:0/2299712638 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c19b290 con 0x7f640c074c50 2024-04-20T10:42:12.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.037+0000 7f640b200640 1 -- 172.21.15.42:0/2299712638 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c1a7600 con 0x7f640c074c50 2024-04-20T10:42:12.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.041+0000 7f64139a2740 1 Processor -- start 2024-04-20T10:42:12.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.041+0000 7f6403400640 1 -- 172.21.15.42:0/2299712638 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f63f4044020 con 0x7f640c074c50 2024-04-20T10:42:12.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.042+0000 7f64139a2740 1 -- start start 2024-04-20T10:42:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.042+0000 7f64139a2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c5391760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.042+0000 7f64139a2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c5391ca0 0x5651c539a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.042+0000 7f64139a2740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c539a880 0x5651c539cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.042+0000 7f64139a2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5651c52f7150 con 0x5651c5391380 2024-04-20T10:42:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.042+0000 7f64139a2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5651c529d370 con 0x5651c5391ca0 2024-04-20T10:42:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.042+0000 7f64139a2740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5651c52f3110 con 0x5651c539a880 2024-04-20T10:42:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.042+0000 7f6409e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c5391ca0 0x5651c539a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.042+0000 7f6409e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c5391ca0 0x5651c539a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59292/0 (socket says 172.21.15.42:59292) 2024-04-20T10:42:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.042+0000 7f6409e00640 1 -- 172.21.15.42:0/2036382356 learned_addr learned my addr 172.21.15.42:0/2036382356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.042+0000 7f6401400640 1 --2- 172.21.15.42:0/2036382356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c539a880 0x5651c539cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.043+0000 7f640a800640 1 --2- 172.21.15.42:0/2036382356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c5391760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.043+0000 7f6409e00640 1 -- 172.21.15.42:0/2036382356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c539a880 msgr2=0x5651c539cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.043+0000 7f6409e00640 1 --2- 172.21.15.42:0/2036382356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c539a880 0x5651c539cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.043+0000 7f6409e00640 1 -- 172.21.15.42:0/2036382356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 msgr2=0x5651c5391760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.043+0000 7f6409e00640 1 --2- 172.21.15.42:0/2036382356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c5391760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.043+0000 7f6409e00640 1 -- 172.21.15.42:0/2036382356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651c50d37d0 con 0x5651c5391ca0 2024-04-20T10:42:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.043+0000 7f6409e00640 1 --2- 172.21.15.42:0/2036382356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c5391ca0 0x5651c539a340 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009c30 tx=0x7f63fc00ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c4e69162e781ed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.044+0000 7f6403400640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f63f00a3da0 0x7f63f00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.044+0000 7f63fb000640 1 -- 172.21.15.42:0/2036382356 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc00cdc0 con 0x5651c5391ca0 2024-04-20T10:42:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.044+0000 7f63fb000640 1 -- 172.21.15.42:0/2036382356 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63fc01d070 con 0x5651c5391ca0 2024-04-20T10:42:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.044+0000 7f63fb000640 1 -- 172.21.15.42:0/2036382356 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc00b660 con 0x5651c5391ca0 2024-04-20T10:42:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.044+0000 7f6403400640 1 -- 172.21.15.42:0/2299712638 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f63f40f1b40 con 0x7f640c074c50 2024-04-20T10:42:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.044+0000 7f640a800640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f63f00a3da0 0x7f63f00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.045+0000 7f64139a2740 1 -- 172.21.15.42:0/2036382356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c5391ca0 msgr2=0x5651c539a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.045+0000 7f64139a2740 1 --2- 172.21.15.42:0/2036382356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c5391ca0 0x5651c539a340 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009c30 tx=0x7f63fc00ede0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.045+0000 7f64139a2740 1 -- 172.21.15.42:0/2036382356 shutdown_connections 2024-04-20T10:42:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.045+0000 7f64139a2740 1 --2- 172.21.15.42:0/2036382356 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c539a880 0x5651c539cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.045+0000 7f64139a2740 1 --2- 172.21.15.42:0/2036382356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c5391ca0 0x5651c539a340 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.045+0000 7f64139a2740 1 --2- 172.21.15.42:0/2036382356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c5391760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.045+0000 7f64139a2740 1 -- 172.21.15.42:0/2036382356 >> 172.21.15.42:0/2036382356 conn(0x5651c538bab0 msgr2=0x5651c538be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.045+0000 7f640a800640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f63f00a3da0 0x7f63f00a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f640c06b4c0 tx=0x7f6404005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.045+0000 7f64139a2740 1 -- 172.21.15.42:0/2036382356 shutdown_connections 2024-04-20T10:42:12.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.045+0000 7f64139a2740 1 -- 172.21.15.42:0/2036382356 wait complete. 2024-04-20T10:42:12.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.046+0000 7f64139a2740 1 -- start start 2024-04-20T10:42:12.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.046+0000 7f64139a2740 1 Processor -- start 2024-04-20T10:42:12.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.046+0000 7f64139a2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c5397950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.046+0000 7f64139a2740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391ca0 0x5651c5397e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.046+0000 7f64139a2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c539a880 0x5651c53983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.046+0000 7f64139a2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5651c539d670 con 0x5651c5391380 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.046+0000 7f64139a2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5651c52b0740 con 0x5651c539a880 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.046+0000 7f64139a2740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5651c539d7f0 con 0x5651c5391ca0 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.046+0000 7f64139a2740 10 client.0 _ll_register_callbacks cb 0x5651c52abb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f6401400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391ca0 0x5651c5397e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f640a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c539a880 0x5651c53983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f6401400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391ca0 0x5651c5397e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:38116/0 (socket says 172.21.15.42:38116) 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f6401400640 1 -- 172.21.15.42:0/4141258160 learned_addr learned my addr 172.21.15.42:0/4141258160 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f6409e00640 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c5397950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f640a800640 1 -- 172.21.15.42:0/4141258160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391ca0 msgr2=0x5651c5397e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f640a800640 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391ca0 0x5651c5397e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f640a800640 1 -- 172.21.15.42:0/4141258160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 msgr2=0x5651c5397950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:12.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f640a800640 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c5397950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f640a800640 1 -- 172.21.15.42:0/4141258160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651c50093f0 con 0x5651c539a880 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f6409e00640 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c5397950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f6401400640 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391ca0 0x5651c5397e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f640a800640 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c539a880 0x5651c53983d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f640c075280 tx=0x7f6404004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f87941b85fb239d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.047+0000 7f63ef400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.048+0000 7f63ef400640 1 -- 172.21.15.42:0/4141258160 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640401e470 con 0x5651c539a880 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.048+0000 7f63ef400640 1 -- 172.21.15.42:0/4141258160 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f640401e610 con 0x5651c539a880 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.048+0000 7f64139a2740 1 -- 172.21.15.42:0/4141258160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63fc01d070 con 0x5651c539a880 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.048+0000 7f63ef400640 1 -- 172.21.15.42:0/4141258160 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f640401e810 con 0x5651c539a880 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.048+0000 7f64139a2740 10 client.26333 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.048+0000 7f64139a2740 1 -- 172.21.15.42:0/4141258160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5651c4fc90f0 con 0x5651c539a880 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.048+0000 7f63ef400640 1 -- 172.21.15.42:0/4141258160 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63fc01d070 con 0x5651c539a880 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.048+0000 7f63ef400640 1 -- 172.21.15.42:0/4141258160 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f6404027070 con 0x5651c539a880 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.049+0000 7f64139a2740 10 client.26333 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:12.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.049+0000 7f64139a2740 10 client.26333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.049+0000 7f64139a2740 20 client.26333 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.049+0000 7f64139a2740 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5651c53aad40 0x5651c53ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.049+0000 7f64139a2740 4 client.26333 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:42:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.049+0000 7f64139a2740 1 -- 172.21.15.42:0/4141258160 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651c5304b70 con 0x5651c53aad40 2024-04-20T10:42:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.049+0000 7f6409e00640 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5651c53aad40 0x5651c53ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.050+0000 7f6409e00640 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5651c53aad40 0x5651c53ad170 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f7f9d34a87e55f server_cookie=57e1241e9c916d0e in_seq=0 out_seq=0 2024-04-20T10:42:12.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.050+0000 7f63ef400640 10 client.26333 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:12.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f63ef400640 1 -- 172.21.15.42:0/4141258160 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651c5304b70 con 0x5651c53aad40 2024-04-20T10:42:12.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f63ef400640 10 client.26333 handle_command_reply: tid=0 2024-04-20T10:42:12.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f64139a2740 1 client.26333 shutdown 2024-04-20T10:42:12.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f64139a2740 1 -- 172.21.15.42:0/4141258160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c539a880 msgr2=0x5651c53983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:12.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f64139a2740 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c539a880 0x5651c53983d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f640c075280 tx=0x7f6404004730 comp rx=0 tx=0).stop 2024-04-20T10:42:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f64139a2740 1 -- 172.21.15.42:0/4141258160 shutdown_connections 2024-04-20T10:42:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f64139a2740 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5651c53aad40 0x5651c53ad170 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f64139a2740 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c539a880 0x5651c53983d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f64139a2740 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391ca0 0x5651c5397e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f64139a2740 1 --2- 172.21.15.42:0/4141258160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c5397950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f63ef400640 0 client.26333 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f64139a2740 1 -- 172.21.15.42:0/4141258160 >> 172.21.15.42:0/4141258160 conn(0x5651c538bab0 msgr2=0x5651c538d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f64139a2740 1 -- 172.21.15.42:0/4141258160 shutdown_connections 2024-04-20T10:42:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f64139a2740 1 -- 172.21.15.42:0/4141258160 wait complete. 2024-04-20T10:42:12.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.052+0000 7f64139a2740 20 client.26333 trim_cache size 0 max 16384 2024-04-20T10:42:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.074+0000 7f64139a2740 1 Processor -- start 2024-04-20T10:42:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.074+0000 7f64139a2740 1 -- start start 2024-04-20T10:42:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.074+0000 7f64139a2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c539a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.074+0000 7f64139a2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c5391ca0 0x5651c53a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.074+0000 7f64139a2740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c539a880 0x5651c5398020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.074+0000 7f64139a2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1ade30 con 0x5651c5391380 2024-04-20T10:42:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.074+0000 7f64139a2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1b0690 con 0x5651c5391ca0 2024-04-20T10:42:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.074+0000 7f64139a2740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1ae480 con 0x5651c539a880 2024-04-20T10:42:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f640a800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c539a880 0x5651c5398020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f6401400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c5391ca0 0x5651c53a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f640a800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c539a880 0x5651c5398020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:38128/0 (socket says 172.21.15.42:38128) 2024-04-20T10:42:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f6409e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c539a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f640a800640 1 -- 172.21.15.42:0/407861496 learned_addr learned my addr 172.21.15.42:0/407861496 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f640a800640 1 -- 172.21.15.42:0/407861496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c5391ca0 msgr2=0x5651c53a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f640a800640 1 --2- 172.21.15.42:0/407861496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c5391ca0 0x5651c53a6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f640a800640 1 -- 172.21.15.42:0/407861496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 msgr2=0x5651c539a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:12.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f640a800640 1 --2- 172.21.15.42:0/407861496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c539a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f640a800640 1 -- 172.21.15.42:0/407861496 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651c538b290 con 0x5651c539a880 2024-04-20T10:42:12.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f640a800640 1 --2- 172.21.15.42:0/407861496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c539a880 0x5651c5398020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6404004630 tx=0x7f6404009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e992fd2cbc0eb616 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:12.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f63fa600640 1 -- 172.21.15.42:0/407861496 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404031430 con 0x5651c539a880 2024-04-20T10:42:12.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f63fa600640 1 -- 172.21.15.42:0/407861496 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f640401e470 con 0x5651c539a880 2024-04-20T10:42:12.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.075+0000 7f63fa600640 1 -- 172.21.15.42:0/407861496 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404005020 con 0x5651c539a880 2024-04-20T10:42:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.076+0000 7f64139a2740 1 -- 172.21.15.42:0/407861496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c539a880 msgr2=0x5651c5398020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.076+0000 7f64139a2740 1 --2- 172.21.15.42:0/407861496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c539a880 0x5651c5398020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6404004630 tx=0x7f6404009620 comp rx=0 tx=0).stop 2024-04-20T10:42:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.076+0000 7f64139a2740 1 -- 172.21.15.42:0/407861496 shutdown_connections 2024-04-20T10:42:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.077+0000 7f64139a2740 1 --2- 172.21.15.42:0/407861496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c539a880 0x5651c5398020 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.077+0000 7f64139a2740 1 --2- 172.21.15.42:0/407861496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c5391ca0 0x5651c53a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.077+0000 7f64139a2740 1 --2- 172.21.15.42:0/407861496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391380 0x5651c539a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.077+0000 7f64139a2740 1 -- 172.21.15.42:0/407861496 >> 172.21.15.42:0/407861496 conn(0x5651c538bab0 msgr2=0x5651c5397910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.077+0000 7f64139a2740 1 -- 172.21.15.42:0/407861496 shutdown_connections 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.077+0000 7f64139a2740 1 -- 172.21.15.42:0/407861496 wait complete. 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.077+0000 7f64139a2740 1 -- start start 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.077+0000 7f64139a2740 1 Processor -- start 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f64139a2740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391380 0x5651c53a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f64139a2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391ca0 0x5651c5394410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f64139a2740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c539a880 0x5651c5394970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f64139a2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1aeb50 con 0x5651c5391ca0 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f64139a2740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1affc0 con 0x5651c539a880 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f64139a2740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1af220 con 0x5651c5391380 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f64139a2740 10 client.0 _ll_register_callbacks cb 0x7f6404009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f6409e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391380 0x5651c53a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f6409e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391380 0x5651c53a2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:38140/0 (socket says 172.21.15.42:38140) 2024-04-20T10:42:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f6409e00640 1 -- 172.21.15.42:0/870117495 learned_addr learned my addr 172.21.15.42:0/870117495 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f6401400640 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391ca0 0x5651c5394410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f640a800640 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c539a880 0x5651c5394970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f6409e00640 1 -- 172.21.15.42:0/870117495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c539a880 msgr2=0x5651c5394970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f6409e00640 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c539a880 0x5651c5394970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f6409e00640 1 -- 172.21.15.42:0/870117495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391ca0 msgr2=0x5651c5394410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f6409e00640 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391ca0 0x5651c5394410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f6409e00640 1 -- 172.21.15.42:0/870117495 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651c53951a0 con 0x5651c5391380 2024-04-20T10:42:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f640a800640 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c539a880 0x5651c5394970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:42:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.078+0000 7f6401400640 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391ca0 0x5651c5394410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:42:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.079+0000 7f6409e00640 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391380 0x5651c53a2eb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00e920 tx=0x7f63fc009d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=413bfd6f0e085899 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.079+0000 7f63ef400640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:42:12.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.079+0000 7f63ef400640 1 -- 172.21.15.42:0/870117495 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc00ce40 con 0x5651c5391380 2024-04-20T10:42:12.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.079+0000 7f63ef400640 1 -- 172.21.15.42:0/870117495 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f63fc02abc0 con 0x5651c5391380 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.079+0000 7f64139a2740 1 -- 172.21.15.42:0/870117495 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f640401e470 con 0x5651c5391380 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.079+0000 7f63ef400640 1 -- 172.21.15.42:0/870117495 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f63fc038050 con 0x5651c5391380 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.080+0000 7f64139a2740 10 client.26353 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.080+0000 7f64139a2740 1 -- 172.21.15.42:0/870117495 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5651c5411350 con 0x5651c5391380 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.080+0000 7f63ef400640 1 -- 172.21.15.42:0/870117495 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f640401e470 con 0x5651c5391380 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.080+0000 7f63ef400640 1 -- 172.21.15.42:0/870117495 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f63fc00b040 con 0x5651c5391380 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.080+0000 7f64139a2740 10 client.26353 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.080+0000 7f64139a2740 10 client.26353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.080+0000 7f64139a2740 20 client.26353 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.080+0000 7f64139a2740 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5651c54118a0 0x5651c5413cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.080+0000 7f64139a2740 4 client.26353 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.080+0000 7f64139a2740 1 -- 172.21.15.42:0/870117495 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651c5387170 con 0x5651c54118a0 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.080+0000 7f6401400640 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5651c54118a0 0x5651c5413cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.081+0000 7f6401400640 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5651c54118a0 0x5651c5413cc0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb848117825f24d4 server_cookie=7bfeb31f2d4dc0e in_seq=0 out_seq=0 2024-04-20T10:42:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.081+0000 7f63ef400640 10 client.26353 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f63ef400640 1 -- 172.21.15.42:0/870117495 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5651c5387170 con 0x5651c54118a0 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f63ef400640 10 client.26353 handle_command_reply: tid=0 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f64139a2740 1 client.26353 shutdown 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f64139a2740 1 -- 172.21.15.42:0/870117495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391380 msgr2=0x5651c53a2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f64139a2740 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391380 0x5651c53a2eb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00e920 tx=0x7f63fc009d70 comp rx=0 tx=0).stop 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f64139a2740 1 -- 172.21.15.42:0/870117495 shutdown_connections 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f64139a2740 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5651c54118a0 0x5651c5413cc0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f64139a2740 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5651c539a880 0x5651c5394970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f64139a2740 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5651c5391ca0 0x5651c5394410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f64139a2740 1 --2- 172.21.15.42:0/870117495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5651c5391380 0x5651c53a2eb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f64139a2740 1 -- 172.21.15.42:0/870117495 >> 172.21.15.42:0/870117495 conn(0x5651c538bab0 msgr2=0x5651c538ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f63ef400640 0 client.26353 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f64139a2740 1 -- 172.21.15.42:0/870117495 shutdown_connections 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.082+0000 7f64139a2740 1 -- 172.21.15.42:0/870117495 wait complete. 2024-04-20T10:42:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f64139a2740 20 client.26353 trim_cache size 0 max 16384 2024-04-20T10:42:12.086 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:42:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (3731 inodes in the stack)", 2024-04-20T10:42:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:42:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "9e68e6f8-78c9-4034-92eb-4562c94f03a2": { 2024-04-20T10:42:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:42:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "9e68e6f8-78c9-4034-92eb-4562c94f03a2", 2024-04-20T10:42:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:42:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:42:12.086 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:42:12.087 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:42:12.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f640b200640 1 -- 172.21.15.42:0/2299712638 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f63f00a3da0 msgr2=0x7f63f00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:12.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f640b200640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f63f00a3da0 0x7f63f00a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f640c06b4c0 tx=0x7f6404005f90 comp rx=0 tx=0).stop 2024-04-20T10:42:12.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f640b200640 1 -- 172.21.15.42:0/2299712638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c074c50 msgr2=0x7f640c1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:12.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f640b200640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c074c50 0x7f640c1a0bc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f63f4037040 tx=0x7f63f40344c0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f640b200640 1 -- 172.21.15.42:0/2299712638 shutdown_connections 2024-04-20T10:42:12.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f640b200640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f63f00a3da0 0x7f63f00a6260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f640b200640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f640c10e6a0 0x7f640c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f640b200640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f640c074c50 0x7f640c1a0bc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f640b200640 1 --2- 172.21.15.42:0/2299712638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c074330 0x7f640c1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:12.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f640b200640 1 -- 172.21.15.42:0/2299712638 >> 172.21.15.42:0/2299712638 conn(0x7f640c0fd930 msgr2=0x7f640c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:12.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f640b200640 1 -- 172.21.15.42:0/2299712638 shutdown_connections 2024-04-20T10:42:12.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:12.083+0000 7f640b200640 1 -- 172.21.15.42:0/2299712638 wait complete. 2024-04-20T10:42:12.102 INFO:tasks.cephfs.filesystem:scrub status for tag:9e68e6f8-78c9-4034-92eb-4562c94f03a2 - {'path': '/', 'tag': '9e68e6f8-78c9-4034-92eb-4562c94f03a2', 'options': 'recursive,force'} 2024-04-20T10:42:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:11 smithi162 ceph-mon[21854]: pgmap v252: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 617 op/s 2024-04-20T10:42:12.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:11 smithi046 ceph-mon[21574]: pgmap v252: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 617 op/s 2024-04-20T10:42:12.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:11 smithi042 ceph-mon[17078]: pgmap v252: 97 pgs: 97 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 617 op/s 2024-04-20T10:42:12.946 INFO:tasks.workunit.client.0.smithi042.stdout:# 2024-04-20T10:42:12.946 INFO:tasks.workunit.client.0.smithi042.stdout:# configuration written to .config 2024-04-20T10:42:12.946 INFO:tasks.workunit.client.0.smithi042.stdout:# 2024-04-20T10:42:12.964 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c processor /proc/cpuinfo 2024-04-20T10:42:12.968 INFO:tasks.workunit.client.0.smithi042.stderr:+ make -j8 2024-04-20T10:42:13.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:12 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:42:13.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:12 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:42:13.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:12 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:42:13.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:12 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:42:13.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:12 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:42:13.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:12 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:42:13.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:12 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:42:13.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:12 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:42:13.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:12 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:42:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:13 smithi162 ceph-mon[21854]: pgmap v253: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.6 MiB/s wr, 719 op/s 2024-04-20T10:42:14.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:13 smithi046 ceph-mon[21574]: pgmap v253: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.6 MiB/s wr, 719 op/s 2024-04-20T10:42:14.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:13 smithi042 ceph-mon[17078]: pgmap v253: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.6 MiB/s wr, 719 op/s 2024-04-20T10:42:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:15 smithi162 ceph-mon[21854]: pgmap v254: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.1 MiB/s wr, 677 op/s 2024-04-20T10:42:16.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:15 smithi046 ceph-mon[21574]: pgmap v254: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.1 MiB/s wr, 677 op/s 2024-04-20T10:42:16.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:15 smithi042 ceph-mon[17078]: pgmap v254: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.1 MiB/s wr, 677 op/s 2024-04-20T10:42:17.253 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/bpf_perf_event.h 2024-04-20T10:42:17.253 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/errno.h 2024-04-20T10:42:17.255 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/fcntl.h 2024-04-20T10:42:17.256 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/ioctl.h 2024-04-20T10:42:17.256 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/ioctls.h 2024-04-20T10:42:17.259 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/ipcbuf.h 2024-04-20T10:42:17.259 INFO:tasks.workunit.client.0.smithi042.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_32.h 2024-04-20T10:42:17.262 INFO:tasks.workunit.client.0.smithi042.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_64.h 2024-04-20T10:42:17.264 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/param.h 2024-04-20T10:42:17.267 INFO:tasks.workunit.client.0.smithi042.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_x32.h 2024-04-20T10:42:17.267 INFO:tasks.workunit.client.0.smithi042.stdout: SYSTBL arch/x86/include/generated/asm/syscalls_32.h 2024-04-20T10:42:17.269 INFO:tasks.workunit.client.0.smithi042.stdout: GEN arch/x86/include/generated/asm/orc_hash.h 2024-04-20T10:42:17.271 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/poll.h 2024-04-20T10:42:17.276 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/resource.h 2024-04-20T10:42:17.293 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/socket.h 2024-04-20T10:42:17.299 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/sockios.h 2024-04-20T10:42:17.308 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/termbits.h 2024-04-20T10:42:17.310 INFO:tasks.workunit.client.0.smithi042.stdout: SYSHDR arch/x86/include/generated/asm/unistd_32_ia32.h 2024-04-20T10:42:17.317 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/termios.h 2024-04-20T10:42:17.323 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/uapi/asm/types.h 2024-04-20T10:42:17.332 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC arch/x86/tools/relocs_32.o 2024-04-20T10:42:17.333 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC arch/x86/tools/relocs_64.o 2024-04-20T10:42:17.345 INFO:tasks.workunit.client.0.smithi042.stdout: SYSHDR arch/x86/include/generated/asm/unistd_64_x32.h 2024-04-20T10:42:17.353 INFO:tasks.workunit.client.0.smithi042.stdout: SYSTBL arch/x86/include/generated/asm/syscalls_64.h 2024-04-20T10:42:17.366 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC arch/x86/tools/relocs_common.o 2024-04-20T10:42:17.411 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/asm/early_ioremap.h 2024-04-20T10:42:17.419 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/asm/export.h 2024-04-20T10:42:17.420 INFO:tasks.workunit.client.0.smithi042.stdout: UPD include/generated/uapi/linux/version.h 2024-04-20T10:42:17.422 INFO:tasks.workunit.client.0.smithi042.stdout: UPD include/config/kernel.release 2024-04-20T10:42:17.426 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/asm/mcs_spinlock.h 2024-04-20T10:42:17.436 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/asm/irq_regs.h 2024-04-20T10:42:17.442 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/asm/kmap_size.h 2024-04-20T10:42:17.445 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/asm/local64.h 2024-04-20T10:42:17.450 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/asm/mmiowb.h 2024-04-20T10:42:17.457 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/asm/module.lds.h 2024-04-20T10:42:17.464 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/asm/rwonce.h 2024-04-20T10:42:17.467 INFO:tasks.workunit.client.0.smithi042.stdout: UPD include/generated/compile.h 2024-04-20T10:42:17.470 INFO:tasks.workunit.client.0.smithi042.stdout: WRAP arch/x86/include/generated/asm/unaligned.h 2024-04-20T10:42:17.495 INFO:tasks.workunit.client.0.smithi042.stdout: UPD include/generated/utsrelease.h 2024-04-20T10:42:17.503 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/kallsyms 2024-04-20T10:42:17.505 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/sorttable 2024-04-20T10:42:17.512 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/asn1_compiler 2024-04-20T10:42:17.547 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/selinux/genheaders/genheaders 2024-04-20T10:42:17.872 INFO:tasks.workunit.client.0.smithi042.stdout: DESCEND objtool 2024-04-20T10:42:17.909 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTLD arch/x86/tools/relocs 2024-04-20T10:42:17.925 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/selinux/mdp/mdp 2024-04-20T10:42:18.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:17 smithi162 ceph-mon[21854]: pgmap v255: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 486 op/s 2024-04-20T10:42:18.164 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep.o 2024-04-20T10:42:18.238 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTLD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep-in.o 2024-04-20T10:42:18.251 INFO:tasks.workunit.client.0.smithi042.stdout: LINK /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep 2024-04-20T10:42:18.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:17 smithi046 ceph-mon[21574]: pgmap v255: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 486 op/s 2024-04-20T10:42:18.339 INFO:tasks.workunit.client.0.smithi042.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/exec-cmd.h 2024-04-20T10:42:18.340 INFO:tasks.workunit.client.0.smithi042.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/help.h 2024-04-20T10:42:18.341 INFO:tasks.workunit.client.0.smithi042.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/pager.h 2024-04-20T10:42:18.343 INFO:tasks.workunit.client.0.smithi042.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/run-command.h 2024-04-20T10:42:18.344 INFO:tasks.workunit.client.0.smithi042.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/parse-options.h 2024-04-20T10:42:18.352 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:17 smithi042 ceph-mon[17078]: pgmap v255: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 486 op/s 2024-04-20T10:42:18.374 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/exec-cmd.o 2024-04-20T10:42:18.377 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/help.o 2024-04-20T10:42:18.378 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/pager.o 2024-04-20T10:42:18.380 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/parse-options.o 2024-04-20T10:42:18.387 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/run-command.o 2024-04-20T10:42:18.390 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/sigchain.o 2024-04-20T10:42:18.394 INFO:tasks.workunit.client.0.smithi042.stdout: INSTALL libsubcmd_headers 2024-04-20T10:42:18.401 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/subcmd-config.o 2024-04-20T10:42:18.951 INFO:tasks.workunit.client.0.smithi042.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/libsubcmd-in.o 2024-04-20T10:42:18.991 INFO:tasks.workunit.client.0.smithi042.stdout: AR /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/libsubcmd.a 2024-04-20T10:42:20.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:19 smithi046 ceph-mon[21574]: pgmap v256: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 518 op/s 2024-04-20T10:42:20.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:19 smithi042 ceph-mon[17078]: pgmap v256: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 518 op/s 2024-04-20T10:42:20.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:19 smithi162 ceph-mon[21854]: pgmap v256: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 518 op/s 2024-04-20T10:42:22.237 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/weak.o 2024-04-20T10:42:22.240 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/check.o 2024-04-20T10:42:22.240 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/special.o 2024-04-20T10:42:22.244 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/builtin-check.o 2024-04-20T10:42:22.246 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/elf.o 2024-04-20T10:42:22.250 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/special.o 2024-04-20T10:42:22.252 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool.o 2024-04-20T10:42:22.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:21 smithi046 ceph-mon[21574]: pgmap v257: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 448 op/s 2024-04-20T10:42:22.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:21 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:42:22.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:21 smithi042 ceph-mon[17078]: pgmap v257: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 448 op/s 2024-04-20T10:42:22.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:21 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:42:22.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:21 smithi162 ceph-mon[21854]: pgmap v257: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 448 op/s 2024-04-20T10:42:22.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:21 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:42:22.466 INFO:tasks.workunit.client.0.smithi042.stdout: MKDIR /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/lib/ 2024-04-20T10:42:22.483 INFO:tasks.workunit.client.0.smithi042.stdout: GEN /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/lib/inat-tables.c 2024-04-20T10:42:22.549 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/orc_gen.o 2024-04-20T10:42:22.562 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/decode.o 2024-04-20T10:42:22.584 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/orc_dump.o 2024-04-20T10:42:22.802 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libstring.o 2024-04-20T10:42:22.807 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libctype.o 2024-04-20T10:42:22.968 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/str_error_r.o 2024-04-20T10:42:23.096 INFO:tasks.workunit.client.0.smithi042.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/librbtree.o 2024-04-20T10:42:23.528 INFO:tasks.workunit.client.0.smithi042.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/objtool-in.o 2024-04-20T10:42:24.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:23 smithi046 ceph-mon[21574]: pgmap v258: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 1.17k op/s 2024-04-20T10:42:24.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:23 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:42:24.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:23 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:42:24.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:23 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:42:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:23 smithi042 ceph-mon[17078]: pgmap v258: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 1.17k op/s 2024-04-20T10:42:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:23 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:42:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:23 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:42:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:23 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:42:24.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:23 smithi162 ceph-mon[21854]: pgmap v258: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 1.17k op/s 2024-04-20T10:42:24.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:23 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:42:24.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:23 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:42:24.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:23 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:42:25.084 INFO:tasks.workunit.client.0.smithi042.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool-in.o 2024-04-20T10:42:25.133 INFO:tasks.workunit.client.0.smithi042.stdout: LINK /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool 2024-04-20T10:42:25.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:25 smithi046 ceph-mon[21574]: pgmap v259: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 1.11k op/s 2024-04-20T10:42:25.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:25 smithi042 ceph-mon[17078]: pgmap v259: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 1.11k op/s 2024-04-20T10:42:25.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:25 smithi162 ceph-mon[21854]: pgmap v259: 97 pgs: 97 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 1.11k op/s 2024-04-20T10:42:27.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:27 smithi162 ceph-mon[21854]: pgmap v260: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 1.44k op/s 2024-04-20T10:42:28.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:27 smithi046 ceph-mon[21574]: pgmap v260: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 1.44k op/s 2024-04-20T10:42:28.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:27 smithi042 ceph-mon[17078]: pgmap v260: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 1.44k op/s 2024-04-20T10:42:29.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:29 smithi162 ceph-mon[21854]: pgmap v261: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 2.10k op/s 2024-04-20T10:42:30.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:29 smithi046 ceph-mon[21574]: pgmap v261: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 2.10k op/s 2024-04-20T10:42:30.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:29 smithi042 ceph-mon[17078]: pgmap v261: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 2.10k op/s 2024-04-20T10:42:31.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:31 smithi162 ceph-mon[21854]: pgmap v262: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 1.89k op/s 2024-04-20T10:42:32.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:31 smithi046 ceph-mon[21574]: pgmap v262: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 1.89k op/s 2024-04-20T10:42:32.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:31 smithi042 ceph-mon[17078]: pgmap v262: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 1.89k op/s 2024-04-20T10:42:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:33 smithi162 ceph-mon[21854]: pgmap v263: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 3.22k op/s 2024-04-20T10:42:34.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:33 smithi046 ceph-mon[21574]: pgmap v263: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 3.22k op/s 2024-04-20T10:42:34.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:33 smithi042 ceph-mon[17078]: pgmap v263: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 3.22k op/s 2024-04-20T10:42:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:35 smithi162 ceph-mon[21854]: pgmap v264: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 2.63k op/s 2024-04-20T10:42:35.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:35 smithi042 ceph-mon[17078]: pgmap v264: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 2.63k op/s 2024-04-20T10:42:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:35 smithi046 ceph-mon[21574]: pgmap v264: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 2.63k op/s 2024-04-20T10:42:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:37 smithi162 ceph-mon[21854]: pgmap v265: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.1 MiB/s wr, 3.12k op/s 2024-04-20T10:42:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:37 smithi046 ceph-mon[21574]: pgmap v265: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.1 MiB/s wr, 3.12k op/s 2024-04-20T10:42:38.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:37 smithi042 ceph-mon[17078]: pgmap v265: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.1 MiB/s wr, 3.12k op/s 2024-04-20T10:42:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:39 smithi046 ceph-mon[21574]: pgmap v266: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.2 MiB/s wr, 3.93k op/s 2024-04-20T10:42:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:39 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:42:40.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:39 smithi042 ceph-mon[17078]: pgmap v266: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.2 MiB/s wr, 3.93k op/s 2024-04-20T10:42:40.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:39 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:42:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:39 smithi162 ceph-mon[21854]: pgmap v266: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.2 MiB/s wr, 3.93k op/s 2024-04-20T10:42:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:39 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:42:41.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:40 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:42:41.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:40 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:42:41.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:40 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:42:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:41 smithi046 ceph-mon[21574]: pgmap v267: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1024 KiB/s wr, 3.20k op/s 2024-04-20T10:42:42.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:42:42.103 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:42:42.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:41 smithi042 ceph-mon[17078]: pgmap v267: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1024 KiB/s wr, 3.20k op/s 2024-04-20T10:42:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:41 smithi162 ceph-mon[21854]: pgmap v267: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1024 KiB/s wr, 3.20k op/s 2024-04-20T10:42:42.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.335+0000 7f63cd800640 1 -- 172.21.15.42:0/2438292634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c8074c20 msgr2=0x7f63c81128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.335+0000 7f63cd800640 1 --2- 172.21.15.42:0/2438292634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c8074c20 0x7f63c81128d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009900 tx=0x7f63b402bcd0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.335+0000 7f63cd800640 1 -- 172.21.15.42:0/2438292634 shutdown_connections 2024-04-20T10:42:42.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.335+0000 7f63cd800640 1 --2- 172.21.15.42:0/2438292634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c8112e10 0x7f63c8115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.335+0000 7f63cd800640 1 --2- 172.21.15.42:0/2438292634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c8074c20 0x7f63c81128d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.335+0000 7f63cd800640 1 --2- 172.21.15.42:0/2438292634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c8074300 0x7f63c80746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.335+0000 7f63cd800640 1 -- 172.21.15.42:0/2438292634 >> 172.21.15.42:0/2438292634 conn(0x7f63c80fd930 msgr2=0x7f63c80ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:42.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.335+0000 7f63cd800640 1 -- 172.21.15.42:0/2438292634 shutdown_connections 2024-04-20T10:42:42.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.335+0000 7f63cd800640 1 -- 172.21.15.42:0/2438292634 wait complete. 2024-04-20T10:42:42.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.336+0000 7f63cd800640 1 Processor -- start 2024-04-20T10:42:42.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.336+0000 7f63cd800640 1 -- start start 2024-04-20T10:42:42.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.337+0000 7f63cd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c8074300 0x7f63c81a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.337+0000 7f63cd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c8074c20 0x7f63c81a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.337+0000 7f63cd800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c8112e10 0x7f63c819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.337+0000 7f63cd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8118830 con 0x7f63c8074300 2024-04-20T10:42:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.337+0000 7f63cd800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81186b0 con 0x7f63c8074c20 2024-04-20T10:42:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.337+0000 7f63cd800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81189b0 con 0x7f63c8112e10 2024-04-20T10:42:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.337+0000 7f63c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c8074c20 0x7f63c81a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.337+0000 7f63c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c8074c20 0x7f63c81a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59198/0 (socket says 172.21.15.42:59198) 2024-04-20T10:42:42.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.337+0000 7f63c7400640 1 -- 172.21.15.42:0/1591009915 learned_addr learned my addr 172.21.15.42:0/1591009915 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:42.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.337+0000 7f63c7e00640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c8074300 0x7f63c81a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.337+0000 7f63cce00640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c8112e10 0x7f63c819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.338+0000 7f63c7400640 1 -- 172.21.15.42:0/1591009915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c8112e10 msgr2=0x7f63c819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.338+0000 7f63c7400640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c8112e10 0x7f63c819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.338+0000 7f63c7400640 1 -- 172.21.15.42:0/1591009915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c8074300 msgr2=0x7f63c81a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.338+0000 7f63c7400640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c8074300 0x7f63c81a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.338+0000 7f63c7400640 1 -- 172.21.15.42:0/1591009915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c819f3f0 con 0x7f63c8074c20 2024-04-20T10:42:42.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.338+0000 7f63cce00640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c8112e10 0x7f63c819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:42.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.338+0000 7f63c7400640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c8074c20 0x7f63c81a4fb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f63b4000c00 tx=0x7f63b4004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:42.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.339+0000 7f63c4c00640 1 -- 172.21.15.42:0/1591009915 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4042020 con 0x7f63c8074c20 2024-04-20T10:42:42.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.339+0000 7f63c4c00640 1 -- 172.21.15.42:0/1591009915 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b40054e0 con 0x7f63c8074c20 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.339+0000 7f63cd800640 1 -- 172.21.15.42:0/1591009915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c819f680 con 0x7f63c8074c20 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.339+0000 7f63c4c00640 1 -- 172.21.15.42:0/1591009915 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4003b20 con 0x7f63c8074c20 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.339+0000 7f63cd800640 1 -- 172.21.15.42:0/1591009915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c81ab9f0 con 0x7f63c8074c20 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.342+0000 7f63d1a5d740 1 Processor -- start 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63d1a5d740 1 -- start start 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63c4c00640 1 -- 172.21.15.42:0/1591009915 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f63b4045020 con 0x7f63c8074c20 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63d1a5d740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c07b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63d1a5d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c07bca0 0x559b8c084340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63d1a5d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c084880 0x559b8c086c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63d1a5d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559b8bfe1150 con 0x559b8c07bca0 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63d1a5d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559b8bf87370 con 0x559b8c084880 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63d1a5d740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559b8bfdd110 con 0x559b8c07b380 2024-04-20T10:42:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c084880 0x559b8c086c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63c7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c07bca0 0x559b8c084340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c084880 0x559b8c086c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59210/0 (socket says 172.21.15.42:59210) 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63c7400640 1 -- 172.21.15.42:0/405221145 learned_addr learned my addr 172.21.15.42:0/405221145 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.343+0000 7f63cce00640 1 --2- 172.21.15.42:0/405221145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.344+0000 7f63c7400640 1 -- 172.21.15.42:0/405221145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 msgr2=0x559b8c07b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.344+0000 7f63c7400640 1 --2- 172.21.15.42:0/405221145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c07b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.344+0000 7f63c7400640 1 -- 172.21.15.42:0/405221145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c07bca0 msgr2=0x559b8c084340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.344+0000 7f63c7400640 1 --2- 172.21.15.42:0/405221145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c07bca0 0x559b8c084340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.344+0000 7f63c7400640 1 -- 172.21.15.42:0/405221145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b8bdbd7d0 con 0x559b8c084880 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.344+0000 7f63c7400640 1 --2- 172.21.15.42:0/405221145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c084880 0x559b8c086c70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f63b40efa00 tx=0x7f63b40fa3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12b1b142d512db32 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.345+0000 7f63bd600640 1 -- 172.21.15.42:0/405221145 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40f40d0 con 0x559b8c084880 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.345+0000 7f63bd600640 1 -- 172.21.15.42:0/405221145 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b40ff030 con 0x559b8c084880 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.345+0000 7f63bd600640 1 -- 172.21.15.42:0/405221145 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b41036b0 con 0x559b8c084880 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.345+0000 7f63c4c00640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f63ac0a3da0 0x7f63ac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.345+0000 7f63c4c00640 1 -- 172.21.15.42:0/1591009915 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f63b40f2260 con 0x7f63c8074c20 2024-04-20T10:42:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.345+0000 7f63cce00640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f63ac0a3da0 0x7f63ac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.345+0000 7f63d1a5d740 1 -- 172.21.15.42:0/405221145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c084880 msgr2=0x559b8c086c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.345+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/405221145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c084880 0x559b8c086c70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f63b40efa00 tx=0x7f63b40fa3f0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.346+0000 7f63d1a5d740 1 -- 172.21.15.42:0/405221145 shutdown_connections 2024-04-20T10:42:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.346+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/405221145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c084880 0x559b8c086c70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.346+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/405221145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c07bca0 0x559b8c084340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.346+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/405221145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c07b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.346+0000 7f63d1a5d740 1 -- 172.21.15.42:0/405221145 >> 172.21.15.42:0/405221145 conn(0x559b8c075ab0 msgr2=0x559b8c075e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.346+0000 7f63cce00640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f63ac0a3da0 0x7f63ac0a6260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009fd0 tx=0x7f63c0009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.346+0000 7f63d1a5d740 1 -- 172.21.15.42:0/405221145 shutdown_connections 2024-04-20T10:42:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.346+0000 7f63d1a5d740 1 -- 172.21.15.42:0/405221145 wait complete. 2024-04-20T10:42:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.346+0000 7f63d1a5d740 1 -- start start 2024-04-20T10:42:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.347+0000 7f63d1a5d740 1 Processor -- start 2024-04-20T10:42:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.347+0000 7f63d1a5d740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c0819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.347+0000 7f63d1a5d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c081ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.347+0000 7f63d1a5d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c084880 0x559b8c082430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.347+0000 7f63d1a5d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559b8c087430 con 0x559b8c084880 2024-04-20T10:42:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.347+0000 7f63d1a5d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559b8bf9a740 con 0x559b8c07bca0 2024-04-20T10:42:42.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.347+0000 7f63d1a5d740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559b8c0875b0 con 0x559b8c07b380 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.347+0000 7f63d1a5d740 10 client.0 _ll_register_callbacks cb 0x559b8bf95b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.347+0000 7f63c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c081ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.348+0000 7f63c7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c0819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.348+0000 7f63c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c081ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59218/0 (socket says 172.21.15.42:59218) 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.348+0000 7f63c7400640 1 -- 172.21.15.42:0/155391822 learned_addr learned my addr 172.21.15.42:0/155391822 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.348+0000 7f63cce00640 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c084880 0x559b8c082430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.348+0000 7f63c7e00640 1 -- 172.21.15.42:0/155391822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 msgr2=0x559b8c081ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.348+0000 7f63c7e00640 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c081ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.348+0000 7f63c7e00640 1 -- 172.21.15.42:0/155391822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c084880 msgr2=0x559b8c082430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.348+0000 7f63c7e00640 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c084880 0x559b8c082430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.348+0000 7f63c7e00640 1 -- 172.21.15.42:0/155391822 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b8bcf33f0 con 0x559b8c07b380 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.348+0000 7f63c7400640 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c081ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.348+0000 7f63c7e00640 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c0819b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f63b80126d0 tx=0x7f63b8012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33dbbe2befd63536 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.348+0000 7f63b2000640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.349+0000 7f63b2000640 1 -- 172.21.15.42:0/155391822 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b800f8c0 con 0x559b8c07b380 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.349+0000 7f63b2000640 1 -- 172.21.15.42:0/155391822 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f63b800fa60 con 0x559b8c07b380 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.349+0000 7f63d1a5d740 1 -- 172.21.15.42:0/155391822 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b40ff030 con 0x559b8c07b380 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.349+0000 7f63b2000640 1 -- 172.21.15.42:0/155391822 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f63b800fc60 con 0x559b8c07b380 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.349+0000 7f63d1a5d740 10 client.26365 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.349+0000 7f63d1a5d740 1 -- 172.21.15.42:0/155391822 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559b8bcb30f0 con 0x559b8c07b380 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.349+0000 7f63b2000640 1 -- 172.21.15.42:0/155391822 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b40ff030 con 0x559b8c07b380 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.350+0000 7f63b2000640 1 -- 172.21.15.42:0/155391822 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f63b8010430 con 0x559b8c07b380 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.350+0000 7f63d1a5d740 10 client.26365 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.350+0000 7f63d1a5d740 10 client.26365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.350+0000 7f63d1a5d740 20 client.26365 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.350+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559b8c094af0 0x559b8c096f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.350+0000 7f63d1a5d740 4 client.26365 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:42:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.350+0000 7f63d1a5d740 1 -- 172.21.15.42:0/155391822 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b8bfeeb70 con 0x559b8c094af0 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.350+0000 7f63c7400640 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559b8c094af0 0x559b8c096f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.351+0000 7f63c7400640 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559b8c094af0 0x559b8c096f80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f972d6c35f1bf105 server_cookie=1aa8db0c9ce755c2 in_seq=0 out_seq=0 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.351+0000 7f63b2000640 10 client.26365 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.351+0000 7f63b2000640 1 -- 172.21.15.42:0/155391822 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b8bfeeb70 con 0x559b8c094af0 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.351+0000 7f63b2000640 10 client.26365 handle_command_reply: tid=0 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63d1a5d740 1 client.26365 shutdown 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63d1a5d740 1 -- 172.21.15.42:0/155391822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 msgr2=0x559b8c0819b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c0819b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f63b80126d0 tx=0x7f63b8012b90 comp rx=0 tx=0).stop 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63d1a5d740 1 -- 172.21.15.42:0/155391822 shutdown_connections 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559b8c094af0 0x559b8c096f80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c084880 0x559b8c082430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c081ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63b2000640 0 client.26365 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/155391822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c0819b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63d1a5d740 1 -- 172.21.15.42:0/155391822 >> 172.21.15.42:0/155391822 conn(0x559b8c075ab0 msgr2=0x559b8c077800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:42.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63d1a5d740 1 -- 172.21.15.42:0/155391822 shutdown_connections 2024-04-20T10:42:42.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63d1a5d740 1 -- 172.21.15.42:0/155391822 wait complete. 2024-04-20T10:42:42.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.352+0000 7f63d1a5d740 20 client.26365 trim_cache size 0 max 16384 2024-04-20T10:42:42.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.372+0000 7f63d1a5d740 1 Processor -- start 2024-04-20T10:42:42.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.372+0000 7f63d1a5d740 1 -- start start 2024-04-20T10:42:42.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.372+0000 7f63d1a5d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c07b380 0x559b8c0840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.372+0000 7f63d1a5d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c0900d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.372+0000 7f63d1a5d740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c084880 0x559b8c074990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.372+0000 7f63d1a5d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81b2270 con 0x559b8c07b380 2024-04-20T10:42:42.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.372+0000 7f63d1a5d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81b4ad0 con 0x559b8c07bca0 2024-04-20T10:42:42.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.372+0000 7f63d1a5d740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81b28c0 con 0x559b8c084880 2024-04-20T10:42:42.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.373+0000 7f63c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c0900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.373+0000 7f63cce00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c084880 0x559b8c074990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.373+0000 7f63c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c0900d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59228/0 (socket says 172.21.15.42:59228) 2024-04-20T10:42:42.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.373+0000 7f63c7400640 1 -- 172.21.15.42:0/2089883701 learned_addr learned my addr 172.21.15.42:0/2089883701 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:42.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.373+0000 7f63c7e00640 1 --2- 172.21.15.42:0/2089883701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c07b380 0x559b8c0840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.373+0000 7f63c7400640 1 -- 172.21.15.42:0/2089883701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c084880 msgr2=0x559b8c074990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:42.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.373+0000 7f63c7400640 1 --2- 172.21.15.42:0/2089883701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c084880 0x559b8c074990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.373+0000 7f63c7400640 1 -- 172.21.15.42:0/2089883701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c07b380 msgr2=0x559b8c0840e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.373+0000 7f63c7400640 1 --2- 172.21.15.42:0/2089883701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c07b380 0x559b8c0840e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.373+0000 7f63c7400640 1 -- 172.21.15.42:0/2089883701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b8c086fe0 con 0x559b8c07bca0 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.373+0000 7f63c7400640 1 --2- 172.21.15.42:0/2089883701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c0900d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f63b40f4530 tx=0x7f63b40f44c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27f371c29f961600 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.374+0000 7f63bcc00640 1 -- 172.21.15.42:0/2089883701 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40f2070 con 0x559b8c07bca0 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.374+0000 7f63bcc00640 1 -- 172.21.15.42:0/2089883701 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b40ff040 con 0x559b8c07bca0 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.374+0000 7f63bcc00640 1 -- 172.21.15.42:0/2089883701 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40bdc40 con 0x559b8c07bca0 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.374+0000 7f63d1a5d740 1 -- 172.21.15.42:0/2089883701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 msgr2=0x559b8c0900d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.374+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/2089883701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c0900d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f63b40f4530 tx=0x7f63b40f44c0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.375+0000 7f63d1a5d740 1 -- 172.21.15.42:0/2089883701 shutdown_connections 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.375+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/2089883701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c084880 0x559b8c074990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.375+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/2089883701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c0900d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.375+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/2089883701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c07b380 0x559b8c0840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.375+0000 7f63d1a5d740 1 -- 172.21.15.42:0/2089883701 >> 172.21.15.42:0/2089883701 conn(0x559b8c075ab0 msgr2=0x559b8c074280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.375+0000 7f63d1a5d740 1 -- 172.21.15.42:0/2089883701 shutdown_connections 2024-04-20T10:42:42.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.375+0000 7f63d1a5d740 1 -- 172.21.15.42:0/2089883701 wait complete. 2024-04-20T10:42:42.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.375+0000 7f63d1a5d740 1 -- start start 2024-04-20T10:42:42.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.375+0000 7f63d1a5d740 1 Processor -- start 2024-04-20T10:42:42.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.376+0000 7f63d1a5d740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c08c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.376+0000 7f63d1a5d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c087c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.376+0000 7f63d1a5d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c084880 0x559b8c088160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.376+0000 7f63d1a5d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81b2f90 con 0x559b8c084880 2024-04-20T10:42:42.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.376+0000 7f63d1a5d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81b4400 con 0x559b8c07bca0 2024-04-20T10:42:42.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.376+0000 7f63d1a5d740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81b3660 con 0x559b8c07b380 2024-04-20T10:42:42.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.376+0000 7f63d1a5d740 10 client.0 _ll_register_callbacks cb 0x7f63b8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.376+0000 7f63c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c087c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.376+0000 7f63c7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c08c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.376+0000 7f63c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c087c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59236/0 (socket says 172.21.15.42:59236) 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.376+0000 7f63c7400640 1 -- 172.21.15.42:0/1477898820 learned_addr learned my addr 172.21.15.42:0/1477898820 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.376+0000 7f63cce00640 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c084880 0x559b8c088160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63c7400640 1 -- 172.21.15.42:0/1477898820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 msgr2=0x559b8c08c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63c7400640 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c08c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63c7400640 1 -- 172.21.15.42:0/1477898820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c084880 msgr2=0x559b8c088160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63c7400640 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c084880 0x559b8c088160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63c7400640 1 -- 172.21.15.42:0/1477898820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b8c088850 con 0x559b8c07bca0 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63cce00640 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c084880 0x559b8c088160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63c7e00640 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c08c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63c7400640 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c087c20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f63b4006010 tx=0x7f63b4004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a65c04c32d32588 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63b2000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:42:42.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63b2000640 1 -- 172.21.15.42:0/1477898820 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40f2040 con 0x559b8c07bca0 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63b2000640 1 -- 172.21.15.42:0/1477898820 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f63b40fae00 con 0x559b8c07bca0 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63d1a5d740 1 -- 172.21.15.42:0/1477898820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b40ff040 con 0x559b8c07bca0 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63b2000640 1 -- 172.21.15.42:0/1477898820 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f63b40bdc40 con 0x559b8c07bca0 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63d1a5d740 10 client.26357 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.377+0000 7f63d1a5d740 1 -- 172.21.15.42:0/1477898820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559b8c0fafd0 con 0x559b8c07bca0 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.378+0000 7f63b2000640 1 -- 172.21.15.42:0/1477898820 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b40ff040 con 0x559b8c07bca0 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.378+0000 7f63b2000640 1 -- 172.21.15.42:0/1477898820 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f63b40ba950 con 0x559b8c07bca0 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.378+0000 7f63d1a5d740 10 client.26357 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.378+0000 7f63d1a5d740 10 client.26357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.378+0000 7f63d1a5d740 20 client.26357 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.378+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559b8c0fb370 0x559b8c0fd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.378+0000 7f63d1a5d740 4 client.26357 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.378+0000 7f63d1a5d740 1 -- 172.21.15.42:0/1477898820 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559b8bfe89e0 con 0x559b8c0fb370 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.378+0000 7f63c7e00640 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559b8c0fb370 0x559b8c0fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.379+0000 7f63c7e00640 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559b8c0fb370 0x559b8c0fd790 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=705e32db2f427fcf server_cookie=471a26226f13fd01 in_seq=0 out_seq=0 2024-04-20T10:42:42.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.379+0000 7f63b2000640 10 client.26357 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:42.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.379+0000 7f63b2000640 1 -- 172.21.15.42:0/1477898820 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559b8bfe89e0 con 0x559b8c0fb370 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.379+0000 7f63b2000640 10 client.26357 handle_command_reply: tid=0 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.379+0000 7f63d1a5d740 1 client.26357 shutdown 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.379+0000 7f63d1a5d740 1 -- 172.21.15.42:0/1477898820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 msgr2=0x559b8c087c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.379+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c087c20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f63b4006010 tx=0x7f63b4004610 comp rx=0 tx=0).stop 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.380+0000 7f63d1a5d740 1 -- 172.21.15.42:0/1477898820 shutdown_connections 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.380+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559b8c0fb370 0x559b8c0fd790 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.380+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559b8c084880 0x559b8c088160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.380+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b8c07bca0 0x559b8c087c20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.380+0000 7f63d1a5d740 1 --2- 172.21.15.42:0/1477898820 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559b8c07b380 0x559b8c08c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.380+0000 7f63b2000640 0 client.26357 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.380+0000 7f63d1a5d740 1 -- 172.21.15.42:0/1477898820 >> 172.21.15.42:0/1477898820 conn(0x559b8c075ab0 msgr2=0x559b8c0821e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.380+0000 7f63d1a5d740 1 -- 172.21.15.42:0/1477898820 shutdown_connections 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.380+0000 7f63d1a5d740 1 -- 172.21.15.42:0/1477898820 wait complete. 2024-04-20T10:42:42.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.380+0000 7f63d1a5d740 20 client.26357 trim_cache size 0 max 16384 2024-04-20T10:42:42.383 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:42:42.383 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:42:42.383 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:42:42.383 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:42:42.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.381+0000 7f63cd800640 1 -- 172.21.15.42:0/1591009915 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f63ac0a3da0 msgr2=0x7f63ac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.381+0000 7f63cd800640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f63ac0a3da0 0x7f63ac0a6260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009fd0 tx=0x7f63c0009500 comp rx=0 tx=0).stop 2024-04-20T10:42:42.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.381+0000 7f63cd800640 1 -- 172.21.15.42:0/1591009915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c8074c20 msgr2=0x7f63c81a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.381+0000 7f63cd800640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c8074c20 0x7f63c81a4fb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f63b4000c00 tx=0x7f63b4004360 comp rx=0 tx=0).stop 2024-04-20T10:42:42.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.381+0000 7f63cd800640 1 -- 172.21.15.42:0/1591009915 shutdown_connections 2024-04-20T10:42:42.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.381+0000 7f63cd800640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f63ac0a3da0 0x7f63ac0a6260 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.381+0000 7f63cd800640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63c8112e10 0x7f63c819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.381+0000 7f63cd800640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63c8074c20 0x7f63c81a4fb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.381+0000 7f63cd800640 1 --2- 172.21.15.42:0/1591009915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63c8074300 0x7f63c81a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.381+0000 7f63cd800640 1 -- 172.21.15.42:0/1591009915 >> 172.21.15.42:0/1591009915 conn(0x7f63c80fd930 msgr2=0x7f63c80ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:42.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.381+0000 7f63cd800640 1 -- 172.21.15.42:0/1591009915 shutdown_connections 2024-04-20T10:42:42.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.381+0000 7f63cd800640 1 -- 172.21.15.42:0/1591009915 wait complete. 2024-04-20T10:42:42.401 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:42:42.402 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:42:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.617+0000 7fd27fe00640 1 -- 172.21.15.42:0/2455329747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280069d10 msgr2=0x7fd2801069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.617+0000 7fd27fe00640 1 --2- 172.21.15.42:0/2455329747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280069d10 0x7fd2801069d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009900 tx=0x7fd26c02bc30 comp rx=0 tx=0).stop 2024-04-20T10:42:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.617+0000 7fd27fe00640 1 -- 172.21.15.42:0/2455329747 shutdown_connections 2024-04-20T10:42:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.617+0000 7fd27fe00640 1 --2- 172.21.15.42:0/2455329747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd280107100 0x7fd28010eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.617+0000 7fd27fe00640 1 --2- 172.21.15.42:0/2455329747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280069d10 0x7fd2801069d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.617+0000 7fd27fe00640 1 --2- 172.21.15.42:0/2455329747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd280069360 0x7fd280069740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.617+0000 7fd27fe00640 1 -- 172.21.15.42:0/2455329747 >> 172.21.15.42:0/2455329747 conn(0x7fd2800767a0 msgr2=0x7fd280076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.617+0000 7fd27fe00640 1 -- 172.21.15.42:0/2455329747 shutdown_connections 2024-04-20T10:42:42.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.617+0000 7fd27fe00640 1 -- 172.21.15.42:0/2455329747 wait complete. 2024-04-20T10:42:42.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.618+0000 7fd27fe00640 1 Processor -- start 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.618+0000 7fd27fe00640 1 -- start start 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.618+0000 7fd27fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd280069360 0x7fd280198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.618+0000 7fd27fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280107100 0x7fd280198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.618+0000 7fd27fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28019cf50 0x7fd28019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.618+0000 7fd27fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd280112190 con 0x7fd28019cf50 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.618+0000 7fd27fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd280112010 con 0x7fd280107100 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.618+0000 7fd27fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd280112310 con 0x7fd280069360 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.619+0000 7fd27ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd280069360 0x7fd280198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.619+0000 7fd27e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280107100 0x7fd280198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.619+0000 7fd27ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd280069360 0x7fd280198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52750/0 (socket says 172.21.15.42:52750) 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.619+0000 7fd27e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280107100 0x7fd280198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59244/0 (socket says 172.21.15.42:59244) 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.619+0000 7fd27ea00640 1 -- 172.21.15.42:0/141248109 learned_addr learned my addr 172.21.15.42:0/141248109 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:42.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.619+0000 7fd27f400640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28019cf50 0x7fd28019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.619+0000 7fd27e000640 1 -- 172.21.15.42:0/141248109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd280069360 msgr2=0x7fd280198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.619+0000 7fd27e000640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd280069360 0x7fd280198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.619+0000 7fd27e000640 1 -- 172.21.15.42:0/141248109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28019cf50 msgr2=0x7fd28019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.619+0000 7fd27e000640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28019cf50 0x7fd28019d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.619+0000 7fd27e000640 1 -- 172.21.15.42:0/141248109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28019da40 con 0x7fd280107100 2024-04-20T10:42:42.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.619+0000 7fd27f400640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28019cf50 0x7fd28019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:42:42.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.620+0000 7fd27e000640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280107100 0x7fd280198bd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009230 tx=0x7fd26c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:42.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.620+0000 7fd284a00640 1 -- 172.21.15.42:0/141248109 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c043020 con 0x7fd280107100 2024-04-20T10:42:42.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.620+0000 7fd284a00640 1 -- 172.21.15.42:0/141248109 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd26c0054e0 con 0x7fd280107100 2024-04-20T10:42:42.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.620+0000 7fd284a00640 1 -- 172.21.15.42:0/141248109 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c003a80 con 0x7fd280107100 2024-04-20T10:42:42.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.620+0000 7fd27fe00640 1 -- 172.21.15.42:0/141248109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28019dcd0 con 0x7fd280107100 2024-04-20T10:42:42.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.620+0000 7fd27fe00640 1 -- 172.21.15.42:0/141248109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2801a54f0 con 0x7fd280107100 2024-04-20T10:42:42.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.621+0000 7fd27fe00640 1 -- 172.21.15.42:0/141248109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd280068ca0 con 0x7fd280107100 2024-04-20T10:42:42.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.623+0000 7fd284a00640 1 -- 172.21.15.42:0/141248109 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd26c046020 con 0x7fd280107100 2024-04-20T10:42:42.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.625+0000 7fd284a00640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd2640a3da0 0x7fd2640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:42.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.625+0000 7fd284a00640 1 -- 172.21.15.42:0/141248109 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fd26c0f2640 con 0x7fd280107100 2024-04-20T10:42:42.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.625+0000 7fd27ea00640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd2640a3da0 0x7fd2640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:42.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.626+0000 7fd27ea00640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd2640a3da0 0x7fd2640a6260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd270009cd0 tx=0x7fd270005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:42.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.629+0000 7fd284a00640 1 -- 172.21.15.42:0/141248109 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd26c04d020 con 0x7fd280107100 2024-04-20T10:42:42.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.895+0000 7fd27fe00640 1 -- 172.21.15.42:0/141248109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd28010a330 con 0x7fd280107100 2024-04-20T10:42:42.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.896+0000 7fd284a00640 1 -- 172.21.15.42:0/141248109 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7fd26c0bed60 con 0x7fd280107100 2024-04-20T10:42:42.896 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:42:42.896 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:42:42.897 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:42:42.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.899+0000 7fd27fe00640 1 -- 172.21.15.42:0/141248109 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd2640a3da0 msgr2=0x7fd2640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.899+0000 7fd27fe00640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd2640a3da0 0x7fd2640a6260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd270009cd0 tx=0x7fd270005f90 comp rx=0 tx=0).stop 2024-04-20T10:42:42.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.899+0000 7fd27fe00640 1 -- 172.21.15.42:0/141248109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280107100 msgr2=0x7fd280198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:42.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.899+0000 7fd27fe00640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280107100 0x7fd280198bd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009230 tx=0x7fd26c004360 comp rx=0 tx=0).stop 2024-04-20T10:42:42.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.899+0000 7fd27fe00640 1 -- 172.21.15.42:0/141248109 shutdown_connections 2024-04-20T10:42:42.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.899+0000 7fd27fe00640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fd2640a3da0 0x7fd2640a6260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.899+0000 7fd27fe00640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd28019cf50 0x7fd28019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.899+0000 7fd27fe00640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280107100 0x7fd280198bd0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.899+0000 7fd27fe00640 1 --2- 172.21.15.42:0/141248109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd280069360 0x7fd280198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:42.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.899+0000 7fd27fe00640 1 -- 172.21.15.42:0/141248109 >> 172.21.15.42:0/141248109 conn(0x7fd2800767a0 msgr2=0x7fd280108230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:42.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.900+0000 7fd27fe00640 1 -- 172.21.15.42:0/141248109 shutdown_connections 2024-04-20T10:42:42.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:42.900+0000 7fd27fe00640 1 -- 172.21.15.42:0/141248109 wait complete. 2024-04-20T10:42:42.919 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:42:43.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.138+0000 7f76cf200640 1 -- 172.21.15.42:0/1065491407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d01038e0 msgr2=0x7f76d010bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:43.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.138+0000 7f76cf200640 1 --2- 172.21.15.42:0/1065491407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d01038e0 0x7f76d010bdc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009900 tx=0x7f76b802bcc0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.138+0000 7f76cf200640 1 -- 172.21.15.42:0/1065491407 shutdown_connections 2024-04-20T10:42:43.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.138+0000 7f76cf200640 1 --2- 172.21.15.42:0/1065491407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76d010c3e0 0x7f76d010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.138+0000 7f76cf200640 1 --2- 172.21.15.42:0/1065491407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d01038e0 0x7f76d010bdc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.138+0000 7f76cf200640 1 --2- 172.21.15.42:0/1065491407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d0102fc0 0x7f76d01033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.138+0000 7f76cf200640 1 -- 172.21.15.42:0/1065491407 >> 172.21.15.42:0/1065491407 conn(0x7f76d00fc7e0 msgr2=0x7f76d00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:43.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.138+0000 7f76cf200640 1 -- 172.21.15.42:0/1065491407 shutdown_connections 2024-04-20T10:42:43.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.138+0000 7f76cf200640 1 -- 172.21.15.42:0/1065491407 wait complete. 2024-04-20T10:42:43.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.139+0000 7f76cf200640 1 Processor -- start 2024-04-20T10:42:43.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.140+0000 7f76cf200640 1 -- start start 2024-04-20T10:42:43.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.140+0000 7f76cf200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76d0102fc0 0x7f76d01a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.140+0000 7f76cf200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d01038e0 0x7f76d01a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.140+0000 7f76cf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d010c3e0 0x7f76d019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.140+0000 7f76cf200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01121f0 con 0x7f76d01038e0 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.140+0000 7f76cf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0112070 con 0x7f76d010c3e0 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.140+0000 7f76cf200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f76d0112370 con 0x7f76d0102fc0 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.140+0000 7f76cde00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76d0102fc0 0x7f76d01a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.140+0000 7f76cde00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76d0102fc0 0x7f76d01a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52764/0 (socket says 172.21.15.42:52764) 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.140+0000 7f76cde00640 1 -- 172.21.15.42:0/2275055120 learned_addr learned my addr 172.21.15.42:0/2275055120 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.140+0000 7f76cd400640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d01038e0 0x7f76d01a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.140+0000 7f76ce800640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d010c3e0 0x7f76d019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.141+0000 7f76cde00640 1 -- 172.21.15.42:0/2275055120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d010c3e0 msgr2=0x7f76d019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.141+0000 7f76cde00640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d010c3e0 0x7f76d019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.141+0000 7f76cde00640 1 -- 172.21.15.42:0/2275055120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d01038e0 msgr2=0x7f76d01a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.141+0000 7f76cde00640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d01038e0 0x7f76d01a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.141+0000 7f76cde00640 1 -- 172.21.15.42:0/2275055120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d019af70 con 0x7f76d0102fc0 2024-04-20T10:42:43.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.141+0000 7f76cde00640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76d0102fc0 0x7f76d01a05f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f76c000ea60 tx=0x7f76c000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:43.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.141+0000 7f76c6a00640 1 -- 172.21.15.42:0/2275055120 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c000cde0 con 0x7f76d0102fc0 2024-04-20T10:42:43.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.141+0000 7f76c6a00640 1 -- 172.21.15.42:0/2275055120 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c00035f0 con 0x7f76d0102fc0 2024-04-20T10:42:43.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.141+0000 7f76cf200640 1 -- 172.21.15.42:0/2275055120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d019b260 con 0x7f76d0102fc0 2024-04-20T10:42:43.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.142+0000 7f76c6a00640 1 -- 172.21.15.42:0/2275055120 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0010670 con 0x7f76d0102fc0 2024-04-20T10:42:43.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.142+0000 7f76cf200640 1 -- 172.21.15.42:0/2275055120 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d01a7570 con 0x7f76d0102fc0 2024-04-20T10:42:43.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.145+0000 7f76c6a00640 1 -- 172.21.15.42:0/2275055120 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f76c00037a0 con 0x7f76d0102fc0 2024-04-20T10:42:43.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.145+0000 7f76d776d740 1 Processor -- start 2024-04-20T10:42:43.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76d776d740 1 -- start start 2024-04-20T10:42:43.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76d776d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91a170 0x559a6e91a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76d776d740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 0x559a6e923130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76d776d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e923670 0x559a6e925a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76d776d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559a6e87f050 con 0x559a6e923670 2024-04-20T10:42:43.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76d776d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559a6e825490 con 0x559a6e91a170 2024-04-20T10:42:43.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76d776d740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559a6e87b010 con 0x559a6e91aa90 2024-04-20T10:42:43.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76cde00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 0x559a6e923130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91a170 0x559a6e91a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76cde00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 0x559a6e923130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52770/0 (socket says 172.21.15.42:52770) 2024-04-20T10:42:43.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91a170 0x559a6e91a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59284/0 (socket says 172.21.15.42:59284) 2024-04-20T10:42:43.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76cde00640 1 -- 172.21.15.42:0/2023649350 learned_addr learned my addr 172.21.15.42:0/2023649350 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:43.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.146+0000 7f76cd400640 1 --2- 172.21.15.42:0/2023649350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e923670 0x559a6e925a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.147+0000 7f76cde00640 1 -- 172.21.15.42:0/2023649350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91a170 msgr2=0x559a6e91a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:43.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.147+0000 7f76cde00640 1 --2- 172.21.15.42:0/2023649350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91a170 0x559a6e91a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.147+0000 7f76cde00640 1 -- 172.21.15.42:0/2023649350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e923670 msgr2=0x559a6e925a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:43.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.147+0000 7f76cde00640 1 --2- 172.21.15.42:0/2023649350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e923670 0x559a6e925a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.147+0000 7f76cde00640 1 -- 172.21.15.42:0/2023649350 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a6e82d070 con 0x559a6e91aa90 2024-04-20T10:42:43.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.147+0000 7f76cde00640 1 --2- 172.21.15.42:0/2023649350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 0x559a6e923130 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76c00ce170 tx=0x7f76c00d9260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4898dcabb51cdd23 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:43.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.147+0000 7f76c6a00640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f76b00a3d20 0x7f76b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.147+0000 7f76bf000640 1 -- 172.21.15.42:0/2023649350 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0098d20 con 0x559a6e91aa90 2024-04-20T10:42:43.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.148+0000 7f76ce800640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f76b00a3d20 0x7f76b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.148+0000 7f76c6a00640 1 -- 172.21.15.42:0/2275055120 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f76c00d0b70 con 0x7f76d0102fc0 2024-04-20T10:42:43.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.148+0000 7f76bf000640 1 -- 172.21.15.42:0/2023649350 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c00e2070 con 0x559a6e91aa90 2024-04-20T10:42:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.148+0000 7f76bf000640 1 -- 172.21.15.42:0/2023649350 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c009bed0 con 0x559a6e91aa90 2024-04-20T10:42:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.148+0000 7f76d776d740 1 -- 172.21.15.42:0/2023649350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 msgr2=0x559a6e923130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.148+0000 7f76d776d740 1 --2- 172.21.15.42:0/2023649350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 0x559a6e923130 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76c00ce170 tx=0x7f76c00d9260 comp rx=0 tx=0).stop 2024-04-20T10:42:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.148+0000 7f76ce800640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f76b00a3d20 0x7f76b00a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f76c8007f00 tx=0x7f76c8007520 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.149+0000 7f76d776d740 1 -- 172.21.15.42:0/2023649350 shutdown_connections 2024-04-20T10:42:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.149+0000 7f76d776d740 1 --2- 172.21.15.42:0/2023649350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e923670 0x559a6e925a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.149+0000 7f76d776d740 1 --2- 172.21.15.42:0/2023649350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 0x559a6e923130 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.149+0000 7f76d776d740 1 --2- 172.21.15.42:0/2023649350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91a170 0x559a6e91a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.149+0000 7f76d776d740 1 -- 172.21.15.42:0/2023649350 >> 172.21.15.42:0/2023649350 conn(0x559a6e9148a0 msgr2=0x559a6e914c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.149+0000 7f76d776d740 1 -- 172.21.15.42:0/2023649350 shutdown_connections 2024-04-20T10:42:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.149+0000 7f76d776d740 1 -- 172.21.15.42:0/2023649350 wait complete. 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.149+0000 7f76d776d740 1 -- start start 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.150+0000 7f76d776d740 1 Processor -- start 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.150+0000 7f76d776d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e91a170 0x559a6e920670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.150+0000 7f76d776d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91aa90 0x559a6e920bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.150+0000 7f76d776d740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e923670 0x559a6e9210f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.150+0000 7f76d776d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559a6e926400 con 0x559a6e91a170 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.150+0000 7f76d776d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559a6e838860 con 0x559a6e91aa90 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.150+0000 7f76d776d740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559a6e926580 con 0x559a6e923670 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.150+0000 7f76d776d740 10 client.0 _ll_register_callbacks cb 0x559a6e806fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.150+0000 7f76cd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91aa90 0x559a6e920bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.151+0000 7f76cd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91aa90 0x559a6e920bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59294/0 (socket says 172.21.15.42:59294) 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.151+0000 7f76cd400640 1 -- 172.21.15.42:0/1824724585 learned_addr learned my addr 172.21.15.42:0/1824724585 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.151+0000 7f76ce800640 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e923670 0x559a6e9210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.151+0000 7f76cde00640 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e91a170 0x559a6e920670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.151+0000 7f76cd400640 1 -- 172.21.15.42:0/1824724585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e923670 msgr2=0x559a6e9210f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.151+0000 7f76cd400640 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e923670 0x559a6e9210f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.151+0000 7f76cd400640 1 -- 172.21.15.42:0/1824724585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e91a170 msgr2=0x559a6e920670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.151+0000 7f76cd400640 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e91a170 0x559a6e920670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.151+0000 7f76cd400640 1 -- 172.21.15.42:0/1824724585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a6e83c950 con 0x559a6e91aa90 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.151+0000 7f76cde00640 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e91a170 0x559a6e920670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.151+0000 7f76cd400640 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91aa90 0x559a6e920bb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009230 tx=0x7f76b8004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76da311682327629 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.151+0000 7f76b7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.152+0000 7f76b7400640 1 -- 172.21.15.42:0/1824724585 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8044070 con 0x559a6e91aa90 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.152+0000 7f76d776d740 1 -- 172.21.15.42:0/1824724585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c00e2070 con 0x559a6e91aa90 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.152+0000 7f76b7400640 1 -- 172.21.15.42:0/1824724585 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f76b80046c0 con 0x559a6e91aa90 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.152+0000 7f76b7400640 1 -- 172.21.15.42:0/1824724585 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f76b80048c0 con 0x559a6e91aa90 2024-04-20T10:42:43.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.152+0000 7f76d776d740 10 client.26375 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:43.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.152+0000 7f76d776d740 1 -- 172.21.15.42:0/1824724585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559a6e80a820 con 0x559a6e91aa90 2024-04-20T10:42:43.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.152+0000 7f76b7400640 1 -- 172.21.15.42:0/1824724585 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c00e2070 con 0x559a6e91aa90 2024-04-20T10:42:43.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.153+0000 7f76b7400640 1 -- 172.21.15.42:0/1824724585 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f76b802cce0 con 0x559a6e91aa90 2024-04-20T10:42:43.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.153+0000 7f76d776d740 10 client.26375 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:43.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.153+0000 7f76d776d740 10 client.26375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:43.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.153+0000 7f76d776d740 20 client.26375 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:43.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.153+0000 7f76d776d740 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559a6e9335d0 0x559a6e935a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.153+0000 7f76d776d740 4 client.26375 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:42:43.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.153+0000 7f76d776d740 1 -- 172.21.15.42:0/1824724585 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a6e88ca70 con 0x559a6e9335d0 2024-04-20T10:42:43.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.153+0000 7f76cde00640 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559a6e9335d0 0x559a6e935a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.153+0000 7f76cde00640 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559a6e9335d0 0x559a6e935a00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994e7a0d79269933 server_cookie=409bfef0008dca3f in_seq=0 out_seq=0 2024-04-20T10:42:43.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.153+0000 7f76b7400640 10 client.26375 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.154+0000 7f76b7400640 1 -- 172.21.15.42:0/1824724585 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a6e88ca70 con 0x559a6e9335d0 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.154+0000 7f76b7400640 10 client.26375 handle_command_reply: tid=0 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.154+0000 7f76d776d740 1 client.26375 shutdown 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.155+0000 7f76d776d740 1 -- 172.21.15.42:0/1824724585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91aa90 msgr2=0x559a6e920bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.155+0000 7f76d776d740 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91aa90 0x559a6e920bb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009230 tx=0x7f76b8004560 comp rx=0 tx=0).stop 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.155+0000 7f76d776d740 1 -- 172.21.15.42:0/1824724585 shutdown_connections 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.155+0000 7f76d776d740 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559a6e9335d0 0x559a6e935a00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.155+0000 7f76d776d740 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e923670 0x559a6e9210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.155+0000 7f76d776d740 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91aa90 0x559a6e920bb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.155+0000 7f76d776d740 1 --2- 172.21.15.42:0/1824724585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e91a170 0x559a6e920670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.155+0000 7f76d776d740 1 -- 172.21.15.42:0/1824724585 >> 172.21.15.42:0/1824724585 conn(0x559a6e9148a0 msgr2=0x559a6e9165f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.155+0000 7f76b7400640 0 client.26375 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:43.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.155+0000 7f76d776d740 1 -- 172.21.15.42:0/1824724585 shutdown_connections 2024-04-20T10:42:43.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.155+0000 7f76d776d740 1 -- 172.21.15.42:0/1824724585 wait complete. 2024-04-20T10:42:43.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.155+0000 7f76d776d740 20 client.26375 trim_cache size 0 max 16384 2024-04-20T10:42:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.171+0000 7f76d776d740 1 Processor -- start 2024-04-20T10:42:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.171+0000 7f76d776d740 1 -- start start 2024-04-20T10:42:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.172+0000 7f76d776d740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91a170 0x559a6e92f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.172+0000 7f76d776d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e91aa90 0x559a6e9137e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.172+0000 7f76d776d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e923670 0x559a6e93b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.172+0000 7f76d776d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01ade50 con 0x559a6e91aa90 2024-04-20T10:42:43.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.172+0000 7f76d776d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01b06b0 con 0x559a6e923670 2024-04-20T10:42:43.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.172+0000 7f76d776d740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01ae4a0 con 0x559a6e91a170 2024-04-20T10:42:43.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.172+0000 7f76ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e923670 0x559a6e93b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.172+0000 7f76ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e923670 0x559a6e93b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59302/0 (socket says 172.21.15.42:59302) 2024-04-20T10:42:43.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.172+0000 7f76cd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e91aa90 0x559a6e9137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.172+0000 7f76ce800640 1 -- 172.21.15.42:0/4277601740 learned_addr learned my addr 172.21.15.42:0/4277601740 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:43.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.172+0000 7f76cde00640 1 --2- 172.21.15.42:0/4277601740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91a170 0x559a6e92f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.173+0000 7f76ce800640 1 -- 172.21.15.42:0/4277601740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91a170 msgr2=0x559a6e92f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:43.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.173+0000 7f76ce800640 1 --2- 172.21.15.42:0/4277601740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91a170 0x559a6e92f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.173+0000 7f76ce800640 1 -- 172.21.15.42:0/4277601740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e91aa90 msgr2=0x559a6e9137e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:43.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.173+0000 7f76ce800640 1 --2- 172.21.15.42:0/4277601740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e91aa90 0x559a6e9137e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.173+0000 7f76ce800640 1 -- 172.21.15.42:0/4277601740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a6e5913f0 con 0x559a6e923670 2024-04-20T10:42:43.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.173+0000 7f76ce800640 1 --2- 172.21.15.42:0/4277601740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e923670 0x559a6e93b2d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f76c801d8f0 tx=0x7f76c8018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad34e703a384505e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:43.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.173+0000 7f76be600640 1 -- 172.21.15.42:0/4277601740 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8003970 con 0x559a6e923670 2024-04-20T10:42:43.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.173+0000 7f76be600640 1 -- 172.21.15.42:0/4277601740 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c800f9a0 con 0x559a6e923670 2024-04-20T10:42:43.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.173+0000 7f76be600640 1 -- 172.21.15.42:0/4277601740 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c80046e0 con 0x559a6e923670 2024-04-20T10:42:43.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.174+0000 7f76d776d740 1 -- 172.21.15.42:0/4277601740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e923670 msgr2=0x559a6e93b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:43.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.174+0000 7f76d776d740 1 --2- 172.21.15.42:0/4277601740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e923670 0x559a6e93b2d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f76c801d8f0 tx=0x7f76c8018000 comp rx=0 tx=0).stop 2024-04-20T10:42:43.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.174+0000 7f76d776d740 1 -- 172.21.15.42:0/4277601740 shutdown_connections 2024-04-20T10:42:43.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.174+0000 7f76d776d740 1 --2- 172.21.15.42:0/4277601740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e923670 0x559a6e93b2d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.174+0000 7f76d776d740 1 --2- 172.21.15.42:0/4277601740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e91aa90 0x559a6e9137e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.174+0000 7f76d776d740 1 --2- 172.21.15.42:0/4277601740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91a170 0x559a6e92f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.174+0000 7f76d776d740 1 -- 172.21.15.42:0/4277601740 >> 172.21.15.42:0/4277601740 conn(0x559a6e9148a0 msgr2=0x559a6e9130d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:43.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.174+0000 7f76d776d740 1 -- 172.21.15.42:0/4277601740 shutdown_connections 2024-04-20T10:42:43.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.174+0000 7f76d776d740 1 -- 172.21.15.42:0/4277601740 wait complete. 2024-04-20T10:42:43.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.175+0000 7f76d776d740 1 -- start start 2024-04-20T10:42:43.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.175+0000 7f76d776d740 1 Processor -- start 2024-04-20T10:42:43.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76d776d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91a170 0x559a6e92bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76d776d740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 0x559a6e927040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76d776d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e923670 0x559a6e927780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76d776d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01aeb70 con 0x559a6e923670 2024-04-20T10:42:43.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76d776d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01affe0 con 0x559a6e91a170 2024-04-20T10:42:43.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76d776d740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f76d01af240 con 0x559a6e91aa90 2024-04-20T10:42:43.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76d776d740 10 client.0 _ll_register_callbacks cb 0x7f76b8004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:43.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91a170 0x559a6e92bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76ce800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e923670 0x559a6e927780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91a170 0x559a6e92bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59308/0 (socket says 172.21.15.42:59308) 2024-04-20T10:42:43.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76cde00640 1 -- 172.21.15.42:0/2233205818 learned_addr learned my addr 172.21.15.42:0/2233205818 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:43.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76cd400640 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 0x559a6e927040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76cd400640 1 -- 172.21.15.42:0/2233205818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91a170 msgr2=0x559a6e92bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76cd400640 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91a170 0x559a6e92bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76cd400640 1 -- 172.21.15.42:0/2233205818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e923670 msgr2=0x559a6e927780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76cd400640 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e923670 0x559a6e927780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.176+0000 7f76cd400640 1 -- 172.21.15.42:0/2233205818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a6e65b7d0 con 0x559a6e91aa90 2024-04-20T10:42:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.177+0000 7f76ce800640 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e923670 0x559a6e927780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.177+0000 7f76cd400640 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 0x559a6e927040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f76b8001eb0 tx=0x7f76b8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd6e80cdedde08c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:43.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.177+0000 7f76b7400640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:42:43.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.177+0000 7f76b7400640 1 -- 172.21.15.42:0/2233205818 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8054040 con 0x559a6e91aa90 2024-04-20T10:42:43.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.177+0000 7f76b7400640 1 -- 172.21.15.42:0/2233205818 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f76b8004ee0 con 0x559a6e91aa90 2024-04-20T10:42:43.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.177+0000 7f76d776d740 1 -- 172.21.15.42:0/2233205818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c800f9a0 con 0x559a6e91aa90 2024-04-20T10:42:43.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.177+0000 7f76b7400640 1 -- 172.21.15.42:0/2233205818 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f76b8044040 con 0x559a6e91aa90 2024-04-20T10:42:43.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.177+0000 7f76d776d740 10 client.26392 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:43.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.177+0000 7f76d776d740 1 -- 172.21.15.42:0/2233205818 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559a6e999aa0 con 0x559a6e91aa90 2024-04-20T10:42:43.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.178+0000 7f76b7400640 1 -- 172.21.15.42:0/2233205818 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c800f9a0 con 0x559a6e91aa90 2024-04-20T10:42:43.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.178+0000 7f76b7400640 1 -- 172.21.15.42:0/2233205818 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f76b802cce0 con 0x559a6e91aa90 2024-04-20T10:42:43.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.178+0000 7f76d776d740 10 client.26392 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:43.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.178+0000 7f76d776d740 10 client.26392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:43.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.178+0000 7f76d776d740 20 client.26392 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:43.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.178+0000 7f76d776d740 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559a6e99a250 0x559a6e99c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:43.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.178+0000 7f76d776d740 4 client.26392 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:42:43.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.178+0000 7f76d776d740 1 -- 172.21.15.42:0/2233205818 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a6e8868e0 con 0x559a6e99a250 2024-04-20T10:42:43.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.179+0000 7f76cde00640 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559a6e99a250 0x559a6e99c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:43.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.179+0000 7f76cde00640 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559a6e99a250 0x559a6e99c670 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c43a0f07ee7c886f server_cookie=c35568972b38d980 in_seq=0 out_seq=0 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.179+0000 7f76b7400640 10 client.26392 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.179+0000 7f76b7400640 1 -- 172.21.15.42:0/2233205818 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a6e8868e0 con 0x559a6e99a250 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.179+0000 7f76b7400640 10 client.26392 handle_command_reply: tid=0 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.179+0000 7f76d776d740 1 client.26392 shutdown 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.180+0000 7f76d776d740 1 -- 172.21.15.42:0/2233205818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 msgr2=0x559a6e927040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.180+0000 7f76d776d740 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 0x559a6e927040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f76b8001eb0 tx=0x7f76b8000bc0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.180+0000 7f76d776d740 1 -- 172.21.15.42:0/2233205818 shutdown_connections 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.180+0000 7f76d776d740 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559a6e99a250 0x559a6e99c670 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.180+0000 7f76d776d740 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559a6e923670 0x559a6e927780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.180+0000 7f76d776d740 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a6e91aa90 0x559a6e927040 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.180+0000 7f76b7400640 0 client.26392 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.180+0000 7f76d776d740 1 --2- 172.21.15.42:0/2233205818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a6e91a170 0x559a6e92bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.180+0000 7f76d776d740 1 -- 172.21.15.42:0/2233205818 >> 172.21.15.42:0/2233205818 conn(0x559a6e9148a0 msgr2=0x559a6e913960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:43.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.180+0000 7f76d776d740 1 -- 172.21.15.42:0/2233205818 shutdown_connections 2024-04-20T10:42:43.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.180+0000 7f76d776d740 1 -- 172.21.15.42:0/2233205818 wait complete. 2024-04-20T10:42:43.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.180+0000 7f76d776d740 20 client.26392 trim_cache size 0 max 16384 2024-04-20T10:42:43.189 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:42:43.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.181+0000 7f76cf200640 1 -- 172.21.15.42:0/2275055120 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f76b00a3d20 msgr2=0x7f76b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:43.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.181+0000 7f76cf200640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f76b00a3d20 0x7f76b00a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f76c8007f00 tx=0x7f76c8007520 comp rx=0 tx=0).stop 2024-04-20T10:42:43.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.181+0000 7f76cf200640 1 -- 172.21.15.42:0/2275055120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76d0102fc0 msgr2=0x7f76d01a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:43.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.181+0000 7f76cf200640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76d0102fc0 0x7f76d01a05f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f76c000ea60 tx=0x7f76c000ef20 comp rx=0 tx=0).stop 2024-04-20T10:42:43.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.181+0000 7f76cf200640 1 -- 172.21.15.42:0/2275055120 shutdown_connections 2024-04-20T10:42:43.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.181+0000 7f76cf200640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f76b00a3d20 0x7f76b00a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.181+0000 7f76cf200640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76d010c3e0 0x7f76d019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.181+0000 7f76cf200640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d01038e0 0x7f76d01a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.181+0000 7f76cf200640 1 --2- 172.21.15.42:0/2275055120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76d0102fc0 0x7f76d01a05f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:43.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.181+0000 7f76cf200640 1 -- 172.21.15.42:0/2275055120 >> 172.21.15.42:0/2275055120 conn(0x7f76d00fc7e0 msgr2=0x7f76d0100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:43.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.181+0000 7f76cf200640 1 -- 172.21.15.42:0/2275055120 shutdown_connections 2024-04-20T10:42:43.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:43.181+0000 7f76cf200640 1 -- 172.21.15.42:0/2275055120 wait complete. 2024-04-20T10:42:44.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:43 smithi046 ceph-mon[21574]: pgmap v268: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 3.81k op/s 2024-04-20T10:42:44.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:43 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/141248109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:42:44.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:43 smithi042 ceph-mon[17078]: pgmap v268: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 3.81k op/s 2024-04-20T10:42:44.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:43 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/141248109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:42:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:43 smithi162 ceph-mon[21854]: pgmap v268: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 3.81k op/s 2024-04-20T10:42:44.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:43 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/141248109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:42:44.201 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:42:44.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:42:44.202 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:42:44.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.440+0000 7f9f67200640 1 -- 172.21.15.42:0/3170582242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68109350 msgr2=0x7f9f68110f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.440+0000 7f9f67200640 1 --2- 172.21.15.42:0/3170582242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68109350 0x7f9f68110f20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009be0 tx=0x7f9f6002bef0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.441+0000 7f9f67200640 1 -- 172.21.15.42:0/3170582242 shutdown_connections 2024-04-20T10:42:44.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.441+0000 7f9f67200640 1 --2- 172.21.15.42:0/3170582242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68109350 0x7f9f68110f20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.441+0000 7f9f67200640 1 --2- 172.21.15.42:0/3170582242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f68104c30 0x7f9f68108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.441+0000 7f9f67200640 1 --2- 172.21.15.42:0/3170582242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f68104280 0x7f9f68104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.441+0000 7f9f67200640 1 -- 172.21.15.42:0/3170582242 >> 172.21.15.42:0/3170582242 conn(0x7f9f680ffaa0 msgr2=0x7f9f68101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:44.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.441+0000 7f9f67200640 1 -- 172.21.15.42:0/3170582242 shutdown_connections 2024-04-20T10:42:44.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.441+0000 7f9f67200640 1 -- 172.21.15.42:0/3170582242 wait complete. 2024-04-20T10:42:44.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.442+0000 7f9f67200640 1 Processor -- start 2024-04-20T10:42:44.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.442+0000 7f9f67200640 1 -- start start 2024-04-20T10:42:44.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.442+0000 7f9f67200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68104280 0x7f9f681a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.442+0000 7f9f67200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f68109350 0x7f9f681a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.442+0000 7f9f67200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f6819a6c0 0x7f9f6819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.442+0000 7f9f67200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f68114440 con 0x7f9f68109350 2024-04-20T10:42:44.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.442+0000 7f9f67200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681142c0 con 0x7f9f68104280 2024-04-20T10:42:44.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.442+0000 7f9f67200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681145c0 con 0x7f9f6819a6c0 2024-04-20T10:42:44.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.442+0000 7f9f65e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68104280 0x7f9f681a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.443+0000 7f9f65e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68104280 0x7f9f681a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59326/0 (socket says 172.21.15.42:59326) 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.443+0000 7f9f65e00640 1 -- 172.21.15.42:0/527088799 learned_addr learned my addr 172.21.15.42:0/527088799 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.443+0000 7f9f66800640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f6819a6c0 0x7f9f6819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.443+0000 7f9f5fe00640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f68109350 0x7f9f681a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.443+0000 7f9f65e00640 1 -- 172.21.15.42:0/527088799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f6819a6c0 msgr2=0x7f9f6819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.443+0000 7f9f65e00640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f6819a6c0 0x7f9f6819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.443+0000 7f9f65e00640 1 -- 172.21.15.42:0/527088799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f68109350 msgr2=0x7f9f681a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.443+0000 7f9f65e00640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f68109350 0x7f9f681a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.443+0000 7f9f65e00640 1 -- 172.21.15.42:0/527088799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6819b2c0 con 0x7f9f68104280 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.443+0000 7f9f66800640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f6819a6c0 0x7f9f6819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.443+0000 7f9f5fe00640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f68109350 0x7f9f681a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.443+0000 7f9f65e00640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68104280 0x7f9f681a0600 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800e9c0 tx=0x7f9f5800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.444+0000 7f9f5ea00640 1 -- 172.21.15.42:0/527088799 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800ccc0 con 0x7f9f68104280 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.444+0000 7f9f5ea00640 1 -- 172.21.15.42:0/527088799 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f58005500 con 0x7f9f68104280 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.444+0000 7f9f67200640 1 -- 172.21.15.42:0/527088799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f681a73a0 con 0x7f9f68104280 2024-04-20T10:42:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.444+0000 7f9f5ea00640 1 -- 172.21.15.42:0/527088799 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58010430 con 0x7f9f68104280 2024-04-20T10:42:44.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.444+0000 7f9f67200640 1 -- 172.21.15.42:0/527088799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f681a7800 con 0x7f9f68104280 2024-04-20T10:42:44.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.447+0000 7f9f5ea00640 1 -- 172.21.15.42:0/527088799 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9f580046c0 con 0x7f9f68104280 2024-04-20T10:42:44.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.447+0000 7f9f6fe0e740 1 Processor -- start 2024-04-20T10:42:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.448+0000 7f9f6fe0e740 1 -- start start 2024-04-20T10:42:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.448+0000 7f9f6fe0e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939d0f0 0x56331939d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.448+0000 7f9f6fe0e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939da10 0x5633193a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.448+0000 7f9f6fe0e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.448+0000 7f9f6fe0e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563319301d20 con 0x5633193a65f0 2024-04-20T10:42:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.448+0000 7f9f6fe0e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5633192a83e0 con 0x56331939da10 2024-04-20T10:42:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.448+0000 7f9f6fe0e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5633192fdce0 con 0x56331939d0f0 2024-04-20T10:42:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.448+0000 7f9f65e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939da10 0x5633193a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.448+0000 7f9f66800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939d0f0 0x56331939d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.448+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.448+0000 7f9f66800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939d0f0 0x56331939d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52806/0 (socket says 172.21.15.42:52806) 2024-04-20T10:42:44.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.448+0000 7f9f66800640 1 -- 172.21.15.42:0/4082486352 learned_addr learned my addr 172.21.15.42:0/4082486352 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:44.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.449+0000 7f9f66800640 1 -- 172.21.15.42:0/4082486352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939da10 msgr2=0x5633193a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.449+0000 7f9f66800640 1 --2- 172.21.15.42:0/4082486352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939da10 0x5633193a60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.449+0000 7f9f66800640 1 -- 172.21.15.42:0/4082486352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 msgr2=0x5633193a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.449+0000 7f9f66800640 1 --2- 172.21.15.42:0/4082486352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193a89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.449+0000 7f9f66800640 1 -- 172.21.15.42:0/4082486352 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633192c7980 con 0x56331939d0f0 2024-04-20T10:42:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.449+0000 7f9f5fe00640 1 --2- 172.21.15.42:0/4082486352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.449+0000 7f9f66800640 1 --2- 172.21.15.42:0/4082486352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939d0f0 0x56331939d4d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009be0 tx=0x7f9f60033f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c92744113d432ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.449+0000 7f9f5ea00640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9f4c0a3da0 0x7f9f4c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.450+0000 7f9f5ea00640 1 -- 172.21.15.42:0/527088799 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f9f580d0b30 con 0x7f9f68104280 2024-04-20T10:42:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.450+0000 7f9f57400640 1 -- 172.21.15.42:0/4082486352 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60043030 con 0x56331939d0f0 2024-04-20T10:42:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.450+0000 7f9f65e00640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9f4c0a3da0 0x7f9f4c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.450+0000 7f9f57400640 1 -- 172.21.15.42:0/4082486352 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f60003970 con 0x56331939d0f0 2024-04-20T10:42:44.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.450+0000 7f9f57400640 1 -- 172.21.15.42:0/4082486352 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6002cc10 con 0x56331939d0f0 2024-04-20T10:42:44.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.451+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/4082486352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939d0f0 msgr2=0x56331939d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.451+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/4082486352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939d0f0 0x56331939d4d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009be0 tx=0x7f9f60033f80 comp rx=0 tx=0).stop 2024-04-20T10:42:44.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.451+0000 7f9f65e00640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9f4c0a3da0 0x7f9f4c0a6260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f580ce300 tx=0x7f9f580d4670 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:44.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.451+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/4082486352 shutdown_connections 2024-04-20T10:42:44.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.451+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/4082486352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.451+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/4082486352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939da10 0x5633193a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.451+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/4082486352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939d0f0 0x56331939d4d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.451+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/4082486352 >> 172.21.15.42:0/4082486352 conn(0x563319397820 msgr2=0x563319397c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:44.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.451+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/4082486352 shutdown_connections 2024-04-20T10:42:44.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.451+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/4082486352 wait complete. 2024-04-20T10:42:44.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.451+0000 7f9f6fe0e740 1 -- start start 2024-04-20T10:42:44.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.452+0000 7f9f6fe0e740 1 Processor -- start 2024-04-20T10:42:44.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.452+0000 7f9f6fe0e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939d0f0 0x5633193a3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.452+0000 7f9f6fe0e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 0x5633193a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.452+0000 7f9f6fe0e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.452+0000 7f9f6fe0e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5633193a9380 con 0x5633193a65f0 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.452+0000 7f9f6fe0e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5633192bb7a0 con 0x56331939d0f0 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.452+0000 7f9f6fe0e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5633193a9500 con 0x56331939da10 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.452+0000 7f9f6fe0e740 10 client.0 _ll_register_callbacks cb 0x5633192bdb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.452+0000 7f9f66800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939d0f0 0x5633193a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.452+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 0x5633193a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.452+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 0x5633193a3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52812/0 (socket says 172.21.15.42:52812) 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.452+0000 7f9f5fe00640 1 -- 172.21.15.42:0/676335495 learned_addr learned my addr 172.21.15.42:0/676335495 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.453+0000 7f9f66800640 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.453+0000 7f9f66800640 1 -- 172.21.15.42:0/676335495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 msgr2=0x5633193a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.453+0000 7f9f66800640 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 0x5633193a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.453+0000 7f9f66800640 1 -- 172.21.15.42:0/676335495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 msgr2=0x5633193a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.453+0000 7f9f66800640 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193a40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.453+0000 7f9f66800640 1 -- 172.21.15.42:0/676335495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633190dc7e0 con 0x56331939d0f0 2024-04-20T10:42:44.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.453+0000 7f9f5fe00640 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 0x5633193a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:44.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.453+0000 7f9f66800640 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939d0f0 0x5633193a3640 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f6002c500 tx=0x7f9f6002c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27c2058446e8d03c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:44.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.453+0000 7f9f4be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:42:44.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.454+0000 7f9f4be00640 1 -- 172.21.15.42:0/676335495 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60053060 con 0x56331939d0f0 2024-04-20T10:42:44.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.454+0000 7f9f4be00640 1 -- 172.21.15.42:0/676335495 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f9f60033680 con 0x56331939d0f0 2024-04-20T10:42:44.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.454+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/676335495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f60003970 con 0x56331939d0f0 2024-04-20T10:42:44.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.454+0000 7f9f4be00640 1 -- 172.21.15.42:0/676335495 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9f60043040 con 0x56331939d0f0 2024-04-20T10:42:44.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.454+0000 7f9f6fe0e740 10 client.26393 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:44.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.454+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/676335495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56331913cf70 con 0x56331939d0f0 2024-04-20T10:42:44.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.454+0000 7f9f4be00640 1 -- 172.21.15.42:0/676335495 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f60003970 con 0x56331939d0f0 2024-04-20T10:42:44.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.455+0000 7f9f4be00640 1 -- 172.21.15.42:0/676335495 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f9f6003f4b0 con 0x56331939d0f0 2024-04-20T10:42:44.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.455+0000 7f9f6fe0e740 10 client.26393 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.455+0000 7f9f6fe0e740 10 client.26393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.455+0000 7f9f6fe0e740 20 client.26393 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.455+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5633193b67c0 0x5633193b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.455+0000 7f9f6fe0e740 4 client.26393 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.455+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/676335495 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56331930f740 con 0x5633193b67c0 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.455+0000 7f9f5fe00640 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5633193b67c0 0x5633193b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.456+0000 7f9f5fe00640 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5633193b67c0 0x5633193b8bb0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b559609f29e5ea server_cookie=10cad693266f925e in_seq=0 out_seq=0 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.456+0000 7f9f4be00640 10 client.26393 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.457+0000 7f9f4be00640 1 -- 172.21.15.42:0/676335495 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56331930f740 con 0x5633193b67c0 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.457+0000 7f9f4be00640 10 client.26393 handle_command_reply: tid=0 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f6fe0e740 1 client.26393 shutdown 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/676335495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939d0f0 msgr2=0x5633193a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939d0f0 0x5633193a3640 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f6002c500 tx=0x7f9f6002c930 comp rx=0 tx=0).stop 2024-04-20T10:42:44.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/676335495 shutdown_connections 2024-04-20T10:42:44.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5633193b67c0 0x5633193b8bb0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 0x5633193a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/676335495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939d0f0 0x5633193a3640 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/676335495 >> 172.21.15.42:0/676335495 conn(0x563319397820 msgr2=0x563319399570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:44.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f4be00640 0 client.26393 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:44.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/676335495 shutdown_connections 2024-04-20T10:42:44.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/676335495 wait complete. 2024-04-20T10:42:44.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.458+0000 7f9f6fe0e740 20 client.26393 trim_cache size 0 max 16384 2024-04-20T10:42:44.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.479+0000 7f9f6fe0e740 1 Processor -- start 2024-04-20T10:42:44.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.479+0000 7f9f6fe0e740 1 -- start start 2024-04-20T10:42:44.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.479+0000 7f9f6fe0e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56331939d0f0 0x5633193a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.479+0000 7f9f6fe0e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939da10 0x5633193a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.479+0000 7f9f6fe0e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5633193a65f0 0x563319396880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.479+0000 7f9f6fe0e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681add60 con 0x56331939d0f0 2024-04-20T10:42:44.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.479+0000 7f9f6fe0e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681b05c0 con 0x56331939da10 2024-04-20T10:42:44.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.479+0000 7f9f6fe0e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681ae3b0 con 0x5633193a65f0 2024-04-20T10:42:44.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.480+0000 7f9f66800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56331939d0f0 0x5633193a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.480+0000 7f9f66800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56331939d0f0 0x5633193a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43914/0 (socket says 172.21.15.42:43914) 2024-04-20T10:42:44.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.480+0000 7f9f66800640 1 -- 172.21.15.42:0/167032046 learned_addr learned my addr 172.21.15.42:0/167032046 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:44.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.480+0000 7f9f5fe00640 1 --2- 172.21.15.42:0/167032046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939da10 0x5633193a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.480+0000 7f9f66800640 1 --2- 172.21.15.42:0/167032046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5633193a65f0 0x563319396880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.480+0000 7f9f66800640 1 -- 172.21.15.42:0/167032046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5633193a65f0 msgr2=0x563319396880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.480+0000 7f9f66800640 1 --2- 172.21.15.42:0/167032046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5633193a65f0 0x563319396880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.480+0000 7f9f66800640 1 -- 172.21.15.42:0/167032046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939da10 msgr2=0x5633193a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.480+0000 7f9f66800640 1 --2- 172.21.15.42:0/167032046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939da10 0x5633193a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.480+0000 7f9f66800640 1 -- 172.21.15.42:0/167032046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563318fd4170 con 0x56331939d0f0 2024-04-20T10:42:44.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.481+0000 7f9f66800640 1 --2- 172.21.15.42:0/167032046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56331939d0f0 0x5633193a5d70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9f60004aa0 tx=0x7f9f60005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1a2424540ff03d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:44.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.481+0000 7f9f56a00640 1 -- 172.21.15.42:0/167032046 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60053070 con 0x56331939d0f0 2024-04-20T10:42:44.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.481+0000 7f9f56a00640 1 -- 172.21.15.42:0/167032046 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f60003970 con 0x56331939d0f0 2024-04-20T10:42:44.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.481+0000 7f9f56a00640 1 -- 172.21.15.42:0/167032046 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6005c040 con 0x56331939d0f0 2024-04-20T10:42:44.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.482+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/167032046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56331939d0f0 msgr2=0x5633193a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.482+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/167032046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56331939d0f0 0x5633193a5d70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9f60004aa0 tx=0x7f9f60005fd0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.482+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/167032046 shutdown_connections 2024-04-20T10:42:44.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.482+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/167032046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5633193a65f0 0x563319396880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.482+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/167032046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939da10 0x5633193a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.482+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/167032046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56331939d0f0 0x5633193a5d70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.482+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/167032046 >> 172.21.15.42:0/167032046 conn(0x563319397820 msgr2=0x563319396170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:44.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.482+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/167032046 shutdown_connections 2024-04-20T10:42:44.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.482+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/167032046 wait complete. 2024-04-20T10:42:44.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.482+0000 7f9f6fe0e740 1 -- start start 2024-04-20T10:42:44.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f6fe0e740 1 Processor -- start 2024-04-20T10:42:44.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f6fe0e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939d0f0 0x5633193b95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f6fe0e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 0x5633193b9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f6fe0e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193ba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f6fe0e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681aea80 con 0x5633193a65f0 2024-04-20T10:42:44.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f6fe0e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681afef0 con 0x56331939d0f0 2024-04-20T10:42:44.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f6fe0e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681af150 con 0x56331939da10 2024-04-20T10:42:44.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f6fe0e740 10 client.0 _ll_register_callbacks cb 0x7f9f50009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:44.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 0x5633193b9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f66800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939d0f0 0x5633193b95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f5fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 0x5633193b9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52834/0 (socket says 172.21.15.42:52834) 2024-04-20T10:42:44.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f5fe00640 1 -- 172.21.15.42:0/532535256 learned_addr learned my addr 172.21.15.42:0/532535256 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:44.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.483+0000 7f9f66800640 1 --2- 172.21.15.42:0/532535256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193ba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.484+0000 7f9f5fe00640 1 -- 172.21.15.42:0/532535256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939d0f0 msgr2=0x5633193b95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.484+0000 7f9f5fe00640 1 --2- 172.21.15.42:0/532535256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939d0f0 0x5633193b95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.484+0000 7f9f5fe00640 1 -- 172.21.15.42:0/532535256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 msgr2=0x5633193ba050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.484+0000 7f9f5fe00640 1 --2- 172.21.15.42:0/532535256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193ba050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.484+0000 7f9f5fe00640 1 -- 172.21.15.42:0/532535256 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331941b9a0 con 0x56331939da10 2024-04-20T10:42:44.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.484+0000 7f9f5fe00640 1 --2- 172.21.15.42:0/532535256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 0x5633193b9b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x5633193a95c0 tx=0x7f9f50013760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=293d91066d032eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:44.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.484+0000 7f9f4be00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:42:44.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.485+0000 7f9f4be00640 1 -- 172.21.15.42:0/532535256 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50021560 con 0x56331939da10 2024-04-20T10:42:44.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.485+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/532535256 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f60003970 con 0x56331939da10 2024-04-20T10:42:44.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.485+0000 7f9f4be00640 1 -- 172.21.15.42:0/532535256 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f9f50021700 con 0x56331939da10 2024-04-20T10:42:44.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.485+0000 7f9f4be00640 1 -- 172.21.15.42:0/532535256 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9f50031050 con 0x56331939da10 2024-04-20T10:42:44.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.485+0000 7f9f6fe0e740 10 client.26410 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.485+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/532535256 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56331941cd00 con 0x56331939da10 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.485+0000 7f9f4be00640 1 -- 172.21.15.42:0/532535256 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f60003970 con 0x56331939da10 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.486+0000 7f9f4be00640 1 -- 172.21.15.42:0/532535256 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f9f50011560 con 0x56331939da10 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.486+0000 7f9f6fe0e740 10 client.26410 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.486+0000 7f9f6fe0e740 10 client.26410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.486+0000 7f9f6fe0e740 20 client.26410 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.486+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/532535256 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56331941d460 0x56331941f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.486+0000 7f9f6fe0e740 4 client.26410 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.486+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/532535256 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5633193095b0 con 0x56331941d460 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.486+0000 7f9f66800640 1 --2- 172.21.15.42:0/532535256 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56331941d460 0x56331941f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.487+0000 7f9f66800640 1 --2- 172.21.15.42:0/532535256 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56331941d460 0x56331941f880 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d308451c3bc954c server_cookie=843bb69dc7b7b535 in_seq=0 out_seq=0 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.487+0000 7f9f4be00640 10 client.26410 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.488+0000 7f9f4be00640 1 -- 172.21.15.42:0/532535256 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5633193095b0 con 0x56331941d460 2024-04-20T10:42:44.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.488+0000 7f9f4be00640 10 client.26410 handle_command_reply: tid=0 2024-04-20T10:42:44.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.488+0000 7f9f6fe0e740 1 client.26410 shutdown 2024-04-20T10:42:44.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.489+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/532535256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 msgr2=0x5633193b9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.489+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/532535256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 0x5633193b9b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x5633193a95c0 tx=0x7f9f50013760 comp rx=0 tx=0).stop 2024-04-20T10:42:44.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.489+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/532535256 shutdown_connections 2024-04-20T10:42:44.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.489+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/532535256 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x56331941d460 0x56331941f880 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.489+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/532535256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5633193a65f0 0x5633193ba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.489+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/532535256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56331939da10 0x5633193b9b10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.489+0000 7f9f6fe0e740 1 --2- 172.21.15.42:0/532535256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331939d0f0 0x5633193b95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.489+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/532535256 >> 172.21.15.42:0/532535256 conn(0x563319397820 msgr2=0x563319399420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:44.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.489+0000 7f9f4be00640 0 client.26410 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:44.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.489+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/532535256 shutdown_connections 2024-04-20T10:42:44.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.489+0000 7f9f6fe0e740 1 -- 172.21.15.42:0/532535256 wait complete. 2024-04-20T10:42:44.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.489+0000 7f9f6fe0e740 20 client.26410 trim_cache size 0 max 16384 2024-04-20T10:42:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:42:44.491 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:42:44.491 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "f553b6b9-7931-40c5-949a-1ea5f3978fe3", 2024-04-20T10:42:44.491 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:42:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:42:44.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.490+0000 7f9f67200640 1 -- 172.21.15.42:0/527088799 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9f4c0a3da0 msgr2=0x7f9f4c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.490+0000 7f9f67200640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9f4c0a3da0 0x7f9f4c0a6260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f580ce300 tx=0x7f9f580d4670 comp rx=0 tx=0).stop 2024-04-20T10:42:44.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.490+0000 7f9f67200640 1 -- 172.21.15.42:0/527088799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68104280 msgr2=0x7f9f681a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.490+0000 7f9f67200640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68104280 0x7f9f681a0600 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800e9c0 tx=0x7f9f5800ee80 comp rx=0 tx=0).stop 2024-04-20T10:42:44.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.490+0000 7f9f67200640 1 -- 172.21.15.42:0/527088799 shutdown_connections 2024-04-20T10:42:44.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.490+0000 7f9f67200640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9f4c0a3da0 0x7f9f4c0a6260 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.490+0000 7f9f67200640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f6819a6c0 0x7f9f6819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.490+0000 7f9f67200640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f68109350 0x7f9f681a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.490+0000 7f9f67200640 1 --2- 172.21.15.42:0/527088799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68104280 0x7f9f681a0600 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.490+0000 7f9f67200640 1 -- 172.21.15.42:0/527088799 >> 172.21.15.42:0/527088799 conn(0x7f9f680ffaa0 msgr2=0x7f9f68101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:44.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.491+0000 7f9f67200640 1 -- 172.21.15.42:0/527088799 shutdown_connections 2024-04-20T10:42:44.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.491+0000 7f9f67200640 1 -- 172.21.15.42:0/527088799 wait complete. 2024-04-20T10:42:44.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:42:44.508 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:42:44.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.730+0000 7f740fe00640 1 -- 172.21.15.42:0/1369483388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7410104c30 msgr2=0x7f7410108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.730+0000 7f740fe00640 1 --2- 172.21.15.42:0/1369483388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7410104c30 0x7f7410108c70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009860 tx=0x7f73f802bb70 comp rx=0 tx=0).stop 2024-04-20T10:42:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.730+0000 7f740fe00640 1 -- 172.21.15.42:0/1369483388 shutdown_connections 2024-04-20T10:42:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.730+0000 7f740fe00640 1 --2- 172.21.15.42:0/1369483388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7410109350 0x7f7410110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.730+0000 7f740fe00640 1 --2- 172.21.15.42:0/1369483388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7410104c30 0x7f7410108c70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.730+0000 7f740fe00640 1 --2- 172.21.15.42:0/1369483388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410104280 0x7f7410104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.730+0000 7f740fe00640 1 -- 172.21.15.42:0/1369483388 >> 172.21.15.42:0/1369483388 conn(0x7f74100ffaa0 msgr2=0x7f7410101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.731+0000 7f740fe00640 1 -- 172.21.15.42:0/1369483388 shutdown_connections 2024-04-20T10:42:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.731+0000 7f740fe00640 1 -- 172.21.15.42:0/1369483388 wait complete. 2024-04-20T10:42:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.732+0000 7f740fe00640 1 Processor -- start 2024-04-20T10:42:44.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.732+0000 7f740fe00640 1 -- start start 2024-04-20T10:42:44.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410104280 0x7f74101a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7410109350 0x7f74101a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f741019a720 0x7f741019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f74101143c0 con 0x7f7410109350 2024-04-20T10:42:44.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7410114240 con 0x7f7410104280 2024-04-20T10:42:44.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7410114540 con 0x7f741019a720 2024-04-20T10:42:44.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740f400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f741019a720 0x7f741019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410104280 0x7f74101a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740f400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f741019a720 0x7f741019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52848/0 (socket says 172.21.15.42:52848) 2024-04-20T10:42:44.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740f400640 1 -- 172.21.15.42:0/2880309105 learned_addr learned my addr 172.21.15.42:0/2880309105 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:44.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f7406000640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7410109350 0x7f74101a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740f400640 1 -- 172.21.15.42:0/2880309105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410104280 msgr2=0x7f74101a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740f400640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410104280 0x7f74101a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740f400640 1 -- 172.21.15.42:0/2880309105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7410109350 msgr2=0x7f74101a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740f400640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7410109350 0x7f74101a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.733+0000 7f740f400640 1 -- 172.21.15.42:0/2880309105 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741019b320 con 0x7f741019a720 2024-04-20T10:42:44.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.734+0000 7f740f400640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f741019a720 0x7f741019aba0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f740800eb10 tx=0x7f740800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:44.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.734+0000 7f7407e00640 1 -- 172.21.15.42:0/2880309105 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408004b20 con 0x7f741019a720 2024-04-20T10:42:44.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.734+0000 7f7407e00640 1 -- 172.21.15.42:0/2880309105 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7408004cc0 con 0x7f741019a720 2024-04-20T10:42:44.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.734+0000 7f7407e00640 1 -- 172.21.15.42:0/2880309105 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408010760 con 0x7f741019a720 2024-04-20T10:42:44.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.734+0000 7f740fe00640 1 -- 172.21.15.42:0/2880309105 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74101a7400 con 0x7f741019a720 2024-04-20T10:42:44.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.734+0000 7f740fe00640 1 -- 172.21.15.42:0/2880309105 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74101a77b0 con 0x7f741019a720 2024-04-20T10:42:44.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.738+0000 7f7407e00640 1 -- 172.21.15.42:0/2880309105 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7408005020 con 0x7f741019a720 2024-04-20T10:42:44.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.739+0000 7f7418581740 1 Processor -- start 2024-04-20T10:42:44.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.740+0000 7f7418581740 1 -- start start 2024-04-20T10:42:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.740+0000 7f7418581740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 0x55a3363a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.740+0000 7f7418581740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 0x55a3363ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.740+0000 7f7418581740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363ad880 0x55a3363afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.740+0000 7f7418581740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a33630a150 con 0x55a3363a4380 2024-04-20T10:42:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.740+0000 7f7418581740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a3362b0370 con 0x55a3363ad880 2024-04-20T10:42:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.740+0000 7f7418581740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a336306110 con 0x55a3363a4ca0 2024-04-20T10:42:44.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.740+0000 7f7407e00640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f73f40a3da0 0x7f73f40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.740+0000 7f7407e00640 1 -- 172.21.15.42:0/2880309105 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f74080d0d40 con 0x7f741019a720 2024-04-20T10:42:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.740+0000 7f740ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 0x55a3363ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.740+0000 7f740ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 0x55a3363ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52856/0 (socket says 172.21.15.42:52856) 2024-04-20T10:42:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.740+0000 7f740ea00640 1 -- 172.21.15.42:0/3618455127 learned_addr learned my addr 172.21.15.42:0/3618455127 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.741+0000 7f7406000640 1 --2- 172.21.15.42:0/3618455127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363ad880 0x55a3363afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.741+0000 7f7406000640 1 --2- 172.21.15.42:0/3618455127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 0x55a3363a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.741+0000 7f740f400640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f73f40a3da0 0x7f73f40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.741+0000 7f740ea00640 1 -- 172.21.15.42:0/3618455127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363ad880 msgr2=0x55a3363afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.741+0000 7f740ea00640 1 --2- 172.21.15.42:0/3618455127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363ad880 0x55a3363afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.741+0000 7f740ea00640 1 -- 172.21.15.42:0/3618455127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 msgr2=0x55a3363a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.741+0000 7f740ea00640 1 --2- 172.21.15.42:0/3618455127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 0x55a3363a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.741+0000 7f740ea00640 1 -- 172.21.15.42:0/3618455127 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3360e67d0 con 0x55a3363a4ca0 2024-04-20T10:42:44.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.741+0000 7f7406000640 1 --2- 172.21.15.42:0/3618455127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 0x55a3363a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:42:44.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.741+0000 7f740ea00640 1 --2- 172.21.15.42:0/3618455127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 0x55a3363ad340 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7400009cc0 tx=0x7f7400011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f945c65342abe077 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:44.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.742+0000 7f740f400640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f73f40a3da0 0x7f73f40a6260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f74080ce460 tx=0x7f7408005b40 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:44.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.742+0000 7f73ffe00640 1 -- 172.21.15.42:0/3618455127 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400005220 con 0x55a3363a4ca0 2024-04-20T10:42:44.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.742+0000 7f73ffe00640 1 -- 172.21.15.42:0/3618455127 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7400004c80 con 0x55a3363a4ca0 2024-04-20T10:42:44.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.742+0000 7f73ffe00640 1 -- 172.21.15.42:0/3618455127 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74000197d0 con 0x55a3363a4ca0 2024-04-20T10:42:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.743+0000 7f7418581740 1 -- 172.21.15.42:0/3618455127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 msgr2=0x55a3363ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.743+0000 7f7418581740 1 --2- 172.21.15.42:0/3618455127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 0x55a3363ad340 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7400009cc0 tx=0x7f7400011810 comp rx=0 tx=0).stop 2024-04-20T10:42:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.743+0000 7f7418581740 1 -- 172.21.15.42:0/3618455127 shutdown_connections 2024-04-20T10:42:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.743+0000 7f7418581740 1 --2- 172.21.15.42:0/3618455127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363ad880 0x55a3363afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.743+0000 7f7418581740 1 --2- 172.21.15.42:0/3618455127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 0x55a3363ad340 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.743+0000 7f7418581740 1 --2- 172.21.15.42:0/3618455127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 0x55a3363a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.743+0000 7f7418581740 1 -- 172.21.15.42:0/3618455127 >> 172.21.15.42:0/3618455127 conn(0x55a33639eab0 msgr2=0x55a33639ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.743+0000 7f7418581740 1 -- 172.21.15.42:0/3618455127 shutdown_connections 2024-04-20T10:42:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.743+0000 7f7418581740 1 -- 172.21.15.42:0/3618455127 wait complete. 2024-04-20T10:42:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.744+0000 7f7418581740 1 -- start start 2024-04-20T10:42:44.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.744+0000 7f7418581740 1 Processor -- start 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7418581740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4380 0x55a3363aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7418581740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4ca0 0x55a3363aae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7418581740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363ad880 0x55a3363ab360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7418581740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a3363b0670 con 0x55a3363a4ca0 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7418581740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a3362c3740 con 0x55a3363a4380 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7418581740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a3363b07f0 con 0x55a3363ad880 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7418581740 10 client.0 _ll_register_callbacks cb 0x55a3362beb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7406000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4380 0x55a3363aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7406000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4380 0x55a3363aa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59380/0 (socket says 172.21.15.42:59380) 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7406000640 1 -- 172.21.15.42:0/3856596352 learned_addr learned my addr 172.21.15.42:0/3856596352 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7406000640 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363ad880 0x55a3363ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f740ea00640 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4ca0 0x55a3363aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7406000640 1 -- 172.21.15.42:0/3856596352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363ad880 msgr2=0x55a3363ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7406000640 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363ad880 0x55a3363ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7406000640 1 -- 172.21.15.42:0/3856596352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4ca0 msgr2=0x55a3363aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.745+0000 7f7406000640 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4ca0 0x55a3363aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.746+0000 7f7406000640 1 -- 172.21.15.42:0/3856596352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a33601c3f0 con 0x55a3363a4380 2024-04-20T10:42:44.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.746+0000 7f7406000640 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4380 0x55a3363aa8e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009860 tx=0x7f73f802bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e2e545f8e92fb5e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:44.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.746+0000 7f73fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:42:44.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.746+0000 7f73fcc00640 1 -- 172.21.15.42:0/3856596352 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8049070 con 0x55a3363a4380 2024-04-20T10:42:44.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.746+0000 7f7418581740 1 -- 172.21.15.42:0/3856596352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7400004c80 con 0x55a3363a4380 2024-04-20T10:42:44.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.746+0000 7f73fcc00640 1 -- 172.21.15.42:0/3856596352 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f73f8004250 con 0x55a3363a4380 2024-04-20T10:42:44.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.747+0000 7f73fcc00640 1 -- 172.21.15.42:0/3856596352 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f73f8052050 con 0x55a3363a4380 2024-04-20T10:42:44.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.747+0000 7f7418581740 10 client.26408 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:44.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.747+0000 7f7418581740 1 -- 172.21.15.42:0/3856596352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a335fdc0f0 con 0x55a3363a4380 2024-04-20T10:42:44.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.747+0000 7f73fcc00640 1 -- 172.21.15.42:0/3856596352 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7400004c80 con 0x55a3363a4380 2024-04-20T10:42:44.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.747+0000 7f73fcc00640 1 -- 172.21.15.42:0/3856596352 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f73f804c020 con 0x55a3363a4380 2024-04-20T10:42:44.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.748+0000 7f7418581740 10 client.26408 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:44.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.748+0000 7f7418581740 10 client.26408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:44.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.748+0000 7f7418581740 20 client.26408 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:44.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.748+0000 7f7418581740 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a3363bdb40 0x55a3363bff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.748+0000 7f7418581740 4 client.26408 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:42:44.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.748+0000 7f7418581740 1 -- 172.21.15.42:0/3856596352 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a336317b70 con 0x55a3363bdb40 2024-04-20T10:42:44.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.748+0000 7f740ea00640 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a3363bdb40 0x55a3363bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.749+0000 7f740ea00640 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a3363bdb40 0x55a3363bff70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160f3ba62bedc9e9 server_cookie=4653835f905f26d7 in_seq=0 out_seq=0 2024-04-20T10:42:44.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.749+0000 7f73fcc00640 10 client.26408 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f73fcc00640 1 -- 172.21.15.42:0/3856596352 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a336317b70 con 0x55a3363bdb40 2024-04-20T10:42:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f73fcc00640 10 client.26408 handle_command_reply: tid=0 2024-04-20T10:42:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f7418581740 1 client.26408 shutdown 2024-04-20T10:42:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f7418581740 1 -- 172.21.15.42:0/3856596352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4380 msgr2=0x55a3363aa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f7418581740 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4380 0x55a3363aa8e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009860 tx=0x7f73f802bb70 comp rx=0 tx=0).stop 2024-04-20T10:42:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f7418581740 1 -- 172.21.15.42:0/3856596352 shutdown_connections 2024-04-20T10:42:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f7418581740 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a3363bdb40 0x55a3363bff70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f7418581740 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363ad880 0x55a3363ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f73fcc00640 0 client.26408 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f7418581740 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4ca0 0x55a3363aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f7418581740 1 --2- 172.21.15.42:0/3856596352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4380 0x55a3363aa8e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f7418581740 1 -- 172.21.15.42:0/3856596352 >> 172.21.15.42:0/3856596352 conn(0x55a33639eab0 msgr2=0x55a3363a0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f7418581740 1 -- 172.21.15.42:0/3856596352 shutdown_connections 2024-04-20T10:42:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f7418581740 1 -- 172.21.15.42:0/3856596352 wait complete. 2024-04-20T10:42:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.756+0000 7f7418581740 20 client.26408 trim_cache size 0 max 16384 2024-04-20T10:42:44.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.776+0000 7f7418581740 1 Processor -- start 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.776+0000 7f7418581740 1 -- start start 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.777+0000 7f7418581740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 0x55a3363ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.777+0000 7f7418581740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 0x55a3363b9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.777+0000 7f7418581740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363ad880 0x55a3363a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.777+0000 7f7418581740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f74101ade00 con 0x55a3363a4380 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.777+0000 7f7418581740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f74101b0660 con 0x55a3363ad880 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.777+0000 7f7418581740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f74101ae450 con 0x55a3363a4ca0 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.777+0000 7f7406000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 0x55a3363ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.777+0000 7f7406000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 0x55a3363ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43954/0 (socket says 172.21.15.42:43954) 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.777+0000 7f7406000640 1 -- 172.21.15.42:0/3951182066 learned_addr learned my addr 172.21.15.42:0/3951182066 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.777+0000 7f740ea00640 1 --2- 172.21.15.42:0/3951182066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 0x55a3363b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.777+0000 7f7406000640 1 --2- 172.21.15.42:0/3951182066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363ad880 0x55a3363a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.778+0000 7f740ea00640 1 -- 172.21.15.42:0/3951182066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363ad880 msgr2=0x55a3363a9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.778+0000 7f740ea00640 1 --2- 172.21.15.42:0/3951182066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363ad880 0x55a3363a9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.778+0000 7f740ea00640 1 -- 172.21.15.42:0/3951182066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 msgr2=0x55a3363ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.778+0000 7f740ea00640 1 --2- 172.21.15.42:0/3951182066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 0x55a3363ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.778+0000 7f740ea00640 1 -- 172.21.15.42:0/3951182066 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3363affe0 con 0x55a3363a4ca0 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.778+0000 7f7406000640 1 --2- 172.21.15.42:0/3951182066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 0x55a3363ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.778+0000 7f740ea00640 1 --2- 172.21.15.42:0/3951182066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 0x55a3363b9530 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f740001b040 tx=0x7f74000056e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e37153e8bcb5f624 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.778+0000 7f73ff400640 1 -- 172.21.15.42:0/3951182066 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400005290 con 0x55a3363a4ca0 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.778+0000 7f73ff400640 1 -- 172.21.15.42:0/3951182066 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740002d900 con 0x55a3363a4ca0 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.778+0000 7f73ff400640 1 -- 172.21.15.42:0/3951182066 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74000048c0 con 0x55a3363a4ca0 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.779+0000 7f7418581740 1 -- 172.21.15.42:0/3951182066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 msgr2=0x55a3363b9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.779+0000 7f7418581740 1 --2- 172.21.15.42:0/3951182066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 0x55a3363b9530 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f740001b040 tx=0x7f74000056e0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.779+0000 7f7418581740 1 -- 172.21.15.42:0/3951182066 shutdown_connections 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.779+0000 7f7418581740 1 --2- 172.21.15.42:0/3951182066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363ad880 0x55a3363a9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.779+0000 7f7418581740 1 --2- 172.21.15.42:0/3951182066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4ca0 0x55a3363b9530 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.779+0000 7f7418581740 1 --2- 172.21.15.42:0/3951182066 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363a4380 0x55a3363ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.779+0000 7f7418581740 1 -- 172.21.15.42:0/3951182066 >> 172.21.15.42:0/3951182066 conn(0x55a33639eab0 msgr2=0x55a33639d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.779+0000 7f7418581740 1 -- 172.21.15.42:0/3951182066 shutdown_connections 2024-04-20T10:42:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.779+0000 7f7418581740 1 -- 172.21.15.42:0/3951182066 wait complete. 2024-04-20T10:42:44.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.779+0000 7f7418581740 1 -- start start 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.780+0000 7f7418581740 1 Processor -- start 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.780+0000 7f7418581740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4380 0x55a3363a7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.780+0000 7f7418581740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4ca0 0x55a3363a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.780+0000 7f7418581740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363ad880 0x55a3363a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.780+0000 7f7418581740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f74101aeb20 con 0x55a3363ad880 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.780+0000 7f7418581740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f74101aff90 con 0x55a3363a4ca0 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.780+0000 7f7418581740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f74101af1f0 con 0x55a3363a4380 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.780+0000 7f7418581740 10 client.0 _ll_register_callbacks cb 0x7f7400005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.780+0000 7f740ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4ca0 0x55a3363a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.780+0000 7f7406000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4380 0x55a3363a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.780+0000 7f740ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4ca0 0x55a3363a7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59396/0 (socket says 172.21.15.42:59396) 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.780+0000 7f740ea00640 1 -- 172.21.15.42:0/3644467039 learned_addr learned my addr 172.21.15.42:0/3644467039 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:42:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.781+0000 7f7406000640 1 --2- 172.21.15.42:0/3644467039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363ad880 0x55a3363a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.781+0000 7f740ea00640 1 -- 172.21.15.42:0/3644467039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4380 msgr2=0x55a3363a7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.781+0000 7f740ea00640 1 --2- 172.21.15.42:0/3644467039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4380 0x55a3363a7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.781+0000 7f740ea00640 1 -- 172.21.15.42:0/3644467039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363ad880 msgr2=0x55a3363a8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.781+0000 7f740ea00640 1 --2- 172.21.15.42:0/3644467039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363ad880 0x55a3363a8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.781+0000 7f740ea00640 1 -- 172.21.15.42:0/3644467039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a336423100 con 0x55a3363a4ca0 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.781+0000 7f740ea00640 1 --2- 172.21.15.42:0/3644467039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4ca0 0x55a3363a7b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f740002f040 tx=0x7f74000092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2363365b7f7f308c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.781+0000 7f73fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.781+0000 7f73fcc00640 1 -- 172.21.15.42:0/3644467039 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400039020 con 0x55a3363a4ca0 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.781+0000 7f7418581740 1 -- 172.21.15.42:0/3644467039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f740002d900 con 0x55a3363a4ca0 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.781+0000 7f73fcc00640 1 -- 172.21.15.42:0/3644467039 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f740002dbd0 con 0x55a3363a4ca0 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.782+0000 7f73fcc00640 1 -- 172.21.15.42:0/3644467039 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7400044050 con 0x55a3363a4ca0 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.782+0000 7f7418581740 10 client.26414 fetch_fsmap learned FSMap version 19 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.782+0000 7f7418581740 1 -- 172.21.15.42:0/3644467039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a3364243f0 con 0x55a3363a4ca0 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.782+0000 7f73fcc00640 1 -- 172.21.15.42:0/3644467039 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740002d900 con 0x55a3363a4ca0 2024-04-20T10:42:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.782+0000 7f73fcc00640 1 -- 172.21.15.42:0/3644467039 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f7400034070 con 0x55a3363a4ca0 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.782+0000 7f7418581740 10 client.26414 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.782+0000 7f7418581740 10 client.26414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.782+0000 7f7418581740 20 client.26414 populate_metadata read hostname 'smithi042' 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.782+0000 7f7418581740 1 --2- 172.21.15.42:0/3644467039 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a336424940 0x55a336426d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.782+0000 7f7418581740 4 client.26414 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.782+0000 7f7418581740 1 -- 172.21.15.42:0/3644467039 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a3363119e0 con 0x55a336424940 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.783+0000 7f7406000640 1 --2- 172.21.15.42:0/3644467039 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a336424940 0x55a336426d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.783+0000 7f7406000640 1 --2- 172.21.15.42:0/3644467039 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a336424940 0x55a336426d60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a54d7fb670c40501 server_cookie=68e5ec88fa822e72 in_seq=0 out_seq=0 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.783+0000 7f73fcc00640 10 client.26414 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.784+0000 7f73fcc00640 1 -- 172.21.15.42:0/3644467039 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55a3363119e0 con 0x55a336424940 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.784+0000 7f73fcc00640 10 client.26414 handle_command_reply: tid=0 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.784+0000 7f7418581740 1 client.26414 shutdown 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.784+0000 7f7418581740 1 -- 172.21.15.42:0/3644467039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4ca0 msgr2=0x55a3363a7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.784+0000 7f7418581740 1 --2- 172.21.15.42:0/3644467039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4ca0 0x55a3363a7b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f740002f040 tx=0x7f74000092d0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.784+0000 7f7418581740 1 -- 172.21.15.42:0/3644467039 shutdown_connections 2024-04-20T10:42:44.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.784+0000 7f7418581740 1 --2- 172.21.15.42:0/3644467039 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55a336424940 0x55a336426d60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.784+0000 7f7418581740 1 --2- 172.21.15.42:0/3644467039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a3363ad880 0x55a3363a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.784+0000 7f7418581740 1 --2- 172.21.15.42:0/3644467039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3363a4ca0 0x55a3363a7b50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.784+0000 7f73fcc00640 0 client.26414 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:42:44.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.784+0000 7f7418581740 1 --2- 172.21.15.42:0/3644467039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a3363a4380 0x55a3363a7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.784+0000 7f7418581740 1 -- 172.21.15.42:0/3644467039 >> 172.21.15.42:0/3644467039 conn(0x55a33639eab0 msgr2=0x55a3363a98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:44.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.785+0000 7f7418581740 1 -- 172.21.15.42:0/3644467039 shutdown_connections 2024-04-20T10:42:44.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.785+0000 7f7418581740 1 -- 172.21.15.42:0/3644467039 wait complete. 2024-04-20T10:42:44.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.785+0000 7f7418581740 20 client.26414 trim_cache size 0 max 16384 2024-04-20T10:42:44.788 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:42:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (5770 inodes in the stack)", 2024-04-20T10:42:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:42:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "f553b6b9-7931-40c5-949a-1ea5f3978fe3": { 2024-04-20T10:42:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:42:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "f553b6b9-7931-40c5-949a-1ea5f3978fe3", 2024-04-20T10:42:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:42:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:42:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:42:44.789 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:42:44.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.785+0000 7f740fe00640 1 -- 172.21.15.42:0/2880309105 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f73f40a3da0 msgr2=0x7f73f40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.785+0000 7f740fe00640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f73f40a3da0 0x7f73f40a6260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f74080ce460 tx=0x7f7408005b40 comp rx=0 tx=0).stop 2024-04-20T10:42:44.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.785+0000 7f740fe00640 1 -- 172.21.15.42:0/2880309105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f741019a720 msgr2=0x7f741019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:42:44.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.785+0000 7f740fe00640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f741019a720 0x7f741019aba0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f740800eb10 tx=0x7f740800c2d0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.786+0000 7f740fe00640 1 -- 172.21.15.42:0/2880309105 shutdown_connections 2024-04-20T10:42:44.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.786+0000 7f740fe00640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f73f40a3da0 0x7f73f40a6260 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.786+0000 7f740fe00640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f741019a720 0x7f741019aba0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.786+0000 7f740fe00640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7410109350 0x7f74101a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.786+0000 7f740fe00640 1 --2- 172.21.15.42:0/2880309105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410104280 0x7f74101a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:42:44.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.786+0000 7f740fe00640 1 -- 172.21.15.42:0/2880309105 >> 172.21.15.42:0/2880309105 conn(0x7f74100ffaa0 msgr2=0x7f7410100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:42:44.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.786+0000 7f740fe00640 1 -- 172.21.15.42:0/2880309105 shutdown_connections 2024-04-20T10:42:44.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:42:44.786+0000 7f740fe00640 1 -- 172.21.15.42:0/2880309105 wait complete. 2024-04-20T10:42:44.803 INFO:tasks.cephfs.filesystem:scrub status for tag:f553b6b9-7931-40c5-949a-1ea5f3978fe3 - {'path': '/', 'tag': 'f553b6b9-7931-40c5-949a-1ea5f3978fe3', 'options': 'recursive,force'} 2024-04-20T10:42:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:45 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:42:46.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:45 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:42:46.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:45 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:42:46.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:45 smithi046 ceph-mon[21574]: pgmap v269: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 2.53k op/s 2024-04-20T10:42:46.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:45 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:42:46.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:45 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:42:46.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:45 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:42:46.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:45 smithi042 ceph-mon[17078]: pgmap v269: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 2.53k op/s 2024-04-20T10:42:46.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:45 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:42:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:45 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:42:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:45 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:42:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:45 smithi162 ceph-mon[21854]: pgmap v269: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 2.53k op/s 2024-04-20T10:42:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:47 smithi046 ceph-mon[21574]: pgmap v270: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 2.43k op/s 2024-04-20T10:42:48.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:47 smithi042 ceph-mon[17078]: pgmap v270: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 2.43k op/s 2024-04-20T10:42:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:47 smithi162 ceph-mon[21854]: pgmap v270: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 2.43k op/s 2024-04-20T10:42:49.950 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:49 smithi162 ceph-mon[21854]: pgmap v271: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 1.96k op/s 2024-04-20T10:42:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:49 smithi046 ceph-mon[21574]: pgmap v271: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 1.96k op/s 2024-04-20T10:42:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:49 smithi042 ceph-mon[17078]: pgmap v271: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 1.96k op/s 2024-04-20T10:42:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:51 smithi046 ceph-mon[21574]: pgmap v272: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 805 op/s 2024-04-20T10:42:52.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:51 smithi042 ceph-mon[17078]: pgmap v272: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 805 op/s 2024-04-20T10:42:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:51 smithi162 ceph-mon[21854]: pgmap v272: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 805 op/s 2024-04-20T10:42:54.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:53 smithi046 ceph-mon[21574]: pgmap v273: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 1.53k op/s 2024-04-20T10:42:54.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:53 smithi042 ceph-mon[17078]: pgmap v273: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 1.53k op/s 2024-04-20T10:42:54.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:53 smithi162 ceph-mon[21854]: pgmap v273: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 1.53k op/s 2024-04-20T10:42:56.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:55 smithi046 ceph-mon[21574]: pgmap v274: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 1.01k op/s 2024-04-20T10:42:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:55 smithi042 ceph-mon[17078]: pgmap v274: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 1.01k op/s 2024-04-20T10:42:56.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:55 smithi162 ceph-mon[21854]: pgmap v274: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 1.01k op/s 2024-04-20T10:42:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:57 smithi046 ceph-mon[21574]: pgmap v275: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 1.43k op/s 2024-04-20T10:42:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:57 smithi042 ceph-mon[17078]: pgmap v275: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 1.43k op/s 2024-04-20T10:42:58.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:57 smithi162 ceph-mon[21854]: pgmap v275: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 1.43k op/s 2024-04-20T10:42:58.641 INFO:tasks.workunit.client.0.smithi042.stdout: CC scripts/mod/empty.o 2024-04-20T10:42:58.641 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/mod/mk_elfconfig 2024-04-20T10:42:58.643 INFO:tasks.workunit.client.0.smithi042.stdout: CC scripts/mod/devicetable-offsets.s 2024-04-20T10:42:58.750 INFO:tasks.workunit.client.0.smithi042.stdout: UPD scripts/mod/devicetable-offsets.h 2024-04-20T10:42:58.751 INFO:tasks.workunit.client.0.smithi042.stdout: MKELF scripts/mod/elfconfig.h 2024-04-20T10:42:58.761 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/mod/modpost.o 2024-04-20T10:42:58.761 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/mod/sumversion.o 2024-04-20T10:42:59.016 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC scripts/mod/file2alias.o 2024-04-20T10:42:59.630 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTLD scripts/mod/modpost 2024-04-20T10:42:59.691 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/bounds.s 2024-04-20T10:42:59.693 INFO:tasks.workunit.client.0.smithi042.stdout: CHKSHA1 include/linux/atomic/atomic-arch-fallback.h 2024-04-20T10:42:59.694 INFO:tasks.workunit.client.0.smithi042.stdout: CHKSHA1 include/linux/atomic/atomic-instrumented.h 2024-04-20T10:42:59.695 INFO:tasks.workunit.client.0.smithi042.stdout: CHKSHA1 include/linux/atomic/atomic-long.h 2024-04-20T10:42:59.715 INFO:tasks.workunit.client.0.smithi042.stdout: UPD include/generated/timeconst.h 2024-04-20T10:42:59.808 INFO:tasks.workunit.client.0.smithi042.stdout: UPD include/generated/bounds.h 2024-04-20T10:42:59.833 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/asm-offsets.s 2024-04-20T10:43:00.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:42:59 smithi046 ceph-mon[21574]: pgmap v276: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 2.40k op/s 2024-04-20T10:43:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:42:59 smithi042 ceph-mon[17078]: pgmap v276: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 2.40k op/s 2024-04-20T10:43:00.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:42:59 smithi162 ceph-mon[21854]: pgmap v276: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 2.40k op/s 2024-04-20T10:43:00.548 INFO:tasks.workunit.client.0.smithi042.stdout: UPD include/generated/asm-offsets.h 2024-04-20T10:43:00.559 INFO:tasks.workunit.client.0.smithi042.stdout: CALL scripts/checksyscalls.sh 2024-04-20T10:43:00.649 INFO:tasks.workunit.client.0.smithi042.stdout: LDS scripts/module.lds 2024-04-20T10:43:00.858 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC usr/gen_init_cpio 2024-04-20T10:43:00.861 INFO:tasks.workunit.client.0.smithi042.stdout: CC init/main.o 2024-04-20T10:43:00.861 INFO:tasks.workunit.client.0.smithi042.stdout: CC certs/system_keyring.o 2024-04-20T10:43:00.865 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/filemap.o 2024-04-20T10:43:00.868 INFO:tasks.workunit.client.0.smithi042.stdout: CC ipc/compat.o 2024-04-20T10:43:01.028 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/notify/dnotify/dnotify.o 2024-04-20T10:43:01.143 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/vdso/vma.o 2024-04-20T10:43:01.460 INFO:tasks.workunit.client.0.smithi042.stdout: GEN usr/initramfs_data.cpio 2024-04-20T10:43:01.474 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/sched/core.o 2024-04-20T10:43:01.542 INFO:tasks.workunit.client.0.smithi042.stdout: COPY usr/initramfs_inc_data 2024-04-20T10:43:01.558 INFO:tasks.workunit.client.0.smithi042.stdout: AS usr/initramfs_data.o 2024-04-20T10:43:01.633 INFO:tasks.workunit.client.0.smithi042.stdout: AR usr/built-in.a 2024-04-20T10:43:01.635 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC certs/extract-cert 2024-04-20T10:43:01.726 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/vsyscall/vsyscall_64.o 2024-04-20T10:43:01.980 INFO:tasks.workunit.client.0.smithi042.stdout: CC ipc/util.o 2024-04-20T10:43:02.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:01 smithi046 ceph-mon[21574]: pgmap v277: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 904 KiB/s wr, 2.30k op/s 2024-04-20T10:43:02.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:01 smithi042 ceph-mon[17078]: pgmap v277: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 904 KiB/s wr, 2.30k op/s 2024-04-20T10:43:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:01 smithi162 ceph-mon[21854]: pgmap v277: 97 pgs: 97 active+clean; 1.7 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 904 KiB/s wr, 2.30k op/s 2024-04-20T10:43:02.180 INFO:tasks.workunit.client.0.smithi042.stdout: CERT certs/x509_certificate_list 2024-04-20T10:43:02.186 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/notify/dnotify/built-in.a 2024-04-20T10:43:02.198 INFO:tasks.workunit.client.0.smithi042.stdout: CERT certs/signing_key.x509 2024-04-20T10:43:02.224 INFO:tasks.workunit.client.0.smithi042.stdout: AS certs/system_certificates.o 2024-04-20T10:43:02.276 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/notify/inotify/inotify_fsnotify.o 2024-04-20T10:43:02.685 INFO:tasks.workunit.client.0.smithi042.stdout: AR certs/built-in.a 2024-04-20T10:43:02.720 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/vdso/extable.o 2024-04-20T10:43:02.755 INFO:tasks.workunit.client.0.smithi042.stdout: UPD init/utsversion-tmp.h 2024-04-20T10:43:02.764 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/sched/fair.o 2024-04-20T10:43:02.781 INFO:tasks.workunit.client.0.smithi042.stdout: CC init/do_mounts.o 2024-04-20T10:43:02.928 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/entry/vsyscall/vsyscall_emu_64.o 2024-04-20T10:43:03.033 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/entry/vsyscall/built-in.a 2024-04-20T10:43:03.108 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/notify/fanotify/built-in.a 2024-04-20T10:43:03.167 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs_common/nfsacl.o 2024-04-20T10:43:03.243 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/notify/inotify/inotify_user.o 2024-04-20T10:43:03.279 INFO:tasks.workunit.client.0.smithi042.stdout: CC ipc/msgutil.o 2024-04-20T10:43:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:03 smithi046 ceph-mon[21574]: pgmap v278: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.1 MiB/s wr, 4.20k op/s 2024-04-20T10:43:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:03 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:43:04.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:03 smithi042 ceph-mon[17078]: pgmap v278: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.1 MiB/s wr, 4.20k op/s 2024-04-20T10:43:04.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:03 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:43:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:03 smithi162 ceph-mon[21854]: pgmap v278: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.1 MiB/s wr, 4.20k op/s 2024-04-20T10:43:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:03 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:43:04.214 INFO:tasks.workunit.client.0.smithi042.stdout: CC ipc/msg.o 2024-04-20T10:43:04.268 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/vdso/vdso32-setup.o 2024-04-20T10:43:04.270 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs_common/grace.o 2024-04-20T10:43:04.278 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/mempool.o 2024-04-20T10:43:04.543 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/notify/inotify/built-in.a 2024-04-20T10:43:04.557 INFO:tasks.workunit.client.0.smithi042.stdout: CC init/do_mounts_initrd.o 2024-04-20T10:43:04.613 INFO:tasks.workunit.client.0.smithi042.stdout: LDS arch/x86/entry/vdso/vdso.lds 2024-04-20T10:43:04.636 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/notify/fsnotify.o 2024-04-20T10:43:04.668 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/entry/vdso/vdso-note.o 2024-04-20T10:43:04.782 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/vdso/vclock_gettime.o 2024-04-20T10:43:05.467 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/oom_kill.o 2024-04-20T10:43:05.477 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/nfs_common/built-in.a 2024-04-20T10:43:05.543 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/amd/core.o 2024-04-20T10:43:05.667 INFO:tasks.workunit.client.0.smithi042.stdout: CC ipc/sem.o 2024-04-20T10:43:05.702 INFO:tasks.workunit.client.0.smithi042.stdout: CC init/initramfs.o 2024-04-20T10:43:05.715 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/vdso/vgetcpu.o 2024-04-20T10:43:05.847 INFO:tasks.workunit.client.0.smithi042.stdout: HOSTCC arch/x86/entry/vdso/vdso2c 2024-04-20T10:43:05.930 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/notify/notification.o 2024-04-20T10:43:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:05 smithi046 ceph-mon[21574]: pgmap v279: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 555 KiB/s wr, 3.68k op/s 2024-04-20T10:43:06.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:05 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:43:06.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:05 smithi042 ceph-mon[17078]: pgmap v279: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 555 KiB/s wr, 3.68k op/s 2024-04-20T10:43:06.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:05 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:43:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:05 smithi162 ceph-mon[21854]: pgmap v279: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 555 KiB/s wr, 3.68k op/s 2024-04-20T10:43:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:05 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:43:06.268 INFO:tasks.workunit.client.0.smithi042.stdout: LDS arch/x86/entry/vdso/vdso32/vdso32.lds 2024-04-20T10:43:06.325 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/entry/vdso/vdso32/note.o 2024-04-20T10:43:06.388 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/entry/vdso/vdso32/system_call.o 2024-04-20T10:43:06.458 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/entry/vdso/vdso32/sigreturn.o 2024-04-20T10:43:06.531 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/vdso/vdso32/vclock_gettime.o 2024-04-20T10:43:06.706 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/amd/lbr.o 2024-04-20T10:43:06.731 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/notify/group.o 2024-04-20T10:43:06.854 INFO:tasks.workunit.client.0.smithi042.stdout: CC init/calibrate.o 2024-04-20T10:43:07.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:06 smithi046 ceph-mon[21574]: osdmap e93: 12 total, 12 up, 12 in 2024-04-20T10:43:07.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:06 smithi042 ceph-mon[17078]: osdmap e93: 12 total, 12 up, 12 in 2024-04-20T10:43:07.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:06 smithi162 ceph-mon[21854]: osdmap e93: 12 total, 12 up, 12 in 2024-04-20T10:43:07.375 INFO:tasks.workunit.client.0.smithi042.stdout: CC ipc/shm.o 2024-04-20T10:43:07.377 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/fadvise.o 2024-04-20T10:43:07.379 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/notify/mark.o 2024-04-20T10:43:07.380 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/locking/mutex.o 2024-04-20T10:43:07.387 INFO:tasks.workunit.client.0.smithi042.stdout: CC init/init_task.o 2024-04-20T10:43:07.387 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/sched/build_policy.o 2024-04-20T10:43:07.400 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/amd/ibs.o 2024-04-20T10:43:07.553 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/vdso/vdso32/vgetcpu.o 2024-04-20T10:43:07.816 INFO:tasks.workunit.client.0.smithi042.stdout: VDSO arch/x86/entry/vdso/vdso64.so.dbg 2024-04-20T10:43:07.963 INFO:tasks.workunit.client.0.smithi042.stdout: VDSO arch/x86/entry/vdso/vdso32.so.dbg 2024-04-20T10:43:08.045 INFO:tasks.workunit.client.0.smithi042.stdout: OBJCOPY arch/x86/entry/vdso/vdso64.so 2024-04-20T10:43:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:07 smithi046 ceph-mon[21574]: pgmap v281: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 990 KiB/s wr, 4.11k op/s 2024-04-20T10:43:08.061 INFO:tasks.workunit.client.0.smithi042.stdout: OBJCOPY arch/x86/entry/vdso/vdso32.so 2024-04-20T10:43:08.075 INFO:tasks.workunit.client.0.smithi042.stdout: VDSO2C arch/x86/entry/vdso/vdso-image-64.c 2024-04-20T10:43:08.089 INFO:tasks.workunit.client.0.smithi042.stdout: VDSO2C arch/x86/entry/vdso/vdso-image-32.c 2024-04-20T10:43:08.105 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/vdso/vdso-image-64.o 2024-04-20T10:43:08.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:07 smithi042 ceph-mon[17078]: pgmap v281: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 990 KiB/s wr, 4.11k op/s 2024-04-20T10:43:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:07 smithi162 ceph-mon[21854]: pgmap v281: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 990 KiB/s wr, 4.11k op/s 2024-04-20T10:43:08.262 INFO:tasks.workunit.client.0.smithi042.stdout: CC init/version.o 2024-04-20T10:43:08.275 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/maccess.o 2024-04-20T10:43:08.363 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/notify/fdinfo.o 2024-04-20T10:43:08.439 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/vdso/vdso-image-32.o 2024-04-20T10:43:08.573 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/amd/uncore.o 2024-04-20T10:43:08.633 INFO:tasks.workunit.client.0.smithi042.stdout: AR init/built-in.a 2024-04-20T10:43:08.644 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/locking/semaphore.o 2024-04-20T10:43:08.764 INFO:tasks.workunit.client.0.smithi042.stdout: CC ipc/syscall.o 2024-04-20T10:43:08.771 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/entry/vdso/built-in.a 2024-04-20T10:43:08.808 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/page-writeback.o 2024-04-20T10:43:08.818 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/entry/entry.o 2024-04-20T10:43:08.888 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/entry/entry_64.o 2024-04-20T10:43:09.024 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/syscall_64.o 2024-04-20T10:43:09.119 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/iomap/trace.o 2024-04-20T10:43:09.123 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/notify/built-in.a 2024-04-20T10:43:09.202 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/realmode/init.o 2024-04-20T10:43:09.957 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/amd/iommu.o 2024-04-20T10:43:09.960 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/locking/rwsem.o 2024-04-20T10:43:09.967 INFO:tasks.workunit.client.0.smithi042.stdout: CC ipc/ipc_sysctl.o 2024-04-20T10:43:09.967 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/common.o 2024-04-20T10:43:09.977 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/realmode/rm/header.o 2024-04-20T10:43:10.047 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/realmode/rm/trampoline_64.o 2024-04-20T10:43:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:09 smithi046 ceph-mon[21574]: pgmap v282: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 828 KiB/s wr, 3.44k op/s 2024-04-20T10:43:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:09 smithi046 ceph-mon[21574]: osdmap e94: 12 total, 12 up, 12 in 2024-04-20T10:43:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:09 smithi042 ceph-mon[17078]: pgmap v282: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 828 KiB/s wr, 3.44k op/s 2024-04-20T10:43:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:09 smithi042 ceph-mon[17078]: osdmap e94: 12 total, 12 up, 12 in 2024-04-20T10:43:10.126 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/realmode/rm/stack.o 2024-04-20T10:43:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:09 smithi162 ceph-mon[21854]: pgmap v282: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 828 KiB/s wr, 3.44k op/s 2024-04-20T10:43:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:09 smithi162 ceph-mon[21854]: osdmap e94: 12 total, 12 up, 12 in 2024-04-20T10:43:10.193 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/realmode/rm/reboot.o 2024-04-20T10:43:10.226 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/iomap/iter.o 2024-04-20T10:43:10.267 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/realmode/rm/wakeup_asm.o 2024-04-20T10:43:10.349 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/realmode/rm/wakemain.o 2024-04-20T10:43:10.434 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/folio-compat.o 2024-04-20T10:43:10.504 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/realmode/rm/video-mode.o 2024-04-20T10:43:10.554 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/sched/build_utility.o 2024-04-20T10:43:10.663 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/realmode/rm/copy.o 2024-04-20T10:43:10.688 INFO:tasks.workunit.client.0.smithi042.stdout: CC ipc/mqueue.o 2024-04-20T10:43:10.736 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/realmode/rm/bioscall.o 2024-04-20T10:43:10.846 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/realmode/rm/regs.o 2024-04-20T10:43:10.912 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/locking/percpu-rwsem.o 2024-04-20T10:43:10.913 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/entry/thunk_64.o 2024-04-20T10:43:10.952 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/realmode/rm/video-vga.o 2024-04-20T10:43:10.968 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/events/amd/built-in.a 2024-04-20T10:43:10.994 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/entry/entry_64_compat.o 2024-04-20T10:43:11.048 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/core.o 2024-04-20T10:43:11.090 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/entry/syscall_32.o 2024-04-20T10:43:11.096 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/realmode/rm/video-vesa.o 2024-04-20T10:43:11.133 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/iomap/buffered-io.o 2024-04-20T10:43:11.227 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/realmode/rm/video-bios.o 2024-04-20T10:43:11.390 INFO:tasks.workunit.client.0.smithi042.stdout: PASYMS arch/x86/realmode/rm/pasyms.h 2024-04-20T10:43:11.406 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/readahead.o 2024-04-20T10:43:11.448 INFO:tasks.workunit.client.0.smithi042.stdout: LDS arch/x86/realmode/rm/realmode.lds 2024-04-20T10:43:11.498 INFO:tasks.workunit.client.0.smithi042.stdout: LD arch/x86/realmode/rm/realmode.elf 2024-04-20T10:43:11.519 INFO:tasks.workunit.client.0.smithi042.stdout: RELOCS arch/x86/realmode/rm/realmode.relocs 2024-04-20T10:43:11.532 INFO:tasks.workunit.client.0.smithi042.stdout: OBJCOPY arch/x86/realmode/rm/realmode.bin 2024-04-20T10:43:11.555 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/realmode/rmpiggy.o 2024-04-20T10:43:11.623 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/realmode/built-in.a 2024-04-20T10:43:11.628 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/locking/spinlock.o 2024-04-20T10:43:11.707 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/zhaoxin/core.o 2024-04-20T10:43:11.920 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/entry/built-in.a 2024-04-20T10:43:11.973 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/core.o 2024-04-20T10:43:12.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:11 smithi046 ceph-mon[21574]: pgmap v284: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 653 KiB/s wr, 1.46k op/s 2024-04-20T10:43:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:11 smithi042 ceph-mon[17078]: pgmap v284: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 653 KiB/s wr, 1.46k op/s 2024-04-20T10:43:12.109 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/locking/osq_lock.o 2024-04-20T10:43:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:11 smithi162 ceph-mon[21854]: pgmap v284: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 653 KiB/s wr, 1.46k op/s 2024-04-20T10:43:12.421 INFO:tasks.workunit.client.0.smithi042.stdout: CC ipc/namespace.o 2024-04-20T10:43:12.496 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/locking/qspinlock.o 2024-04-20T10:43:12.503 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/swap.o 2024-04-20T10:43:12.676 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/events/zhaoxin/built-in.a 2024-04-20T10:43:12.744 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/quota/dquot.o 2024-04-20T10:43:12.835 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/iomap/direct-io.o 2024-04-20T10:43:13.096 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/locking/rtmutex_api.o 2024-04-20T10:43:13.122 INFO:tasks.workunit.client.0.smithi042.stdout: CC ipc/mq_sysctl.o 2024-04-20T10:43:13.546 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/bts.o 2024-04-20T10:43:13.649 INFO:tasks.workunit.client.0.smithi042.stdout: AR ipc/built-in.a 2024-04-20T10:43:13.711 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/quota/quota_v2.o 2024-04-20T10:43:13.849 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/power/qos.o 2024-04-20T10:43:13.857 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/sched/built-in.a 2024-04-20T10:43:13.902 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/iomap/fiemap.o 2024-04-20T10:43:13.961 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/truncate.o 2024-04-20T10:43:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:13 smithi046 ceph-mon[21574]: pgmap v285: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 1.21k op/s 2024-04-20T10:43:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:13 smithi042 ceph-mon[17078]: pgmap v285: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 1.21k op/s 2024-04-20T10:43:14.121 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/locking/qrwlock.o 2024-04-20T10:43:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:13 smithi162 ceph-mon[21854]: pgmap v285: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 1.21k op/s 2024-04-20T10:43:14.252 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/task_mmu.o 2024-04-20T10:43:14.457 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/ds.o 2024-04-20T10:43:14.559 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/gc.o 2024-04-20T10:43:14.563 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/quota/quota_tree.o 2024-04-20T10:43:14.672 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/iomap/seek.o 2024-04-20T10:43:14.699 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/locking/built-in.a 2024-04-20T10:43:14.774 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/iomap/swapfile.o 2024-04-20T10:43:14.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:43:14.804 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:43:14.952 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/power/main.o 2024-04-20T10:43:15.124 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/vmscan.o 2024-04-20T10:43:15.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.245+0000 7f7ac7200640 1 -- 172.21.15.42:0/1309299256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac8110600 msgr2=0x7f7ac81109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:15.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.245+0000 7f7ac7200640 1 --2- 172.21.15.42:0/1309299256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac8110600 0x7f7ac81109e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009900 tx=0x7f7ab802bcd0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.245+0000 7f7ac7200640 1 -- 172.21.15.42:0/1309299256 shutdown_connections 2024-04-20T10:43:15.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.245+0000 7f7ac7200640 1 --2- 172.21.15.42:0/1309299256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac81099e0 0x7f7ac811d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.245+0000 7f7ac7200640 1 --2- 172.21.15.42:0/1309299256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac8108f60 0x7f7ac81093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.245+0000 7f7ac7200640 1 --2- 172.21.15.42:0/1309299256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac8110600 0x7f7ac81109e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.245+0000 7f7ac7200640 1 -- 172.21.15.42:0/1309299256 >> 172.21.15.42:0/1309299256 conn(0x7f7ac806cee0 msgr2=0x7f7ac806d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:15.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.245+0000 7f7ac7200640 1 -- 172.21.15.42:0/1309299256 shutdown_connections 2024-04-20T10:43:15.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.246+0000 7f7ac7200640 1 -- 172.21.15.42:0/1309299256 wait complete. 2024-04-20T10:43:15.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.247+0000 7f7ac7200640 1 Processor -- start 2024-04-20T10:43:15.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.247+0000 7f7ac7200640 1 -- start start 2024-04-20T10:43:15.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.248+0000 7f7ac7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac8108f60 0x7f7ac81a7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.248+0000 7f7ac7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac81099e0 0x7f7ac81a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.248+0000 7f7ac7200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac8110600 0x7f7ac81abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.248+0000 7f7ac7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8121060 con 0x7f7ac81099e0 2024-04-20T10:43:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.248+0000 7f7ac7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8120ee0 con 0x7f7ac8108f60 2024-04-20T10:43:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.248+0000 7f7ac7200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81211e0 con 0x7f7ac8110600 2024-04-20T10:43:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.248+0000 7f7ac5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac8108f60 0x7f7ac81a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.248+0000 7f7ac5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac8108f60 0x7f7ac81a7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42800/0 (socket says 172.21.15.42:42800) 2024-04-20T10:43:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.248+0000 7f7ac5e00640 1 -- 172.21.15.42:0/2656987359 learned_addr learned my addr 172.21.15.42:0/2656987359 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.249+0000 7f7ac6800640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac8110600 0x7f7ac81abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.249+0000 7f7ac5400640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac81099e0 0x7f7ac81a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.249+0000 7f7ac5e00640 1 -- 172.21.15.42:0/2656987359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac8110600 msgr2=0x7f7ac81abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.249+0000 7f7ac5e00640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac8110600 0x7f7ac81abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.249+0000 7f7ac5e00640 1 -- 172.21.15.42:0/2656987359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac81099e0 msgr2=0x7f7ac81a7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:15.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.250+0000 7f7ac5e00640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac81099e0 0x7f7ac81a7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.250+0000 7f7ac5e00640 1 -- 172.21.15.42:0/2656987359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac81ac4e0 con 0x7f7ac8108f60 2024-04-20T10:43:15.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.250+0000 7f7ac5400640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac81099e0 0x7f7ac81a7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:15.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.250+0000 7f7ac5e00640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac8108f60 0x7f7ac81a7520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7ab802c2e0 tx=0x7f7ab80045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:15.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.252+0000 7f7abea00640 1 -- 172.21.15.42:0/2656987359 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8041020 con 0x7f7ac8108f60 2024-04-20T10:43:15.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.252+0000 7f7abea00640 1 -- 172.21.15.42:0/2656987359 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab80054e0 con 0x7f7ac8108f60 2024-04-20T10:43:15.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.252+0000 7f7abea00640 1 -- 172.21.15.42:0/2656987359 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8003c50 con 0x7f7ac8108f60 2024-04-20T10:43:15.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.252+0000 7f7ac7200640 1 -- 172.21.15.42:0/2656987359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac81ac770 con 0x7f7ac8108f60 2024-04-20T10:43:15.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.253+0000 7f7ac7200640 1 -- 172.21.15.42:0/2656987359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac81acb50 con 0x7f7ac8108f60 2024-04-20T10:43:15.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.260+0000 7f7acffd4740 1 Processor -- start 2024-04-20T10:43:15.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.261+0000 7f7abea00640 1 -- 172.21.15.42:0/2656987359 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7ab8044020 con 0x7f7ac8108f60 2024-04-20T10:43:15.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.262+0000 7f7acffd4740 1 -- start start 2024-04-20T10:43:15.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.262+0000 7f7acffd4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.262+0000 7f7acffd4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7acca0 0x55d8aa7b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.262+0000 7f7acffd4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7b5880 0x55d8aa7b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.262+0000 7f7acffd4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d8aa712150 con 0x55d8aa7ac380 2024-04-20T10:43:15.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.262+0000 7f7acffd4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d8aa6b8370 con 0x55d8aa7acca0 2024-04-20T10:43:15.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.262+0000 7f7acffd4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d8aa70e110 con 0x55d8aa7b5880 2024-04-20T10:43:15.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.263+0000 7f7ac5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7acca0 0x55d8aa7b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.263+0000 7f7ac5400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7b5880 0x55d8aa7b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.263+0000 7f7ac5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7acca0 0x55d8aa7b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42816/0 (socket says 172.21.15.42:42816) 2024-04-20T10:43:15.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.263+0000 7f7ac5e00640 1 -- 172.21.15.42:0/237682205 learned_addr learned my addr 172.21.15.42:0/237682205 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:15.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.263+0000 7f7ac6800640 1 --2- 172.21.15.42:0/237682205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.264+0000 7f7ac5400640 1 -- 172.21.15.42:0/237682205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7acca0 msgr2=0x55d8aa7b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:15.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.264+0000 7f7ac5400640 1 --2- 172.21.15.42:0/237682205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7acca0 0x55d8aa7b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.264+0000 7f7ac5400640 1 -- 172.21.15.42:0/237682205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 msgr2=0x55d8aa7ac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:15.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.264+0000 7f7abea00640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7aac0a3de0 0x7f7aac0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.264+0000 7f7abea00640 1 -- 172.21.15.42:0/2656987359 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f7ab80f1b10 con 0x7f7ac8108f60 2024-04-20T10:43:15.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.265+0000 7f7ac5400640 1 --2- 172.21.15.42:0/237682205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7ac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.265+0000 7f7ac5400640 1 -- 172.21.15.42:0/237682205 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8aa4ee7d0 con 0x55d8aa7b5880 2024-04-20T10:43:15.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.265+0000 7f7ac5400640 1 --2- 172.21.15.42:0/237682205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7b5880 0x55d8aa7b7c70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0005fd0 tx=0x7f7ab000e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=777c7a223c7cb848 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:15.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.265+0000 7f7ab7400640 1 -- 172.21.15.42:0/237682205 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0016070 con 0x55d8aa7b5880 2024-04-20T10:43:15.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.265+0000 7f7ab7400640 1 -- 172.21.15.42:0/237682205 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab0005500 con 0x55d8aa7b5880 2024-04-20T10:43:15.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.265+0000 7f7ab7400640 1 -- 172.21.15.42:0/237682205 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0011470 con 0x55d8aa7b5880 2024-04-20T10:43:15.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.266+0000 7f7ac6800640 1 --2- 172.21.15.42:0/237682205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:15.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.267+0000 7f7ac6800640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7aac0a3de0 0x7f7aac0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.268+0000 7f7acffd4740 1 -- 172.21.15.42:0/237682205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7b5880 msgr2=0x55d8aa7b7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:15.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.268+0000 7f7acffd4740 1 --2- 172.21.15.42:0/237682205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7b5880 0x55d8aa7b7c70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0005fd0 tx=0x7f7ab000e6c0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.269+0000 7f7ac6800640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7aac0a3de0 0x7f7aac0a62a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009fd0 tx=0x7f7ac000c040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:15.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.269+0000 7f7acffd4740 1 -- 172.21.15.42:0/237682205 shutdown_connections 2024-04-20T10:43:15.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.269+0000 7f7acffd4740 1 --2- 172.21.15.42:0/237682205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7b5880 0x55d8aa7b7c70 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0005fd0 tx=0x7f7ab000e6c0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.269+0000 7f7acffd4740 1 --2- 172.21.15.42:0/237682205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7acca0 0x55d8aa7b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.269+0000 7f7acffd4740 1 --2- 172.21.15.42:0/237682205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.269+0000 7f7acffd4740 1 -- 172.21.15.42:0/237682205 >> 172.21.15.42:0/237682205 conn(0x55d8aa7a6ab0 msgr2=0x55d8aa7a6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:15.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.270+0000 7f7acffd4740 1 -- 172.21.15.42:0/237682205 shutdown_connections 2024-04-20T10:43:15.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.270+0000 7f7acffd4740 1 -- 172.21.15.42:0/237682205 wait complete. 2024-04-20T10:43:15.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.270+0000 7f7acffd4740 1 -- start start 2024-04-20T10:43:15.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.271+0000 7f7acffd4740 1 Processor -- start 2024-04-20T10:43:15.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.275+0000 7f7acffd4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.275+0000 7f7acffd4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7acca0 0x55d8aa7b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.275+0000 7f7acffd4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 0x55d8aa7c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.275+0000 7f7acffd4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d8aa7b8670 con 0x55d8aa7ac380 2024-04-20T10:43:15.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.275+0000 7f7acffd4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d8aa6cb740 con 0x55d8aa7b3460 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.275+0000 7f7acffd4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d8aa7b87f0 con 0x55d8aa7acca0 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.275+0000 7f7acffd4740 10 client.0 _ll_register_callbacks cb 0x55d8aa6c6b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.275+0000 7f7ac6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 0x55d8aa7c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.275+0000 7f7ac6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 0x55d8aa7c1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42820/0 (socket says 172.21.15.42:42820) 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.275+0000 7f7ac6800640 1 -- 172.21.15.42:0/163309026 learned_addr learned my addr 172.21.15.42:0/163309026 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.275+0000 7f7ac5400640 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7acca0 0x55d8aa7b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.275+0000 7f7ac5e00640 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.276+0000 7f7ac6800640 1 -- 172.21.15.42:0/163309026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7acca0 msgr2=0x55d8aa7b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.276+0000 7f7ac6800640 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7acca0 0x55d8aa7b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.276+0000 7f7ac6800640 1 -- 172.21.15.42:0/163309026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 msgr2=0x55d8aa7b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.276+0000 7f7ac6800640 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.276+0000 7f7ac6800640 1 -- 172.21.15.42:0/163309026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8aa4243f0 con 0x55d8aa7b3460 2024-04-20T10:43:15.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.276+0000 7f7ac5e00640 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:43:15.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.277+0000 7f7ac6800640 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 0x55d8aa7c1ae0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0007f20 tx=0x7f7ac000e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba8d0f881bc416a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:15.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.277+0000 7f7aabe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:43:15.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.277+0000 7f7aabe00640 1 -- 172.21.15.42:0/163309026 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0027060 con 0x55d8aa7b3460 2024-04-20T10:43:15.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.277+0000 7f7aabe00640 1 -- 172.21.15.42:0/163309026 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f7ac0017420 con 0x55d8aa7b3460 2024-04-20T10:43:15.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.277+0000 7f7acffd4740 1 -- 172.21.15.42:0/163309026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab0005500 con 0x55d8aa7b3460 2024-04-20T10:43:15.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.279+0000 7f7aabe00640 1 -- 172.21.15.42:0/163309026 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7ac0017620 con 0x55d8aa7b3460 2024-04-20T10:43:15.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.279+0000 7f7acffd4740 10 client.26426 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:15.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.279+0000 7f7acffd4740 1 -- 172.21.15.42:0/163309026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d8aa3e40f0 con 0x55d8aa7b3460 2024-04-20T10:43:15.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.279+0000 7f7aabe00640 1 -- 172.21.15.42:0/163309026 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab0005500 con 0x55d8aa7b3460 2024-04-20T10:43:15.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.280+0000 7f7aabe00640 1 -- 172.21.15.42:0/163309026 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f7ac002b600 con 0x55d8aa7b3460 2024-04-20T10:43:15.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.283+0000 7f7acffd4740 10 client.26426 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:15.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.283+0000 7f7acffd4740 10 client.26426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:15.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.283+0000 7f7acffd4740 20 client.26426 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:15.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.283+0000 7f7acffd4740 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d8aa7c6200 0x55d8aa7c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.283+0000 7f7acffd4740 4 client.26426 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:43:15.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.283+0000 7f7acffd4740 1 -- 172.21.15.42:0/163309026 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8aa71fb70 con 0x55d8aa7c6200 2024-04-20T10:43:15.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.283+0000 7f7ac5e00640 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d8aa7c6200 0x55d8aa7c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.284+0000 7f7ac5e00640 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d8aa7c6200 0x55d8aa7c85f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c088d3c9ac0800c server_cookie=9cceb40299dffe42 in_seq=0 out_seq=0 2024-04-20T10:43:15.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.284+0000 7f7aabe00640 10 client.26426 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:15.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.285+0000 7f7aabe00640 1 -- 172.21.15.42:0/163309026 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8aa71fb70 con 0x55d8aa7c6200 2024-04-20T10:43:15.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.285+0000 7f7aabe00640 10 client.26426 handle_command_reply: tid=0 2024-04-20T10:43:15.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.285+0000 7f7acffd4740 1 client.26426 shutdown 2024-04-20T10:43:15.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.286+0000 7f7acffd4740 1 -- 172.21.15.42:0/163309026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 msgr2=0x55d8aa7c1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:15.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.286+0000 7f7acffd4740 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 0x55d8aa7c1ae0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0007f20 tx=0x7f7ac000e040 comp rx=0 tx=0).stop 2024-04-20T10:43:15.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.286+0000 7f7acffd4740 1 -- 172.21.15.42:0/163309026 shutdown_connections 2024-04-20T10:43:15.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.286+0000 7f7acffd4740 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d8aa7c6200 0x55d8aa7c85f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.286+0000 7f7acffd4740 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 0x55d8aa7c1ae0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.286+0000 7f7acffd4740 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7acca0 0x55d8aa7b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.286+0000 7f7aabe00640 0 client.26426 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:15.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.286+0000 7f7acffd4740 1 --2- 172.21.15.42:0/163309026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.286+0000 7f7acffd4740 1 -- 172.21.15.42:0/163309026 >> 172.21.15.42:0/163309026 conn(0x55d8aa7a6ab0 msgr2=0x55d8aa7a8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:15.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.287+0000 7f7acffd4740 1 -- 172.21.15.42:0/163309026 shutdown_connections 2024-04-20T10:43:15.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.287+0000 7f7acffd4740 1 -- 172.21.15.42:0/163309026 wait complete. 2024-04-20T10:43:15.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.287+0000 7f7acffd4740 20 client.26426 trim_cache size 0 max 16384 2024-04-20T10:43:15.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.331+0000 7f7acffd4740 1 Processor -- start 2024-04-20T10:43:15.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.332+0000 7f7acffd4740 1 -- start start 2024-04-20T10:43:15.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.332+0000 7f7acffd4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7ac380 0x55d8aa7a8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.332+0000 7f7acffd4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7acca0 0x55d8aa7cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.333+0000 7f7acffd4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 0x55d8aa7ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.333+0000 7f7ac5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7acca0 0x55d8aa7cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.333+0000 7f7ac5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7acca0 0x55d8aa7cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49892/0 (socket says 172.21.15.42:49892) 2024-04-20T10:43:15.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.334+0000 7f7ac5400640 1 -- 172.21.15.42:0/3022882076 learned_addr learned my addr 172.21.15.42:0/3022882076 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:15.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.334+0000 7f7acffd4740 1 -- 172.21.15.42:0/3022882076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8117830 con 0x55d8aa7acca0 2024-04-20T10:43:15.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.334+0000 7f7acffd4740 1 -- 172.21.15.42:0/3022882076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b4a60 con 0x55d8aa7b3460 2024-04-20T10:43:15.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.334+0000 7f7acffd4740 1 -- 172.21.15.42:0/3022882076 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8117ee0 con 0x55d8aa7ac380 2024-04-20T10:43:15.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.335+0000 7f7ac6800640 1 --2- 172.21.15.42:0/3022882076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 0x55d8aa7ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.335+0000 7f7ac5400640 1 -- 172.21.15.42:0/3022882076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7ac380 msgr2=0x55d8aa7a8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:15.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.335+0000 7f7ac5400640 1 --2- 172.21.15.42:0/3022882076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7ac380 0x55d8aa7a8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.335+0000 7f7ac5400640 1 -- 172.21.15.42:0/3022882076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 msgr2=0x55d8aa7ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:15.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.335+0000 7f7ac5400640 1 --2- 172.21.15.42:0/3022882076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 0x55d8aa7ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.335+0000 7f7ac5400640 1 -- 172.21.15.42:0/3022882076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8aa7b3280 con 0x55d8aa7acca0 2024-04-20T10:43:15.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.335+0000 7f7ac6800640 1 --2- 172.21.15.42:0/3022882076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 0x55d8aa7ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:43:15.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.336+0000 7f7ac5400640 1 --2- 172.21.15.42:0/3022882076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7acca0 0x55d8aa7cdd60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0000c00 tx=0x7f7ab0005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4a8fabd514f794 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:15.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.337+0000 7f7ab6a00640 1 -- 172.21.15.42:0/3022882076 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0016040 con 0x55d8aa7acca0 2024-04-20T10:43:15.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.337+0000 7f7ab6a00640 1 -- 172.21.15.42:0/3022882076 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab0005000 con 0x55d8aa7acca0 2024-04-20T10:43:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.337+0000 7f7ab6a00640 1 -- 172.21.15.42:0/3022882076 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00048e0 con 0x55d8aa7acca0 2024-04-20T10:43:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.338+0000 7f7acffd4740 1 -- 172.21.15.42:0/3022882076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7acca0 msgr2=0x55d8aa7cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.338+0000 7f7acffd4740 1 --2- 172.21.15.42:0/3022882076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7acca0 0x55d8aa7cdd60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0000c00 tx=0x7f7ab0005da0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.339+0000 7f7acffd4740 1 -- 172.21.15.42:0/3022882076 shutdown_connections 2024-04-20T10:43:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.339+0000 7f7acffd4740 1 --2- 172.21.15.42:0/3022882076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7b3460 0x55d8aa7ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.339+0000 7f7acffd4740 1 --2- 172.21.15.42:0/3022882076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7acca0 0x55d8aa7cdd60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.339+0000 7f7acffd4740 1 --2- 172.21.15.42:0/3022882076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7ac380 0x55d8aa7a8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.339+0000 7f7acffd4740 1 -- 172.21.15.42:0/3022882076 >> 172.21.15.42:0/3022882076 conn(0x55d8aa7a6ab0 msgr2=0x55d8aa7a5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.340+0000 7f7acffd4740 1 -- 172.21.15.42:0/3022882076 shutdown_connections 2024-04-20T10:43:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.340+0000 7f7acffd4740 1 -- 172.21.15.42:0/3022882076 wait complete. 2024-04-20T10:43:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.341+0000 7f7acffd4740 1 -- start start 2024-04-20T10:43:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.342+0000 7f7acffd4740 1 Processor -- start 2024-04-20T10:43:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.343+0000 7f7acffd4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.343+0000 7f7acffd4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7acca0 0x55d8aa7b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.343+0000 7f7acffd4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7b3460 0x55d8aa7ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.343+0000 7f7acffd4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81185b0 con 0x55d8aa7ac380 2024-04-20T10:43:15.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.343+0000 7f7acffd4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b4390 con 0x55d8aa7acca0 2024-04-20T10:43:15.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.343+0000 7f7acffd4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8118c80 con 0x55d8aa7b3460 2024-04-20T10:43:15.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.343+0000 7f7ac5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.343+0000 7f7acffd4740 10 client.0 _ll_register_callbacks cb 0x7f7ac0009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:15.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.344+0000 7f7ac5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7b96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49900/0 (socket says 172.21.15.42:49900) 2024-04-20T10:43:15.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.344+0000 7f7ac5e00640 1 -- 172.21.15.42:0/1035860814 learned_addr learned my addr 172.21.15.42:0/1035860814 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:15.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.344+0000 7f7ac6800640 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7b3460 0x55d8aa7ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.344+0000 7f7ac5e00640 1 -- 172.21.15.42:0/1035860814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7b3460 msgr2=0x55d8aa7ba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:15.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.344+0000 7f7ac5e00640 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7b3460 0x55d8aa7ba130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.344+0000 7f7ac5e00640 1 -- 172.21.15.42:0/1035860814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7acca0 msgr2=0x55d8aa7b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:15.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.344+0000 7f7ac5e00640 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7acca0 0x55d8aa7b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.344+0000 7f7ac5e00640 1 -- 172.21.15.42:0/1035860814 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8aa82b7d0 con 0x55d8aa7ac380 2024-04-20T10:43:15.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.344+0000 7f7ac6800640 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7b3460 0x55d8aa7ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:15.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.345+0000 7f7ac5e00640 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7b96b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8004650 tx=0x7f7ab80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4092edf25a3ce6c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:15.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.345+0000 7f7aabe00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:43:15.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.345+0000 7f7aabe00640 1 -- 172.21.15.42:0/1035860814 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80f6040 con 0x55d8aa7ac380 2024-04-20T10:43:15.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.345+0000 7f7aabe00640 1 -- 172.21.15.42:0/1035860814 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f7ab80bbc30 con 0x55d8aa7ac380 2024-04-20T10:43:15.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.346+0000 7f7acffd4740 1 -- 172.21.15.42:0/1035860814 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ab0005000 con 0x55d8aa7ac380 2024-04-20T10:43:15.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.346+0000 7f7aabe00640 1 -- 172.21.15.42:0/1035860814 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7ab80be290 con 0x55d8aa7ac380 2024-04-20T10:43:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.346+0000 7f7acffd4740 10 client.16311 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.346+0000 7f7acffd4740 1 -- 172.21.15.42:0/1035860814 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d8aa82ca80 con 0x55d8aa7ac380 2024-04-20T10:43:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.347+0000 7f7aabe00640 1 -- 172.21.15.42:0/1035860814 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab0005000 con 0x55d8aa7ac380 2024-04-20T10:43:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.347+0000 7f7aabe00640 1 -- 172.21.15.42:0/1035860814 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f7ab80ef4b0 con 0x55d8aa7ac380 2024-04-20T10:43:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.347+0000 7f7acffd4740 10 client.16311 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.347+0000 7f7acffd4740 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.347+0000 7f7acffd4740 20 client.16311 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.347+0000 7f7acffd4740 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d8aa82d1d0 0x55d8aa82f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.347+0000 7f7acffd4740 4 client.16311 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:43:15.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.347+0000 7f7acffd4740 1 -- 172.21.15.42:0/1035860814 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d8aa7199e0 con 0x55d8aa82d1d0 2024-04-20T10:43:15.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.348+0000 7f7ac5400640 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d8aa82d1d0 0x55d8aa82f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.349+0000 7f7ac5400640 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d8aa82d1d0 0x55d8aa82f630 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d04a6727c84eafb5 server_cookie=e34c81c65a899085 in_seq=0 out_seq=0 2024-04-20T10:43:15.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.349+0000 7f7aabe00640 10 client.16311 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:15.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.360+0000 7f7aabe00640 1 -- 172.21.15.42:0/1035860814 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d8aa7199e0 con 0x55d8aa82d1d0 2024-04-20T10:43:15.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.360+0000 7f7aabe00640 10 client.16311 handle_command_reply: tid=0 2024-04-20T10:43:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.360+0000 7f7acffd4740 1 client.16311 shutdown 2024-04-20T10:43:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.360+0000 7f7acffd4740 1 -- 172.21.15.42:0/1035860814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 msgr2=0x55d8aa7b96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:15.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.360+0000 7f7acffd4740 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7b96b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8004650 tx=0x7f7ab80036f0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.361+0000 7f7acffd4740 1 -- 172.21.15.42:0/1035860814 shutdown_connections 2024-04-20T10:43:15.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.361+0000 7f7acffd4740 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d8aa82d1d0 0x55d8aa82f630 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.361+0000 7f7acffd4740 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8aa7b3460 0x55d8aa7ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.361+0000 7f7acffd4740 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8aa7acca0 0x55d8aa7b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.361+0000 7f7acffd4740 1 --2- 172.21.15.42:0/1035860814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d8aa7ac380 0x55d8aa7b96b0 secure :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8004650 tx=0x7f7ab80036f0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.361+0000 7f7aabe00640 0 client.16311 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:15.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.361+0000 7f7acffd4740 1 -- 172.21.15.42:0/1035860814 >> 172.21.15.42:0/1035860814 conn(0x55d8aa7a6ab0 msgr2=0x55d8aa7b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:15.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.363+0000 7f7acffd4740 1 -- 172.21.15.42:0/1035860814 shutdown_connections 2024-04-20T10:43:15.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.363+0000 7f7acffd4740 1 -- 172.21.15.42:0/1035860814 wait complete. 2024-04-20T10:43:15.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.363+0000 7f7acffd4740 20 client.16311 trim_cache size 0 max 16384 2024-04-20T10:43:15.363 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:43:15.363 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:43:15.363 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:43:15.363 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:43:15.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.364+0000 7f7ac7200640 1 -- 172.21.15.42:0/2656987359 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7aac0a3de0 msgr2=0x7f7aac0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:15.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.365+0000 7f7ac7200640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7aac0a3de0 0x7f7aac0a62a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009fd0 tx=0x7f7ac000c040 comp rx=0 tx=0).stop 2024-04-20T10:43:15.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.365+0000 7f7ac7200640 1 -- 172.21.15.42:0/2656987359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac8108f60 msgr2=0x7f7ac81a7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:15.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.365+0000 7f7ac7200640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac8108f60 0x7f7ac81a7520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7ab802c2e0 tx=0x7f7ab80045e0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.365+0000 7f7ac7200640 1 -- 172.21.15.42:0/2656987359 shutdown_connections 2024-04-20T10:43:15.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.365+0000 7f7ac7200640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f7aac0a3de0 0x7f7aac0a62a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.365+0000 7f7ac7200640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ac8110600 0x7f7ac81abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.366+0000 7f7ac7200640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ac81099e0 0x7f7ac81a7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.366+0000 7f7ac7200640 1 --2- 172.21.15.42:0/2656987359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ac8108f60 0x7f7ac81a7520 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.366+0000 7f7ac7200640 1 -- 172.21.15.42:0/2656987359 >> 172.21.15.42:0/2656987359 conn(0x7f7ac806cee0 msgr2=0x7f7ac8070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:15.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.366+0000 7f7ac7200640 1 -- 172.21.15.42:0/2656987359 shutdown_connections 2024-04-20T10:43:15.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.366+0000 7f7ac7200640 1 -- 172.21.15.42:0/2656987359 wait complete. 2024-04-20T10:43:15.382 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/key.o 2024-04-20T10:43:15.399 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:43:15.400 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:43:15.543 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/quota/quota.o 2024-04-20T10:43:15.586 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/power/console.o 2024-04-20T10:43:15.707 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/iomap/built-in.a 2024-04-20T10:43:15.789 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/power/process.o 2024-04-20T10:43:15.791 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:15 smithi042 ceph-mon[17078]: pgmap v286: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 1.10k op/s 2024-04-20T10:43:15.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.838+0000 7f90d0e00640 1 -- 172.21.15.42:0/3110063515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90c40096b0 msgr2=0x7f90c4007a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:15.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.838+0000 7f90d0e00640 1 --2- 172.21.15.42:0/3110063515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90c40096b0 0x7f90c4007a80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009900 tx=0x7f90b802bd70 comp rx=0 tx=0).stop 2024-04-20T10:43:15.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.838+0000 7f90d0e00640 1 -- 172.21.15.42:0/3110063515 shutdown_connections 2024-04-20T10:43:15.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.838+0000 7f90d0e00640 1 --2- 172.21.15.42:0/3110063515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c40a6b90 0x7f90c40a8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.838+0000 7f90d0e00640 1 --2- 172.21.15.42:0/3110063515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90c4007fc0 0x7f90c4008440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.838+0000 7f90d0e00640 1 --2- 172.21.15.42:0/3110063515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90c40096b0 0x7f90c4007a80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.838+0000 7f90d0e00640 1 -- 172.21.15.42:0/3110063515 >> 172.21.15.42:0/3110063515 conn(0x7f90c400b840 msgr2=0x7f90c400bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:15.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.839+0000 7f90d0e00640 1 -- 172.21.15.42:0/3110063515 shutdown_connections 2024-04-20T10:43:15.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.839+0000 7f90d0e00640 1 -- 172.21.15.42:0/3110063515 wait complete. 2024-04-20T10:43:15.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.841+0000 7f90d0e00640 1 Processor -- start 2024-04-20T10:43:15.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.841+0000 7f90d0e00640 1 -- start start 2024-04-20T10:43:15.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.841+0000 7f90d0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c4007fc0 0x7f90c4138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.841+0000 7f90d0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90c40a6b90 0x7f90c4138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.841+0000 7f90d0e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90c41327d0 0x7f90c4132c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.841+0000 7f90d0e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f90c40ac810 con 0x7f90c40a6b90 2024-04-20T10:43:15.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.842+0000 7f90d0e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90c40ac690 con 0x7f90c4007fc0 2024-04-20T10:43:15.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.842+0000 7f90d0e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f90c40ac990 con 0x7f90c41327d0 2024-04-20T10:43:15.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.842+0000 7f90cbe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90c41327d0 0x7f90c4132c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.842+0000 7f90cbe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90c41327d0 0x7f90c4132c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:57722/0 (socket says 172.21.15.42:57722) 2024-04-20T10:43:15.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.842+0000 7f90cbe00640 1 -- 172.21.15.42:0/448625228 learned_addr learned my addr 172.21.15.42:0/448625228 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:15.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.842+0000 7f90caa00640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90c40a6b90 0x7f90c4138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.843+0000 7f90cbe00640 1 -- 172.21.15.42:0/448625228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c4007fc0 msgr2=0x7f90c4138660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T10:43:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.843+0000 7f90cbe00640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c4007fc0 0x7f90c4138660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.843+0000 7f90cbe00640 1 -- 172.21.15.42:0/448625228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90c40a6b90 msgr2=0x7f90c4138ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.843+0000 7f90cbe00640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90c40a6b90 0x7f90c4138ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:15.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.843+0000 7f90cbe00640 1 -- 172.21.15.42:0/448625228 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c4133380 con 0x7f90c41327d0 2024-04-20T10:43:15.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.843+0000 7f90cbe00640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90c41327d0 0x7f90c4132c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f90cc02db60 tx=0x7f90cc0775f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:15.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.847+0000 7f90caa00640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90c40a6b90 0x7f90c4138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:15.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.847+0000 7f90bfe00640 1 -- 172.21.15.42:0/448625228 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc081070 con 0x7f90c41327d0 2024-04-20T10:43:15.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.847+0000 7f90bfe00640 1 -- 172.21.15.42:0/448625228 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90cc079e40 con 0x7f90c41327d0 2024-04-20T10:43:15.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.847+0000 7f90bfe00640 1 -- 172.21.15.42:0/448625228 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc07b8f0 con 0x7f90c41327d0 2024-04-20T10:43:15.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.850+0000 7f90d0e00640 1 -- 172.21.15.42:0/448625228 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c409a360 con 0x7f90c41327d0 2024-04-20T10:43:15.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.850+0000 7f90d0e00640 1 -- 172.21.15.42:0/448625228 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c409a830 con 0x7f90c41327d0 2024-04-20T10:43:15.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.854+0000 7f90d0e00640 1 -- 172.21.15.42:0/448625228 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c4008f50 con 0x7f90c41327d0 2024-04-20T10:43:15.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.860+0000 7f90bfe00640 1 -- 172.21.15.42:0/448625228 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f90cc084070 con 0x7f90c41327d0 2024-04-20T10:43:15.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.863+0000 7f90bfe00640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f90b40a3ea0 0x7f90b40a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:15.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.863+0000 7f90bfe00640 1 -- 172.21.15.42:0/448625228 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f90cc13c860 con 0x7f90c41327d0 2024-04-20T10:43:15.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.863+0000 7f90cb400640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f90b40a3ea0 0x7f90b40a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:15.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.865+0000 7f90cb400640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f90b40a3ea0 0x7f90b40a6360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90b8005ec0 tx=0x7f90b8000bc0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:15.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:15.868+0000 7f90bfe00640 1 -- 172.21.15.42:0/448625228 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f90cc08b020 con 0x7f90c41327d0 2024-04-20T10:43:15.910 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/inode.o 2024-04-20T10:43:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:15 smithi046 ceph-mon[21574]: pgmap v286: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 1.10k op/s 2024-04-20T10:43:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:15 smithi162 ceph-mon[21854]: pgmap v286: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 1.10k op/s 2024-04-20T10:43:16.322 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/knc.o 2024-04-20T10:43:16.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.405+0000 7f90d0e00640 1 -- 172.21.15.42:0/448625228 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90c41335b0 con 0x7f90c41327d0 2024-04-20T10:43:16.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.407+0000 7f90bfe00640 1 -- 172.21.15.42:0/448625228 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f90cc108ed0 con 0x7f90c41327d0 2024-04-20T10:43:16.407 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:43:16.407 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:43:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:43:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.412+0000 7f90d0e00640 1 -- 172.21.15.42:0/448625228 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f90b40a3ea0 msgr2=0x7f90b40a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.412+0000 7f90d0e00640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f90b40a3ea0 0x7f90b40a6360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90b8005ec0 tx=0x7f90b8000bc0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.412+0000 7f90d0e00640 1 -- 172.21.15.42:0/448625228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90c41327d0 msgr2=0x7f90c4132c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.412+0000 7f90d0e00640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90c41327d0 0x7f90c4132c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f90cc02db60 tx=0x7f90cc0775f0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.413+0000 7f90d0e00640 1 -- 172.21.15.42:0/448625228 shutdown_connections 2024-04-20T10:43:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.413+0000 7f90d0e00640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f90b40a3ea0 0x7f90b40a6360 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.413+0000 7f90d0e00640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90c41327d0 0x7f90c4132c80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.413+0000 7f90d0e00640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90c40a6b90 0x7f90c4138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.413+0000 7f90d0e00640 1 --2- 172.21.15.42:0/448625228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90c4007fc0 0x7f90c4138660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.413+0000 7f90d0e00640 1 -- 172.21.15.42:0/448625228 >> 172.21.15.42:0/448625228 conn(0x7f90c400b840 msgr2=0x7f90c40a5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:16.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.416+0000 7f90d0e00640 1 -- 172.21.15.42:0/448625228 shutdown_connections 2024-04-20T10:43:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.417+0000 7f90d0e00640 1 -- 172.21.15.42:0/448625228 wait complete. 2024-04-20T10:43:16.447 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:43:16.497 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/printk/printk.o 2024-04-20T10:43:16.500 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/keyring.o 2024-04-20T10:43:16.594 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/asymmetric_type.o 2024-04-20T10:43:16.802 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:16 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/448625228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:43:16.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.876+0000 7f2b3be00640 1 -- 172.21.15.42:0/1587951612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b3c108f60 msgr2=0x7f2b3c109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:16.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.876+0000 7f2b3be00640 1 --2- 172.21.15.42:0/1587951612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b3c108f60 0x7f2b3c109340 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009900 tx=0x7f2b2802bc60 comp rx=0 tx=0).stop 2024-04-20T10:43:16.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.879+0000 7f2b3be00640 1 -- 172.21.15.42:0/1587951612 shutdown_connections 2024-04-20T10:43:16.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.879+0000 7f2b3be00640 1 --2- 172.21.15.42:0/1587951612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1150b0 0x7f2b3c1174a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.879+0000 7f2b3be00640 1 --2- 172.21.15.42:0/1587951612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b3c109880 0x7f2b3c114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.879+0000 7f2b3be00640 1 --2- 172.21.15.42:0/1587951612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b3c108f60 0x7f2b3c109340 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.879+0000 7f2b3be00640 1 -- 172.21.15.42:0/1587951612 >> 172.21.15.42:0/1587951612 conn(0x7f2b3c06cee0 msgr2=0x7f2b3c06d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:16.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.880+0000 7f2b3be00640 1 -- 172.21.15.42:0/1587951612 shutdown_connections 2024-04-20T10:43:16.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.880+0000 7f2b3be00640 1 -- 172.21.15.42:0/1587951612 wait complete. 2024-04-20T10:43:16.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.881+0000 7f2b3be00640 1 Processor -- start 2024-04-20T10:43:16.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.881+0000 7f2b3be00640 1 -- start start 2024-04-20T10:43:16.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.882+0000 7f2b3be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b3c108f60 0x7f2b3c1a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.882+0000 7f2b3be00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b3c109880 0x7f2b3c1a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.882+0000 7f2b3be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1150b0 0x7f2b3c1a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.882+0000 7f2b3be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c11aa80 con 0x7f2b3c108f60 2024-04-20T10:43:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.883+0000 7f2b3be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c11a900 con 0x7f2b3c1150b0 2024-04-20T10:43:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.883+0000 7f2b3be00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c11ac00 con 0x7f2b3c109880 2024-04-20T10:43:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.883+0000 7f2b3a000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b3c109880 0x7f2b3c1a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.883+0000 7f2b3a000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b3c109880 0x7f2b3c1a3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:57734/0 (socket says 172.21.15.42:57734) 2024-04-20T10:43:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.883+0000 7f2b3a000640 1 -- 172.21.15.42:0/174595925 learned_addr learned my addr 172.21.15.42:0/174595925 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.883+0000 7f2b3b400640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1150b0 0x7f2b3c1a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.883+0000 7f2b3aa00640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b3c108f60 0x7f2b3c1a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.883+0000 7f2b3a000640 1 -- 172.21.15.42:0/174595925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1150b0 msgr2=0x7f2b3c1a7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:16.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.883+0000 7f2b3a000640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1150b0 0x7f2b3c1a7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.883+0000 7f2b3a000640 1 -- 172.21.15.42:0/174595925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b3c108f60 msgr2=0x7f2b3c1a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:16.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.884+0000 7f2b3a000640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b3c108f60 0x7f2b3c1a31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.884+0000 7f2b3a000640 1 -- 172.21.15.42:0/174595925 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c1a81d0 con 0x7f2b3c109880 2024-04-20T10:43:16.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.884+0000 7f2b3aa00640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b3c108f60 0x7f2b3c1a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:43:16.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.885+0000 7f2b3a000640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b3c109880 0x7f2b3c1a3720 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400ba50 tx=0x7f2b3400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:16.890 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/quota/kqid.o 2024-04-20T10:43:16.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.890+0000 7f2b33400640 1 -- 172.21.15.42:0/174595925 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34019070 con 0x7f2b3c109880 2024-04-20T10:43:16.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.891+0000 7f2b33400640 1 -- 172.21.15.42:0/174595925 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b3400cbe0 con 0x7f2b3c109880 2024-04-20T10:43:16.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.891+0000 7f2b33400640 1 -- 172.21.15.42:0/174595925 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b340076e0 con 0x7f2b3c109880 2024-04-20T10:43:16.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.892+0000 7f2b3be00640 1 -- 172.21.15.42:0/174595925 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3c1a84c0 con 0x7f2b3c109880 2024-04-20T10:43:16.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.893+0000 7f2b3be00640 1 -- 172.21.15.42:0/174595925 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3c1afd50 con 0x7f2b3c109880 2024-04-20T10:43:16.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.898+0000 7f2b33400640 1 -- 172.21.15.42:0/174595925 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2b3401c070 con 0x7f2b3c109880 2024-04-20T10:43:16.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.901+0000 7f2b33400640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2b200a3ea0 0x7f2b200a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.902+0000 7f2b33400640 1 -- 172.21.15.42:0/174595925 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f2b34099ce0 con 0x7f2b3c109880 2024-04-20T10:43:16.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.903+0000 7f2b3aa00640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2b200a3ea0 0x7f2b200a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.904+0000 7f2b44a39740 1 Processor -- start 2024-04-20T10:43:16.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.904+0000 7f2b44a39740 1 -- start start 2024-04-20T10:43:16.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.905+0000 7f2b3aa00640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2b200a3ea0 0x7f2b200a6360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2b28006010 tx=0x7f2b2802c180 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:16.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.905+0000 7f2b44a39740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6170 0x55b7ff4f6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.905+0000 7f2b44a39740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff4ff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.906+0000 7f2b44a39740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4ff670 0x55b7ff501a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.906+0000 7f2b3aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6170 0x55b7ff4f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.906+0000 7f2b3aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6170 0x55b7ff4f6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42880/0 (socket says 172.21.15.42:42880) 2024-04-20T10:43:16.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.906+0000 7f2b3aa00640 1 -- 172.21.15.42:0/804512808 learned_addr learned my addr 172.21.15.42:0/804512808 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:16.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.906+0000 7f2b44a39740 1 -- 172.21.15.42:0/804512808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b7ff45b050 con 0x55b7ff4f6a90 2024-04-20T10:43:16.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.907+0000 7f2b3b400640 1 --2- 172.21.15.42:0/804512808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4ff670 0x55b7ff501a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.907+0000 7f2b3a000640 1 --2- 172.21.15.42:0/804512808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff4ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.907+0000 7f2b44a39740 1 -- 172.21.15.42:0/804512808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b7ff401490 con 0x55b7ff4f6170 2024-04-20T10:43:16.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.907+0000 7f2b44a39740 1 -- 172.21.15.42:0/804512808 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b7ff457010 con 0x55b7ff4ff670 2024-04-20T10:43:16.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.908+0000 7f2b3a000640 1 -- 172.21.15.42:0/804512808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4ff670 msgr2=0x55b7ff501a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:16.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.908+0000 7f2b3a000640 1 --2- 172.21.15.42:0/804512808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4ff670 0x55b7ff501a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.908+0000 7f2b3a000640 1 -- 172.21.15.42:0/804512808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6170 msgr2=0x55b7ff4f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:16.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.908+0000 7f2b3a000640 1 --2- 172.21.15.42:0/804512808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6170 0x55b7ff4f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.908+0000 7f2b3a000640 1 -- 172.21.15.42:0/804512808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ff409070 con 0x55b7ff4f6a90 2024-04-20T10:43:16.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.908+0000 7f2b3aa00640 1 --2- 172.21.15.42:0/804512808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6170 0x55b7ff4f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:43:16.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.908+0000 7f2b3a000640 1 --2- 172.21.15.42:0/804512808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff4ff130 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2b340cd3f0 tx=0x7f2b340a01e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ef9dd9a30c9fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:16.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.909+0000 7f2b27e00640 1 -- 172.21.15.42:0/804512808 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34098670 con 0x55b7ff4f6a90 2024-04-20T10:43:16.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.909+0000 7f2b27e00640 1 -- 172.21.15.42:0/804512808 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b34017e50 con 0x55b7ff4f6a90 2024-04-20T10:43:16.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.909+0000 7f2b27e00640 1 -- 172.21.15.42:0/804512808 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34098810 con 0x55b7ff4f6a90 2024-04-20T10:43:16.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.910+0000 7f2b44a39740 1 -- 172.21.15.42:0/804512808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 msgr2=0x55b7ff4ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:16.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.911+0000 7f2b44a39740 1 --2- 172.21.15.42:0/804512808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff4ff130 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2b340cd3f0 tx=0x7f2b340a01e0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.911+0000 7f2b44a39740 1 -- 172.21.15.42:0/804512808 shutdown_connections 2024-04-20T10:43:16.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.911+0000 7f2b44a39740 1 --2- 172.21.15.42:0/804512808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4ff670 0x55b7ff501a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.911+0000 7f2b44a39740 1 --2- 172.21.15.42:0/804512808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff4ff130 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.911+0000 7f2b44a39740 1 --2- 172.21.15.42:0/804512808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6170 0x55b7ff4f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.911+0000 7f2b44a39740 1 -- 172.21.15.42:0/804512808 >> 172.21.15.42:0/804512808 conn(0x55b7ff4f08a0 msgr2=0x55b7ff4f0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:16.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.911+0000 7f2b44a39740 1 -- 172.21.15.42:0/804512808 shutdown_connections 2024-04-20T10:43:16.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.911+0000 7f2b44a39740 1 -- 172.21.15.42:0/804512808 wait complete. 2024-04-20T10:43:16.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.912+0000 7f2b44a39740 1 -- start start 2024-04-20T10:43:16.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.912+0000 7f2b44a39740 1 Processor -- start 2024-04-20T10:43:16.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.916+0000 7f2b44a39740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 0x55b7ff4fc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.916+0000 7f2b44a39740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff4fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.916+0000 7f2b44a39740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 0x55b7ff4fd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.916+0000 7f2b44a39740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b7ff502460 con 0x55b7ff4f6a90 2024-04-20T10:43:16.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.916+0000 7f2b44a39740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b7ff414860 con 0x55b7ff4ff670 2024-04-20T10:43:16.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.916+0000 7f2b44a39740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b7ff5025e0 con 0x55b7ff4f6170 2024-04-20T10:43:16.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.917+0000 7f2b44a39740 10 client.0 _ll_register_callbacks cb 0x55b7ff3e2fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:16.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.917+0000 7f2b3aa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 0x55b7ff4fc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.917+0000 7f2b3aa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 0x55b7ff4fc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:57764/0 (socket says 172.21.15.42:57764) 2024-04-20T10:43:16.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.917+0000 7f2b3aa00640 1 -- 172.21.15.42:0/3117064478 learned_addr learned my addr 172.21.15.42:0/3117064478 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:16.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.917+0000 7f2b3b400640 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 0x55b7ff4fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.918+0000 7f2b3b400640 1 -- 172.21.15.42:0/3117064478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 msgr2=0x55b7ff4fc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:16.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.918+0000 7f2b3b400640 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 0x55b7ff4fc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.918+0000 7f2b3b400640 1 -- 172.21.15.42:0/3117064478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 msgr2=0x55b7ff4fcc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:43:16.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.918+0000 7f2b3a000640 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff4fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.919+0000 7f2b3b400640 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff4fcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.919+0000 7f2b3b400640 1 -- 172.21.15.42:0/3117064478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7ff418950 con 0x55b7ff4ff670 2024-04-20T10:43:16.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.919+0000 7f2b3aa00640 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 0x55b7ff4fc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:16.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.919+0000 7f2b3a000640 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff4fcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:16.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.921+0000 7f2b24c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:43:16.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.921+0000 7f2b3b400640 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 0x55b7ff4fd170 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c06b4b0 tx=0x7f2b2c012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=229d946f24825be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:16.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.922+0000 7f2b24c00640 1 -- 172.21.15.42:0/3117064478 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c00f8a0 con 0x55b7ff4ff670 2024-04-20T10:43:16.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.922+0000 7f2b44a39740 1 -- 172.21.15.42:0/3117064478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b34017e50 con 0x55b7ff4ff670 2024-04-20T10:43:16.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.922+0000 7f2b24c00640 1 -- 172.21.15.42:0/3117064478 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f2b2c00f6a0 con 0x55b7ff4ff670 2024-04-20T10:43:16.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.923+0000 7f2b24c00640 1 -- 172.21.15.42:0/3117064478 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2b2c00f8a0 con 0x55b7ff4ff670 2024-04-20T10:43:16.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.923+0000 7f2b44a39740 10 client.26441 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:16.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.923+0000 7f2b44a39740 1 -- 172.21.15.42:0/3117064478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b7ff3e6820 con 0x55b7ff4ff670 2024-04-20T10:43:16.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.924+0000 7f2b24c00640 1 -- 172.21.15.42:0/3117064478 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b34017e50 con 0x55b7ff4ff670 2024-04-20T10:43:16.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.924+0000 7f2b24c00640 1 -- 172.21.15.42:0/3117064478 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f2b2c019070 con 0x55b7ff4ff670 2024-04-20T10:43:16.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.924+0000 7f2b44a39740 10 client.26441 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:16.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.924+0000 7f2b44a39740 10 client.26441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:16.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.924+0000 7f2b44a39740 20 client.26441 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:16.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.925+0000 7f2b44a39740 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b7ff520770 0x55b7ff522b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.925+0000 7f2b44a39740 4 client.26441 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:43:16.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.925+0000 7f2b44a39740 1 -- 172.21.15.42:0/3117064478 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7ff468a70 con 0x55b7ff520770 2024-04-20T10:43:16.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.928+0000 7f2b3aa00640 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b7ff520770 0x55b7ff522b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.928+0000 7f2b3aa00640 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b7ff520770 0x55b7ff522b90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2757b63d419c9049 server_cookie=d746cb24746dd35c in_seq=0 out_seq=0 2024-04-20T10:43:16.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.929+0000 7f2b24c00640 10 client.26441 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:16.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.929+0000 7f2b24c00640 1 -- 172.21.15.42:0/3117064478 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7ff468a70 con 0x55b7ff520770 2024-04-20T10:43:16.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.929+0000 7f2b24c00640 10 client.26441 handle_command_reply: tid=0 2024-04-20T10:43:16.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.932+0000 7f2b44a39740 1 client.26441 shutdown 2024-04-20T10:43:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.932+0000 7f2b44a39740 1 -- 172.21.15.42:0/3117064478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 msgr2=0x55b7ff4fd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.932+0000 7f2b44a39740 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 0x55b7ff4fd170 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c06b4b0 tx=0x7f2b2c012820 comp rx=0 tx=0).stop 2024-04-20T10:43:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.932+0000 7f2b44a39740 1 -- 172.21.15.42:0/3117064478 shutdown_connections 2024-04-20T10:43:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.932+0000 7f2b44a39740 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b7ff520770 0x55b7ff522b90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.932+0000 7f2b44a39740 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 0x55b7ff4fd170 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.932+0000 7f2b44a39740 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff4fcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.932+0000 7f2b44a39740 1 --2- 172.21.15.42:0/3117064478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 0x55b7ff4fc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.932+0000 7f2b44a39740 1 -- 172.21.15.42:0/3117064478 >> 172.21.15.42:0/3117064478 conn(0x55b7ff4f08a0 msgr2=0x55b7ff4f25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.932+0000 7f2b24c00640 0 client.26441 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.933+0000 7f2b44a39740 1 -- 172.21.15.42:0/3117064478 shutdown_connections 2024-04-20T10:43:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.933+0000 7f2b44a39740 1 -- 172.21.15.42:0/3117064478 wait complete. 2024-04-20T10:43:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.933+0000 7f2b44a39740 20 client.26441 trim_cache size 0 max 16384 2024-04-20T10:43:16.973 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/root.o 2024-04-20T10:43:16.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.975+0000 7f2b44a39740 1 Processor -- start 2024-04-20T10:43:16.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.975+0000 7f2b44a39740 1 -- start start 2024-04-20T10:43:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.976+0000 7f2b44a39740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 0x55b7ff4fee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.976+0000 7f2b44a39740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff50b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.976+0000 7f2b44a39740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 0x55b7ff5284a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.976+0000 7f2b44a39740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c00c920 con 0x55b7ff4f6a90 2024-04-20T10:43:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.976+0000 7f2b44a39740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c00f1e0 con 0x55b7ff4ff670 2024-04-20T10:43:16.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.976+0000 7f2b44a39740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c00cfd0 con 0x55b7ff4f6170 2024-04-20T10:43:16.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.977+0000 7f2b3aa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 0x55b7ff4fee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.977+0000 7f2b3b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 0x55b7ff5284a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.977+0000 7f2b3b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 0x55b7ff5284a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42910/0 (socket says 172.21.15.42:42910) 2024-04-20T10:43:16.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.977+0000 7f2b3b400640 1 -- 172.21.15.42:0/4011076889 learned_addr learned my addr 172.21.15.42:0/4011076889 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:16.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.977+0000 7f2b3aa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 0x55b7ff4fee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:57768/0 (socket says 172.21.15.42:57768) 2024-04-20T10:43:16.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.977+0000 7f2b3a000640 1 --2- 172.21.15.42:0/4011076889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff50b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.977+0000 7f2b3b400640 1 -- 172.21.15.42:0/4011076889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 msgr2=0x55b7ff4fee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:16.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.977+0000 7f2b3b400640 1 --2- 172.21.15.42:0/4011076889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 0x55b7ff4fee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.977+0000 7f2b3b400640 1 -- 172.21.15.42:0/4011076889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 msgr2=0x55b7ff50b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:16.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.977+0000 7f2b3b400640 1 --2- 172.21.15.42:0/4011076889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff50b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.977+0000 7f2b3b400640 1 -- 172.21.15.42:0/4011076889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ff16d3f0 con 0x55b7ff4ff670 2024-04-20T10:43:16.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.978+0000 7f2b3aa00640 1 --2- 172.21.15.42:0/4011076889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 0x55b7ff4fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:16.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.978+0000 7f2b3a000640 1 --2- 172.21.15.42:0/4011076889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff50b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:16.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.982+0000 7f2b3b400640 1 --2- 172.21.15.42:0/4011076889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 0x55b7ff5284a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c000c00 tx=0x7f2b2c012d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2410775ad9ffc16 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:16.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.982+0000 7f2b27400640 1 -- 172.21.15.42:0/4011076889 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c00fa40 con 0x55b7ff4ff670 2024-04-20T10:43:16.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.983+0000 7f2b27400640 1 -- 172.21.15.42:0/4011076889 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b2c01b920 con 0x55b7ff4ff670 2024-04-20T10:43:16.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.983+0000 7f2b27400640 1 -- 172.21.15.42:0/4011076889 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c024a10 con 0x55b7ff4ff670 2024-04-20T10:43:16.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.984+0000 7f2b44a39740 1 -- 172.21.15.42:0/4011076889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 msgr2=0x55b7ff5284a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:16.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.984+0000 7f2b44a39740 1 --2- 172.21.15.42:0/4011076889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 0x55b7ff5284a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c000c00 tx=0x7f2b2c012d20 comp rx=0 tx=0).stop 2024-04-20T10:43:16.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.985+0000 7f2b44a39740 1 -- 172.21.15.42:0/4011076889 shutdown_connections 2024-04-20T10:43:16.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.985+0000 7f2b44a39740 1 --2- 172.21.15.42:0/4011076889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4ff670 0x55b7ff5284a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.986+0000 7f2b44a39740 1 --2- 172.21.15.42:0/4011076889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6a90 0x55b7ff50b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.986+0000 7f2b44a39740 1 --2- 172.21.15.42:0/4011076889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4f6170 0x55b7ff4fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.986+0000 7f2b44a39740 1 -- 172.21.15.42:0/4011076889 >> 172.21.15.42:0/4011076889 conn(0x55b7ff4f08a0 msgr2=0x55b7ff4f7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:16.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.986+0000 7f2b44a39740 1 -- 172.21.15.42:0/4011076889 shutdown_connections 2024-04-20T10:43:16.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.986+0000 7f2b44a39740 1 -- 172.21.15.42:0/4011076889 wait complete. 2024-04-20T10:43:16.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.987+0000 7f2b44a39740 1 -- start start 2024-04-20T10:43:16.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.988+0000 7f2b44a39740 1 Processor -- start 2024-04-20T10:43:16.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.989+0000 7f2b44a39740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6170 0x55b7ff507890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.990+0000 7f2b44a39740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6a90 0x55b7ff507dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.990+0000 7f2b44a39740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4ff670 0x55b7ff4f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.991+0000 7f2b3aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6170 0x55b7ff507890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.991+0000 7f2b3a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6a90 0x55b7ff507dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.991+0000 7f2b3a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6a90 0x55b7ff507dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:42918/0 (socket says 172.21.15.42:42918) 2024-04-20T10:43:16.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.991+0000 7f2b44a39740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c00d6a0 con 0x55b7ff4f6170 2024-04-20T10:43:16.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.991+0000 7f2b3a000640 1 -- 172.21.15.42:0/1182588931 learned_addr learned my addr 172.21.15.42:0/1182588931 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:16.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.991+0000 7f2b44a39740 1 -- 172.21.15.42:0/1182588931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c00eb10 con 0x55b7ff4f6a90 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.991+0000 7f2b44a39740 1 -- 172.21.15.42:0/1182588931 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c00dd70 con 0x55b7ff4ff670 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.991+0000 7f2b44a39740 10 client.0 _ll_register_callbacks cb 0x7f2b2c00db00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.991+0000 7f2b3b400640 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4ff670 0x55b7ff4f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.992+0000 7f2b3a000640 1 -- 172.21.15.42:0/1182588931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4ff670 msgr2=0x55b7ff4f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.992+0000 7f2b3a000640 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4ff670 0x55b7ff4f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.992+0000 7f2b3a000640 1 -- 172.21.15.42:0/1182588931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6170 msgr2=0x55b7ff507890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.992+0000 7f2b3a000640 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6170 0x55b7ff507890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.992+0000 7f2b3a000640 1 -- 172.21.15.42:0/1182588931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7ff2377d0 con 0x55b7ff4f6a90 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.992+0000 7f2b3aa00640 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6170 0x55b7ff507890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.992+0000 7f2b3a000640 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6a90 0x55b7ff507dd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2b34098140 tx=0x7f2b3400c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc453bd579a4ee3c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.992+0000 7f2b24c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.992+0000 7f2b24c00640 1 -- 172.21.15.42:0/1182588931 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b340b4070 con 0x55b7ff4f6a90 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.993+0000 7f2b24c00640 1 -- 172.21.15.42:0/1182588931 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f2b34099ee0 con 0x55b7ff4f6a90 2024-04-20T10:43:16.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.993+0000 7f2b44a39740 1 -- 172.21.15.42:0/1182588931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b2c01b920 con 0x55b7ff4f6a90 2024-04-20T10:43:16.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.993+0000 7f2b24c00640 1 -- 172.21.15.42:0/1182588931 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2b340989b0 con 0x55b7ff4f6a90 2024-04-20T10:43:16.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.993+0000 7f2b44a39740 10 client.26447 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:16.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.993+0000 7f2b44a39740 1 -- 172.21.15.42:0/1182588931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b7ff586770 con 0x55b7ff4f6a90 2024-04-20T10:43:16.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.997+0000 7f2b24c00640 1 -- 172.21.15.42:0/1182588931 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b2c01b920 con 0x55b7ff4f6a90 2024-04-20T10:43:16.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.997+0000 7f2b24c00640 1 -- 172.21.15.42:0/1182588931 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f2b340cd5f0 con 0x55b7ff4f6a90 2024-04-20T10:43:16.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.997+0000 7f2b44a39740 10 client.26447 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:16.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.997+0000 7f2b44a39740 10 client.26447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:16.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.997+0000 7f2b44a39740 20 client.26447 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:16.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.997+0000 7f2b44a39740 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b7ff586aa0 0x55b7ff588e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:16.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.997+0000 7f2b44a39740 4 client.26447 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:43:16.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.997+0000 7f2b44a39740 1 -- 172.21.15.42:0/1182588931 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7ff4628e0 con 0x55b7ff586aa0 2024-04-20T10:43:16.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:16.998+0000 7f2b3aa00640 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b7ff586aa0 0x55b7ff588e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:16.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.000+0000 7f2b24c00640 10 client.26447 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:16.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.000+0000 7f2b3aa00640 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b7ff586aa0 0x55b7ff588e90 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c877a4ad2e15c5 server_cookie=48d08f3b4595e957 in_seq=0 out_seq=0 2024-04-20T10:43:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.002+0000 7f2b24c00640 1 -- 172.21.15.42:0/1182588931 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7ff4628e0 con 0x55b7ff586aa0 2024-04-20T10:43:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.002+0000 7f2b24c00640 10 client.26447 handle_command_reply: tid=0 2024-04-20T10:43:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.002+0000 7f2b44a39740 1 client.26447 shutdown 2024-04-20T10:43:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.003+0000 7f2b44a39740 1 -- 172.21.15.42:0/1182588931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6a90 msgr2=0x55b7ff507dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.003+0000 7f2b44a39740 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6a90 0x55b7ff507dd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2b34098140 tx=0x7f2b3400c550 comp rx=0 tx=0).stop 2024-04-20T10:43:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.003+0000 7f2b44a39740 1 -- 172.21.15.42:0/1182588931 shutdown_connections 2024-04-20T10:43:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.003+0000 7f2b44a39740 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b7ff586aa0 0x55b7ff588e90 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:17.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.003+0000 7f2b44a39740 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7ff4ff670 0x55b7ff4f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:17.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.003+0000 7f2b44a39740 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7ff4f6a90 0x55b7ff507dd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:17.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.003+0000 7f2b44a39740 1 --2- 172.21.15.42:0/1182588931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b7ff4f6170 0x55b7ff507890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:17.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.003+0000 7f2b44a39740 1 -- 172.21.15.42:0/1182588931 >> 172.21.15.42:0/1182588931 conn(0x55b7ff4f08a0 msgr2=0x55b7ff4fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:17.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.003+0000 7f2b24c00640 0 client.26447 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:17.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.003+0000 7f2b44a39740 1 -- 172.21.15.42:0/1182588931 shutdown_connections 2024-04-20T10:43:17.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.003+0000 7f2b44a39740 1 -- 172.21.15.42:0/1182588931 wait complete. 2024-04-20T10:43:17.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.004+0000 7f2b44a39740 20 client.26447 trim_cache size 0 max 16384 2024-04-20T10:43:17.008 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:43:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.010+0000 7f2b3be00640 1 -- 172.21.15.42:0/174595925 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2b200a3ea0 msgr2=0x7f2b200a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.010+0000 7f2b3be00640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2b200a3ea0 0x7f2b200a6360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2b28006010 tx=0x7f2b2802c180 comp rx=0 tx=0).stop 2024-04-20T10:43:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.010+0000 7f2b3be00640 1 -- 172.21.15.42:0/174595925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b3c109880 msgr2=0x7f2b3c1a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.010+0000 7f2b3be00640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b3c109880 0x7f2b3c1a3720 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400ba50 tx=0x7f2b3400bf10 comp rx=0 tx=0).stop 2024-04-20T10:43:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.010+0000 7f2b3be00640 1 -- 172.21.15.42:0/174595925 shutdown_connections 2024-04-20T10:43:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.010+0000 7f2b3be00640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f2b200a3ea0 0x7f2b200a6360 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.010+0000 7f2b3be00640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1150b0 0x7f2b3c1a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.010+0000 7f2b3be00640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b3c109880 0x7f2b3c1a3720 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.010+0000 7f2b3be00640 1 --2- 172.21.15.42:0/174595925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2b3c108f60 0x7f2b3c1a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:17.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.010+0000 7f2b3be00640 1 -- 172.21.15.42:0/174595925 >> 172.21.15.42:0/174595925 conn(0x7f2b3c06cee0 msgr2=0x7f2b3c10a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.011+0000 7f2b3be00640 1 -- 172.21.15.42:0/174595925 shutdown_connections 2024-04-20T10:43:17.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:17.011+0000 7f2b3be00640 1 -- 172.21.15.42:0/174595925 wait complete. 2024-04-20T10:43:17.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:16 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/448625228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:43:17.063 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/power/suspend.o 2024-04-20T10:43:17.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:16 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/448625228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:43:17.238 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/lbr.o 2024-04-20T10:43:17.425 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/restrict.o 2024-04-20T10:43:17.557 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/quota/netlink.o 2024-04-20T10:43:17.778 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/base.o 2024-04-20T10:43:17.782 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/keyctl.o 2024-04-20T10:43:17.992 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/signature.o 2024-04-20T10:43:18.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:17 smithi046 ceph-mon[21574]: pgmap v287: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.0 MiB/s wr, 620 op/s 2024-04-20T10:43:18.063 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:43:18.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:43:18.063 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:43:18.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:17 smithi042 ceph-mon[17078]: pgmap v287: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.0 MiB/s wr, 620 op/s 2024-04-20T10:43:18.120 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/shmem.o 2024-04-20T10:43:18.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:17 smithi162 ceph-mon[21854]: pgmap v287: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.0 MiB/s wr, 620 op/s 2024-04-20T10:43:18.274 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/power/hibernate.o 2024-04-20T10:43:18.420 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/printk/printk_safe.o 2024-04-20T10:43:18.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.492+0000 7f305f400640 1 -- 172.21.15.42:0/2675510262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3058109170 msgr2=0x7f3058109550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:18.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.492+0000 7f305f400640 1 --2- 172.21.15.42:0/2675510262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3058109170 0x7f3058109550 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3048009930 tx=0x7f304802bc20 comp rx=0 tx=0).stop 2024-04-20T10:43:18.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.493+0000 7f305f400640 1 -- 172.21.15.42:0/2675510262 shutdown_connections 2024-04-20T10:43:18.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.494+0000 7f305f400640 1 --2- 172.21.15.42:0/2675510262 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30581174a0 0x7f3058119890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.494+0000 7f305f400640 1 --2- 172.21.15.42:0/2675510262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3058109a90 0x7f3058116f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.494+0000 7f305f400640 1 --2- 172.21.15.42:0/2675510262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3058109170 0x7f3058109550 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.494+0000 7f305f400640 1 -- 172.21.15.42:0/2675510262 >> 172.21.15.42:0/2675510262 conn(0x7f305806d070 msgr2=0x7f305806d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.494+0000 7f305f400640 1 -- 172.21.15.42:0/2675510262 shutdown_connections 2024-04-20T10:43:18.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.498+0000 7f305f400640 1 -- 172.21.15.42:0/2675510262 wait complete. 2024-04-20T10:43:18.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.500+0000 7f305f400640 1 Processor -- start 2024-04-20T10:43:18.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.500+0000 7f305f400640 1 -- start start 2024-04-20T10:43:18.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.503+0000 7f305f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3058109170 0x7f30581a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.503+0000 7f305f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3058109a90 0x7f30581a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.503+0000 7f305f400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30581174a0 0x7f30581a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.503+0000 7f305f400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f305811cd60 con 0x7f3058109a90 2024-04-20T10:43:18.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.503+0000 7f305f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f305811cbe0 con 0x7f3058109170 2024-04-20T10:43:18.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.503+0000 7f305f400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f305811cee0 con 0x7f30581174a0 2024-04-20T10:43:18.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.504+0000 7f3057e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3058109a90 0x7f30581a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.504+0000 7f3057e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3058109a90 0x7f30581a3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49988/0 (socket says 172.21.15.42:49988) 2024-04-20T10:43:18.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.504+0000 7f3057e00640 1 -- 172.21.15.42:0/1094755367 learned_addr learned my addr 172.21.15.42:0/1094755367 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:18.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.504+0000 7f305ca00640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3058109170 0x7f30581a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.504+0000 7f305d400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30581174a0 0x7f30581a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.504+0000 7f3057e00640 1 -- 172.21.15.42:0/1094755367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30581174a0 msgr2=0x7f30581a7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:18.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.504+0000 7f3057e00640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30581174a0 0x7f30581a7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.504+0000 7f3057e00640 1 -- 172.21.15.42:0/1094755367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3058109170 msgr2=0x7f30581a3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:18.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.504+0000 7f3057e00640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3058109170 0x7f30581a3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.504+0000 7f3057e00640 1 -- 172.21.15.42:0/1094755367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30581a81f0 con 0x7f3058109a90 2024-04-20T10:43:18.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.504+0000 7f305d400640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30581174a0 0x7f30581a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:43:18.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.505+0000 7f3057e00640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3058109a90 0x7f30581a3740 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f304400e9f0 tx=0x7f304400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:18.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.505+0000 7f3055600640 1 -- 172.21.15.42:0/1094755367 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304400cd80 con 0x7f3058109a90 2024-04-20T10:43:18.508 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/quota/built-in.a 2024-04-20T10:43:18.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.505+0000 7f3055600640 1 -- 172.21.15.42:0/1094755367 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3044010910 con 0x7f3058109a90 2024-04-20T10:43:18.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.506+0000 7f3055600640 1 -- 172.21.15.42:0/1094755367 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044018990 con 0x7f3058109a90 2024-04-20T10:43:18.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.507+0000 7f305f400640 1 -- 172.21.15.42:0/1094755367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30581a84e0 con 0x7f3058109a90 2024-04-20T10:43:18.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.507+0000 7f305f400640 1 -- 172.21.15.42:0/1094755367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30581afd70 con 0x7f3058109a90 2024-04-20T10:43:18.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.515+0000 7f3055600640 1 -- 172.21.15.42:0/1094755367 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3044010ac0 con 0x7f3058109a90 2024-04-20T10:43:18.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.520+0000 7f3055600640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f303c0a3de0 0x7f303c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.520+0000 7f3055600640 1 -- 172.21.15.42:0/1094755367 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f30440d0fa0 con 0x7f3058109a90 2024-04-20T10:43:18.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.521+0000 7f305ca00640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f303c0a3de0 0x7f303c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.523+0000 7f305ca00640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f303c0a3de0 0x7f303c0a62a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3048006010 tx=0x7f3048000bc0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:18.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.523+0000 7f30621b7740 1 Processor -- start 2024-04-20T10:43:18.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.524+0000 7f30621b7740 1 -- start start 2024-04-20T10:43:18.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.524+0000 7f30621b7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 0x563d85a70100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.524+0000 7f30621b7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a816e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.525+0000 7f30621b7740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a84010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.525+0000 7f305ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 0x563d85a70100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.525+0000 7f305ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 0x563d85a70100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49996/0 (socket says 172.21.15.42:49996) 2024-04-20T10:43:18.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.525+0000 7f305ca00640 1 -- 172.21.15.42:0/4239653975 learned_addr learned my addr 172.21.15.42:0/4239653975 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:18.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.525+0000 7f30621b7740 1 -- 172.21.15.42:0/4239653975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563d859d4d20 con 0x563d85a6fd20 2024-04-20T10:43:18.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.525+0000 7f30621b7740 1 -- 172.21.15.42:0/4239653975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563d8597b3e0 con 0x563d85a706d0 2024-04-20T10:43:18.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.525+0000 7f30621b7740 1 -- 172.21.15.42:0/4239653975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563d859d0ce0 con 0x563d85a81c20 2024-04-20T10:43:18.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.526+0000 7f305d400640 1 --2- 172.21.15.42:0/4239653975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a84010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.526+0000 7f305d400640 1 -- 172.21.15.42:0/4239653975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 msgr2=0x563d85a816e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:43:18.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.526+0000 7f305d400640 1 --2- 172.21.15.42:0/4239653975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a816e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.526+0000 7f305d400640 1 -- 172.21.15.42:0/4239653975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 msgr2=0x563d85a70100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:18.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.526+0000 7f305d400640 1 --2- 172.21.15.42:0/4239653975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 0x563d85a70100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.526+0000 7f305d400640 1 -- 172.21.15.42:0/4239653975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d8599a980 con 0x563d85a81c20 2024-04-20T10:43:18.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.526+0000 7f305ca00640 1 --2- 172.21.15.42:0/4239653975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 0x563d85a70100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:43:18.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.527+0000 7f305d400640 1 --2- 172.21.15.42:0/4239653975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a84010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3050009f10 tx=0x7f30500107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e76ca18c7c1557a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:18.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.527+0000 7f304dc00640 1 -- 172.21.15.42:0/4239653975 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305001c680 con 0x563d85a81c20 2024-04-20T10:43:18.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.527+0000 7f304dc00640 1 -- 172.21.15.42:0/4239653975 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305001cd00 con 0x563d85a81c20 2024-04-20T10:43:18.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.527+0000 7f304dc00640 1 -- 172.21.15.42:0/4239653975 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305001a380 con 0x563d85a81c20 2024-04-20T10:43:18.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.528+0000 7f30621b7740 1 -- 172.21.15.42:0/4239653975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 msgr2=0x563d85a84010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:18.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.528+0000 7f30621b7740 1 --2- 172.21.15.42:0/4239653975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a84010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3050009f10 tx=0x7f30500107a0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.529+0000 7f30621b7740 1 -- 172.21.15.42:0/4239653975 shutdown_connections 2024-04-20T10:43:18.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.529+0000 7f30621b7740 1 --2- 172.21.15.42:0/4239653975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a84010 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.529+0000 7f30621b7740 1 --2- 172.21.15.42:0/4239653975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a816e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.529+0000 7f30621b7740 1 --2- 172.21.15.42:0/4239653975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 0x563d85a70100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.529+0000 7f30621b7740 1 -- 172.21.15.42:0/4239653975 >> 172.21.15.42:0/4239653975 conn(0x563d85a6a640 msgr2=0x563d85a6aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:18.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.529+0000 7f30621b7740 1 -- 172.21.15.42:0/4239653975 shutdown_connections 2024-04-20T10:43:18.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.529+0000 7f30621b7740 1 -- 172.21.15.42:0/4239653975 wait complete. 2024-04-20T10:43:18.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.530+0000 7f30621b7740 1 -- start start 2024-04-20T10:43:18.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.530+0000 7f30621b7740 1 Processor -- start 2024-04-20T10:43:18.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.532+0000 7f30621b7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 0x563d85a7ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.533+0000 7f305ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 0x563d85a7ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.533+0000 7f305ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 0x563d85a7ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50000/0 (socket says 172.21.15.42:50000) 2024-04-20T10:43:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.533+0000 7f30621b7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a7f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.533+0000 7f30621b7740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a7f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.533+0000 7f305ca00640 1 -- 172.21.15.42:0/1520556906 learned_addr learned my addr 172.21.15.42:0/1520556906 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.534+0000 7f3057e00640 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a7f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.534+0000 7f30621b7740 1 -- 172.21.15.42:0/1520556906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563d85a849e0 con 0x563d85a6fd20 2024-04-20T10:43:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.534+0000 7f305d400640 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a7f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.534+0000 7f30621b7740 1 -- 172.21.15.42:0/1520556906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563d8598e7a0 con 0x563d85a706d0 2024-04-20T10:43:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.534+0000 7f30621b7740 1 -- 172.21.15.42:0/1520556906 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563d85a84b60 con 0x563d85a81c20 2024-04-20T10:43:18.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.534+0000 7f30621b7740 10 client.0 _ll_register_callbacks cb 0x563d85990b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.535+0000 7f305ca00640 1 -- 172.21.15.42:0/1520556906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 msgr2=0x563d85a7f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.535+0000 7f305ca00640 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a7f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.535+0000 7f305ca00640 1 -- 172.21.15.42:0/1520556906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 msgr2=0x563d85a7f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.535+0000 7f305ca00640 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a7f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.535+0000 7f305ca00640 1 -- 172.21.15.42:0/1520556906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d857af7e0 con 0x563d85a6fd20 2024-04-20T10:43:18.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.535+0000 7f3057e00640 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a7f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:18.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.535+0000 7f305d400640 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a7f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:18.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.536+0000 7f305ca00640 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 0x563d85a7ed30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f304802c230 tx=0x7f3048009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723ecc7ca8cc8eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:18.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.536+0000 7f3042000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:43:18.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.536+0000 7f3042000640 1 -- 172.21.15.42:0/1520556906 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304804f070 con 0x563d85a6fd20 2024-04-20T10:43:18.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.536+0000 7f3042000640 1 -- 172.21.15.42:0/1520556906 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f3048004e70 con 0x563d85a6fd20 2024-04-20T10:43:18.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.537+0000 7f30621b7740 1 -- 172.21.15.42:0/1520556906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f305001cd00 con 0x563d85a6fd20 2024-04-20T10:43:18.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.537+0000 7f3042000640 1 -- 172.21.15.42:0/1520556906 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3048005070 con 0x563d85a6fd20 2024-04-20T10:43:18.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.538+0000 7f30621b7740 10 client.16338 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:18.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.538+0000 7f30621b7740 1 -- 172.21.15.42:0/1520556906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563d8580ff70 con 0x563d85a6fd20 2024-04-20T10:43:18.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.538+0000 7f3042000640 1 -- 172.21.15.42:0/1520556906 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305001cd00 con 0x563d85a6fd20 2024-04-20T10:43:18.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.538+0000 7f3042000640 1 -- 172.21.15.42:0/1520556906 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f304805b070 con 0x563d85a6fd20 2024-04-20T10:43:18.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.538+0000 7f30621b7740 10 client.16338 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:18.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.539+0000 7f30621b7740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:18.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.539+0000 7f30621b7740 20 client.16338 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:18.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.539+0000 7f30621b7740 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563d85a92040 0x563d85a94430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.539+0000 7f30621b7740 4 client.16338 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:43:18.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.539+0000 7f30621b7740 1 -- 172.21.15.42:0/1520556906 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d859e2740 con 0x563d85a92040 2024-04-20T10:43:18.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.539+0000 7f3057e00640 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563d85a92040 0x563d85a94430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.545+0000 7f3057e00640 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563d85a92040 0x563d85a94430 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb23ffcc645159f2 server_cookie=94a64e9f12507afb in_seq=0 out_seq=0 2024-04-20T10:43:18.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.545+0000 7f3042000640 10 client.16338 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.548+0000 7f3042000640 1 -- 172.21.15.42:0/1520556906 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d859e2740 con 0x563d85a92040 2024-04-20T10:43:18.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.548+0000 7f3042000640 10 client.16338 handle_command_reply: tid=0 2024-04-20T10:43:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.548+0000 7f30621b7740 1 client.16338 shutdown 2024-04-20T10:43:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.548+0000 7f30621b7740 1 -- 172.21.15.42:0/1520556906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 msgr2=0x563d85a7ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:18.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.548+0000 7f30621b7740 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 0x563d85a7ed30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f304802c230 tx=0x7f3048009260 comp rx=0 tx=0).stop 2024-04-20T10:43:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.549+0000 7f30621b7740 1 -- 172.21.15.42:0/1520556906 shutdown_connections 2024-04-20T10:43:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.549+0000 7f30621b7740 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563d85a92040 0x563d85a94430 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.549+0000 7f30621b7740 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a7f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.549+0000 7f30621b7740 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a7f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.549+0000 7f30621b7740 1 --2- 172.21.15.42:0/1520556906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a6fd20 0x563d85a7ed30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.549+0000 7f3042000640 0 client.16338 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.549+0000 7f30621b7740 1 -- 172.21.15.42:0/1520556906 >> 172.21.15.42:0/1520556906 conn(0x563d85a6a640 msgr2=0x563d85a6c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.549+0000 7f30621b7740 1 -- 172.21.15.42:0/1520556906 shutdown_connections 2024-04-20T10:43:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.549+0000 7f30621b7740 1 -- 172.21.15.42:0/1520556906 wait complete. 2024-04-20T10:43:18.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.550+0000 7f30621b7740 20 client.16338 trim_cache size 0 max 16384 2024-04-20T10:43:18.552 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/p4.o 2024-04-20T10:43:18.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.593+0000 7f30621b7740 1 Processor -- start 2024-04-20T10:43:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.593+0000 7f30621b7740 1 -- start start 2024-04-20T10:43:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.594+0000 7f30621b7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a81460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.594+0000 7f30621b7740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a8d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.594+0000 7f30621b7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 0x563d85a6c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.594+0000 7f30621b7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f30581b23f0 con 0x563d85a7de90 2024-04-20T10:43:18.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.594+0000 7f30621b7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30581b4c10 con 0x563d85a706d0 2024-04-20T10:43:18.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.594+0000 7f30621b7740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f30581b2a40 con 0x563d85a81c20 2024-04-20T10:43:18.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.594+0000 7f3057e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a8d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.594+0000 7f305d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 0x563d85a6c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.594+0000 7f3057e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a8d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:57836/0 (socket says 172.21.15.42:57836) 2024-04-20T10:43:18.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.594+0000 7f3057e00640 1 -- 172.21.15.42:0/2257313835 learned_addr learned my addr 172.21.15.42:0/2257313835 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:18.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.595+0000 7f3057e00640 1 -- 172.21.15.42:0/2257313835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 msgr2=0x563d85a81460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:43:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.595+0000 7f305ca00640 1 --2- 172.21.15.42:0/2257313835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a81460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.595+0000 7f3057e00640 1 --2- 172.21.15.42:0/2257313835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a81460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.596+0000 7f3057e00640 1 -- 172.21.15.42:0/2257313835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 msgr2=0x563d85a6c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.596+0000 7f3057e00640 1 --2- 172.21.15.42:0/2257313835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 0x563d85a6c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.596+0000 7f3057e00640 1 -- 172.21.15.42:0/2257313835 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d856a7170 con 0x563d85a81c20 2024-04-20T10:43:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.596+0000 7f305d400640 1 --2- 172.21.15.42:0/2257313835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 0x563d85a6c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.596+0000 7f305ca00640 1 --2- 172.21.15.42:0/2257313835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a81460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.596+0000 7f3057e00640 1 --2- 172.21.15.42:0/2257313835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a8d940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f30440998e0 tx=0x7f304401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8661279a8f5bd78b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:18.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.596+0000 7f304d200640 1 -- 172.21.15.42:0/2257313835 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30440a5b80 con 0x563d85a81c20 2024-04-20T10:43:18.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.596+0000 7f304d200640 1 -- 172.21.15.42:0/2257313835 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f304409b210 con 0x563d85a81c20 2024-04-20T10:43:18.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.596+0000 7f304d200640 1 -- 172.21.15.42:0/2257313835 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044018d90 con 0x563d85a81c20 2024-04-20T10:43:18.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.597+0000 7f30621b7740 1 -- 172.21.15.42:0/2257313835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 msgr2=0x563d85a8d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:18.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.597+0000 7f30621b7740 1 --2- 172.21.15.42:0/2257313835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a8d940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f30440998e0 tx=0x7f304401a040 comp rx=0 tx=0).stop 2024-04-20T10:43:18.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.598+0000 7f30621b7740 1 -- 172.21.15.42:0/2257313835 shutdown_connections 2024-04-20T10:43:18.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.598+0000 7f30621b7740 1 --2- 172.21.15.42:0/2257313835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 0x563d85a6c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.598+0000 7f30621b7740 1 --2- 172.21.15.42:0/2257313835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a8d940 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.598+0000 7f30621b7740 1 --2- 172.21.15.42:0/2257313835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a81460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.598+0000 7f30621b7740 1 -- 172.21.15.42:0/2257313835 >> 172.21.15.42:0/2257313835 conn(0x563d85a6a640 msgr2=0x563d85a6ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:18.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.599+0000 7f30621b7740 1 -- 172.21.15.42:0/2257313835 shutdown_connections 2024-04-20T10:43:18.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.599+0000 7f30621b7740 1 -- 172.21.15.42:0/2257313835 wait complete. 2024-04-20T10:43:18.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.599+0000 7f30621b7740 1 -- start start 2024-04-20T10:43:18.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.600+0000 7f30621b7740 1 Processor -- start 2024-04-20T10:43:18.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.601+0000 7f30621b7740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a85a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.601+0000 7f30621b7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 0x563d85a85f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.601+0000 7f30621b7740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.602+0000 7f30621b7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f30581b30d0 con 0x563d85a7de90 2024-04-20T10:43:18.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.602+0000 7f30621b7740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30581b4540 con 0x563d85a706d0 2024-04-20T10:43:18.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.602+0000 7f30621b7740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f30581b37a0 con 0x563d85a81c20 2024-04-20T10:43:18.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.602+0000 7f30621b7740 10 client.0 _ll_register_callbacks cb 0x7f3048042050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:18.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.602+0000 7f3057e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 0x563d85a85f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.602+0000 7f3057e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 0x563d85a85f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50004/0 (socket says 172.21.15.42:50004) 2024-04-20T10:43:18.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.602+0000 7f3057e00640 1 -- 172.21.15.42:0/122082597 learned_addr learned my addr 172.21.15.42:0/122082597 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:18.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.602+0000 7f305ca00640 1 --2- 172.21.15.42:0/122082597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a85a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.603+0000 7f3057e00640 1 -- 172.21.15.42:0/122082597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 msgr2=0x563d85a864b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:43:18.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.603+0000 7f3057e00640 1 --2- 172.21.15.42:0/122082597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a864b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.603+0000 7f3057e00640 1 -- 172.21.15.42:0/122082597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 msgr2=0x563d85a85a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:18.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.603+0000 7f3057e00640 1 --2- 172.21.15.42:0/122082597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a85a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.603+0000 7f3057e00640 1 -- 172.21.15.42:0/122082597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d85af76e0 con 0x563d85a7de90 2024-04-20T10:43:18.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.603+0000 7f3057e00640 1 --2- 172.21.15.42:0/122082597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 0x563d85a85f70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f304409cc10 tx=0x7f304409b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3f25761a7207d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:18.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.607+0000 7f3042000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:43:18.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.607+0000 7f3042000640 1 -- 172.21.15.42:0/122082597 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044018b30 con 0x563d85a7de90 2024-04-20T10:43:18.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.607+0000 7f3042000640 1 -- 172.21.15.42:0/122082597 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f3044018cd0 con 0x563d85a7de90 2024-04-20T10:43:18.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.607+0000 7f30621b7740 1 -- 172.21.15.42:0/122082597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f304409b210 con 0x563d85a7de90 2024-04-20T10:43:18.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.608+0000 7f3042000640 1 -- 172.21.15.42:0/122082597 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3044087bd0 con 0x563d85a7de90 2024-04-20T10:43:18.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.610+0000 7f30621b7740 10 client.16344 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:18.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.610+0000 7f30621b7740 1 -- 172.21.15.42:0/122082597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563d85af8950 con 0x563d85a7de90 2024-04-20T10:43:18.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.611+0000 7f3042000640 1 -- 172.21.15.42:0/122082597 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f304409b210 con 0x563d85a7de90 2024-04-20T10:43:18.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.611+0000 7f3042000640 1 -- 172.21.15.42:0/122082597 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f304409bdd0 con 0x563d85a7de90 2024-04-20T10:43:18.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.611+0000 7f30621b7740 10 client.16344 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:18.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.611+0000 7f30621b7740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:18.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.611+0000 7f30621b7740 20 client.16344 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:18.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.611+0000 7f30621b7740 1 --2- 172.21.15.42:0/122082597 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563d85af9040 0x563d85afb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:18.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.611+0000 7f30621b7740 4 client.16344 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:43:18.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.611+0000 7f30621b7740 1 -- 172.21.15.42:0/122082597 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563d859dc5b0 con 0x563d85af9040 2024-04-20T10:43:18.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.614+0000 7f305ca00640 1 --2- 172.21.15.42:0/122082597 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563d85af9040 0x563d85afb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:18.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.615+0000 7f305ca00640 1 --2- 172.21.15.42:0/122082597 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563d85af9040 0x563d85afb460 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3a449e23f4fee4 server_cookie=54495f3668efc64f in_seq=0 out_seq=0 2024-04-20T10:43:18.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.615+0000 7f3042000640 10 client.16344 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:18.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.618+0000 7f3042000640 1 -- 172.21.15.42:0/122082597 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563d859dc5b0 con 0x563d85af9040 2024-04-20T10:43:18.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.618+0000 7f3042000640 10 client.16344 handle_command_reply: tid=0 2024-04-20T10:43:18.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.619+0000 7f30621b7740 1 client.16344 shutdown 2024-04-20T10:43:18.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.620+0000 7f30621b7740 1 -- 172.21.15.42:0/122082597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 msgr2=0x563d85a85f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:18.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.620+0000 7f30621b7740 1 --2- 172.21.15.42:0/122082597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 0x563d85a85f70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f304409cc10 tx=0x7f304409b3c0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.620+0000 7f30621b7740 1 -- 172.21.15.42:0/122082597 shutdown_connections 2024-04-20T10:43:18.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.620+0000 7f30621b7740 1 --2- 172.21.15.42:0/122082597 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x563d85af9040 0x563d85afb460 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.620+0000 7f30621b7740 1 --2- 172.21.15.42:0/122082597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d85a81c20 0x563d85a864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.620+0000 7f30621b7740 1 --2- 172.21.15.42:0/122082597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563d85a7de90 0x563d85a85f70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.620+0000 7f30621b7740 1 --2- 172.21.15.42:0/122082597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d85a706d0 0x563d85a85a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.620+0000 7f30621b7740 1 -- 172.21.15.42:0/122082597 >> 172.21.15.42:0/122082597 conn(0x563d85a6a640 msgr2=0x563d85a7f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:18.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.620+0000 7f3042000640 0 client.16344 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:18.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.620+0000 7f30621b7740 1 -- 172.21.15.42:0/122082597 shutdown_connections 2024-04-20T10:43:18.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.621+0000 7f30621b7740 1 -- 172.21.15.42:0/122082597 wait complete. 2024-04-20T10:43:18.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.621+0000 7f30621b7740 20 client.16344 trim_cache size 0 max 16384 2024-04-20T10:43:18.621 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:43:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:43:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "176cb6ef-7a4a-462f-afb5-325ae28ba12f", 2024-04-20T10:43:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:43:18.621 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:43:18.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.622+0000 7f305f400640 1 -- 172.21.15.42:0/1094755367 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f303c0a3de0 msgr2=0x7f303c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:18.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.622+0000 7f305f400640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f303c0a3de0 0x7f303c0a62a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3048006010 tx=0x7f3048000bc0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.622+0000 7f305f400640 1 -- 172.21.15.42:0/1094755367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3058109a90 msgr2=0x7f30581a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:18.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.622+0000 7f305f400640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3058109a90 0x7f30581a3740 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f304400e9f0 tx=0x7f304400eeb0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.623+0000 7f305f400640 1 -- 172.21.15.42:0/1094755367 shutdown_connections 2024-04-20T10:43:18.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.623+0000 7f305f400640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f303c0a3de0 0x7f303c0a62a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.623+0000 7f305f400640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f30581174a0 0x7f30581a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.623+0000 7f305f400640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3058109a90 0x7f30581a3740 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.623+0000 7f305f400640 1 --2- 172.21.15.42:0/1094755367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3058109170 0x7f30581a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:18.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.623+0000 7f305f400640 1 -- 172.21.15.42:0/1094755367 >> 172.21.15.42:0/1094755367 conn(0x7f305806d070 msgr2=0x7f30581179c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:18.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.623+0000 7f305f400640 1 -- 172.21.15.42:0/1094755367 shutdown_connections 2024-04-20T10:43:18.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:18.624+0000 7f305f400640 1 -- 172.21.15.42:0/1094755367 wait complete. 2024-04-20T10:43:18.632 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/public_key.o 2024-04-20T10:43:18.664 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/kernfs/mount.o 2024-04-20T10:43:18.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:43:18.665 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:43:19.130 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/permission.o 2024-04-20T10:43:19.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.194+0000 7fb3eea00640 1 -- 172.21.15.42:0/3987636868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3e810c7e0 msgr2=0x7fb3e810cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:19.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.194+0000 7fb3eea00640 1 --2- 172.21.15.42:0/3987636868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3e810c7e0 0x7fb3e810cc60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009ca0 tx=0x7fb3d802c100 comp rx=0 tx=0).stop 2024-04-20T10:43:19.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.194+0000 7fb3eea00640 1 -- 172.21.15.42:0/3987636868 shutdown_connections 2024-04-20T10:43:19.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.194+0000 7fb3eea00640 1 --2- 172.21.15.42:0/3987636868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e810d1a0 0x7fb3e81196b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.194+0000 7fb3eea00640 1 --2- 172.21.15.42:0/3987636868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3e810c7e0 0x7fb3e810cc60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.195+0000 7fb3eea00640 1 --2- 172.21.15.42:0/3987636868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e81127f0 0x7fb3e8112bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.195+0000 7fb3eea00640 1 -- 172.21.15.42:0/3987636868 >> 172.21.15.42:0/3987636868 conn(0x7fb3e806cdf0 msgr2=0x7fb3e810af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:19.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.195+0000 7fb3eea00640 1 -- 172.21.15.42:0/3987636868 shutdown_connections 2024-04-20T10:43:19.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.198+0000 7fb3eea00640 1 -- 172.21.15.42:0/3987636868 wait complete. 2024-04-20T10:43:19.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.199+0000 7fb3eea00640 1 Processor -- start 2024-04-20T10:43:19.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.203+0000 7fb3eea00640 1 -- start start 2024-04-20T10:43:19.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.203+0000 7fb3eea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e810c7e0 0x7fb3e8119580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.203+0000 7fb3eea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e810d1a0 0x7fb3e81166c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.203+0000 7fb3eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3e81127f0 0x7fb3e8116c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.203+0000 7fb3eea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e811cc50 con 0x7fb3e810d1a0 2024-04-20T10:43:19.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.203+0000 7fb3eea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e811cad0 con 0x7fb3e81127f0 2024-04-20T10:43:19.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.203+0000 7fb3eea00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e811cdd0 con 0x7fb3e810c7e0 2024-04-20T10:43:19.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.204+0000 7fb3ecc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e810d1a0 0x7fb3e81166c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.204+0000 7fb3ecc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e810d1a0 0x7fb3e81166c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57432/0 (socket says 172.21.15.42:57432) 2024-04-20T10:43:19.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.204+0000 7fb3ecc00640 1 -- 172.21.15.42:0/3000229240 learned_addr learned my addr 172.21.15.42:0/3000229240 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:19.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.204+0000 7fb3ed600640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e810c7e0 0x7fb3e8119580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.204+0000 7fb3ecc00640 1 -- 172.21.15.42:0/3000229240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e810c7e0 msgr2=0x7fb3e8119580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:19.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.204+0000 7fb3ecc00640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e810c7e0 0x7fb3e8119580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.204+0000 7fb3ecc00640 1 -- 172.21.15.42:0/3000229240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3e81127f0 msgr2=0x7fb3e8116c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:43:19.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.204+0000 7fb3ee000640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3e81127f0 0x7fb3e8116c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.205+0000 7fb3ecc00640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3e81127f0 0x7fb3e8116c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.205+0000 7fb3ecc00640 1 -- 172.21.15.42:0/3000229240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e81174e0 con 0x7fb3e810d1a0 2024-04-20T10:43:19.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.205+0000 7fb3ed600640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e810c7e0 0x7fb3e8119580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:19.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.206+0000 7fb3ecc00640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e810d1a0 0x7fb3e81166c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb3d802c690 tx=0x7fb3d8034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:19.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.206+0000 7fb3e6000640 1 -- 172.21.15.42:0/3000229240 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8039050 con 0x7fb3e810d1a0 2024-04-20T10:43:19.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.206+0000 7fb3e6000640 1 -- 172.21.15.42:0/3000229240 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d8045070 con 0x7fb3e810d1a0 2024-04-20T10:43:19.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.206+0000 7fb3e6000640 1 -- 172.21.15.42:0/3000229240 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80045e0 con 0x7fb3e810d1a0 2024-04-20T10:43:19.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.206+0000 7fb3eea00640 1 -- 172.21.15.42:0/3000229240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e81a91d0 con 0x7fb3e810d1a0 2024-04-20T10:43:19.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.207+0000 7fb3eea00640 1 -- 172.21.15.42:0/3000229240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e81a9670 con 0x7fb3e810d1a0 2024-04-20T10:43:19.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.215+0000 7fb3e6000640 1 -- 172.21.15.42:0/3000229240 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb3d8040450 con 0x7fb3e810d1a0 2024-04-20T10:43:19.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.222+0000 7fb3e6000640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb3cc0a3de0 0x7fb3cc0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.222+0000 7fb3f2e29740 1 Processor -- start 2024-04-20T10:43:19.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.222+0000 7fb3f2e29740 1 -- start start 2024-04-20T10:43:19.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.223+0000 7fb3f2e29740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030380 0x5631e1030760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.223+0000 7fb3f2e29740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e1039340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.223+0000 7fb3e6000640 1 -- 172.21.15.42:0/3000229240 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fb3d80f3d20 con 0x7fb3e810d1a0 2024-04-20T10:43:19.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.223+0000 7fb3ed600640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb3cc0a3de0 0x7fb3cc0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.224+0000 7fb3ecc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e1039340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.224+0000 7fb3ecc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e1039340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:51506/0 (socket says 172.21.15.42:51506) 2024-04-20T10:43:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.225+0000 7fb3ed600640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb3cc0a3de0 0x7fb3cc0a62a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009b20 tx=0x7fb3d4005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:19.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.226+0000 7fb3ed600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030380 0x5631e1030760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.223+0000 7fb3f2e29740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 0x5631e103bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.227+0000 7fb3ecc00640 1 -- 172.21.15.42:0/589161738 learned_addr learned my addr 172.21.15.42:0/589161738 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.227+0000 7fb3f2e29740 1 -- 172.21.15.42:0/589161738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5631e0f96150 con 0x5631e1030380 2024-04-20T10:43:19.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.227+0000 7fb3f2e29740 1 -- 172.21.15.42:0/589161738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5631e0f3c370 con 0x5631e1030ca0 2024-04-20T10:43:19.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.227+0000 7fb3f2e29740 1 -- 172.21.15.42:0/589161738 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5631e0f92110 con 0x5631e1039880 2024-04-20T10:43:19.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.227+0000 7fb3ee000640 1 --2- 172.21.15.42:0/589161738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 0x5631e103bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.229+0000 7fb3ee000640 1 -- 172.21.15.42:0/589161738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 msgr2=0x5631e1039340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:19.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.229+0000 7fb3ee000640 1 --2- 172.21.15.42:0/589161738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e1039340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.229+0000 7fb3ee000640 1 -- 172.21.15.42:0/589161738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030380 msgr2=0x5631e1030760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:19.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.229+0000 7fb3ee000640 1 --2- 172.21.15.42:0/589161738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030380 0x5631e1030760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.229+0000 7fb3ee000640 1 -- 172.21.15.42:0/589161738 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631e0d727d0 con 0x5631e1039880 2024-04-20T10:43:19.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.229+0000 7fb3ee000640 1 --2- 172.21.15.42:0/589161738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 0x5631e103bc70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009a40 tx=0x7fb3e0009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a5d65c7ea16271a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:19.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.230+0000 7fb3de600640 1 -- 172.21.15.42:0/589161738 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0016070 con 0x5631e1039880 2024-04-20T10:43:19.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.230+0000 7fb3de600640 1 -- 172.21.15.42:0/589161738 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e000ec50 con 0x5631e1039880 2024-04-20T10:43:19.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.231+0000 7fb3f2e29740 1 -- 172.21.15.42:0/589161738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 msgr2=0x5631e103bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:19.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.231+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/589161738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 0x5631e103bc70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009a40 tx=0x7fb3e0009f00 comp rx=0 tx=0).stop 2024-04-20T10:43:19.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.231+0000 7fb3f2e29740 1 -- 172.21.15.42:0/589161738 shutdown_connections 2024-04-20T10:43:19.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.231+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/589161738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 0x5631e103bc70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.231+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/589161738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e1039340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.231+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/589161738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030380 0x5631e1030760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.231+0000 7fb3f2e29740 1 -- 172.21.15.42:0/589161738 >> 172.21.15.42:0/589161738 conn(0x5631e102aab0 msgr2=0x5631e102ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:19.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.231+0000 7fb3f2e29740 1 -- 172.21.15.42:0/589161738 shutdown_connections 2024-04-20T10:43:19.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.232+0000 7fb3f2e29740 1 -- 172.21.15.42:0/589161738 wait complete. 2024-04-20T10:43:19.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.232+0000 7fb3f2e29740 1 -- start start 2024-04-20T10:43:19.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.232+0000 7fb3f2e29740 1 Processor -- start 2024-04-20T10:43:19.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.233+0000 7fb3f2e29740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030380 0x5631e1039330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.234+0000 7fb3f2e29740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e10366c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.234+0000 7fb3ed600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030380 0x5631e1039330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.234+0000 7fb3ed600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030380 0x5631e1039330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57440/0 (socket says 172.21.15.42:57440) 2024-04-20T10:43:19.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.234+0000 7fb3ecc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e10366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.234+0000 7fb3ecc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e10366c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:51516/0 (socket says 172.21.15.42:51516) 2024-04-20T10:43:19.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.237+0000 7fb3f2e29740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 0x5631e1036e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.237+0000 7fb3f2e29740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5631e103c710 con 0x5631e1030380 2024-04-20T10:43:19.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.237+0000 7fb3f2e29740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5631e0f4f740 con 0x5631e1030ca0 2024-04-20T10:43:19.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.237+0000 7fb3f2e29740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5631e103c890 con 0x5631e1039880 2024-04-20T10:43:19.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.237+0000 7fb3f2e29740 10 client.0 _ll_register_callbacks cb 0x5631e0f4ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:19.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.237+0000 7fb3ed600640 1 -- 172.21.15.42:0/2241128034 learned_addr learned my addr 172.21.15.42:0/2241128034 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:19.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.238+0000 7fb3ecc00640 1 -- 172.21.15.42:0/2241128034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 msgr2=0x5631e1036e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:43:19.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.238+0000 7fb3ecc00640 1 --2- 172.21.15.42:0/2241128034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 0x5631e1036e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.238+0000 7fb3ecc00640 1 -- 172.21.15.42:0/2241128034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030380 msgr2=0x5631e1039330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:19.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.238+0000 7fb3ecc00640 1 --2- 172.21.15.42:0/2241128034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030380 0x5631e1039330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.238+0000 7fb3ecc00640 1 -- 172.21.15.42:0/2241128034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631e0ca83f0 con 0x5631e1030ca0 2024-04-20T10:43:19.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.238+0000 7fb3ed600640 1 --2- 172.21.15.42:0/2241128034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030380 0x5631e1039330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:19.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.238+0000 7fb3ecc00640 1 --2- 172.21.15.42:0/2241128034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e10366c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80bc770 tx=0x7fb3d80bf700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef50a2abf090d155 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:19.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.238+0000 7fb3d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:43:19.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.238+0000 7fb3d2a00640 1 -- 172.21.15.42:0/2241128034 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80c3070 con 0x5631e1030ca0 2024-04-20T10:43:19.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.239+0000 7fb3d2a00640 1 -- 172.21.15.42:0/2241128034 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fb3d80f16c0 con 0x5631e1030ca0 2024-04-20T10:43:19.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.239+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2241128034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e000ec50 con 0x5631e1030ca0 2024-04-20T10:43:19.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.239+0000 7fb3d2a00640 1 -- 172.21.15.42:0/2241128034 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb3d80bfda0 con 0x5631e1030ca0 2024-04-20T10:43:19.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.241+0000 7fb3f2e29740 10 client.26468 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:19.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.241+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2241128034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5631e0c680f0 con 0x5631e1030ca0 2024-04-20T10:43:19.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.242+0000 7fb3d2a00640 1 -- 172.21.15.42:0/2241128034 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e000ec50 con 0x5631e1030ca0 2024-04-20T10:43:19.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.242+0000 7fb3d2a00640 1 -- 172.21.15.42:0/2241128034 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fb3d80c8580 con 0x5631e1030ca0 2024-04-20T10:43:19.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.242+0000 7fb3f2e29740 10 client.26468 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:19.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.242+0000 7fb3f2e29740 10 client.26468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:19.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.242+0000 7fb3f2e29740 20 client.26468 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:19.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.242+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/2241128034 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5631e1049de0 0x5631e104c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.242+0000 7fb3f2e29740 4 client.26468 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:43:19.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.242+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2241128034 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631e0fa3b70 con 0x5631e1049de0 2024-04-20T10:43:19.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.243+0000 7fb3ed600640 1 --2- 172.21.15.42:0/2241128034 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5631e1049de0 0x5631e104c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.246+0000 7fb3ed600640 1 --2- 172.21.15.42:0/2241128034 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5631e1049de0 0x5631e104c280 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=567b43489310ebc3 server_cookie=bf3f27011939dc4c in_seq=0 out_seq=0 2024-04-20T10:43:19.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.246+0000 7fb3d2a00640 10 client.26468 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:19.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.248+0000 7fb3d2a00640 1 -- 172.21.15.42:0/2241128034 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631e0fa3b70 con 0x5631e1049de0 2024-04-20T10:43:19.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.248+0000 7fb3d2a00640 10 client.26468 handle_command_reply: tid=0 2024-04-20T10:43:19.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.248+0000 7fb3f2e29740 1 client.26468 shutdown 2024-04-20T10:43:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.251+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2241128034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 msgr2=0x5631e10366c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.251+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/2241128034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e10366c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80bc770 tx=0x7fb3d80bf700 comp rx=0 tx=0).stop 2024-04-20T10:43:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.251+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2241128034 shutdown_connections 2024-04-20T10:43:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.251+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/2241128034 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5631e1049de0 0x5631e104c280 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.251+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/2241128034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 0x5631e1036e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.251+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/2241128034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e10366c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.251+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/2241128034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030380 0x5631e1039330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.251+0000 7fb3d2a00640 0 client.26468 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:19.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.251+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2241128034 >> 172.21.15.42:0/2241128034 conn(0x5631e102aab0 msgr2=0x5631e102c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:19.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.251+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2241128034 shutdown_connections 2024-04-20T10:43:19.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.252+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2241128034 wait complete. 2024-04-20T10:43:19.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.252+0000 7fb3f2e29740 20 client.26468 trim_cache size 0 max 16384 2024-04-20T10:43:19.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.306+0000 7fb3f2e29740 1 Processor -- start 2024-04-20T10:43:19.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.306+0000 7fb3f2e29740 1 -- start start 2024-04-20T10:43:19.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.307+0000 7fb3f2e29740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030380 0x5631e10456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.307+0000 7fb3f2e29740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030ca0 0x5631e1038df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.308+0000 7fb3ecc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030ca0 0x5631e1038df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.308+0000 7fb3ecc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030ca0 0x5631e1038df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57456/0 (socket says 172.21.15.42:57456) 2024-04-20T10:43:19.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.308+0000 7fb3ed600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030380 0x5631e10456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.308+0000 7fb3ed600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030380 0x5631e10456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:51518/0 (socket says 172.21.15.42:51518) 2024-04-20T10:43:19.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.309+0000 7fb3f2e29740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 0x5631e1039330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.309+0000 7fb3ed600640 1 -- 172.21.15.42:0/1979237174 learned_addr learned my addr 172.21.15.42:0/1979237174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:19.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.309+0000 7fb3ee000640 1 --2- 172.21.15.42:0/1979237174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 0x5631e1039330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.309+0000 7fb3f2e29740 1 -- 172.21.15.42:0/1979237174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c800c8e0 con 0x5631e1030ca0 2024-04-20T10:43:19.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.310+0000 7fb3f2e29740 1 -- 172.21.15.42:0/1979237174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c800f140 con 0x5631e1030380 2024-04-20T10:43:19.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.310+0000 7fb3f2e29740 1 -- 172.21.15.42:0/1979237174 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c800cf30 con 0x5631e1039880 2024-04-20T10:43:19.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.310+0000 7fb3ed600640 1 -- 172.21.15.42:0/1979237174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 msgr2=0x5631e1039330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:19.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.310+0000 7fb3ed600640 1 --2- 172.21.15.42:0/1979237174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 0x5631e1039330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.310+0000 7fb3ed600640 1 -- 172.21.15.42:0/1979237174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030ca0 msgr2=0x5631e1038df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:19.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.311+0000 7fb3ed600640 1 --2- 172.21.15.42:0/1979237174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030ca0 0x5631e1038df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.311+0000 7fb3ed600640 1 -- 172.21.15.42:0/1979237174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631e1030ad0 con 0x5631e1030380 2024-04-20T10:43:19.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.311+0000 7fb3ecc00640 1 --2- 172.21.15.42:0/1979237174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030ca0 0x5631e1038df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:43:19.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.312+0000 7fb3ed600640 1 --2- 172.21.15.42:0/1979237174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030380 0x5631e10456f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5631e104ca90 tx=0x7fb3d4016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94489b800491d6a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:19.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.312+0000 7fb3ddc00640 1 -- 172.21.15.42:0/1979237174 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4012ba0 con 0x5631e1030380 2024-04-20T10:43:19.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.312+0000 7fb3ddc00640 1 -- 172.21.15.42:0/1979237174 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5631e0fa3b70 con 0x5631e1030380 2024-04-20T10:43:19.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.313+0000 7fb3ddc00640 1 -- 172.21.15.42:0/1979237174 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40209d0 con 0x5631e1030380 2024-04-20T10:43:19.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.314+0000 7fb3f2e29740 1 -- 172.21.15.42:0/1979237174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030380 msgr2=0x5631e10456f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:19.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.314+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/1979237174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030380 0x5631e10456f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5631e104ca90 tx=0x7fb3d4016000 comp rx=0 tx=0).stop 2024-04-20T10:43:19.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.314+0000 7fb3f2e29740 1 -- 172.21.15.42:0/1979237174 shutdown_connections 2024-04-20T10:43:19.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.315+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/1979237174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1039880 0x5631e1039330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.315+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/1979237174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1030ca0 0x5631e1038df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.315+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/1979237174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030380 0x5631e10456f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.315+0000 7fb3f2e29740 1 -- 172.21.15.42:0/1979237174 >> 172.21.15.42:0/1979237174 conn(0x5631e102aab0 msgr2=0x5631e10319c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:19.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.315+0000 7fb3f2e29740 1 -- 172.21.15.42:0/1979237174 shutdown_connections 2024-04-20T10:43:19.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.316+0000 7fb3f2e29740 1 -- 172.21.15.42:0/1979237174 wait complete. 2024-04-20T10:43:19.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.317+0000 7fb3f2e29740 1 -- start start 2024-04-20T10:43:19.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.317+0000 7fb3f2e29740 1 Processor -- start 2024-04-20T10:43:19.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.318+0000 7fb3f2e29740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1030380 0x5631e1032ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.319+0000 7fb3f2e29740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e1033010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.319+0000 7fb3f2e29740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1039880 0x5631e1033550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.319+0000 7fb3f2e29740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c800d600 con 0x5631e1039880 2024-04-20T10:43:19.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.319+0000 7fb3ee000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1039880 0x5631e1033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.319+0000 7fb3ee000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1039880 0x5631e1033550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57464/0 (socket says 172.21.15.42:57464) 2024-04-20T10:43:19.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.320+0000 7fb3ee000640 1 -- 172.21.15.42:0/2682203854 learned_addr learned my addr 172.21.15.42:0/2682203854 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:19.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.320+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2682203854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c800ea70 con 0x5631e1030ca0 2024-04-20T10:43:19.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.320+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2682203854 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c800dcd0 con 0x5631e1030380 2024-04-20T10:43:19.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.320+0000 7fb3f2e29740 10 client.0 _ll_register_callbacks cb 0x7fb3d80bfb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:19.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.320+0000 7fb3ed600640 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1030380 0x5631e1032ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.320+0000 7fb3ecc00640 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e1033010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.321+0000 7fb3ee000640 1 -- 172.21.15.42:0/2682203854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1030380 msgr2=0x5631e1032ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:19.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.321+0000 7fb3ee000640 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1030380 0x5631e1032ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.321+0000 7fb3ee000640 1 -- 172.21.15.42:0/2682203854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 msgr2=0x5631e1033010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:19.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.321+0000 7fb3ee000640 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e1033010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.322+0000 7fb3ee000640 1 -- 172.21.15.42:0/2682203854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631e1040e80 con 0x5631e1039880 2024-04-20T10:43:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.322+0000 7fb3ed600640 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1030380 0x5631e1032ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.322+0000 7fb3ecc00640 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e1033010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:19.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.323+0000 7fb3d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:43:19.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.323+0000 7fb3ee000640 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1039880 0x5631e1033550 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5631e102bc40 tx=0x7fb3e0005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18498af5275e2608 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:19.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.324+0000 7fb3d2a00640 1 -- 172.21.15.42:0/2682203854 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0016040 con 0x5631e1039880 2024-04-20T10:43:19.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.324+0000 7fb3d2a00640 1 -- 172.21.15.42:0/2682203854 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fb3e00052d0 con 0x5631e1039880 2024-04-20T10:43:19.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.324+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2682203854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5631e0fa3b70 con 0x5631e1039880 2024-04-20T10:43:19.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.325+0000 7fb3d2a00640 1 -- 172.21.15.42:0/2682203854 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb3e00054d0 con 0x5631e1039880 2024-04-20T10:43:19.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.325+0000 7fb3f2e29740 10 client.16362 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:19.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.325+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2682203854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5631e1042190 con 0x5631e1039880 2024-04-20T10:43:19.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.326+0000 7fb3d2a00640 1 -- 172.21.15.42:0/2682203854 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5631e0fa3b70 con 0x5631e1039880 2024-04-20T10:43:19.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.326+0000 7fb3d2a00640 1 -- 172.21.15.42:0/2682203854 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fb3e000f040 con 0x5631e1039880 2024-04-20T10:43:19.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.326+0000 7fb3f2e29740 10 client.16362 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:19.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.326+0000 7fb3f2e29740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:19.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.327+0000 7fb3f2e29740 20 client.16362 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:19.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.327+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5631e10b03c0 0x5631e10b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:19.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.327+0000 7fb3f2e29740 4 client.16362 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:43:19.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.327+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2682203854 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631e0f9d9e0 con 0x5631e10b03c0 2024-04-20T10:43:19.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.328+0000 7fb3ed600640 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5631e10b03c0 0x5631e10b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:19.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.330+0000 7fb3ed600640 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5631e10b03c0 0x5631e10b27b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6513e923680ffd0a server_cookie=d0301172c6d64a48 in_seq=0 out_seq=0 2024-04-20T10:43:19.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.330+0000 7fb3d2a00640 10 client.16362 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:19.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.330+0000 7fb3d2a00640 1 -- 172.21.15.42:0/2682203854 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x5631e0f9d9e0 con 0x5631e10b03c0 2024-04-20T10:43:19.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.330+0000 7fb3d2a00640 10 client.16362 handle_command_reply: tid=0 2024-04-20T10:43:19.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.331+0000 7fb3f2e29740 1 client.16362 shutdown 2024-04-20T10:43:19.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.331+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2682203854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1039880 msgr2=0x5631e1033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:19.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.331+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1039880 0x5631e1033550 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5631e102bc40 tx=0x7fb3e0005cb0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.333+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2682203854 shutdown_connections 2024-04-20T10:43:19.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.334+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5631e10b03c0 0x5631e10b27b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.334+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631e1039880 0x5631e1033550 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.334+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e1030ca0 0x5631e1033010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.334+0000 7fb3f2e29740 1 --2- 172.21.15.42:0/2682203854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5631e1030380 0x5631e1032ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.334+0000 7fb3d2a00640 0 client.16362 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:19.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.334+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2682203854 >> 172.21.15.42:0/2682203854 conn(0x5631e102aab0 msgr2=0x5631e1034f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:19.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.335+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2682203854 shutdown_connections 2024-04-20T10:43:19.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.335+0000 7fb3f2e29740 1 -- 172.21.15.42:0/2682203854 wait complete. 2024-04-20T10:43:19.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.336+0000 7fb3f2e29740 20 client.16362 trim_cache size 0 max 16384 2024-04-20T10:43:19.336 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:43:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (16959 inodes in the stack)", 2024-04-20T10:43:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:43:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "176cb6ef-7a4a-462f-afb5-325ae28ba12f": { 2024-04-20T10:43:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:43:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "176cb6ef-7a4a-462f-afb5-325ae28ba12f", 2024-04-20T10:43:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:43:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:43:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:43:19.337 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:43:19.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.338+0000 7fb3eea00640 1 -- 172.21.15.42:0/3000229240 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb3cc0a3de0 msgr2=0x7fb3cc0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:19.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.338+0000 7fb3eea00640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb3cc0a3de0 0x7fb3cc0a62a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009b20 tx=0x7fb3d4005f90 comp rx=0 tx=0).stop 2024-04-20T10:43:19.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.338+0000 7fb3eea00640 1 -- 172.21.15.42:0/3000229240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e810d1a0 msgr2=0x7fb3e81166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:19.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.338+0000 7fb3eea00640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e810d1a0 0x7fb3e81166c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb3d802c690 tx=0x7fb3d8034670 comp rx=0 tx=0).stop 2024-04-20T10:43:19.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.339+0000 7fb3eea00640 1 -- 172.21.15.42:0/3000229240 shutdown_connections 2024-04-20T10:43:19.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.339+0000 7fb3eea00640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb3cc0a3de0 0x7fb3cc0a62a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.339+0000 7fb3eea00640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3e81127f0 0x7fb3e8116c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.339+0000 7fb3eea00640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e810d1a0 0x7fb3e81166c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.339+0000 7fb3eea00640 1 --2- 172.21.15.42:0/3000229240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3e810c7e0 0x7fb3e8119580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:19.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.339+0000 7fb3eea00640 1 -- 172.21.15.42:0/3000229240 >> 172.21.15.42:0/3000229240 conn(0x7fb3e806cdf0 msgr2=0x7fb3e810f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:19.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.339+0000 7fb3eea00640 1 -- 172.21.15.42:0/3000229240 shutdown_connections 2024-04-20T10:43:19.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:19.339+0000 7fb3eea00640 1 -- 172.21.15.42:0/3000229240 wait complete. 2024-04-20T10:43:19.389 INFO:tasks.cephfs.filesystem:scrub status for tag:176cb6ef-7a4a-462f-afb5-325ae28ba12f - {'path': '/', 'tag': '176cb6ef-7a4a-462f-afb5-325ae28ba12f', 'options': 'recursive,force'} 2024-04-20T10:43:19.404 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/printk/printk_ringbuffer.o 2024-04-20T10:43:19.608 INFO:tasks.workunit.client.0.smithi042.stdout: ASN.1 crypto/asymmetric_keys/x509.asn1.[ch] 2024-04-20T10:43:19.631 INFO:tasks.workunit.client.0.smithi042.stdout: ASN.1 crypto/asymmetric_keys/x509_akid.asn1.[ch] 2024-04-20T10:43:19.649 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/x509_loader.o 2024-04-20T10:43:19.679 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/kernfs/inode.o 2024-04-20T10:43:19.803 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:19 smithi042 ceph-mon[17078]: pgmap v288: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 875 KiB/s wr, 74 op/s 2024-04-20T10:43:19.803 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:19 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:43:19.803 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:19 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:43:19.803 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:19 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:43:19.830 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/p6.o 2024-04-20T10:43:19.850 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/generic.o 2024-04-20T10:43:19.977 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/power/snapshot.o 2024-04-20T10:43:20.036 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/process_keys.o 2024-04-20T10:43:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:19 smithi046 ceph-mon[21574]: pgmap v288: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 875 KiB/s wr, 74 op/s 2024-04-20T10:43:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:19 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:43:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:19 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:43:20.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:19 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:43:20.056 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/x509_public_key.o 2024-04-20T10:43:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:19 smithi162 ceph-mon[21854]: pgmap v288: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 875 KiB/s wr, 74 op/s 2024-04-20T10:43:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:19 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:43:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:19 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:43:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:19 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:43:20.258 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/printk/sysctl.o 2024-04-20T10:43:20.692 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/kernfs/dir.o 2024-04-20T10:43:20.698 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/util.o 2024-04-20T10:43:20.804 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/printk/built-in.a 2024-04-20T10:43:20.862 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/mmzone.o 2024-04-20T10:43:20.915 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/pt.o 2024-04-20T10:43:21.015 INFO:tasks.workunit.client.0.smithi042.stdout: ASN.1 crypto/asymmetric_keys/pkcs7.asn1.[ch] 2024-04-20T10:43:21.033 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/pkcs7_trust.o 2024-04-20T10:43:21.080 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/array.o 2024-04-20T10:43:21.376 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/request_key.o 2024-04-20T10:43:21.597 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/pkcs7_verify.o 2024-04-20T10:43:21.746 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/fpu/init.o 2024-04-20T10:43:21.985 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/power/swap.o 2024-04-20T10:43:22.073 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/vmstat.o 2024-04-20T10:43:22.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:21 smithi042 ceph-mon[17078]: pgmap v289: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 757 KiB/s wr, 64 op/s 2024-04-20T10:43:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:21 smithi162 ceph-mon[21854]: pgmap v289: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 757 KiB/s wr, 64 op/s 2024-04-20T10:43:22.182 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/kernfs/file.o 2024-04-20T10:43:22.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:21 smithi046 ceph-mon[21574]: pgmap v289: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 757 KiB/s wr, 64 op/s 2024-04-20T10:43:22.305 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/x509.asn1.o 2024-04-20T10:43:22.314 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/fpu/bugs.o 2024-04-20T10:43:22.410 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/x509_akid.asn1.o 2024-04-20T10:43:22.507 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/x509_cert_parser.o 2024-04-20T10:43:22.583 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/fd.o 2024-04-20T10:43:22.597 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/uncore.o 2024-04-20T10:43:22.599 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/request_key_auth.o 2024-04-20T10:43:22.636 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/fpu/core.o 2024-04-20T10:43:23.389 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/pkcs7.asn1.o 2024-04-20T10:43:23.448 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/kernfs/symlink.o 2024-04-20T10:43:23.497 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/asymmetric_keys/pkcs7_parser.o 2024-04-20T10:43:23.655 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/user_defined.o 2024-04-20T10:43:23.815 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/proc_tty.o 2024-04-20T10:43:23.851 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/power/user.o 2024-04-20T10:43:24.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:23 smithi042 ceph-mon[17078]: pgmap v290: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 847 KiB/s wr, 510 op/s 2024-04-20T10:43:24.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:23 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:43:24.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:23 smithi162 ceph-mon[21854]: pgmap v290: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 847 KiB/s wr, 510 op/s 2024-04-20T10:43:24.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:23 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:43:24.158 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/backing-dev.o 2024-04-20T10:43:24.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:23 smithi046 ceph-mon[21574]: pgmap v290: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 847 KiB/s wr, 510 op/s 2024-04-20T10:43:24.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:23 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:43:24.427 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/fpu/regset.o 2024-04-20T10:43:24.559 INFO:tasks.workunit.client.0.smithi042.stdout: AR crypto/asymmetric_keys/built-in.a 2024-04-20T10:43:24.604 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/uncore_nhmex.o 2024-04-20T10:43:24.618 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/kernfs/built-in.a 2024-04-20T10:43:24.670 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/compat.o 2024-04-20T10:43:24.727 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/api.o 2024-04-20T10:43:24.805 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/cmdline.o 2024-04-20T10:43:24.819 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/mce/core.o 2024-04-20T10:43:25.267 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/power/poweroff.o 2024-04-20T10:43:25.730 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/fpu/signal.o 2024-04-20T10:43:25.832 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/power/built-in.a 2024-04-20T10:43:25.899 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/consoles.o 2024-04-20T10:43:25.906 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/mm_init.o 2024-04-20T10:43:25.931 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/proc.o 2024-04-20T10:43:25.959 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/irqdesc.o 2024-04-20T10:43:26.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:25 smithi042 ceph-mon[17078]: pgmap v291: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 336 KiB/s wr, 529 op/s 2024-04-20T10:43:26.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:25 smithi162 ceph-mon[21854]: pgmap v291: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 336 KiB/s wr, 529 op/s 2024-04-20T10:43:26.220 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/cipher.o 2024-04-20T10:43:26.245 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/uncore_snb.o 2024-04-20T10:43:26.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:25 smithi046 ceph-mon[21574]: pgmap v291: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 336 KiB/s wr, 529 op/s 2024-04-20T10:43:26.949 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/cpuinfo.o 2024-04-20T10:43:26.998 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/sysctl.o 2024-04-20T10:43:27.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:26 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:43:27.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:26 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:43:27.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:26 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:43:27.131 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/handle.o 2024-04-20T10:43:27.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:26 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:43:27.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:26 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:43:27.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:26 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:43:27.150 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/fpu/xstate.o 2024-04-20T10:43:27.261 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/compress.o 2024-04-20T10:43:27.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:26 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:43:27.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:26 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:43:27.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:26 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:43:27.317 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/mce/severity.o 2024-04-20T10:43:27.731 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/uncore_snbep.o 2024-04-20T10:43:27.787 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/devices.o 2024-04-20T10:43:27.835 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/percpu.o 2024-04-20T10:43:27.918 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/keys/keyctl_pkey.o 2024-04-20T10:43:28.048 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/manage.o 2024-04-20T10:43:28.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:27 smithi042 ceph-mon[17078]: pgmap v292: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 618 KiB/s wr, 1.02k op/s 2024-04-20T10:43:28.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:27 smithi162 ceph-mon[21854]: pgmap v292: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 618 KiB/s wr, 1.02k op/s 2024-04-20T10:43:28.241 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/algapi.o 2024-04-20T10:43:28.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:27 smithi046 ceph-mon[21574]: pgmap v292: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 618 KiB/s wr, 1.02k op/s 2024-04-20T10:43:28.498 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/mce/genpool.o 2024-04-20T10:43:28.723 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/interrupts.o 2024-04-20T10:43:28.914 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/kernel/fpu/built-in.a 2024-04-20T10:43:29.006 INFO:tasks.workunit.client.0.smithi042.stdout: AR security/keys/built-in.a 2024-04-20T10:43:29.059 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/mtrr/mtrr.o 2024-04-20T10:43:29.081 INFO:tasks.workunit.client.0.smithi042.stdout: GEN security/selinux/flask.h security/selinux/av_permissions.h 2024-04-20T10:43:29.123 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/avc.o 2024-04-20T10:43:29.441 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/mce/intel.o 2024-04-20T10:43:29.482 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/loadavg.o 2024-04-20T10:43:29.696 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/spurious.o 2024-04-20T10:43:29.861 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/scatterwalk.o 2024-04-20T10:43:30.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:29 smithi042 ceph-mon[17078]: pgmap v293: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 2.02k op/s 2024-04-20T10:43:30.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:29 smithi162 ceph-mon[21854]: pgmap v293: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 2.02k op/s 2024-04-20T10:43:30.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:29 smithi046 ceph-mon[21574]: pgmap v293: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 2.02k op/s 2024-04-20T10:43:30.337 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/mtrr/if.o 2024-04-20T10:43:30.431 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/meminfo.o 2024-04-20T10:43:30.448 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/uncore_discovery.o 2024-04-20T10:43:30.469 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/mce/amd.o 2024-04-20T10:43:30.487 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/resend.o 2024-04-20T10:43:30.654 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/slab_common.o 2024-04-20T10:43:30.909 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/proc.o 2024-04-20T10:43:31.278 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/chip.o 2024-04-20T10:43:31.291 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/mtrr/generic.o 2024-04-20T10:43:31.355 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/hooks.o 2024-04-20T10:43:31.537 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/stat.o 2024-04-20T10:43:31.871 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/intel/cstate.o 2024-04-20T10:43:31.931 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/aead.o 2024-04-20T10:43:32.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:31 smithi162 ceph-mon[21854]: pgmap v294: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 2.02k op/s 2024-04-20T10:43:32.207 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/mce/threshold.o 2024-04-20T10:43:32.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:31 smithi046 ceph-mon[21574]: pgmap v294: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 2.02k op/s 2024-04-20T10:43:32.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:31 smithi042 ceph-mon[17078]: pgmap v294: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 2.02k op/s 2024-04-20T10:43:32.563 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/uptime.o 2024-04-20T10:43:32.606 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/dummychip.o 2024-04-20T10:43:32.780 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/mtrr/cleanup.o 2024-04-20T10:43:32.896 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/compaction.o 2024-04-20T10:43:33.132 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/events/intel/built-in.a 2024-04-20T10:43:33.166 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/geniv.o 2024-04-20T10:43:33.277 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/probe.o 2024-04-20T10:43:33.319 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/util.o 2024-04-20T10:43:33.384 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/devres.o 2024-04-20T10:43:33.658 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/kernel/cpu/mce/built-in.a 2024-04-20T10:43:33.796 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/pat/set_memory.o 2024-04-20T10:43:33.861 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/kernel/cpu/mtrr/built-in.a 2024-04-20T10:43:33.872 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/utils.o 2024-04-20T10:43:33.950 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/microcode/core.o 2024-04-20T10:43:34.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:33 smithi162 ceph-mon[21854]: pgmap v295: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 3.81k op/s 2024-04-20T10:43:34.181 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/autoprobe.o 2024-04-20T10:43:34.189 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/version.o 2024-04-20T10:43:34.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:33 smithi046 ceph-mon[21574]: pgmap v295: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 3.81k op/s 2024-04-20T10:43:34.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:33 smithi042 ceph-mon[17078]: pgmap v295: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 3.81k op/s 2024-04-20T10:43:34.435 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/skcipher.o 2024-04-20T10:43:34.875 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/irqdomain.o 2024-04-20T10:43:34.908 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/rapl.o 2024-04-20T10:43:34.919 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/softirqs.o 2024-04-20T10:43:35.014 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/microcode/intel.o 2024-04-20T10:43:35.582 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/show_mem.o 2024-04-20T10:43:35.583 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/namespaces.o 2024-04-20T10:43:35.599 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/selinuxfs.o 2024-04-20T10:43:35.605 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/pat/memtype.o 2024-04-20T10:43:35.609 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/seqiv.o 2024-04-20T10:43:35.842 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/events/msr.o 2024-04-20T10:43:35.907 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/microcode/amd.o 2024-04-20T10:43:36.082 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/proc.o 2024-04-20T10:43:36.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:35 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:43:36.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:35 smithi162 ceph-mon[21854]: pgmap v296: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 3.59k op/s 2024-04-20T10:43:36.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:35 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:43:36.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:35 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:43:36.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:35 smithi042 ceph-mon[17078]: pgmap v296: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 3.59k op/s 2024-04-20T10:43:36.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:35 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:43:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:35 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:43:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:35 smithi046 ceph-mon[21574]: pgmap v296: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 3.59k op/s 2024-04-20T10:43:36.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:35 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:43:36.386 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/echainiv.o 2024-04-20T10:43:36.500 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/self.o 2024-04-20T10:43:36.572 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/pat/memtype_interval.o 2024-04-20T10:43:36.635 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/interval_tree.o 2024-04-20T10:43:36.702 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/events/built-in.a 2024-04-20T10:43:36.828 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/rcu/update.o 2024-04-20T10:43:36.860 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/kernel/cpu/microcode/built-in.a 2024-04-20T10:43:36.890 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/migration.o 2024-04-20T10:43:36.917 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/cacheinfo.o 2024-04-20T10:43:37.133 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/ahash.o 2024-04-20T10:43:37.181 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/thread_self.o 2024-04-20T10:43:37.332 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/mm/pat/built-in.a 2024-04-20T10:43:37.388 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/init.o 2024-04-20T10:43:37.440 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/netlink.o 2024-04-20T10:43:37.490 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/cpuhotplug.o 2024-04-20T10:43:37.555 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/list_lru.o 2024-04-20T10:43:37.871 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/proc_sysctl.o 2024-04-20T10:43:38.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:37 smithi162 ceph-mon[21854]: pgmap v297: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.2 MiB/s wr, 3.91k op/s 2024-04-20T10:43:38.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:37 smithi046 ceph-mon[21574]: pgmap v297: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.2 MiB/s wr, 3.91k op/s 2024-04-20T10:43:38.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:37 smithi042 ceph-mon[17078]: pgmap v297: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.2 MiB/s wr, 3.91k op/s 2024-04-20T10:43:38.541 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/scattered.o 2024-04-20T10:43:38.557 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/init_64.o 2024-04-20T10:43:38.558 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/shash.o 2024-04-20T10:43:38.563 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/workingset.o 2024-04-20T10:43:38.567 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/pm.o 2024-04-20T10:43:38.578 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/nlmsgtab.o 2024-04-20T10:43:38.762 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/rcu/sync.o 2024-04-20T10:43:39.072 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/topology.o 2024-04-20T10:43:39.184 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/rcu/srcutree.o 2024-04-20T10:43:39.196 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/proc_net.o 2024-04-20T10:43:39.432 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/msi.o 2024-04-20T10:43:39.458 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/debug.o 2024-04-20T10:43:39.614 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/akcipher.o 2024-04-20T10:43:39.626 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/common.o 2024-04-20T10:43:39.654 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/netif.o 2024-04-20T10:43:40.008 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/fault.o 2024-04-20T10:43:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:39 smithi162 ceph-mon[21854]: pgmap v298: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.0 MiB/s wr, 3.94k op/s 2024-04-20T10:43:40.217 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/kcore.o 2024-04-20T10:43:40.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:39 smithi046 ceph-mon[21574]: pgmap v298: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.0 MiB/s wr, 3.94k op/s 2024-04-20T10:43:40.343 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/rcu/tree.o 2024-04-20T10:43:40.351 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/gup.o 2024-04-20T10:43:40.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:39 smithi042 ceph-mon[17078]: pgmap v298: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.0 MiB/s wr, 3.94k op/s 2024-04-20T10:43:40.563 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/sig.o 2024-04-20T10:43:40.880 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/affinity.o 2024-04-20T10:43:40.958 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/netnode.o 2024-04-20T10:43:41.089 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/rdrand.o 2024-04-20T10:43:41.174 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/vmcore.o 2024-04-20T10:43:41.338 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/kpp.o 2024-04-20T10:43:41.339 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/match.o 2024-04-20T10:43:41.450 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/ioremap.o 2024-04-20T10:43:41.480 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq/matrix.o 2024-04-20T10:43:41.901 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/bugs.o 2024-04-20T10:43:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:41 smithi162 ceph-mon[21854]: pgmap v299: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 257 KiB/s wr, 2.93k op/s 2024-04-20T10:43:42.217 INFO:tasks.workunit.client.0.smithi042.stdout: ASN.1 crypto/rsapubkey.asn1.[ch] 2024-04-20T10:43:42.220 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/netport.o 2024-04-20T10:43:42.222 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/kmsg.o 2024-04-20T10:43:42.230 INFO:tasks.workunit.client.0.smithi042.stdout: ASN.1 crypto/rsaprivkey.asn1.[ch] 2024-04-20T10:43:42.244 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/rsa.o 2024-04-20T10:43:42.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:41 smithi046 ceph-mon[21574]: pgmap v299: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 257 KiB/s wr, 2.93k op/s 2024-04-20T10:43:42.352 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:41 smithi042 ceph-mon[17078]: pgmap v299: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 257 KiB/s wr, 2.93k op/s 2024-04-20T10:43:42.359 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/mmap_lock.o 2024-04-20T10:43:42.465 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/extable.o 2024-04-20T10:43:42.710 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/irq/built-in.a 2024-04-20T10:43:42.820 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/rsa_helper.o 2024-04-20T10:43:42.849 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc/page.o 2024-04-20T10:43:42.990 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/integrity/iint.o 2024-04-20T10:43:43.033 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/rsa-pkcs1pad.o 2024-04-20T10:43:43.306 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/highmem.o 2024-04-20T10:43:43.412 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/mmap.o 2024-04-20T10:43:43.437 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/aperfmperf.o 2024-04-20T10:43:43.462 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/status.o 2024-04-20T10:43:43.738 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/integrity/integrity_audit.o 2024-04-20T10:43:43.769 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/rcu/rcu_segcblist.o 2024-04-20T10:43:43.871 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/proc/built-in.a 2024-04-20T10:43:43.922 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/acompress.o 2024-04-20T10:43:43.981 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/sysfs/file.o 2024-04-20T10:43:44.113 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/memory.o 2024-04-20T10:43:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:43 smithi162 ceph-mon[21854]: pgmap v300: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 349 KiB/s wr, 2.93k op/s 2024-04-20T10:43:44.164 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/cpuid-deps.o 2024-04-20T10:43:44.202 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/pgtable.o 2024-04-20T10:43:44.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:43 smithi046 ceph-mon[21574]: pgmap v300: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 349 KiB/s wr, 2.93k op/s 2024-04-20T10:43:44.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:43 smithi042 ceph-mon[17078]: pgmap v300: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 349 KiB/s wr, 2.93k op/s 2024-04-20T10:43:44.418 INFO:tasks.workunit.client.0.smithi042.stdout: AR security/integrity/built-in.a 2024-04-20T10:43:44.438 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/rcu/built-in.a 2024-04-20T10:43:44.494 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/sysfs/dir.o 2024-04-20T10:43:44.506 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/livepatch/built-in.a 2024-04-20T10:43:44.548 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/dma/mapping.o 2024-04-20T10:43:44.651 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/ss/ebitmap.o 2024-04-20T10:43:44.678 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/umwait.o 2024-04-20T10:43:44.814 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/scompress.o 2024-04-20T10:43:44.814 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/dma/direct.o 2024-04-20T10:43:45.145 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/sysfs/symlink.o 2024-04-20T10:43:45.200 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/physaddr.o 2024-04-20T10:43:45.495 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/cpu/proc.o 2024-04-20T10:43:45.793 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/partitions/core.o 2024-04-20T10:43:45.987 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/algboss.o 2024-04-20T10:43:46.019 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/dma/ops_helpers.o 2024-04-20T10:43:46.083 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/sysfs/mount.o 2024-04-20T10:43:46.166 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/tlb.o 2024-04-20T10:43:46.192 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/ss/hashtab.o 2024-04-20T10:43:46.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:45 smithi046 ceph-mon[21574]: pgmap v301: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 249 KiB/s wr, 1.14k op/s 2024-04-20T10:43:46.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:45 smithi042 ceph-mon[17078]: pgmap v301: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 249 KiB/s wr, 1.14k op/s 2024-04-20T10:43:46.398 INFO:tasks.workunit.client.0.smithi042.stdout: MKCAP arch/x86/kernel/cpu/capflags.c 2024-04-20T10:43:46.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:45 smithi162 ceph-mon[21854]: pgmap v301: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 249 KiB/s wr, 1.14k op/s 2024-04-20T10:43:46.754 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/ss/symtab.o 2024-04-20T10:43:46.814 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/dma/dummy.o 2024-04-20T10:43:46.844 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/partitions/msdos.o 2024-04-20T10:43:46.966 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/sysfs/group.o 2024-04-20T10:43:47.011 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/testmgr.o 2024-04-20T10:43:47.148 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/ss/sidtab.o 2024-04-20T10:43:47.240 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/cpu_entry_area.o 2024-04-20T10:43:47.402 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/mincore.o 2024-04-20T10:43:47.514 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/dma/swiotlb.o 2024-04-20T10:43:47.710 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/cmac.o 2024-04-20T10:43:47.751 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/sysfs/built-in.a 2024-04-20T10:43:47.776 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/partitions/efi.o 2024-04-20T10:43:47.995 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/devpts/inode.o 2024-04-20T10:43:48.034 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/maccess.o 2024-04-20T10:43:48.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:47 smithi046 ceph-mon[21574]: pgmap v302: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 518 KiB/s wr, 918 op/s 2024-04-20T10:43:48.326 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/hmac.o 2024-04-20T10:43:48.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:47 smithi042 ceph-mon[17078]: pgmap v302: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 518 KiB/s wr, 918 op/s 2024-04-20T10:43:48.384 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/mlock.o 2024-04-20T10:43:48.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:47 smithi162 ceph-mon[21854]: pgmap v302: 97 pgs: 97 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 518 KiB/s wr, 918 op/s 2024-04-20T10:43:48.459 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/pgprot.o 2024-04-20T10:43:48.505 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/ss/avtab.o 2024-04-20T10:43:48.799 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/dma/remap.o 2024-04-20T10:43:48.827 INFO:tasks.workunit.client.0.smithi042.stdout: AR block/partitions/built-in.a 2024-04-20T10:43:48.901 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/bdev.o 2024-04-20T10:43:48.950 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/devpts/built-in.a 2024-04-20T10:43:49.018 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/netfs/buffered_read.o 2024-04-20T10:43:49.147 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/hugetlbpage.o 2024-04-20T10:43:49.200 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/crypto_null.o 2024-04-20T10:43:49.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:43:49.391 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:43:49.552 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/dma/built-in.a 2024-04-20T10:43:49.630 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/entry/common.o 2024-04-20T10:43:49.749 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/mmap.o 2024-04-20T10:43:49.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.780+0000 7f0c23800640 1 -- 172.21.15.42:0/3903402009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c109a90 msgr2=0x7f0c1c114b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:49.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.780+0000 7f0c23800640 1 --2- 172.21.15.42:0/3903402009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c109a90 0x7f0c1c114b70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009900 tx=0x7f0c0802bc30 comp rx=0 tx=0).stop 2024-04-20T10:43:49.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.782+0000 7f0c23800640 1 -- 172.21.15.42:0/3903402009 shutdown_connections 2024-04-20T10:43:49.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.782+0000 7f0c23800640 1 --2- 172.21.15.42:0/3903402009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c1150b0 0x7f0c1c1174a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.782+0000 7f0c23800640 1 --2- 172.21.15.42:0/3903402009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c109a90 0x7f0c1c114b70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.782+0000 7f0c23800640 1 --2- 172.21.15.42:0/3903402009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1c109170 0x7f0c1c109550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.782+0000 7f0c23800640 1 -- 172.21.15.42:0/3903402009 >> 172.21.15.42:0/3903402009 conn(0x7f0c1c06d070 msgr2=0x7f0c1c06d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:49.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.783+0000 7f0c23800640 1 -- 172.21.15.42:0/3903402009 shutdown_connections 2024-04-20T10:43:49.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.783+0000 7f0c23800640 1 -- 172.21.15.42:0/3903402009 wait complete. 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.785+0000 7f0c23800640 1 Processor -- start 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.785+0000 7f0c23800640 1 -- start start 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.786+0000 7f0c23800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1c109170 0x7f0c1c114580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.786+0000 7f0c23800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c109a90 0x7f0c1c114ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.786+0000 7f0c23800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c1150b0 0x7f0c1c10b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.786+0000 7f0c20e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1c109170 0x7f0c1c114580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.786+0000 7f0c20e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1c109170 0x7f0c1c114580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52856/0 (socket says 172.21.15.42:52856) 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.787+0000 7f0c1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c109a90 0x7f0c1c114ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.787+0000 7f0c1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c109a90 0x7f0c1c114ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:60036/0 (socket says 172.21.15.42:60036) 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.787+0000 7f0c20e00640 1 -- 172.21.15.42:0/2862821526 learned_addr learned my addr 172.21.15.42:0/2862821526 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.787+0000 7f0c23800640 1 -- 172.21.15.42:0/2862821526 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c11abd0 con 0x7f0c1c1150b0 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.787+0000 7f0c23800640 1 -- 172.21.15.42:0/2862821526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c11aa50 con 0x7f0c1c109a90 2024-04-20T10:43:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.787+0000 7f0c23800640 1 -- 172.21.15.42:0/2862821526 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c11ad50 con 0x7f0c1c109170 2024-04-20T10:43:49.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.787+0000 7f0c21800640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c1150b0 0x7f0c1c10b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.787+0000 7f0c1be00640 1 -- 172.21.15.42:0/2862821526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1c109170 msgr2=0x7f0c1c114580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:49.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.787+0000 7f0c1be00640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1c109170 0x7f0c1c114580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.787+0000 7f0c1be00640 1 -- 172.21.15.42:0/2862821526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c1150b0 msgr2=0x7f0c1c10b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:49.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.787+0000 7f0c1be00640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c1150b0 0x7f0c1c10b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.787+0000 7f0c1be00640 1 -- 172.21.15.42:0/2862821526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c10bf10 con 0x7f0c1c109a90 2024-04-20T10:43:49.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.788+0000 7f0c20e00640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1c109170 0x7f0c1c114580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:43:49.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.788+0000 7f0c1be00640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c109a90 0x7f0c1c114ac0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009850 tx=0x7f0c080042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:49.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.790+0000 7f0c19600640 1 -- 172.21.15.42:0/2862821526 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08043020 con 0x7f0c1c109a90 2024-04-20T10:43:49.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.790+0000 7f0c19600640 1 -- 172.21.15.42:0/2862821526 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c080044d0 con 0x7f0c1c109a90 2024-04-20T10:43:49.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.790+0000 7f0c19600640 1 -- 172.21.15.42:0/2862821526 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08003bb0 con 0x7f0c1c109a90 2024-04-20T10:43:49.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.790+0000 7f0c23800640 1 -- 172.21.15.42:0/2862821526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1c10c1a0 con 0x7f0c1c109a90 2024-04-20T10:43:49.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.790+0000 7f0c23800640 1 -- 172.21.15.42:0/2862821526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1c1101a0 con 0x7f0c1c109a90 2024-04-20T10:43:49.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.796+0000 7f0c26628740 1 Processor -- start 2024-04-20T10:43:49.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.796+0000 7f0c26628740 1 -- start start 2024-04-20T10:43:49.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.799+0000 7f0c19600640 1 -- 172.21.15.42:0/2862821526 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0c08046020 con 0x7f0c1c109a90 2024-04-20T10:43:49.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.801+0000 7f0c26628740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7f1a0 0x55842bc7f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.801+0000 7f0c26628740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc7fac0 0x55842bc88160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.801+0000 7f0c26628740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc8aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.802+0000 7f0c26628740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55842bbe5150 con 0x55842bc7fac0 2024-04-20T10:43:49.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.802+0000 7f0c26628740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55842bb8b370 con 0x55842bc886a0 2024-04-20T10:43:49.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.802+0000 7f0c26628740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55842bbe1110 con 0x55842bc7f1a0 2024-04-20T10:43:49.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.803+0000 7f0c19600640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0c040a3f20 0x7f0c040a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.803+0000 7f0c19600640 1 -- 172.21.15.42:0/2862821526 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f0c0804e060 con 0x7f0c1c109a90 2024-04-20T10:43:49.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.804+0000 7f0c21800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7f1a0 0x55842bc7f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.804+0000 7f0c21800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7f1a0 0x55842bc7f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52864/0 (socket says 172.21.15.42:52864) 2024-04-20T10:43:49.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.804+0000 7f0c21800640 1 -- 172.21.15.42:0/3578073409 learned_addr learned my addr 172.21.15.42:0/3578073409 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:49.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.804+0000 7f0c21800640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0c040a3f20 0x7f0c040a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.805+0000 7f0c20e00640 1 --2- 172.21.15.42:0/3578073409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc7fac0 0x55842bc88160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.805+0000 7f0c21800640 1 -- 172.21.15.42:0/3578073409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 msgr2=0x55842bc8aa90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:43:49.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.805+0000 7f0c21800640 1 --2- 172.21.15.42:0/3578073409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc8aa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.805+0000 7f0c21800640 1 -- 172.21.15.42:0/3578073409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc7fac0 msgr2=0x55842bc88160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:49.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.805+0000 7f0c21800640 1 --2- 172.21.15.42:0/3578073409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc7fac0 0x55842bc88160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.805+0000 7f0c21800640 1 -- 172.21.15.42:0/3578073409 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55842b9c17d0 con 0x55842bc7f1a0 2024-04-20T10:43:49.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.805+0000 7f0c21800640 1 --2- 172.21.15.42:0/3578073409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7f1a0 0x55842bc7f580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0c14013dc0 tx=0x7f0c1400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd30993f4e7b3e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:49.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.806+0000 7f0c21800640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0c040a3f20 0x7f0c040a63e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400fc90 tx=0x7f0c1401e000 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:49.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.806+0000 7f0c0e000640 1 -- 172.21.15.42:0/3578073409 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140138a0 con 0x55842bc7f1a0 2024-04-20T10:43:49.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.806+0000 7f0c0e000640 1 -- 172.21.15.42:0/3578073409 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c14013170 con 0x55842bc7f1a0 2024-04-20T10:43:49.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.807+0000 7f0c26628740 1 -- 172.21.15.42:0/3578073409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7f1a0 msgr2=0x55842bc7f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:49.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.807+0000 7f0c26628740 1 --2- 172.21.15.42:0/3578073409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7f1a0 0x55842bc7f580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0c14013dc0 tx=0x7f0c1400f790 comp rx=0 tx=0).stop 2024-04-20T10:43:49.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.808+0000 7f0c26628740 1 -- 172.21.15.42:0/3578073409 shutdown_connections 2024-04-20T10:43:49.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.808+0000 7f0c26628740 1 --2- 172.21.15.42:0/3578073409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc8aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.808+0000 7f0c26628740 1 --2- 172.21.15.42:0/3578073409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc7fac0 0x55842bc88160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.808+0000 7f0c26628740 1 --2- 172.21.15.42:0/3578073409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7f1a0 0x55842bc7f580 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.808+0000 7f0c26628740 1 -- 172.21.15.42:0/3578073409 >> 172.21.15.42:0/3578073409 conn(0x55842bc798d0 msgr2=0x55842bc79cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:49.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.808+0000 7f0c26628740 1 -- 172.21.15.42:0/3578073409 shutdown_connections 2024-04-20T10:43:49.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.810+0000 7f0c26628740 1 -- 172.21.15.42:0/3578073409 wait complete. 2024-04-20T10:43:49.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.810+0000 7f0c26628740 1 -- start start 2024-04-20T10:43:49.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.811+0000 7f0c26628740 1 Processor -- start 2024-04-20T10:43:49.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.812+0000 7f0c26628740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc7fac0 0x55842bc880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.812+0000 7f0c26628740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc85390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.813+0000 7f0c26628740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 0x55842bc85d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.813+0000 7f0c26628740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55842bc8b4d0 con 0x55842bc7fac0 2024-04-20T10:43:49.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.813+0000 7f0c26628740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55842bb9e740 con 0x55842bc886a0 2024-04-20T10:43:49.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.813+0000 7f0c26628740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55842bc8b650 con 0x55842bc85920 2024-04-20T10:43:49.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.813+0000 7f0c21800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 0x55842bc85d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.813+0000 7f0c21800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 0x55842bc85d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52870/0 (socket says 172.21.15.42:52870) 2024-04-20T10:43:49.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.813+0000 7f0c21800640 1 -- 172.21.15.42:0/4247005003 learned_addr learned my addr 172.21.15.42:0/4247005003 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:49.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.813+0000 7f0c20e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc7fac0 0x55842bc880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.813+0000 7f0c26628740 10 client.0 _ll_register_callbacks cb 0x55842bb99b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:49.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.814+0000 7f0c21800640 1 -- 172.21.15.42:0/4247005003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 msgr2=0x55842bc85390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:49.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.814+0000 7f0c1be00640 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc85390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.814+0000 7f0c21800640 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc85390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.814+0000 7f0c21800640 1 -- 172.21.15.42:0/4247005003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc7fac0 msgr2=0x55842bc880a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:49.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.814+0000 7f0c21800640 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc7fac0 0x55842bc880a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.814+0000 7f0c21800640 1 -- 172.21.15.42:0/4247005003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55842b8f73f0 con 0x55842bc85920 2024-04-20T10:43:49.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.814+0000 7f0c20e00640 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc7fac0 0x55842bc880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:43:49.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.815+0000 7f0c1be00640 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc85390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:49.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.815+0000 7f0c21800640 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 0x55842bc85d80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0c14014010 tx=0x7f0c14009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76858c5f997133ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:49.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.815+0000 7f0c02a00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:43:49.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.817+0000 7f0c02a00640 1 -- 172.21.15.42:0/4247005003 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14013270 con 0x55842bc85920 2024-04-20T10:43:49.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.817+0000 7f0c02a00640 1 -- 172.21.15.42:0/4247005003 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f0c1400ee20 con 0x55842bc85920 2024-04-20T10:43:49.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.817+0000 7f0c26628740 1 -- 172.21.15.42:0/4247005003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55842bbf2b70 con 0x55842bc85920 2024-04-20T10:43:49.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.817+0000 7f0c02a00640 1 -- 172.21.15.42:0/4247005003 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0c14005720 con 0x55842bc85920 2024-04-20T10:43:49.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.818+0000 7f0c26628740 10 client.26497 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:49.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.818+0000 7f0c26628740 1 -- 172.21.15.42:0/4247005003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55842b8b70f0 con 0x55842bc85920 2024-04-20T10:43:49.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.818+0000 7f0c02a00640 1 -- 172.21.15.42:0/4247005003 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55842bbf2b70 con 0x55842bc85920 2024-04-20T10:43:49.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.818+0000 7f0c02a00640 1 -- 172.21.15.42:0/4247005003 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f0c14036020 con 0x55842bc85920 2024-04-20T10:43:49.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.819+0000 7f0c26628740 10 client.26497 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:49.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.819+0000 7f0c26628740 10 client.26497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:49.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.819+0000 7f0c26628740 20 client.26497 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:49.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.819+0000 7f0c26628740 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55842bc98c50 0x55842bc9b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.819+0000 7f0c26628740 4 client.26497 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:43:49.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.819+0000 7f0c26628740 1 -- 172.21.15.42:0/4247005003 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55842bbec9e0 con 0x55842bc98c50 2024-04-20T10:43:49.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.819+0000 7f0c20e00640 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55842bc98c50 0x55842bc9b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.820+0000 7f0c20e00640 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55842bc98c50 0x55842bc9b040 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7aa1de6847c514d server_cookie=5e9239aaa9c89409 in_seq=0 out_seq=0 2024-04-20T10:43:49.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.821+0000 7f0c02a00640 10 client.26497 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.822+0000 7f0c02a00640 1 -- 172.21.15.42:0/4247005003 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55842bbec9e0 con 0x55842bc98c50 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.822+0000 7f0c02a00640 10 client.26497 handle_command_reply: tid=0 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.822+0000 7f0c26628740 1 client.26497 shutdown 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.822+0000 7f0c26628740 1 -- 172.21.15.42:0/4247005003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 msgr2=0x55842bc85d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.822+0000 7f0c26628740 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 0x55842bc85d80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0c14014010 tx=0x7f0c14009fa0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.823+0000 7f0c26628740 1 -- 172.21.15.42:0/4247005003 shutdown_connections 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.823+0000 7f0c26628740 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55842bc98c50 0x55842bc9b040 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.823+0000 7f0c26628740 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 0x55842bc85d80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.823+0000 7f0c26628740 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc85390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.823+0000 7f0c02a00640 0 client.26497 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.823+0000 7f0c26628740 1 --2- 172.21.15.42:0/4247005003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc7fac0 0x55842bc880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.823+0000 7f0c26628740 1 -- 172.21.15.42:0/4247005003 >> 172.21.15.42:0/4247005003 conn(0x55842bc798d0 msgr2=0x55842bc7ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:49.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.823+0000 7f0c26628740 1 -- 172.21.15.42:0/4247005003 shutdown_connections 2024-04-20T10:43:49.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.823+0000 7f0c26628740 1 -- 172.21.15.42:0/4247005003 wait complete. 2024-04-20T10:43:49.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.823+0000 7f0c26628740 20 client.26497 trim_cache size 0 max 16384 2024-04-20T10:43:49.857 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/ss/policydb.o 2024-04-20T10:43:49.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.867+0000 7f0c26628740 1 Processor -- start 2024-04-20T10:43:49.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.869+0000 7f0c26628740 1 -- start start 2024-04-20T10:43:49.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.870+0000 7f0c26628740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc7fac0 0x55842bc842d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.871+0000 7f0c26628740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 0x55842bc7b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.871+0000 7f0c26628740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc886a0 0x55842bc87bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.871+0000 7f0c26628740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc00c8b0 con 0x55842bc886a0 2024-04-20T10:43:49.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.871+0000 7f0c26628740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc00f110 con 0x55842bc7fac0 2024-04-20T10:43:49.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.871+0000 7f0c26628740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc00cf00 con 0x55842bc85920 2024-04-20T10:43:49.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.871+0000 7f0c20e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc7fac0 0x55842bc842d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.871+0000 7f0c20e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc7fac0 0x55842bc842d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:60054/0 (socket says 172.21.15.42:60054) 2024-04-20T10:43:49.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.871+0000 7f0c20e00640 1 -- 172.21.15.42:0/1263727975 learned_addr learned my addr 172.21.15.42:0/1263727975 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:49.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.871+0000 7f0c1be00640 1 --2- 172.21.15.42:0/1263727975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 0x55842bc7b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.872+0000 7f0c1be00640 1 -- 172.21.15.42:0/1263727975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc7fac0 msgr2=0x55842bc842d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:49.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.872+0000 7f0c1be00640 1 --2- 172.21.15.42:0/1263727975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc7fac0 0x55842bc842d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.872+0000 7f0c1be00640 1 -- 172.21.15.42:0/1263727975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc886a0 msgr2=0x55842bc87bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:49.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.872+0000 7f0c21800640 1 --2- 172.21.15.42:0/1263727975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc886a0 0x55842bc87bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.872+0000 7f0c1be00640 1 --2- 172.21.15.42:0/1263727975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc886a0 0x55842bc87bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.872+0000 7f0c1be00640 1 -- 172.21.15.42:0/1263727975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55842bc8b6d0 con 0x55842bc85920 2024-04-20T10:43:49.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.872+0000 7f0c21800640 1 --2- 172.21.15.42:0/1263727975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc886a0 0x55842bc87bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:49.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.872+0000 7f0c1be00640 1 --2- 172.21.15.42:0/1263727975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 0x55842bc7b430 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0c080ef470 tx=0x7f0c080bdb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1545c3185900460e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:49.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.873+0000 7f0c0d600640 1 -- 172.21.15.42:0/1263727975 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08004c80 con 0x55842bc85920 2024-04-20T10:43:49.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.873+0000 7f0c0d600640 1 -- 172.21.15.42:0/1263727975 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c0804d020 con 0x55842bc85920 2024-04-20T10:43:49.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.873+0000 7f0c0d600640 1 -- 172.21.15.42:0/1263727975 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08004c80 con 0x55842bc85920 2024-04-20T10:43:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.875+0000 7f0c26628740 1 -- 172.21.15.42:0/1263727975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 msgr2=0x55842bc7b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.875+0000 7f0c26628740 1 --2- 172.21.15.42:0/1263727975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 0x55842bc7b430 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0c080ef470 tx=0x7f0c080bdb50 comp rx=0 tx=0).stop 2024-04-20T10:43:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.875+0000 7f0c26628740 1 -- 172.21.15.42:0/1263727975 shutdown_connections 2024-04-20T10:43:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.875+0000 7f0c26628740 1 --2- 172.21.15.42:0/1263727975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc886a0 0x55842bc87bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.875+0000 7f0c26628740 1 --2- 172.21.15.42:0/1263727975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc85920 0x55842bc7b430 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.875+0000 7f0c26628740 1 --2- 172.21.15.42:0/1263727975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc7fac0 0x55842bc842d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.875+0000 7f0c26628740 1 -- 172.21.15.42:0/1263727975 >> 172.21.15.42:0/1263727975 conn(0x55842bc798d0 msgr2=0x55842bc7ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.876+0000 7f0c26628740 1 -- 172.21.15.42:0/1263727975 shutdown_connections 2024-04-20T10:43:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.876+0000 7f0c26628740 1 -- 172.21.15.42:0/1263727975 wait complete. 2024-04-20T10:43:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.876+0000 7f0c26628740 1 -- start start 2024-04-20T10:43:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.877+0000 7f0c26628740 1 Processor -- start 2024-04-20T10:43:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.877+0000 7f0c26628740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7fac0 0x55842bc895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.877+0000 7f0c26628740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc85920 0x55842bc89af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.878+0000 7f0c26628740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc8a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.878+0000 7f0c26628740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc00d5d0 con 0x55842bc85920 2024-04-20T10:43:49.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.878+0000 7f0c26628740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc00ea40 con 0x55842bc886a0 2024-04-20T10:43:49.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.878+0000 7f0c26628740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc00dca0 con 0x55842bc7fac0 2024-04-20T10:43:49.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.878+0000 7f0c26628740 10 client.0 _ll_register_callbacks cb 0x7f0c14013b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:49.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.878+0000 7f0c21800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc8a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.878+0000 7f0c20e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7fac0 0x55842bc895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.878+0000 7f0c21800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc8a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:60062/0 (socket says 172.21.15.42:60062) 2024-04-20T10:43:49.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.878+0000 7f0c21800640 1 -- 172.21.15.42:0/345097775 learned_addr learned my addr 172.21.15.42:0/345097775 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:49.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.878+0000 7f0c1be00640 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc85920 0x55842bc89af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.879+0000 7f0c21800640 1 -- 172.21.15.42:0/345097775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7fac0 msgr2=0x55842bc895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:49.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.879+0000 7f0c21800640 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7fac0 0x55842bc895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.880+0000 7f0c21800640 1 -- 172.21.15.42:0/345097775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc85920 msgr2=0x55842bc89af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:49.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.880+0000 7f0c21800640 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc85920 0x55842bc89af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.880+0000 7f0c21800640 1 -- 172.21.15.42:0/345097775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55842bc8ef60 con 0x55842bc886a0 2024-04-20T10:43:49.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.880+0000 7f0c20e00640 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7fac0 0x55842bc895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:43:49.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.880+0000 7f0c1be00640 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc85920 0x55842bc89af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:49.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.880+0000 7f0c21800640 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc8a030 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c14013dc0 tx=0x7f0c14009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e20b0198f91ad14 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:49.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.882+0000 7f0c02a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:43:49.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.882+0000 7f0c02a00640 1 -- 172.21.15.42:0/345097775 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1403d060 con 0x55842bc886a0 2024-04-20T10:43:49.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.882+0000 7f0c02a00640 1 -- 172.21.15.42:0/345097775 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f0c1401c040 con 0x55842bc886a0 2024-04-20T10:43:49.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.882+0000 7f0c26628740 1 -- 172.21.15.42:0/345097775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c0804d020 con 0x55842bc886a0 2024-04-20T10:43:49.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.883+0000 7f0c02a00640 1 -- 172.21.15.42:0/345097775 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0c14038460 con 0x55842bc886a0 2024-04-20T10:43:49.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.883+0000 7f0c26628740 10 client.26483 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:49.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.883+0000 7f0c26628740 1 -- 172.21.15.42:0/345097775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55842bcff340 con 0x55842bc886a0 2024-04-20T10:43:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.883+0000 7f0c02a00640 1 -- 172.21.15.42:0/345097775 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c0804d020 con 0x55842bc886a0 2024-04-20T10:43:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.884+0000 7f0c02a00640 1 -- 172.21.15.42:0/345097775 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f0c14036040 con 0x55842bc886a0 2024-04-20T10:43:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.884+0000 7f0c26628740 10 client.26483 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:49.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.884+0000 7f0c26628740 10 client.26483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:49.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.884+0000 7f0c26628740 20 client.26483 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:49.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.884+0000 7f0c26628740 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55842bcffa20 0x55842bd01e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:49.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.884+0000 7f0c26628740 4 client.26483 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:43:49.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.884+0000 7f0c26628740 1 -- 172.21.15.42:0/345097775 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55842bb68450 con 0x55842bcffa20 2024-04-20T10:43:49.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.884+0000 7f0c20e00640 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55842bcffa20 0x55842bd01e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:49.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.886+0000 7f0c20e00640 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55842bcffa20 0x55842bd01e40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9e97d12d8ae5ba server_cookie=3dfb5f0b8c4d6059 in_seq=0 out_seq=0 2024-04-20T10:43:49.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.886+0000 7f0c02a00640 10 client.26483 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:49.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.889+0000 7f0c02a00640 1 -- 172.21.15.42:0/345097775 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55842bb68450 con 0x55842bcffa20 2024-04-20T10:43:49.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.889+0000 7f0c02a00640 10 client.26483 handle_command_reply: tid=0 2024-04-20T10:43:49.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.889+0000 7f0c26628740 1 client.26483 shutdown 2024-04-20T10:43:49.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.891+0000 7f0c26628740 1 -- 172.21.15.42:0/345097775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 msgr2=0x55842bc8a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:49.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.891+0000 7f0c26628740 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc8a030 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c14013dc0 tx=0x7f0c14009f00 comp rx=0 tx=0).stop 2024-04-20T10:43:49.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.894+0000 7f0c26628740 1 -- 172.21.15.42:0/345097775 shutdown_connections 2024-04-20T10:43:49.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.894+0000 7f0c26628740 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55842bcffa20 0x55842bd01e40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.894+0000 7f0c26628740 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55842bc886a0 0x55842bc8a030 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.894+0000 7f0c26628740 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55842bc85920 0x55842bc89af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.894+0000 7f0c26628740 1 --2- 172.21.15.42:0/345097775 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55842bc7fac0 0x55842bc895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.894+0000 7f0c02a00640 0 client.26483 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:49.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.894+0000 7f0c26628740 1 -- 172.21.15.42:0/345097775 >> 172.21.15.42:0/345097775 conn(0x55842bc798d0 msgr2=0x55842bc78030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:49.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.894+0000 7f0c26628740 1 -- 172.21.15.42:0/345097775 shutdown_connections 2024-04-20T10:43:49.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.894+0000 7f0c26628740 1 -- 172.21.15.42:0/345097775 wait complete. 2024-04-20T10:43:49.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.895+0000 7f0c26628740 20 client.26483 trim_cache size 0 max 16384 2024-04-20T10:43:49.899 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:43:49.899 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:43:49.899 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:43:49.899 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:43:49.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.903+0000 7f0c23800640 1 -- 172.21.15.42:0/2862821526 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0c040a3f20 msgr2=0x7f0c040a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:49.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.903+0000 7f0c23800640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0c040a3f20 0x7f0c040a63e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400fc90 tx=0x7f0c1401e000 comp rx=0 tx=0).stop 2024-04-20T10:43:49.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.903+0000 7f0c23800640 1 -- 172.21.15.42:0/2862821526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c109a90 msgr2=0x7f0c1c114ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:49.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.903+0000 7f0c23800640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c109a90 0x7f0c1c114ac0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009850 tx=0x7f0c080042e0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.904+0000 7f0c23800640 1 -- 172.21.15.42:0/2862821526 shutdown_connections 2024-04-20T10:43:49.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.904+0000 7f0c23800640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0c040a3f20 0x7f0c040a63e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.904+0000 7f0c23800640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c1c1150b0 0x7f0c1c10b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.904+0000 7f0c23800640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c109a90 0x7f0c1c114ac0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.904+0000 7f0c23800640 1 --2- 172.21.15.42:0/2862821526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1c109170 0x7f0c1c114580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:49.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.904+0000 7f0c23800640 1 -- 172.21.15.42:0/2862821526 >> 172.21.15.42:0/2862821526 conn(0x7f0c1c06d070 msgr2=0x7f0c1c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:49.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.905+0000 7f0c23800640 1 -- 172.21.15.42:0/2862821526 shutdown_connections 2024-04-20T10:43:49.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:49.905+0000 7f0c23800640 1 -- 172.21.15.42:0/2862821526 wait complete. 2024-04-20T10:43:49.937 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:43:49.938 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:43:49.985 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/md5.o 2024-04-20T10:43:50.071 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/netfs/io.o 2024-04-20T10:43:50.094 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/dump_pagetables.o 2024-04-20T10:43:50.096 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/fops.o 2024-04-20T10:43:50.202 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:49 smithi162 ceph-mon[21854]: pgmap v303: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.4 MiB/s wr, 571 op/s 2024-04-20T10:43:50.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.270+0000 7f4ba8e00640 1 -- 172.21.15.42:0/381027022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba410eb50 msgr2=0x7f4ba411d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:50.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.270+0000 7f4ba8e00640 1 --2- 172.21.15.42:0/381027022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba410eb50 0x7f4ba411d750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c009ca0 tx=0x7f4b9c02c190 comp rx=0 tx=0).stop 2024-04-20T10:43:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.270+0000 7f4ba8e00640 1 -- 172.21.15.42:0/381027022 shutdown_connections 2024-04-20T10:43:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.270+0000 7f4ba8e00640 1 --2- 172.21.15.42:0/381027022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba410eb50 0x7f4ba411d750 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.270+0000 7f4ba8e00640 1 --2- 172.21.15.42:0/381027022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba410e0d0 0x7f4ba410e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.270+0000 7f4ba8e00640 1 --2- 172.21.15.42:0/381027022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba4110560 0x7f4ba4110940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.270+0000 7f4ba8e00640 1 -- 172.21.15.42:0/381027022 >> 172.21.15.42:0/381027022 conn(0x7f4ba406ce90 msgr2=0x7f4ba406d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.271+0000 7f4ba8e00640 1 -- 172.21.15.42:0/381027022 shutdown_connections 2024-04-20T10:43:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.271+0000 7f4ba8e00640 1 -- 172.21.15.42:0/381027022 wait complete. 2024-04-20T10:43:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.272+0000 7f4ba8e00640 1 Processor -- start 2024-04-20T10:43:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.272+0000 7f4ba8e00640 1 -- start start 2024-04-20T10:43:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.273+0000 7f4ba8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba410e0d0 0x7f4ba41a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.273+0000 7f4ba8e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba410eb50 0x7f4ba41a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.273+0000 7f4ba8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba4110560 0x7f4ba41abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.273+0000 7f4ba8e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba4120f70 con 0x7f4ba410e0d0 2024-04-20T10:43:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.273+0000 7f4ba8e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba4120df0 con 0x7f4ba4110560 2024-04-20T10:43:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.273+0000 7f4ba8e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba41210f0 con 0x7f4ba410eb50 2024-04-20T10:43:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.273+0000 7f4ba1400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba410eb50 0x7f4ba41a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.273+0000 7f4ba1400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba410eb50 0x7f4ba41a78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52906/0 (socket says 172.21.15.42:52906) 2024-04-20T10:43:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.273+0000 7f4ba1400640 1 -- 172.21.15.42:0/486382803 learned_addr learned my addr 172.21.15.42:0/486382803 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.273+0000 7f4ba2800640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba4110560 0x7f4ba41abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.274+0000 7f4ba1e00640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba410e0d0 0x7f4ba41a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.275+0000 7f4ba2800640 1 -- 172.21.15.42:0/486382803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba410eb50 msgr2=0x7f4ba41a78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:50.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.275+0000 7f4ba2800640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba410eb50 0x7f4ba41a78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:50.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.275+0000 7f4ba2800640 1 -- 172.21.15.42:0/486382803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba410e0d0 msgr2=0x7f4ba41a73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:50.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.275+0000 7f4ba2800640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba410e0d0 0x7f4ba41a73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:50.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.276+0000 7f4ba2800640 1 -- 172.21.15.42:0/486382803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba41ac350 con 0x7f4ba4110560 2024-04-20T10:43:50.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.276+0000 7f4ba1e00640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba410e0d0 0x7f4ba41a73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:50.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.276+0000 7f4ba2800640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba4110560 0x7f4ba41abc20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c02c6d0 tx=0x7f4b9c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:50.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.277+0000 7f4b9aa00640 1 -- 172.21.15.42:0/486382803 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c039050 con 0x7f4ba4110560 2024-04-20T10:43:50.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.277+0000 7f4b9aa00640 1 -- 172.21.15.42:0/486382803 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b9c03a040 con 0x7f4ba4110560 2024-04-20T10:43:50.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.277+0000 7f4b9aa00640 1 -- 172.21.15.42:0/486382803 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c004460 con 0x7f4ba4110560 2024-04-20T10:43:50.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.278+0000 7f4ba8e00640 1 -- 172.21.15.42:0/486382803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba41ac5e0 con 0x7f4ba4110560 2024-04-20T10:43:50.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.278+0000 7f4ba8e00640 1 -- 172.21.15.42:0/486382803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba41aca50 con 0x7f4ba4110560 2024-04-20T10:43:50.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.280+0000 7f4b8fe00640 1 -- 172.21.15.42:0/486382803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba4068ca0 con 0x7f4ba4110560 2024-04-20T10:43:50.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.285+0000 7f4b9aa00640 1 -- 172.21.15.42:0/486382803 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4b9c047020 con 0x7f4ba4110560 2024-04-20T10:43:50.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.287+0000 7f4b9aa00640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4b880a3e20 0x7f4b880a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:50.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.287+0000 7f4b9aa00640 1 -- 172.21.15.42:0/486382803 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f4b9c0f22c0 con 0x7f4ba4110560 2024-04-20T10:43:50.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.288+0000 7f4ba1e00640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4b880a3e20 0x7f4b880a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:50.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.293+0000 7f4ba1e00640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4b880a3e20 0x7f4b880a62e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400af30 tx=0x7f4b9400a2d0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:50.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.293+0000 7f4b9aa00640 1 -- 172.21.15.42:0/486382803 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4ba41aca50 con 0x7f4ba4110560 2024-04-20T10:43:50.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:49 smithi046 ceph-mon[21574]: pgmap v303: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.4 MiB/s wr, 571 op/s 2024-04-20T10:43:50.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:49 smithi042 ceph-mon[17078]: pgmap v303: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.4 MiB/s wr, 571 op/s 2024-04-20T10:43:50.756 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/sha256_generic.o 2024-04-20T10:43:50.819 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/entry/syscall_user_dispatch.o 2024-04-20T10:43:50.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.867+0000 7f4b8fe00640 1 -- 172.21.15.42:0/486382803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ba4062b60 con 0x7f4ba4110560 2024-04-20T10:43:50.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.868+0000 7f4b9aa00640 1 -- 172.21.15.42:0/486382803 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f4b9c0bea70 con 0x7f4ba4110560 2024-04-20T10:43:50.869 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:43:50.870 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:43:50.870 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:43:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.874+0000 7f4b8fe00640 1 -- 172.21.15.42:0/486382803 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4b880a3e20 msgr2=0x7f4b880a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.874+0000 7f4b8fe00640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4b880a3e20 0x7f4b880a62e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400af30 tx=0x7f4b9400a2d0 comp rx=0 tx=0).stop 2024-04-20T10:43:50.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.874+0000 7f4b8fe00640 1 -- 172.21.15.42:0/486382803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba4110560 msgr2=0x7f4ba41abc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:50.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.874+0000 7f4b8fe00640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba4110560 0x7f4ba41abc20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c02c6d0 tx=0x7f4b9c003ba0 comp rx=0 tx=0).stop 2024-04-20T10:43:50.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.874+0000 7f4b8fe00640 1 -- 172.21.15.42:0/486382803 shutdown_connections 2024-04-20T10:43:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.874+0000 7f4b8fe00640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4b880a3e20 0x7f4b880a62e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.874+0000 7f4b8fe00640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba4110560 0x7f4ba41abc20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.874+0000 7f4b8fe00640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ba410eb50 0x7f4ba41a78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.874+0000 7f4b8fe00640 1 --2- 172.21.15.42:0/486382803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba410e0d0 0x7f4ba41a73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:50.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.874+0000 7f4b8fe00640 1 -- 172.21.15.42:0/486382803 >> 172.21.15.42:0/486382803 conn(0x7f4ba406ce90 msgr2=0x7f4ba4071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:50.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.878+0000 7f4b8fe00640 1 -- 172.21.15.42:0/486382803 shutdown_connections 2024-04-20T10:43:50.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:50.878+0000 7f4b8fe00640 1 -- 172.21.15.42:0/486382803 wait complete. 2024-04-20T10:43:50.919 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:43:51.139 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/numa.o 2024-04-20T10:43:51.209 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/netfs/iterator.o 2024-04-20T10:43:51.278 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/bio.o 2024-04-20T10:43:51.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:50 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/486382803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:43:51.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.348+0000 7f9f1da00640 1 -- 172.21.15.42:0/2124805846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f18108f60 msgr2=0x7f9f181093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:51.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.348+0000 7f9f1da00640 1 --2- 172.21.15.42:0/2124805846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f18108f60 0x7f9f181093c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009900 tx=0x7f9f0402bcf0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.348+0000 7f9f1da00640 1 -- 172.21.15.42:0/2124805846 shutdown_connections 2024-04-20T10:43:51.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.348+0000 7f9f1da00640 1 --2- 172.21.15.42:0/2124805846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f181099e0 0x7f9f1811d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.349+0000 7f9f1da00640 1 --2- 172.21.15.42:0/2124805846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f18108f60 0x7f9f181093c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.349+0000 7f9f1da00640 1 --2- 172.21.15.42:0/2124805846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f18110600 0x7f9f181109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.349+0000 7f9f1da00640 1 -- 172.21.15.42:0/2124805846 >> 172.21.15.42:0/2124805846 conn(0x7f9f1806cee0 msgr2=0x7f9f1806d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:51.352 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:50 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/486382803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:43:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.353+0000 7f9f1da00640 1 -- 172.21.15.42:0/2124805846 shutdown_connections 2024-04-20T10:43:51.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.356+0000 7f9f1da00640 1 -- 172.21.15.42:0/2124805846 wait complete. 2024-04-20T10:43:51.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.357+0000 7f9f1da00640 1 Processor -- start 2024-04-20T10:43:51.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.358+0000 7f9f1da00640 1 -- start start 2024-04-20T10:43:51.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.358+0000 7f9f1da00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f18108f60 0x7f9f1811d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.358+0000 7f9f1da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f181099e0 0x7f9f18116390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.358+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f18108f60 0x7f9f1811d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.358+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f18108f60 0x7f9f1811d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52936/0 (socket says 172.21.15.42:52936) 2024-04-20T10:43:51.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.359+0000 7f9f17e00640 1 -- 172.21.15.42:0/3343764347 learned_addr learned my addr 172.21.15.42:0/3343764347 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:51.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.359+0000 7f9f1da00640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f18110600 0x7f9f181168d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.359+0000 7f9f1da00640 1 -- 172.21.15.42:0/3343764347 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18121080 con 0x7f9f18110600 2024-04-20T10:43:51.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.359+0000 7f9f1da00640 1 -- 172.21.15.42:0/3343764347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18120f00 con 0x7f9f181099e0 2024-04-20T10:43:51.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.359+0000 7f9f1da00640 1 -- 172.21.15.42:0/3343764347 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9f18121200 con 0x7f9f18108f60 2024-04-20T10:43:51.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.359+0000 7f9f17400640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f181099e0 0x7f9f18116390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.359+0000 7f9f1d000640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f18110600 0x7f9f181168d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.360+0000 7f9f17400640 1 -- 172.21.15.42:0/3343764347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f18108f60 msgr2=0x7f9f1811d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:51.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.360+0000 7f9f17400640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f18108f60 0x7f9f1811d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.360+0000 7f9f17400640 1 -- 172.21.15.42:0/3343764347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f18110600 msgr2=0x7f9f181168d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:51.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.360+0000 7f9f17400640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f18110600 0x7f9f181168d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.360+0000 7f9f17400640 1 -- 172.21.15.42:0/3343764347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f18116e60 con 0x7f9f181099e0 2024-04-20T10:43:51.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.360+0000 7f9f17e00640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f18108f60 0x7f9f1811d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:51.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.360+0000 7f9f1d000640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f18110600 0x7f9f181168d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:43:51.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.360+0000 7f9f17400640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f181099e0 0x7f9f18116390 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009850 tx=0x7f9f040043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:51.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.361+0000 7f9f14c00640 1 -- 172.21.15.42:0/3343764347 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04042020 con 0x7f9f181099e0 2024-04-20T10:43:51.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.361+0000 7f9f14c00640 1 -- 172.21.15.42:0/3343764347 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f040054e0 con 0x7f9f181099e0 2024-04-20T10:43:51.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.361+0000 7f9f14c00640 1 -- 172.21.15.42:0/3343764347 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04003ac0 con 0x7f9f181099e0 2024-04-20T10:43:51.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.361+0000 7f9f1da00640 1 -- 172.21.15.42:0/3343764347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f181170f0 con 0x7f9f181099e0 2024-04-20T10:43:51.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.363+0000 7f9f1da00640 1 -- 172.21.15.42:0/3343764347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1811b070 con 0x7f9f181099e0 2024-04-20T10:43:51.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.368+0000 7f9f21dcb740 1 Processor -- start 2024-04-20T10:43:51.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.369+0000 7f9f21dcb740 1 -- start start 2024-04-20T10:43:51.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.371+0000 7f9f21dcb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 0x55cfb127e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.371+0000 7f9f21dcb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 0x55cfb1287130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.372+0000 7f9f21dcb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb1287670 0x55cfb1289a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.372+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 0x55cfb1287130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.372+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 0x55cfb1287130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52946/0 (socket says 172.21.15.42:52946) 2024-04-20T10:43:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.372+0000 7f9f17e00640 1 -- 172.21.15.42:0/914876076 learned_addr learned my addr 172.21.15.42:0/914876076 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:51.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.372+0000 7f9f21dcb740 1 -- 172.21.15.42:0/914876076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cfb11e3050 con 0x55cfb127e170 2024-04-20T10:43:51.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.374+0000 7f9f1d000640 1 --2- 172.21.15.42:0/914876076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 0x55cfb127e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.374+0000 7f9f14c00640 1 -- 172.21.15.42:0/3343764347 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9f04045020 con 0x7f9f181099e0 2024-04-20T10:43:51.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.374+0000 7f9f21dcb740 1 -- 172.21.15.42:0/914876076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfb1189490 con 0x55cfb1287670 2024-04-20T10:43:51.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.374+0000 7f9f21dcb740 1 -- 172.21.15.42:0/914876076 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55cfb11df010 con 0x55cfb127ea90 2024-04-20T10:43:51.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.374+0000 7f9f1d000640 1 -- 172.21.15.42:0/914876076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 msgr2=0x55cfb1287130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:51.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.374+0000 7f9f1d000640 1 --2- 172.21.15.42:0/914876076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 0x55cfb1287130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.374+0000 7f9f1d000640 1 -- 172.21.15.42:0/914876076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb1287670 msgr2=0x55cfb1289a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:43:51.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.374+0000 7f9f1d000640 1 --2- 172.21.15.42:0/914876076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb1287670 0x55cfb1289a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.374+0000 7f9f1d000640 1 -- 172.21.15.42:0/914876076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb1191070 con 0x55cfb127e170 2024-04-20T10:43:51.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.375+0000 7f9f17e00640 1 --2- 172.21.15.42:0/914876076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 0x55cfb1287130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:51.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.375+0000 7f9f1d000640 1 --2- 172.21.15.42:0/914876076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 0x55cfb127e550 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009bb0 tx=0x7f9f080117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f226db8fe19b3a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:51.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.376+0000 7f9f14c00640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9efc0a3f20 0x7f9efc0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.376+0000 7f9f14c00640 1 -- 172.21.15.42:0/3343764347 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f9f040f1bf0 con 0x7f9f181099e0 2024-04-20T10:43:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.378+0000 7f9f0d600640 1 -- 172.21.15.42:0/914876076 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08005020 con 0x55cfb127e170 2024-04-20T10:43:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.378+0000 7f9f0d600640 1 -- 172.21.15.42:0/914876076 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f080047a0 con 0x55cfb127e170 2024-04-20T10:43:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.378+0000 7f9f0d600640 1 -- 172.21.15.42:0/914876076 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f080127f0 con 0x55cfb127e170 2024-04-20T10:43:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.378+0000 7f9f17e00640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9efc0a3f20 0x7f9efc0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.380+0000 7f9f21dcb740 1 -- 172.21.15.42:0/914876076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 msgr2=0x55cfb127e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.380+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/914876076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 0x55cfb127e550 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009bb0 tx=0x7f9f080117b0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.380+0000 7f9f17e00640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9efc0a3f20 0x7f9efc0a63e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009500 tx=0x7f9f1000b040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.381+0000 7f9f21dcb740 1 -- 172.21.15.42:0/914876076 shutdown_connections 2024-04-20T10:43:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.381+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/914876076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb1287670 0x55cfb1289a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.381+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/914876076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 0x55cfb1287130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.381+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/914876076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 0x55cfb127e550 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.381+0000 7f9f21dcb740 1 -- 172.21.15.42:0/914876076 >> 172.21.15.42:0/914876076 conn(0x55cfb12788a0 msgr2=0x55cfb1278c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.382+0000 7f9f21dcb740 1 -- 172.21.15.42:0/914876076 shutdown_connections 2024-04-20T10:43:51.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.382+0000 7f9f21dcb740 1 -- 172.21.15.42:0/914876076 wait complete. 2024-04-20T10:43:51.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.382+0000 7f9f21dcb740 1 -- start start 2024-04-20T10:43:51.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.383+0000 7f9f21dcb740 1 Processor -- start 2024-04-20T10:43:51.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.384+0000 7f9f21dcb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb127e170 0x55cfb1284840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.385+0000 7f9f21dcb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 0x55cfb1284d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.385+0000 7f9f21dcb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb1287670 0x55cfb12852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.386+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb127e170 0x55cfb1284840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.386+0000 7f9f1d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb1287670 0x55cfb12852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.386+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb127e170 0x55cfb1284840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:60102/0 (socket says 172.21.15.42:60102) 2024-04-20T10:43:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.386+0000 7f9f1d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb1287670 0x55cfb12852c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58546/0 (socket says 172.21.15.42:58546) 2024-04-20T10:43:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.386+0000 7f9f17400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 0x55cfb1284d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.386+0000 7f9f17400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 0x55cfb1284d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52958/0 (socket says 172.21.15.42:52958) 2024-04-20T10:43:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.386+0000 7f9f17e00640 1 -- 172.21.15.42:0/3475604552 learned_addr learned my addr 172.21.15.42:0/3475604552 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.386+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3475604552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cfb128a460 con 0x55cfb1287670 2024-04-20T10:43:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.386+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3475604552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfb119c860 con 0x55cfb127e170 2024-04-20T10:43:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.386+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3475604552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55cfb128a5e0 con 0x55cfb127ea90 2024-04-20T10:43:51.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.387+0000 7f9f17e00640 1 -- 172.21.15.42:0/3475604552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 msgr2=0x55cfb1284d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:51.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.387+0000 7f9f17e00640 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 0x55cfb1284d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.387+0000 7f9f17e00640 1 -- 172.21.15.42:0/3475604552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb1287670 msgr2=0x55cfb12852c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:51.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.387+0000 7f9f17e00640 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb1287670 0x55cfb12852c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.387+0000 7f9f17e00640 1 -- 172.21.15.42:0/3475604552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb11a0950 con 0x55cfb127e170 2024-04-20T10:43:51.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.387+0000 7f9f17400640 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 0x55cfb1284d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:51.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.387+0000 7f9f1d000640 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb1287670 0x55cfb12852c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:51.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.387+0000 7f9f21dcb740 10 client.0 _ll_register_callbacks cb 0x55cfb116afe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:51.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.387+0000 7f9f17e00640 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb127e170 0x55cfb1284840 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9f10017670 tx=0x7f9f10004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ba1b2e47b496404 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:51.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.388+0000 7f9f02000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:43:51.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.388+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3475604552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f080047a0 con 0x55cfb127e170 2024-04-20T10:43:51.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.388+0000 7f9f02000640 1 -- 172.21.15.42:0/3475604552 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1002f050 con 0x55cfb127e170 2024-04-20T10:43:51.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.388+0000 7f9f02000640 1 -- 172.21.15.42:0/3475604552 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f9f10015820 con 0x55cfb127e170 2024-04-20T10:43:51.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.388+0000 7f9f02000640 1 -- 172.21.15.42:0/3475604552 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9f10025070 con 0x55cfb127e170 2024-04-20T10:43:51.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.389+0000 7f9f21dcb740 10 client.26495 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:51.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.389+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3475604552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=19}) -- 0x55cfb116e820 con 0x55cfb127e170 2024-04-20T10:43:51.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.392+0000 7f9f02000640 1 -- 172.21.15.42:0/3475604552 <== mon.2 v2:172.21.15.46:3300/0 4 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f9f10020070 con 0x55cfb127e170 2024-04-20T10:43:51.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.392+0000 7f9f21dcb740 10 client.26495 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:51.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.392+0000 7f9f21dcb740 10 client.26495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:51.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.392+0000 7f9f21dcb740 20 client.26495 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:51.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.393+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55cfb12973c0 0x55cfb1299810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.393+0000 7f9f21dcb740 4 client.26495 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:43:51.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.393+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3475604552 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfb11f0a70 con 0x55cfb12973c0 2024-04-20T10:43:51.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.393+0000 7f9f17400640 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55cfb12973c0 0x55cfb1299810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.394+0000 7f9f17400640 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55cfb12973c0 0x55cfb1299810 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25260dd73615ae8d server_cookie=af10c65eb05e7394 in_seq=0 out_seq=0 2024-04-20T10:43:51.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.394+0000 7f9f02000640 10 client.26495 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:51.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.396+0000 7f9f02000640 1 -- 172.21.15.42:0/3475604552 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfb11f0a70 con 0x55cfb12973c0 2024-04-20T10:43:51.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.396+0000 7f9f02000640 10 client.26495 handle_command_reply: tid=0 2024-04-20T10:43:51.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.397+0000 7f9f21dcb740 1 client.26495 shutdown 2024-04-20T10:43:51.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.397+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3475604552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb127e170 msgr2=0x55cfb1284840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:51.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.397+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb127e170 0x55cfb1284840 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9f10017670 tx=0x7f9f10004a60 comp rx=0 tx=0).stop 2024-04-20T10:43:51.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.398+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3475604552 shutdown_connections 2024-04-20T10:43:51.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.398+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55cfb12973c0 0x55cfb1299810 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.398+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb1287670 0x55cfb12852c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.398+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127ea90 0x55cfb1284d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.398+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/3475604552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb127e170 0x55cfb1284840 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.398+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3475604552 >> 172.21.15.42:0/3475604552 conn(0x55cfb12788a0 msgr2=0x55cfb127a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:51.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:50 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/486382803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:43:51.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.399+0000 7f9f02000640 0 client.26495 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:51.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.399+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3475604552 shutdown_connections 2024-04-20T10:43:51.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.399+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3475604552 wait complete. 2024-04-20T10:43:51.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.400+0000 7f9f21dcb740 20 client.26495 trim_cache size 0 max 16384 2024-04-20T10:43:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.432+0000 7f9f21dcb740 1 Processor -- start 2024-04-20T10:43:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.432+0000 7f9f21dcb740 1 -- start start 2024-04-20T10:43:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.433+0000 7f9f21dcb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 0x55cfb1286f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.433+0000 7f9f21dcb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb127ea90 0x55cfb1293350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.433+0000 7f9f21dcb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb1287670 0x55cfb129f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.433+0000 7f9f21dcb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181c3100 con 0x55cfb127e170 2024-04-20T10:43:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.433+0000 7f9f21dcb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181c5960 con 0x55cfb127ea90 2024-04-20T10:43:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.433+0000 7f9f21dcb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181c3750 con 0x55cfb1287670 2024-04-20T10:43:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.434+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 0x55cfb1286f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.434+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 0x55cfb1286f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58554/0 (socket says 172.21.15.42:58554) 2024-04-20T10:43:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.434+0000 7f9f17e00640 1 -- 172.21.15.42:0/2152646808 learned_addr learned my addr 172.21.15.42:0/2152646808 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:51.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.434+0000 7f9f17400640 1 --2- 172.21.15.42:0/2152646808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb127ea90 0x55cfb1293350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.435+0000 7f9f1d000640 1 --2- 172.21.15.42:0/2152646808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb1287670 0x55cfb129f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.435+0000 7f9f17e00640 1 -- 172.21.15.42:0/2152646808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb1287670 msgr2=0x55cfb129f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:51.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.435+0000 7f9f17e00640 1 --2- 172.21.15.42:0/2152646808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb1287670 0x55cfb129f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.435+0000 7f9f17e00640 1 -- 172.21.15.42:0/2152646808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb127ea90 msgr2=0x55cfb1293350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:51.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.435+0000 7f9f17e00640 1 --2- 172.21.15.42:0/2152646808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb127ea90 0x55cfb1293350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.435+0000 7f9f17e00640 1 -- 172.21.15.42:0/2152646808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb0ef53f0 con 0x55cfb127e170 2024-04-20T10:43:51.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.439+0000 7f9f17e00640 1 --2- 172.21.15.42:0/2152646808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 0x55cfb1286f70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9f100070a0 tx=0x7f9f10020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11ac2e8437cc819 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:51.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.440+0000 7f9f0cc00640 1 -- 172.21.15.42:0/2152646808 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1002f040 con 0x55cfb127e170 2024-04-20T10:43:51.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.440+0000 7f9f0cc00640 1 -- 172.21.15.42:0/2152646808 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f10038070 con 0x55cfb127e170 2024-04-20T10:43:51.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.440+0000 7f9f0cc00640 1 -- 172.21.15.42:0/2152646808 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1002f040 con 0x55cfb127e170 2024-04-20T10:43:51.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.441+0000 7f9f21dcb740 1 -- 172.21.15.42:0/2152646808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 msgr2=0x55cfb1286f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:51.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.441+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/2152646808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 0x55cfb1286f70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9f100070a0 tx=0x7f9f10020000 comp rx=0 tx=0).stop 2024-04-20T10:43:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.442+0000 7f9f21dcb740 1 -- 172.21.15.42:0/2152646808 shutdown_connections 2024-04-20T10:43:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.442+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/2152646808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb1287670 0x55cfb129f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.443+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/2152646808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb127ea90 0x55cfb1293350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.443+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/2152646808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127e170 0x55cfb1286f70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.443+0000 7f9f21dcb740 1 -- 172.21.15.42:0/2152646808 >> 172.21.15.42:0/2152646808 conn(0x55cfb12788a0 msgr2=0x55cfb127fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:51.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.447+0000 7f9f21dcb740 1 -- 172.21.15.42:0/2152646808 shutdown_connections 2024-04-20T10:43:51.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.447+0000 7f9f21dcb740 1 -- 172.21.15.42:0/2152646808 wait complete. 2024-04-20T10:43:51.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.447+0000 7f9f21dcb740 1 -- start start 2024-04-20T10:43:51.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.448+0000 7f9f21dcb740 1 Processor -- start 2024-04-20T10:43:51.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.448+0000 7f9f21dcb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127e170 0x55cfb128fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.448+0000 7f9f21dcb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127ea90 0x55cfb1280ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.448+0000 7f9f21dcb740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb1287670 0x55cfb1281400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.449+0000 7f9f21dcb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181c3e20 con 0x55cfb127ea90 2024-04-20T10:43:51.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.449+0000 7f9f21dcb740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181c5290 con 0x55cfb1287670 2024-04-20T10:43:51.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.449+0000 7f9f21dcb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181c44f0 con 0x55cfb127e170 2024-04-20T10:43:51.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.449+0000 7f9f21dcb740 10 client.0 _ll_register_callbacks cb 0x7f9f10011e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:51.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.449+0000 7f9f1d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb1287670 0x55cfb1281400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.449+0000 7f9f1d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb1287670 0x55cfb1281400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:60122/0 (socket says 172.21.15.42:60122) 2024-04-20T10:43:51.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.449+0000 7f9f1d000640 1 -- 172.21.15.42:0/3048866700 learned_addr learned my addr 172.21.15.42:0/3048866700 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:51.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.450+0000 7f9f1d000640 1 -- 172.21.15.42:0/3048866700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127e170 msgr2=0x55cfb128fa50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:43:51.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.450+0000 7f9f1d000640 1 --2- 172.21.15.42:0/3048866700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127e170 0x55cfb128fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.450+0000 7f9f1d000640 1 -- 172.21.15.42:0/3048866700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127ea90 msgr2=0x55cfb1280ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:43:51.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.450+0000 7f9f1d000640 1 --2- 172.21.15.42:0/3048866700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127ea90 0x55cfb1280ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.450+0000 7f9f1d000640 1 -- 172.21.15.42:0/3048866700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb0fbf7d0 con 0x55cfb1287670 2024-04-20T10:43:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.450+0000 7f9f1d000640 1 --2- 172.21.15.42:0/3048866700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb1287670 0x55cfb1281400 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009bb0 tx=0x7f9f08009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c10a80b6ace9eec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.450+0000 7f9f02000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:43:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.451+0000 7f9f02000640 1 -- 172.21.15.42:0/3048866700 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0802a030 con 0x55cfb1287670 2024-04-20T10:43:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.451+0000 7f9f02000640 1 -- 172.21.15.42:0/3048866700 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f9f08004f30 con 0x55cfb1287670 2024-04-20T10:43:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.451+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3048866700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f10038070 con 0x55cfb1287670 2024-04-20T10:43:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.451+0000 7f9f02000640 1 -- 172.21.15.42:0/3048866700 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9f0801f040 con 0x55cfb1287670 2024-04-20T10:43:51.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.451+0000 7f9f21dcb740 10 client.26501 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:51.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.451+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3048866700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55cfb12fd470 con 0x55cfb1287670 2024-04-20T10:43:51.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.452+0000 7f9f02000640 1 -- 172.21.15.42:0/3048866700 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f10038070 con 0x55cfb1287670 2024-04-20T10:43:51.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.454+0000 7f9f02000640 1 -- 172.21.15.42:0/3048866700 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f9f08026ba0 con 0x55cfb1287670 2024-04-20T10:43:51.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.454+0000 7f9f21dcb740 10 client.26501 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:51.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.454+0000 7f9f21dcb740 10 client.26501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:51.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.454+0000 7f9f21dcb740 20 client.26501 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:51.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.454+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/3048866700 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55cfb12fd8b0 0x55cfb12ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:51.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.454+0000 7f9f21dcb740 4 client.26501 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:43:51.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.454+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3048866700 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfb1299ba0 con 0x55cfb12fd8b0 2024-04-20T10:43:51.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.454+0000 7f9f17e00640 1 --2- 172.21.15.42:0/3048866700 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55cfb12fd8b0 0x55cfb12ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.455+0000 7f9f17e00640 1 --2- 172.21.15.42:0/3048866700 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55cfb12fd8b0 0x55cfb12ffca0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c8463bc7d50b614 server_cookie=ef33a2a5112b3a31 in_seq=0 out_seq=0 2024-04-20T10:43:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.455+0000 7f9f02000640 10 client.26501 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.456+0000 7f9f02000640 1 -- 172.21.15.42:0/3048866700 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfb1299ba0 con 0x55cfb12fd8b0 2024-04-20T10:43:51.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.456+0000 7f9f02000640 10 client.26501 handle_command_reply: tid=0 2024-04-20T10:43:51.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.460+0000 7f9f21dcb740 1 client.26501 shutdown 2024-04-20T10:43:51.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.460+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3048866700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb1287670 msgr2=0x55cfb1281400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.460+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/3048866700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb1287670 0x55cfb1281400 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009bb0 tx=0x7f9f08009ed0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.460+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3048866700 shutdown_connections 2024-04-20T10:43:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.460+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/3048866700 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55cfb12fd8b0 0x55cfb12ffca0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.460+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/3048866700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfb1287670 0x55cfb1281400 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.460+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/3048866700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cfb127ea90 0x55cfb1280ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.460+0000 7f9f21dcb740 1 --2- 172.21.15.42:0/3048866700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cfb127e170 0x55cfb128fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.460+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3048866700 >> 172.21.15.42:0/3048866700 conn(0x55cfb12788a0 msgr2=0x55cfb12770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.460+0000 7f9f02000640 0 client.26501 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:51.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.461+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3048866700 shutdown_connections 2024-04-20T10:43:51.464 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:43:51.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.461+0000 7f9f21dcb740 1 -- 172.21.15.42:0/3048866700 wait complete. 2024-04-20T10:43:51.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.461+0000 7f9f21dcb740 20 client.26501 trim_cache size 0 max 16384 2024-04-20T10:43:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.464+0000 7f9f1da00640 1 -- 172.21.15.42:0/3343764347 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9efc0a3f20 msgr2=0x7f9efc0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.464+0000 7f9f1da00640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9efc0a3f20 0x7f9efc0a63e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009500 tx=0x7f9f1000b040 comp rx=0 tx=0).stop 2024-04-20T10:43:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.464+0000 7f9f1da00640 1 -- 172.21.15.42:0/3343764347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f181099e0 msgr2=0x7f9f18116390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.464+0000 7f9f1da00640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f181099e0 0x7f9f18116390 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009850 tx=0x7f9f040043b0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.465+0000 7f9f1da00640 1 -- 172.21.15.42:0/3343764347 shutdown_connections 2024-04-20T10:43:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.465+0000 7f9f1da00640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f9efc0a3f20 0x7f9efc0a63e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.465+0000 7f9f1da00640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f18110600 0x7f9f181168d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.465+0000 7f9f1da00640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f181099e0 0x7f9f18116390 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.465+0000 7f9f1da00640 1 --2- 172.21.15.42:0/3343764347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f18108f60 0x7f9f1811d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:51.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.465+0000 7f9f1da00640 1 -- 172.21.15.42:0/3343764347 >> 172.21.15.42:0/3343764347 conn(0x7f9f1806cee0 msgr2=0x7f9f18070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:51.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.469+0000 7f9f1da00640 1 -- 172.21.15.42:0/3343764347 shutdown_connections 2024-04-20T10:43:51.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:51.469+0000 7f9f1da00640 1 -- 172.21.15.42:0/3343764347 wait complete. 2024-04-20T10:43:51.557 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/sha512_generic.o 2024-04-20T10:43:51.598 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/entry/built-in.a 2024-04-20T10:43:51.690 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/module/main.o 2024-04-20T10:43:52.049 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/mmu_gather.o 2024-04-20T10:43:52.077 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/netfs/main.o 2024-04-20T10:43:52.247 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/ss/services.o 2024-04-20T10:43:52.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:51 smithi046 ceph-mon[21574]: pgmap v304: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.2 MiB/s wr, 50 op/s 2024-04-20T10:43:52.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:51 smithi042 ceph-mon[17078]: pgmap v304: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.2 MiB/s wr, 50 op/s 2024-04-20T10:43:52.370 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/numa_64.o 2024-04-20T10:43:52.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:51 smithi162 ceph-mon[21854]: pgmap v304: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.2 MiB/s wr, 50 op/s 2024-04-20T10:43:52.441 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/sha3_generic.o 2024-04-20T10:43:52.508 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:43:52.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:43:52.509 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:43:52.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.908+0000 7f2a19000640 1 -- 172.21.15.42:0/1790316016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0b7890 msgr2=0x7f2a0c0b9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:52.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.908+0000 7f2a19000640 1 --2- 172.21.15.42:0/1790316016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0b7890 0x7f2a0c0b9d30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2a14067c60 tx=0x7f2a1408f550 comp rx=0 tx=0).stop 2024-04-20T10:43:52.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.910+0000 7f2a19000640 1 -- 172.21.15.42:0/1790316016 shutdown_connections 2024-04-20T10:43:52.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.910+0000 7f2a19000640 1 --2- 172.21.15.42:0/1790316016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0b7890 0x7f2a0c0b9d30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.910+0000 7f2a19000640 1 --2- 172.21.15.42:0/1790316016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0c0a5b40 0x7f2a0c0ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.910+0000 7f2a19000640 1 --2- 172.21.15.42:0/1790316016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c0a5190 0x7f2a0c0a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.910+0000 7f2a19000640 1 -- 172.21.15.42:0/1790316016 >> 172.21.15.42:0/1790316016 conn(0x7f2a0c01a810 msgr2=0x7f2a0c01ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:52.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.911+0000 7f2a19000640 1 -- 172.21.15.42:0/1790316016 shutdown_connections 2024-04-20T10:43:52.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.912+0000 7f2a19000640 1 -- 172.21.15.42:0/1790316016 wait complete. 2024-04-20T10:43:52.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.913+0000 7f2a19000640 1 Processor -- start 2024-04-20T10:43:52.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.914+0000 7f2a19000640 1 -- start start 2024-04-20T10:43:52.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.915+0000 7f2a19000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0a5190 0x7f2a0c149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:52.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.915+0000 7f2a19000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c0a5b40 0x7f2a0c149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:52.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.915+0000 7f2a19000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0c0b7890 0x7f2a0c143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:52.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.915+0000 7f2a12a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0a5190 0x7f2a0c149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:52.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.915+0000 7f2a12a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0a5190 0x7f2a0c149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:52976/0 (socket says 172.21.15.42:52976) 2024-04-20T10:43:52.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.915+0000 7f2a19000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c0bd620 con 0x7f2a0c0b7890 2024-04-20T10:43:52.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.915+0000 7f2a19000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c0bd4a0 con 0x7f2a0c0a5b40 2024-04-20T10:43:52.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.916+0000 7f2a19000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c0bd7a0 con 0x7f2a0c0a5190 2024-04-20T10:43:52.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.916+0000 7f2a13400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0c0b7890 0x7f2a0c143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:52.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.916+0000 7f2a13400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0c0b7890 0x7f2a0c143980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58568/0 (socket says 172.21.15.42:58568) 2024-04-20T10:43:52.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.916+0000 7f2a13400640 1 -- 172.21.15.42:0/1790555330 learned_addr learned my addr 172.21.15.42:0/1790555330 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:52.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.917+0000 7f2a13400640 1 -- 172.21.15.42:0/1790555330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0a5190 msgr2=0x7f2a0c149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:52.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.917+0000 7f2a13400640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0a5190 0x7f2a0c149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.917+0000 7f2a13400640 1 -- 172.21.15.42:0/1790555330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c0a5b40 msgr2=0x7f2a0c149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:52.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.917+0000 7f2a12a00640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0a5190 0x7f2a0c149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-20T10:43:52.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.915+0000 7f2a12000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c0a5b40 0x7f2a0c149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:52.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.918+0000 7f2a13400640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c0a5b40 0x7f2a0c149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.918+0000 7f2a13400640 1 -- 172.21.15.42:0/1790555330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c144100 con 0x7f2a0c0b7890 2024-04-20T10:43:52.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.918+0000 7f2a12000640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c0a5b40 0x7f2a0c149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:52.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.918+0000 7f2a13400640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0c0b7890 0x7f2a0c143980 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a140516a0 tx=0x7f2a14068810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:52.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.919+0000 7f2a07e00640 1 -- 172.21.15.42:0/1790555330 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a140a8070 con 0x7f2a0c0b7890 2024-04-20T10:43:52.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.919+0000 7f2a07e00640 1 -- 172.21.15.42:0/1790555330 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a1409d040 con 0x7f2a0c0b7890 2024-04-20T10:43:52.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.920+0000 7f2a19000640 1 -- 172.21.15.42:0/1790555330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c144390 con 0x7f2a0c0b7890 2024-04-20T10:43:52.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.920+0000 7f2a19000640 1 -- 172.21.15.42:0/1790555330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c150640 con 0x7f2a0c0b7890 2024-04-20T10:43:52.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.920+0000 7f2a07e00640 1 -- 172.21.15.42:0/1790555330 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14097830 con 0x7f2a0c0b7890 2024-04-20T10:43:52.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.927+0000 7f2a07e00640 1 -- 172.21.15.42:0/1790555330 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2a140979d0 con 0x7f2a0c0b7890 2024-04-20T10:43:52.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.930+0000 7f2a07e00640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f29fc0a3e60 0x7f29fc0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:52.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.931+0000 7f2a1d223740 1 Processor -- start 2024-04-20T10:43:52.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.932+0000 7f2a07e00640 1 -- 172.21.15.42:0/1790555330 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f2a14156740 con 0x7f2a0c0b7890 2024-04-20T10:43:52.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.933+0000 7f2a12a00640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f29fc0a3e60 0x7f29fc0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.933+0000 7f2a1d223740 1 -- start start 2024-04-20T10:43:52.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.934+0000 7f2a1d223740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f150f0 0x561106f154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:52.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.934+0000 7f2a1d223740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f15a10 0x561106f1e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:52.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.934+0000 7f2a1d223740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 0x561106f209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:52.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.934+0000 7f2a1d223740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561106e79d20 con 0x561106f15a10 2024-04-20T10:43:52.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.934+0000 7f2a1d223740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561106e203e0 con 0x561106f1e5f0 2024-04-20T10:43:52.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.934+0000 7f2a1d223740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561106e75ce0 con 0x561106f150f0 2024-04-20T10:43:52.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.934+0000 7f2a13400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 0x561106f209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:52.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.935+0000 7f2a12000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f15a10 0x561106f1e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:52.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.935+0000 7f2a13400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 0x561106f209e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:60150/0 (socket says 172.21.15.42:60150) 2024-04-20T10:43:52.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.935+0000 7f2a13400640 1 -- 172.21.15.42:0/1586529456 learned_addr learned my addr 172.21.15.42:0/1586529456 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.935+0000 7f2a12000640 1 -- 172.21.15.42:0/1586529456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f150f0 msgr2=0x561106f154d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:43:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.935+0000 7f2a12000640 1 --2- 172.21.15.42:0/1586529456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f150f0 0x561106f154d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.935+0000 7f2a12000640 1 -- 172.21.15.42:0/1586529456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 msgr2=0x561106f209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.935+0000 7f2a12000640 1 --2- 172.21.15.42:0/1586529456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 0x561106f209e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.935+0000 7f2a12000640 1 -- 172.21.15.42:0/1586529456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561106e3f980 con 0x561106f15a10 2024-04-20T10:43:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.936+0000 7f2a12000640 1 --2- 172.21.15.42:0/1586529456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f15a10 0x561106f1e0b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c145120 tx=0x7f2a0800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7f089b5421637d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.937+0000 7f2a04c00640 1 -- 172.21.15.42:0/1586529456 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08003970 con 0x561106f15a10 2024-04-20T10:43:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.937+0000 7f2a04c00640 1 -- 172.21.15.42:0/1586529456 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a08004030 con 0x561106f15a10 2024-04-20T10:43:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.937+0000 7f2a04c00640 1 -- 172.21.15.42:0/1586529456 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a080117f0 con 0x561106f15a10 2024-04-20T10:43:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.938+0000 7f2a1d223740 1 -- 172.21.15.42:0/1586529456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f15a10 msgr2=0x561106f1e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.938+0000 7f2a12a00640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f29fc0a3e60 0x7f29fc0a6320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0b9f90 tx=0x7f2a00009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.938+0000 7f2a1d223740 1 --2- 172.21.15.42:0/1586529456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f15a10 0x561106f1e0b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c145120 tx=0x7f2a0800f730 comp rx=0 tx=0).stop 2024-04-20T10:43:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.938+0000 7f2a1d223740 1 -- 172.21.15.42:0/1586529456 shutdown_connections 2024-04-20T10:43:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.938+0000 7f2a1d223740 1 --2- 172.21.15.42:0/1586529456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 0x561106f209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.938+0000 7f2a1d223740 1 --2- 172.21.15.42:0/1586529456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f15a10 0x561106f1e0b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.938+0000 7f2a1d223740 1 --2- 172.21.15.42:0/1586529456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f150f0 0x561106f154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.938+0000 7f2a1d223740 1 -- 172.21.15.42:0/1586529456 >> 172.21.15.42:0/1586529456 conn(0x561106f0f820 msgr2=0x561106f0fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.938+0000 7f2a1d223740 1 -- 172.21.15.42:0/1586529456 shutdown_connections 2024-04-20T10:43:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.938+0000 7f2a1d223740 1 -- 172.21.15.42:0/1586529456 wait complete. 2024-04-20T10:43:52.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.939+0000 7f2a1d223740 1 -- start start 2024-04-20T10:43:52.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.939+0000 7f2a1d223740 1 Processor -- start 2024-04-20T10:43:52.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.940+0000 7f2a1d223740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f150f0 0x561106f1b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:52.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.941+0000 7f2a1d223740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 0x561106f1bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.941+0000 7f2a1d223740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f1c1a0 0x561106f2a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.941+0000 7f2a12a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f150f0 0x561106f1b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.941+0000 7f2a12a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f150f0 0x561106f1b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58594/0 (socket says 172.21.15.42:58594) 2024-04-20T10:43:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.941+0000 7f2a12000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 0x561106f1bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.941+0000 7f2a12000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 0x561106f1bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:60160/0 (socket says 172.21.15.42:60160) 2024-04-20T10:43:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.942+0000 7f2a12a00640 1 -- 172.21.15.42:0/990273962 learned_addr learned my addr 172.21.15.42:0/990273962 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.942+0000 7f2a1d223740 1 -- 172.21.15.42:0/990273962 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561106f213e0 con 0x561106f150f0 2024-04-20T10:43:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.942+0000 7f2a13400640 1 --2- 172.21.15.42:0/990273962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f1c1a0 0x561106f2a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.942+0000 7f2a1d223740 1 -- 172.21.15.42:0/990273962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561106e337a0 con 0x561106f1e5f0 2024-04-20T10:43:52.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.942+0000 7f2a1d223740 1 -- 172.21.15.42:0/990273962 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561106f21560 con 0x561106f1c1a0 2024-04-20T10:43:52.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.943+0000 7f2a1d223740 10 client.0 _ll_register_callbacks cb 0x561106e35b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:52.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.943+0000 7f2a12000640 1 -- 172.21.15.42:0/990273962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f1c1a0 msgr2=0x561106f2a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:52.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.943+0000 7f2a12000640 1 --2- 172.21.15.42:0/990273962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f1c1a0 0x561106f2a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.943+0000 7f2a12000640 1 -- 172.21.15.42:0/990273962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f150f0 msgr2=0x561106f1b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:52.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.943+0000 7f2a12000640 1 --2- 172.21.15.42:0/990273962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f150f0 0x561106f1b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.943+0000 7f2a12000640 1 -- 172.21.15.42:0/990273962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561106c547e0 con 0x561106f1e5f0 2024-04-20T10:43:52.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.943+0000 7f2a1d223740 1 -- 172.21.15.42:0/990273962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a08004030 con 0x561106f1e5f0 2024-04-20T10:43:52.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.948+0000 7f2a12000640 1 --2- 172.21.15.42:0/990273962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 0x561106f1bc60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x561106f15f30 tx=0x7f2a08004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26cdaeebf7f7ff52 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:52.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.948+0000 7f29f9600640 10 client.26510 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:43:52.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.948+0000 7f29f9600640 1 -- 172.21.15.42:0/990273962 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08003d80 con 0x561106f1e5f0 2024-04-20T10:43:52.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.948+0000 7f29f9600640 1 -- 172.21.15.42:0/990273962 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f2a080041e0 con 0x561106f1e5f0 2024-04-20T10:43:52.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.949+0000 7f29f9600640 1 -- 172.21.15.42:0/990273962 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2a08004420 con 0x561106f1e5f0 2024-04-20T10:43:52.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.952+0000 7f2a1d223740 10 client.26510 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:52.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.952+0000 7f2a1d223740 1 -- 172.21.15.42:0/990273962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=19}) -- 0x561106cb4f70 con 0x561106f1e5f0 2024-04-20T10:43:52.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.952+0000 7f29f9600640 1 -- 172.21.15.42:0/990273962 <== mon.2 v2:172.21.15.46:3300/0 4 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f2a080045c0 con 0x561106f1e5f0 2024-04-20T10:43:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.953+0000 7f2a1d223740 10 client.26510 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.953+0000 7f2a1d223740 10 client.26510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.953+0000 7f2a1d223740 20 client.26510 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.953+0000 7f2a1d223740 1 --2- 172.21.15.42:0/990273962 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561106f2ecf0 0x561106f310e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.953+0000 7f2a1d223740 4 client.26510 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:43:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.953+0000 7f2a1d223740 1 -- 172.21.15.42:0/990273962 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561106e87740 con 0x561106f2ecf0 2024-04-20T10:43:52.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.953+0000 7f2a12a00640 1 --2- 172.21.15.42:0/990273962 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561106f2ecf0 0x561106f310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:52.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.957+0000 7f2a12a00640 1 --2- 172.21.15.42:0/990273962 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561106f2ecf0 0x561106f310e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=892bf773a657fa22 server_cookie=a0aee95873fa2d72 in_seq=0 out_seq=0 2024-04-20T10:43:52.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.957+0000 7f29f9600640 10 client.26510 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.958+0000 7f29f9600640 1 -- 172.21.15.42:0/990273962 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561106e87740 con 0x561106f2ecf0 2024-04-20T10:43:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.958+0000 7f29f9600640 10 client.26510 handle_command_reply: tid=0 2024-04-20T10:43:52.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.958+0000 7f2a1d223740 1 client.26510 shutdown 2024-04-20T10:43:52.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.959+0000 7f2a1d223740 1 -- 172.21.15.42:0/990273962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 msgr2=0x561106f1bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:52.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.959+0000 7f2a1d223740 1 --2- 172.21.15.42:0/990273962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 0x561106f1bc60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x561106f15f30 tx=0x7f2a08004c70 comp rx=0 tx=0).stop 2024-04-20T10:43:52.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.961+0000 7f2a1d223740 1 -- 172.21.15.42:0/990273962 shutdown_connections 2024-04-20T10:43:52.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.962+0000 7f2a1d223740 1 --2- 172.21.15.42:0/990273962 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561106f2ecf0 0x561106f310e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.962+0000 7f2a1d223740 1 --2- 172.21.15.42:0/990273962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f1c1a0 0x561106f2a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.962+0000 7f2a1d223740 1 --2- 172.21.15.42:0/990273962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1e5f0 0x561106f1bc60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.962+0000 7f2a1d223740 1 --2- 172.21.15.42:0/990273962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f150f0 0x561106f1b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.962+0000 7f2a1d223740 1 -- 172.21.15.42:0/990273962 >> 172.21.15.42:0/990273962 conn(0x561106f0f820 msgr2=0x561106f11570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:52.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.962+0000 7f29f9600640 0 client.26510 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.966+0000 7f2a1d223740 1 -- 172.21.15.42:0/990273962 shutdown_connections 2024-04-20T10:43:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.966+0000 7f2a1d223740 1 -- 172.21.15.42:0/990273962 wait complete. 2024-04-20T10:43:52.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:52.966+0000 7f2a1d223740 20 client.26510 trim_cache size 0 max 16384 2024-04-20T10:43:52.982 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/elevator.o 2024-04-20T10:43:52.987 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/mprotect.o 2024-04-20T10:43:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.007+0000 7f2a1d223740 1 Processor -- start 2024-04-20T10:43:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.008+0000 7f2a1d223740 1 -- start start 2024-04-20T10:43:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.008+0000 7f2a1d223740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f150f0 0x561106f2a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.008+0000 7f2a1d223740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 0x561106f33d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.008+0000 7f2a1d223740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f1e5f0 0x561106f34260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.008+0000 7f2a12000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 0x561106f33d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.008+0000 7f2a12a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f150f0 0x561106f2a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.008+0000 7f2a12000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 0x561106f33d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:60172/0 (socket says 172.21.15.42:60172) 2024-04-20T10:43:53.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.008+0000 7f2a12a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f150f0 0x561106f2a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:53000/0 (socket says 172.21.15.42:53000) 2024-04-20T10:43:53.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.011+0000 7f2a12000640 1 -- 172.21.15.42:0/779014238 learned_addr learned my addr 172.21.15.42:0/779014238 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:53.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.011+0000 7f2a1d223740 1 -- 172.21.15.42:0/779014238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f29f400c850 con 0x561106f1e5f0 2024-04-20T10:43:53.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.011+0000 7f2a1d223740 1 -- 172.21.15.42:0/779014238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29f400f0b0 con 0x561106f1c1a0 2024-04-20T10:43:53.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.011+0000 7f2a1d223740 1 -- 172.21.15.42:0/779014238 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f29f400cea0 con 0x561106f150f0 2024-04-20T10:43:53.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.011+0000 7f2a12000640 1 -- 172.21.15.42:0/779014238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f150f0 msgr2=0x561106f2a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:53.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.011+0000 7f2a12000640 1 --2- 172.21.15.42:0/779014238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f150f0 0x561106f2a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.011+0000 7f2a12000640 1 -- 172.21.15.42:0/779014238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f1e5f0 msgr2=0x561106f34260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:43:53.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.011+0000 7f2a13400640 1 --2- 172.21.15.42:0/779014238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f1e5f0 0x561106f34260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.011+0000 7f2a12000640 1 --2- 172.21.15.42:0/779014238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f1e5f0 0x561106f34260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.011+0000 7f2a12000640 1 -- 172.21.15.42:0/779014238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561106b4c170 con 0x561106f1c1a0 2024-04-20T10:43:53.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.011+0000 7f2a13400640 1 --2- 172.21.15.42:0/779014238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f1e5f0 0x561106f34260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:53.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.014+0000 7f2a12000640 1 --2- 172.21.15.42:0/779014238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 0x561106f33d20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2a080096d0 tx=0x7f2a0801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b38d6287408d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:53.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.014+0000 7f29fbe00640 1 -- 172.21.15.42:0/779014238 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08003b80 con 0x561106f1c1a0 2024-04-20T10:43:53.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.014+0000 7f29fbe00640 1 -- 172.21.15.42:0/779014238 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a08004030 con 0x561106f1c1a0 2024-04-20T10:43:53.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.014+0000 7f29fbe00640 1 -- 172.21.15.42:0/779014238 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08023460 con 0x561106f1c1a0 2024-04-20T10:43:53.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.015+0000 7f2a1d223740 1 -- 172.21.15.42:0/779014238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 msgr2=0x561106f33d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.015+0000 7f2a1d223740 1 --2- 172.21.15.42:0/779014238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 0x561106f33d20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2a080096d0 tx=0x7f2a0801a000 comp rx=0 tx=0).stop 2024-04-20T10:43:53.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.015+0000 7f2a1d223740 1 -- 172.21.15.42:0/779014238 shutdown_connections 2024-04-20T10:43:53.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.015+0000 7f2a1d223740 1 --2- 172.21.15.42:0/779014238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f1e5f0 0x561106f34260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.015+0000 7f2a1d223740 1 --2- 172.21.15.42:0/779014238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 0x561106f33d20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.015+0000 7f2a1d223740 1 --2- 172.21.15.42:0/779014238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f150f0 0x561106f2a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.015+0000 7f2a1d223740 1 -- 172.21.15.42:0/779014238 >> 172.21.15.42:0/779014238 conn(0x561106f0f820 msgr2=0x561106f1a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:53.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.016+0000 7f2a1d223740 1 -- 172.21.15.42:0/779014238 shutdown_connections 2024-04-20T10:43:53.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.016+0000 7f2a1d223740 1 -- 172.21.15.42:0/779014238 wait complete. 2024-04-20T10:43:53.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.016+0000 7f2a1d223740 1 -- start start 2024-04-20T10:43:53.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.017+0000 7f2a1d223740 1 Processor -- start 2024-04-20T10:43:53.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.017+0000 7f2a1d223740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f150f0 0x561106f33ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.017+0000 7f2a1d223740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 0x561106f34220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.018+0000 7f2a1d223740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f1e5f0 0x561106f32a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.018+0000 7f2a1d223740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f29f400d570 con 0x561106f150f0 2024-04-20T10:43:53.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.018+0000 7f2a1d223740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29f400e9e0 con 0x561106f1c1a0 2024-04-20T10:43:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.018+0000 7f2a1d223740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f29f400dc40 con 0x561106f1e5f0 2024-04-20T10:43:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.018+0000 7f2a1d223740 10 client.0 _ll_register_callbacks cb 0x7f2a14068f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:53.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.018+0000 7f2a12a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f150f0 0x561106f33ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.018+0000 7f2a12000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 0x561106f34220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.018+0000 7f2a12000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 0x561106f34220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:60174/0 (socket says 172.21.15.42:60174) 2024-04-20T10:43:53.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.019+0000 7f2a12000640 1 -- 172.21.15.42:0/3741893790 learned_addr learned my addr 172.21.15.42:0/3741893790 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:53.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.019+0000 7f2a12000640 1 -- 172.21.15.42:0/3741893790 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f1e5f0 msgr2=0x561106f32a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:53.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.019+0000 7f2a12000640 1 --2- 172.21.15.42:0/3741893790 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f1e5f0 0x561106f32a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.019+0000 7f2a12000640 1 -- 172.21.15.42:0/3741893790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f150f0 msgr2=0x561106f33ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:53.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.019+0000 7f2a12000640 1 --2- 172.21.15.42:0/3741893790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f150f0 0x561106f33ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.020+0000 7f2a12000640 1 -- 172.21.15.42:0/3741893790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561106f33240 con 0x561106f1c1a0 2024-04-20T10:43:53.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.020+0000 7f2a12a00640 1 --2- 172.21.15.42:0/3741893790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f150f0 0x561106f33ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:53.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.022+0000 7f2a12000640 1 --2- 172.21.15.42:0/3741893790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 0x561106f34220 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2a08003f00 tx=0x7f2a08009a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=803e586af9465d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:53.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.022+0000 7f29f9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:43:53.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.022+0000 7f29f9600640 1 -- 172.21.15.42:0/3741893790 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a080116d0 con 0x561106f1c1a0 2024-04-20T10:43:53.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.022+0000 7f29f9600640 1 -- 172.21.15.42:0/3741893790 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f2a08011870 con 0x561106f1c1a0 2024-04-20T10:43:53.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.023+0000 7f2a1d223740 1 -- 172.21.15.42:0/3741893790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a08004030 con 0x561106f1c1a0 2024-04-20T10:43:53.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.023+0000 7f29f9600640 1 -- 172.21.15.42:0/3741893790 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2a08011a70 con 0x561106f1c1a0 2024-04-20T10:43:53.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.026+0000 7f2a1d223740 10 client.26516 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:53.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.026+0000 7f2a1d223740 1 -- 172.21.15.42:0/3741893790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561106f94ad0 con 0x561106f1c1a0 2024-04-20T10:43:53.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.026+0000 7f29f9600640 1 -- 172.21.15.42:0/3741893790 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a08004030 con 0x561106f1c1a0 2024-04-20T10:43:53.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.026+0000 7f29f9600640 1 -- 172.21.15.42:0/3741893790 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f2a08023480 con 0x561106f1c1a0 2024-04-20T10:43:53.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.026+0000 7f2a1d223740 10 client.26516 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:53.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.026+0000 7f2a1d223740 10 client.26516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:53.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.026+0000 7f2a1d223740 20 client.26516 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:53.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.026+0000 7f2a1d223740 1 --2- 172.21.15.42:0/3741893790 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561106f95160 0x561106f97580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.026+0000 7f2a1d223740 4 client.26516 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:43:53.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.026+0000 7f2a1d223740 1 -- 172.21.15.42:0/3741893790 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561106e815b0 con 0x561106f95160 2024-04-20T10:43:53.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.029+0000 7f2a12a00640 1 --2- 172.21.15.42:0/3741893790 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561106f95160 0x561106f97580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.030+0000 7f29f9600640 10 client.26516 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:53.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.030+0000 7f2a12a00640 1 --2- 172.21.15.42:0/3741893790 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561106f95160 0x561106f97580 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff4c9e1c012ccd3 server_cookie=433b610cd1d4beac in_seq=0 out_seq=0 2024-04-20T10:43:53.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.032+0000 7f29f9600640 1 -- 172.21.15.42:0/3741893790 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561106e815b0 con 0x561106f95160 2024-04-20T10:43:53.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.032+0000 7f29f9600640 10 client.26516 handle_command_reply: tid=0 2024-04-20T10:43:53.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.034+0000 7f2a1d223740 1 client.26516 shutdown 2024-04-20T10:43:53.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.039+0000 7f2a1d223740 1 -- 172.21.15.42:0/3741893790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 msgr2=0x561106f34220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.039+0000 7f2a1d223740 1 --2- 172.21.15.42:0/3741893790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 0x561106f34220 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2a08003f00 tx=0x7f2a08009a90 comp rx=0 tx=0).stop 2024-04-20T10:43:53.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.042+0000 7f2a1d223740 1 -- 172.21.15.42:0/3741893790 shutdown_connections 2024-04-20T10:43:53.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.042+0000 7f2a1d223740 1 --2- 172.21.15.42:0/3741893790 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x561106f95160 0x561106f97580 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.042+0000 7f2a1d223740 1 --2- 172.21.15.42:0/3741893790 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561106f1e5f0 0x561106f32a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.042+0000 7f2a1d223740 1 --2- 172.21.15.42:0/3741893790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561106f1c1a0 0x561106f34220 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.042+0000 7f2a1d223740 1 --2- 172.21.15.42:0/3741893790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561106f150f0 0x561106f33ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.042+0000 7f2a1d223740 1 -- 172.21.15.42:0/3741893790 >> 172.21.15.42:0/3741893790 conn(0x561106f0f820 msgr2=0x561106f0e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:53.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.043+0000 7f29f9600640 0 client.26516 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:53.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.043+0000 7f2a1d223740 1 -- 172.21.15.42:0/3741893790 shutdown_connections 2024-04-20T10:43:53.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.043+0000 7f2a1d223740 1 -- 172.21.15.42:0/3741893790 wait complete. 2024-04-20T10:43:53.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.044+0000 7f2a1d223740 20 client.26516 trim_cache size 0 max 16384 2024-04-20T10:43:53.045 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:43:53.045 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:43:53.045 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "36820ea4-a691-45ca-9778-729e7058e196", 2024-04-20T10:43:53.045 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:43:53.045 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:43:53.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.046+0000 7f2a19000640 1 -- 172.21.15.42:0/1790555330 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f29fc0a3e60 msgr2=0x7f29fc0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.046+0000 7f2a19000640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f29fc0a3e60 0x7f29fc0a6320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0b9f90 tx=0x7f2a00009260 comp rx=0 tx=0).stop 2024-04-20T10:43:53.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.046+0000 7f2a19000640 1 -- 172.21.15.42:0/1790555330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0c0b7890 msgr2=0x7f2a0c143980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.046+0000 7f2a19000640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0c0b7890 0x7f2a0c143980 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a140516a0 tx=0x7f2a14068810 comp rx=0 tx=0).stop 2024-04-20T10:43:53.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.046+0000 7f2a19000640 1 -- 172.21.15.42:0/1790555330 shutdown_connections 2024-04-20T10:43:53.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.046+0000 7f2a19000640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f29fc0a3e60 0x7f29fc0a6320 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.046+0000 7f2a19000640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0c0b7890 0x7f2a0c143980 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.047+0000 7f2a19000640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0c0a5b40 0x7f2a0c149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.047+0000 7f2a19000640 1 --2- 172.21.15.42:0/1790555330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0a5190 0x7f2a0c149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.047+0000 7f2a19000640 1 -- 172.21.15.42:0/1790555330 >> 172.21.15.42:0/1790555330 conn(0x7f2a0c01a810 msgr2=0x7f2a0c0ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:53.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.047+0000 7f2a19000640 1 -- 172.21.15.42:0/1790555330 shutdown_connections 2024-04-20T10:43:53.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.047+0000 7f2a19000640 1 -- 172.21.15.42:0/1790555330 wait complete. 2024-04-20T10:43:53.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:43:53.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:43:53.106 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/cbc.o 2024-04-20T10:43:53.128 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/amdtopology.o 2024-04-20T10:43:53.202 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/netfs/objects.o 2024-04-20T10:43:53.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.493+0000 7f9a01600640 1 -- 172.21.15.42:0/3564378448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99fc11b740 msgr2=0x7f99fc11db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.493+0000 7f9a01600640 1 --2- 172.21.15.42:0/3564378448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99fc11b740 0x7f99fc11db30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f99e801a940 tx=0x7f99e803cc80 comp rx=0 tx=0).stop 2024-04-20T10:43:53.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.495+0000 7f9a01600640 1 -- 172.21.15.42:0/3564378448 shutdown_connections 2024-04-20T10:43:53.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.495+0000 7f9a01600640 1 --2- 172.21.15.42:0/3564378448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99fc11b740 0x7f99fc11db30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.495+0000 7f9a01600640 1 --2- 172.21.15.42:0/3564378448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99fc10e7e0 0x7f99fc10ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.497+0000 7f9a01600640 1 --2- 172.21.15.42:0/3564378448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99fc1107b0 0x7f99fc10e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.497+0000 7f9a01600640 1 -- 172.21.15.42:0/3564378448 >> 172.21.15.42:0/3564378448 conn(0x7f99fc06d070 msgr2=0x7f99fc06d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:53.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.498+0000 7f9a01600640 1 -- 172.21.15.42:0/3564378448 shutdown_connections 2024-04-20T10:43:53.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.498+0000 7f9a01600640 1 -- 172.21.15.42:0/3564378448 wait complete. 2024-04-20T10:43:53.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.499+0000 7f9a01600640 1 Processor -- start 2024-04-20T10:43:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.500+0000 7f9a01600640 1 -- start start 2024-04-20T10:43:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.500+0000 7f9a01600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99fc10e7e0 0x7f99fc11ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.500+0000 7f9a01600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99fc1107b0 0x7f99fc116f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.500+0000 7f9a01600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99fc11b740 0x7f99fc117470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.501+0000 7f99fb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99fc11b740 0x7f99fc117470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.501+0000 7f99fb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99fc11b740 0x7f99fc117470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58638/0 (socket says 172.21.15.42:58638) 2024-04-20T10:43:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.501+0000 7f99fb400640 1 -- 172.21.15.42:0/3395665382 learned_addr learned my addr 172.21.15.42:0/3395665382 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.501+0000 7f9a01600640 1 -- 172.21.15.42:0/3395665382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc113940 con 0x7f99fc11b740 2024-04-20T10:43:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.501+0000 7f9a01600640 1 -- 172.21.15.42:0/3395665382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1137c0 con 0x7f99fc1107b0 2024-04-20T10:43:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.501+0000 7f9a01600640 1 -- 172.21.15.42:0/3395665382 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc113ac0 con 0x7f99fc10e7e0 2024-04-20T10:43:53.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.501+0000 7f99faa00640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99fc10e7e0 0x7f99fc11ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.502+0000 7f99fa000640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99fc1107b0 0x7f99fc116f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.504+0000 7f99fa000640 1 -- 172.21.15.42:0/3395665382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99fc10e7e0 msgr2=0x7f99fc11ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.504+0000 7f99fa000640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99fc10e7e0 0x7f99fc11ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.504+0000 7f99fa000640 1 -- 172.21.15.42:0/3395665382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99fc11b740 msgr2=0x7f99fc117470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.504+0000 7f99fa000640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99fc11b740 0x7f99fc117470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.504+0000 7f99fa000640 1 -- 172.21.15.42:0/3395665382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc117ba0 con 0x7f99fc1107b0 2024-04-20T10:43:53.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.504+0000 7f99fa000640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99fc1107b0 0x7f99fc116f30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00c970 tx=0x7f99ec00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:53.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.505+0000 7f99f3400640 1 -- 172.21.15.42:0/3395665382 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec007bb0 con 0x7f99fc1107b0 2024-04-20T10:43:53.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.505+0000 7f99f3400640 1 -- 172.21.15.42:0/3395665382 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99ec007d50 con 0x7f99fc1107b0 2024-04-20T10:43:53.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.505+0000 7f99f3400640 1 -- 172.21.15.42:0/3395665382 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0188c0 con 0x7f99fc1107b0 2024-04-20T10:43:53.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.505+0000 7f99fb400640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99fc11b740 0x7f99fc117470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:53.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.506+0000 7f9a01600640 1 -- 172.21.15.42:0/3395665382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99fc074190 con 0x7f99fc1107b0 2024-04-20T10:43:53.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.506+0000 7f9a01600640 1 -- 172.21.15.42:0/3395665382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99fc074690 con 0x7f99fc1107b0 2024-04-20T10:43:53.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.522+0000 7f99f3400640 1 -- 172.21.15.42:0/3395665382 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f99ec018a60 con 0x7f99fc1107b0 2024-04-20T10:43:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.523+0000 7f9a043d9740 1 Processor -- start 2024-04-20T10:43:53.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.523+0000 7f9a043d9740 1 -- start start 2024-04-20T10:43:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.525+0000 7f9a043d9740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e1195f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.525+0000 7f9a043d9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195f6ac0 0x55e1195ff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.525+0000 7f9a043d9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195ff6a0 0x55e119601a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.525+0000 7f9a043d9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e11955c150 con 0x55e1195ff6a0 2024-04-20T10:43:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.525+0000 7f9a043d9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e119502370 con 0x55e1195f6ac0 2024-04-20T10:43:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.525+0000 7f9a043d9740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55e119558110 con 0x55e1195f61a0 2024-04-20T10:43:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.525+0000 7f99fb400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e1195f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.525+0000 7f99fb400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e1195f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:53050/0 (socket says 172.21.15.42:53050) 2024-04-20T10:43:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.525+0000 7f99fb400640 1 -- 172.21.15.42:0/2295754555 learned_addr learned my addr 172.21.15.42:0/2295754555 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.525+0000 7f99faa00640 1 --2- 172.21.15.42:0/2295754555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195f6ac0 0x55e1195ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.526+0000 7f99fb400640 1 -- 172.21.15.42:0/2295754555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195f6ac0 msgr2=0x55e1195ff160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:53.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.526+0000 7f99fb400640 1 --2- 172.21.15.42:0/2295754555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195f6ac0 0x55e1195ff160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.526+0000 7f99fa000640 1 --2- 172.21.15.42:0/2295754555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195ff6a0 0x55e119601a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.526+0000 7f99fb400640 1 -- 172.21.15.42:0/2295754555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195ff6a0 msgr2=0x55e119601a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.527+0000 7f99fb400640 1 --2- 172.21.15.42:0/2295754555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195ff6a0 0x55e119601a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.527+0000 7f99fb400640 1 -- 172.21.15.42:0/2295754555 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1193387d0 con 0x55e1195f61a0 2024-04-20T10:43:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.527+0000 7f99fa000640 1 --2- 172.21.15.42:0/2295754555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195ff6a0 0x55e119601a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.528+0000 7f99fb400640 1 --2- 172.21.15.42:0/2295754555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e1195f6580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f99e8006010 tx=0x7f99e8004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf515c6e8fd4a896 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.528+0000 7f99f3400640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f99e00a3ef0 0x7f99e00a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.528+0000 7f99e7400640 1 -- 172.21.15.42:0/2295754555 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8052020 con 0x55e1195f61a0 2024-04-20T10:43:53.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.528+0000 7f99e7400640 1 -- 172.21.15.42:0/2295754555 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e80053e0 con 0x55e1195f61a0 2024-04-20T10:43:53.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.528+0000 7f99e7400640 1 -- 172.21.15.42:0/2295754555 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8004950 con 0x55e1195f61a0 2024-04-20T10:43:53.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.529+0000 7f9a043d9740 1 -- 172.21.15.42:0/2295754555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 msgr2=0x55e1195f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.529+0000 7f9a043d9740 1 --2- 172.21.15.42:0/2295754555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e1195f6580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f99e8006010 tx=0x7f99e8004840 comp rx=0 tx=0).stop 2024-04-20T10:43:53.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.530+0000 7f99f3400640 1 -- 172.21.15.42:0/3395665382 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f99ec0d0d50 con 0x7f99fc1107b0 2024-04-20T10:43:53.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.530+0000 7f99faa00640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f99e00a3ef0 0x7f99e00a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.531+0000 7f9a043d9740 1 -- 172.21.15.42:0/2295754555 shutdown_connections 2024-04-20T10:43:53.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.531+0000 7f9a043d9740 1 --2- 172.21.15.42:0/2295754555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195ff6a0 0x55e119601a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.531+0000 7f9a043d9740 1 --2- 172.21.15.42:0/2295754555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195f6ac0 0x55e1195ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.531+0000 7f9a043d9740 1 --2- 172.21.15.42:0/2295754555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e1195f6580 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.531+0000 7f9a043d9740 1 -- 172.21.15.42:0/2295754555 >> 172.21.15.42:0/2295754555 conn(0x55e1195f08d0 msgr2=0x55e1195f0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:53.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.533+0000 7f99faa00640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f99e00a3ef0 0x7f99e00a63b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55e1195f5c70 tx=0x7f99f4030040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:53.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.533+0000 7f9a043d9740 1 -- 172.21.15.42:0/2295754555 shutdown_connections 2024-04-20T10:43:53.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.534+0000 7f9a043d9740 1 -- 172.21.15.42:0/2295754555 wait complete. 2024-04-20T10:43:53.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.534+0000 7f9a043d9740 1 -- start start 2024-04-20T10:43:53.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.535+0000 7f9a043d9740 1 Processor -- start 2024-04-20T10:43:53.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.536+0000 7f9a043d9740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e1195fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.536+0000 7f9a043d9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f6ac0 0x55e1195fcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.536+0000 7f9a043d9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e1195fd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.536+0000 7f9a043d9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e119602430 con 0x55e1195f6ac0 2024-04-20T10:43:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.536+0000 7f99faa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e1195fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.536+0000 7f99fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e1195fd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.536+0000 7f99faa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e1195fc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:53066/0 (socket says 172.21.15.42:53066) 2024-04-20T10:43:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.536+0000 7f99faa00640 1 -- 172.21.15.42:0/357872902 learned_addr learned my addr 172.21.15.42:0/357872902 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.536+0000 7f9a043d9740 1 -- 172.21.15.42:0/357872902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e119515740 con 0x55e1195ff6a0 2024-04-20T10:43:53.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.536+0000 7f9a043d9740 1 -- 172.21.15.42:0/357872902 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55e1196025b0 con 0x55e1195f61a0 2024-04-20T10:43:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.536+0000 7f9a043d9740 10 client.0 _ll_register_callbacks cb 0x55e119510b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.537+0000 7f99fa000640 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f6ac0 0x55e1195fcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.537+0000 7f99faa00640 1 -- 172.21.15.42:0/357872902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 msgr2=0x55e1195fd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.537+0000 7f99faa00640 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e1195fd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.537+0000 7f99faa00640 1 -- 172.21.15.42:0/357872902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f6ac0 msgr2=0x55e1195fcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:53.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.537+0000 7f99faa00640 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f6ac0 0x55e1195fcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.537+0000 7f99faa00640 1 -- 172.21.15.42:0/357872902 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e11926e3f0 con 0x55e1195f61a0 2024-04-20T10:43:53.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.537+0000 7f99fa000640 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f6ac0 0x55e1195fcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:53.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.539+0000 7f99faa00640 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e1195fc6e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f99f4029c40 tx=0x7f99f4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b11d587b36c5476b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:53.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.539+0000 7f99dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:43:53.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.539+0000 7f99dbe00640 1 -- 172.21.15.42:0/357872902 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f404a060 con 0x55e1195f61a0 2024-04-20T10:43:53.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.540+0000 7f99dbe00640 1 -- 172.21.15.42:0/357872902 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f99f40453f0 con 0x55e1195f61a0 2024-04-20T10:43:53.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.540+0000 7f9a043d9740 1 -- 172.21.15.42:0/357872902 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99e80053e0 con 0x55e1195f61a0 2024-04-20T10:43:53.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.542+0000 7f99dbe00640 1 -- 172.21.15.42:0/357872902 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f99f4029e70 con 0x55e1195f61a0 2024-04-20T10:43:53.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.542+0000 7f9a043d9740 10 client.26533 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:53.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.542+0000 7f9a043d9740 1 -- 172.21.15.42:0/357872902 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e11922e0f0 con 0x55e1195f61a0 2024-04-20T10:43:53.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.543+0000 7f99dbe00640 1 -- 172.21.15.42:0/357872902 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e80053e0 con 0x55e1195f61a0 2024-04-20T10:43:53.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.543+0000 7f99dbe00640 1 -- 172.21.15.42:0/357872902 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f99f4044070 con 0x55e1195f61a0 2024-04-20T10:43:53.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.543+0000 7f9a043d9740 10 client.26533 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:53.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.543+0000 7f9a043d9740 10 client.26533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:53.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.543+0000 7f9a043d9740 20 client.26533 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:53.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.543+0000 7f9a043d9740 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e11960fa90 0x55e119611f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.544+0000 7f9a043d9740 4 client.26533 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:43:53.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.544+0000 7f9a043d9740 1 -- 172.21.15.42:0/357872902 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e119569b70 con 0x55e11960fa90 2024-04-20T10:43:53.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.545+0000 7f99fa000640 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e11960fa90 0x55e119611f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.546+0000 7f99fa000640 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e11960fa90 0x55e119611f10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33475fbbd89f983e server_cookie=66b585c113ec1e63 in_seq=0 out_seq=0 2024-04-20T10:43:53.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.546+0000 7f99dbe00640 10 client.26533 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:53.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.548+0000 7f99dbe00640 1 -- 172.21.15.42:0/357872902 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e119569b70 con 0x55e11960fa90 2024-04-20T10:43:53.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.549+0000 7f99dbe00640 10 client.26533 handle_command_reply: tid=0 2024-04-20T10:43:53.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.549+0000 7f9a043d9740 1 client.26533 shutdown 2024-04-20T10:43:53.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.551+0000 7f9a043d9740 1 -- 172.21.15.42:0/357872902 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 msgr2=0x55e1195fc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.551+0000 7f9a043d9740 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e1195fc6e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f99f4029c40 tx=0x7f99f4004ad0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.551+0000 7f9a043d9740 1 -- 172.21.15.42:0/357872902 shutdown_connections 2024-04-20T10:43:53.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.551+0000 7f9a043d9740 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e11960fa90 0x55e119611f10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.551+0000 7f9a043d9740 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e1195fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.551+0000 7f9a043d9740 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f6ac0 0x55e1195fcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.551+0000 7f9a043d9740 1 --2- 172.21.15.42:0/357872902 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e1195fc6e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.551+0000 7f9a043d9740 1 -- 172.21.15.42:0/357872902 >> 172.21.15.42:0/357872902 conn(0x55e1195f08d0 msgr2=0x55e1195f2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:53.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.552+0000 7f99dbe00640 0 client.26533 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:53.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.552+0000 7f9a043d9740 1 -- 172.21.15.42:0/357872902 shutdown_connections 2024-04-20T10:43:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.552+0000 7f9a043d9740 1 -- 172.21.15.42:0/357872902 wait complete. 2024-04-20T10:43:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.553+0000 7f9a043d9740 20 client.26533 trim_cache size 0 max 16384 2024-04-20T10:43:53.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.598+0000 7f9a043d9740 1 Processor -- start 2024-04-20T10:43:53.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.598+0000 7f9a043d9740 1 -- start start 2024-04-20T10:43:53.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.598+0000 7f9a043d9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f61a0 0x55e1195fee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.598+0000 7f9a043d9740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f6ac0 0x55e11960b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.598+0000 7f9a043d9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e119617820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.598+0000 7f9a043d9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc00c9a0 con 0x55e1195f61a0 2024-04-20T10:43:53.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.598+0000 7f9a043d9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc00f200 con 0x55e1195ff6a0 2024-04-20T10:43:53.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.598+0000 7f9a043d9740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc00cff0 con 0x55e1195f6ac0 2024-04-20T10:43:53.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.599+0000 7f99fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e119617820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.599+0000 7f99fa000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f6ac0 0x55e11960b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.599+0000 7f99fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e119617820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:60218/0 (socket says 172.21.15.42:60218) 2024-04-20T10:43:53.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.599+0000 7f99fb400640 1 -- 172.21.15.42:0/1331726888 learned_addr learned my addr 172.21.15.42:0/1331726888 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:53.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.599+0000 7f99fb400640 1 -- 172.21.15.42:0/1331726888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f6ac0 msgr2=0x55e11960b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:53.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.599+0000 7f99fb400640 1 --2- 172.21.15.42:0/1331726888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f6ac0 0x55e11960b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.599+0000 7f99fb400640 1 -- 172.21.15.42:0/1331726888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f61a0 msgr2=0x55e1195fee10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:43:53.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.600+0000 7f99faa00640 1 --2- 172.21.15.42:0/1331726888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f61a0 0x55e1195fee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.600+0000 7f99fb400640 1 --2- 172.21.15.42:0/1331726888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f61a0 0x55e1195fee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.600+0000 7f99fb400640 1 -- 172.21.15.42:0/1331726888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1195fb620 con 0x55e1195ff6a0 2024-04-20T10:43:53.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.600+0000 7f99faa00640 1 --2- 172.21.15.42:0/1331726888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f61a0 0x55e1195fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:53.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.600+0000 7f99fa000640 1 --2- 172.21.15.42:0/1331726888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f6ac0 0x55e11960b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:43:53.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.602+0000 7f99fb400640 1 --2- 172.21.15.42:0/1331726888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e119617820 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f99e8005ab0 tx=0x7f99e8005570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb352d2e33815cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:53.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.602+0000 7f99e6a00640 1 -- 172.21.15.42:0/1331726888 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e805c070 con 0x55e1195ff6a0 2024-04-20T10:43:53.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.602+0000 7f99e6a00640 1 -- 172.21.15.42:0/1331726888 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e803d830 con 0x55e1195ff6a0 2024-04-20T10:43:53.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.602+0000 7f99e6a00640 1 -- 172.21.15.42:0/1331726888 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8052040 con 0x55e1195ff6a0 2024-04-20T10:43:53.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.603+0000 7f9a043d9740 1 -- 172.21.15.42:0/1331726888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 msgr2=0x55e119617820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.603+0000 7f9a043d9740 1 --2- 172.21.15.42:0/1331726888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e119617820 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f99e8005ab0 tx=0x7f99e8005570 comp rx=0 tx=0).stop 2024-04-20T10:43:53.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.606+0000 7f9a043d9740 1 -- 172.21.15.42:0/1331726888 shutdown_connections 2024-04-20T10:43:53.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.606+0000 7f9a043d9740 1 --2- 172.21.15.42:0/1331726888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e119617820 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.606+0000 7f9a043d9740 1 --2- 172.21.15.42:0/1331726888 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f6ac0 0x55e11960b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.606+0000 7f9a043d9740 1 --2- 172.21.15.42:0/1331726888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f61a0 0x55e1195fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.606+0000 7f9a043d9740 1 -- 172.21.15.42:0/1331726888 >> 172.21.15.42:0/1331726888 conn(0x55e1195f08d0 msgr2=0x55e1195ef8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:53.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.606+0000 7f9a043d9740 1 -- 172.21.15.42:0/1331726888 shutdown_connections 2024-04-20T10:43:53.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.606+0000 7f9a043d9740 1 -- 172.21.15.42:0/1331726888 wait complete. 2024-04-20T10:43:53.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.607+0000 7f9a043d9740 1 -- start start 2024-04-20T10:43:53.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.607+0000 7f9a043d9740 1 Processor -- start 2024-04-20T10:43:53.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.611+0000 7f9a043d9740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e119607890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.611+0000 7f9a043d9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f6ac0 0x55e119607dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.611+0000 7f9a043d9740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e119604860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.611+0000 7f9a043d9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc00d6c0 con 0x55e1195f6ac0 2024-04-20T10:43:53.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.611+0000 7f9a043d9740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc00eb30 con 0x55e1195ff6a0 2024-04-20T10:43:53.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.611+0000 7f9a043d9740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc00dd90 con 0x55e1195f61a0 2024-04-20T10:43:53.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.611+0000 7f9a043d9740 10 client.0 _ll_register_callbacks cb 0x7f99f40298c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:43:53.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.612+0000 7f99fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e119604860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.612+0000 7f99fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e119604860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:60224/0 (socket says 172.21.15.42:60224) 2024-04-20T10:43:53.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.612+0000 7f99fb400640 1 -- 172.21.15.42:0/988185522 learned_addr learned my addr 172.21.15.42:0/988185522 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:43:53.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.612+0000 7f99fa000640 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f6ac0 0x55e119607dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.612+0000 7f99faa00640 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e119607890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.613+0000 7f99fb400640 1 -- 172.21.15.42:0/988185522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 msgr2=0x55e119607890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:53.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.614+0000 7f99fb400640 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e119607890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.614+0000 7f99fb400640 1 -- 172.21.15.42:0/988185522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f6ac0 msgr2=0x55e119607dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:43:53.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.614+0000 7f99fb400640 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f6ac0 0x55e119607dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.614+0000 7f99fb400640 1 -- 172.21.15.42:0/988185522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e119604f20 con 0x55e1195ff6a0 2024-04-20T10:43:53.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.614+0000 7f99fa000640 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f6ac0 0x55e119607dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:43:53.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.616+0000 7f99fb400640 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e119604860 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f99e805a040 tx=0x7f99e8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ebe65a2710c3bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:43:53.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.616+0000 7f99faa00640 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e119607890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:43:53.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.616+0000 7f99dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:43:53.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.620+0000 7f99dbe00640 1 -- 172.21.15.42:0/988185522 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e805c040 con 0x55e1195ff6a0 2024-04-20T10:43:53.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.620+0000 7f99dbe00640 1 -- 172.21.15.42:0/988185522 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f99e803d9e0 con 0x55e1195ff6a0 2024-04-20T10:43:53.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.620+0000 7f9a043d9740 1 -- 172.21.15.42:0/988185522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99e803d830 con 0x55e1195ff6a0 2024-04-20T10:43:53.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.620+0000 7f99dbe00640 1 -- 172.21.15.42:0/988185522 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f99e806f050 con 0x55e1195ff6a0 2024-04-20T10:43:53.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.620+0000 7f9a043d9740 10 client.26534 fetch_fsmap learned FSMap version 19 2024-04-20T10:43:53.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.620+0000 7f9a043d9740 1 -- 172.21.15.42:0/988185522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e1196769e0 con 0x55e1195ff6a0 2024-04-20T10:43:53.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.621+0000 7f99dbe00640 1 -- 172.21.15.42:0/988185522 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e8004950 con 0x55e1195ff6a0 2024-04-20T10:43:53.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.623+0000 7f99dbe00640 1 -- 172.21.15.42:0/988185522 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f99e806a030 con 0x55e1195ff6a0 2024-04-20T10:43:53.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.624+0000 7f9a043d9740 10 client.26534 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:43:53.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.624+0000 7f9a043d9740 10 client.26534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:43:53.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.624+0000 7f9a043d9740 20 client.26534 populate_metadata read hostname 'smithi042' 2024-04-20T10:43:53.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.624+0000 7f9a043d9740 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e119676c10 0x55e119679000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:43:53.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.624+0000 7f9a043d9740 4 client.26534 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:43:53.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.624+0000 7f9a043d9740 1 -- 172.21.15.42:0/988185522 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f99e803d830 con 0x55e119676c10 2024-04-20T10:43:53.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.626+0000 7f99faa00640 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e119676c10 0x55e119679000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:43:53.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.626+0000 7f99faa00640 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e119676c10 0x55e119679000 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f2812328308df4 server_cookie=c219c33bca5d3949 in_seq=0 out_seq=0 2024-04-20T10:43:53.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.626+0000 7f99dbe00640 10 client.26534 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:53.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.629+0000 7f99dbe00640 1 -- 172.21.15.42:0/988185522 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x7f99e803d830 con 0x55e119676c10 2024-04-20T10:43:53.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.629+0000 7f99dbe00640 10 client.26534 handle_command_reply: tid=0 2024-04-20T10:43:53.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.630+0000 7f9a043d9740 1 client.26534 shutdown 2024-04-20T10:43:53.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.634+0000 7f9a043d9740 1 -- 172.21.15.42:0/988185522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 msgr2=0x55e119604860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.634+0000 7f9a043d9740 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e119604860 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f99e805a040 tx=0x7f99e8000bc0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.635+0000 7f9a043d9740 1 -- 172.21.15.42:0/988185522 shutdown_connections 2024-04-20T10:43:53.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.635+0000 7f9a043d9740 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55e119676c10 0x55e119679000 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.635+0000 7f9a043d9740 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1195ff6a0 0x55e119604860 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.635+0000 7f9a043d9740 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e1195f6ac0 0x55e119607dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.635+0000 7f9a043d9740 1 --2- 172.21.15.42:0/988185522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e1195f61a0 0x55e119607890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.635+0000 7f99dbe00640 0 client.26534 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:43:53.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.635+0000 7f9a043d9740 1 -- 172.21.15.42:0/988185522 >> 172.21.15.42:0/988185522 conn(0x55e1195f08d0 msgr2=0x55e1195ef030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:53.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.639+0000 7f9a043d9740 1 -- 172.21.15.42:0/988185522 shutdown_connections 2024-04-20T10:43:53.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.639+0000 7f9a043d9740 1 -- 172.21.15.42:0/988185522 wait complete. 2024-04-20T10:43:53.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.639+0000 7f9a043d9740 20 client.26534 trim_cache size 0 max 16384 2024-04-20T10:43:53.641 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:43:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (16990 inodes in the stack)", 2024-04-20T10:43:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:43:53.641 INFO:teuthology.orchestra.run.smithi042.stdout: "36820ea4-a691-45ca-9778-729e7058e196": { 2024-04-20T10:43:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:43:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "36820ea4-a691-45ca-9778-729e7058e196", 2024-04-20T10:43:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:43:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:43:53.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:43:53.642 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:43:53.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.642+0000 7f9a01600640 1 -- 172.21.15.42:0/3395665382 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f99e00a3ef0 msgr2=0x7f99e00a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.642+0000 7f9a01600640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f99e00a3ef0 0x7f99e00a63b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55e1195f5c70 tx=0x7f99f4030040 comp rx=0 tx=0).stop 2024-04-20T10:43:53.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.643+0000 7f9a01600640 1 -- 172.21.15.42:0/3395665382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99fc1107b0 msgr2=0x7f99fc116f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:43:53.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.643+0000 7f9a01600640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99fc1107b0 0x7f99fc116f30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00c970 tx=0x7f99ec00ce30 comp rx=0 tx=0).stop 2024-04-20T10:43:53.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.644+0000 7f9a01600640 1 -- 172.21.15.42:0/3395665382 shutdown_connections 2024-04-20T10:43:53.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.644+0000 7f9a01600640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f99e00a3ef0 0x7f99e00a63b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.644+0000 7f9a01600640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99fc11b740 0x7f99fc117470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.644+0000 7f9a01600640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99fc1107b0 0x7f99fc116f30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.644+0000 7f9a01600640 1 --2- 172.21.15.42:0/3395665382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99fc10e7e0 0x7f99fc11ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:43:53.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.644+0000 7f9a01600640 1 -- 172.21.15.42:0/3395665382 >> 172.21.15.42:0/3395665382 conn(0x7f99fc06d070 msgr2=0x7f99fc072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:43:53.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.645+0000 7f9a01600640 1 -- 172.21.15.42:0/3395665382 shutdown_connections 2024-04-20T10:43:53.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:43:53.646+0000 7f9a01600640 1 -- 172.21.15.42:0/3395665382 wait complete. 2024-04-20T10:43:53.684 INFO:tasks.cephfs.filesystem:scrub status for tag:36820ea4-a691-45ca-9778-729e7058e196 - {'path': '/', 'tag': '36820ea4-a691-45ca-9778-729e7058e196', 'options': 'recursive,force'} 2024-04-20T10:43:53.873 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/ctr.o 2024-04-20T10:43:53.876 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/module/strict_rwx.o 2024-04-20T10:43:53.928 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:53 smithi042 ceph-mon[17078]: pgmap v305: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.5 MiB/s wr, 65 op/s 2024-04-20T10:43:53.928 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:53 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:43:53.928 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:53 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:43:53.928 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:53 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:43:54.224 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/srat.o 2024-04-20T10:43:54.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:53 smithi046 ceph-mon[21574]: pgmap v305: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.5 MiB/s wr, 65 op/s 2024-04-20T10:43:54.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:53 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:43:54.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:53 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:43:54.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:53 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:43:54.350 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/netfs/built-in.a 2024-04-20T10:43:54.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:53 smithi162 ceph-mon[21854]: pgmap v305: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.5 MiB/s wr, 65 op/s 2024-04-20T10:43:54.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:53 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:43:54.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:53 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:43:54.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:53 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:43:54.443 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/balloc.o 2024-04-20T10:43:54.542 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-core.o 2024-04-20T10:43:54.594 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/mremap.o 2024-04-20T10:43:54.615 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/gcm.o 2024-04-20T10:43:54.794 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/module/kmod.o 2024-04-20T10:43:55.200 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/ss/conditional.o 2024-04-20T10:43:55.236 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/pkeys.o 2024-04-20T10:43:55.960 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/module/tree_lookup.o 2024-04-20T10:43:56.056 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/ccm.o 2024-04-20T10:43:56.204 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/kaslr.o 2024-04-20T10:43:56.206 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/msync.o 2024-04-20T10:43:56.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:55 smithi046 ceph-mon[21574]: pgmap v306: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.4 MiB/s wr, 62 op/s 2024-04-20T10:43:56.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:55 smithi042 ceph-mon[17078]: pgmap v306: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.4 MiB/s wr, 62 op/s 2024-04-20T10:43:56.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:55 smithi162 ceph-mon[21854]: pgmap v306: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.4 MiB/s wr, 62 op/s 2024-04-20T10:43:56.498 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/bitmap.o 2024-04-20T10:43:56.836 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/module/kallsyms.o 2024-04-20T10:43:56.875 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/ss/mls.o 2024-04-20T10:43:56.940 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-sysfs.o 2024-04-20T10:43:57.138 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/mm/pti.o 2024-04-20T10:43:57.245 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/page_vma_mapped.o 2024-04-20T10:43:57.424 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/aes_generic.o 2024-04-20T10:43:57.601 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/block_validity.o 2024-04-20T10:43:57.956 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/module/procfs.o 2024-04-20T10:43:58.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:57 smithi046 ceph-mon[21574]: pgmap v307: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.4 MiB/s wr, 248 op/s 2024-04-20T10:43:58.339 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-flush.o 2024-04-20T10:43:58.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:57 smithi042 ceph-mon[17078]: pgmap v307: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.4 MiB/s wr, 248 op/s 2024-04-20T10:43:58.368 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/mm/built-in.a 2024-04-20T10:43:58.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:57 smithi162 ceph-mon[21854]: pgmap v307: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.4 MiB/s wr, 248 op/s 2024-04-20T10:43:58.473 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/pagewalk.o 2024-04-20T10:43:58.530 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/dir.o 2024-04-20T10:43:58.630 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/ss/context.o 2024-04-20T10:43:58.706 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/crc32c_generic.o 2024-04-20T10:43:58.851 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/module/sysfs.o 2024-04-20T10:43:58.866 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/acpi/boot.o 2024-04-20T10:43:59.449 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/authenc.o 2024-04-20T10:43:59.535 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-settings.o 2024-04-20T10:43:59.820 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/pgtable-generic.o 2024-04-20T10:43:59.874 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/module/built-in.a 2024-04-20T10:43:59.956 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/ext4_jbd2.o 2024-04-20T10:43:59.994 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/time.o 2024-04-20T10:44:00.066 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/selinux/netlabel.o 2024-04-20T10:44:00.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:43:59 smithi046 ceph-mon[21574]: pgmap v308: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 640 op/s 2024-04-20T10:44:00.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:43:59 smithi042 ceph-mon[17078]: pgmap v308: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 640 op/s 2024-04-20T10:44:00.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:43:59 smithi162 ceph-mon[21854]: pgmap v308: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 640 op/s 2024-04-20T10:44:00.549 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/acpi/sleep.o 2024-04-20T10:44:00.823 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/authencesn.o 2024-04-20T10:44:00.853 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-ioc.o 2024-04-20T10:44:00.867 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/rmap.o 2024-04-20T10:44:01.286 INFO:tasks.workunit.client.0.smithi042.stderr:./arch/x86/kernel/cpu/mkcapflags.sh: line 43: printf: write error: Resource temporarily unavailable 2024-04-20T10:44:01.301 INFO:tasks.workunit.client.0.smithi042.stderr:make[5]: *** [arch/x86/kernel/cpu/Makefile:64: arch/x86/kernel/cpu/capflags.c] Error 1 2024-04-20T10:44:01.302 INFO:tasks.workunit.client.0.smithi042.stderr:make[4]: *** [scripts/Makefile.build:480: arch/x86/kernel/cpu] Error 2 2024-04-20T10:44:01.302 INFO:tasks.workunit.client.0.smithi042.stderr:make[4]: *** Waiting for unfinished jobs.... 2024-04-20T10:44:01.309 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/commoncap.o 2024-04-20T10:44:01.386 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/timer.o 2024-04-20T10:44:01.700 INFO:tasks.workunit.client.0.smithi042.stdout: AS arch/x86/kernel/acpi/wakeup_64.o 2024-04-20T10:44:01.794 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/acpi/cppc.o 2024-04-20T10:44:01.974 INFO:tasks.workunit.client.0.smithi042.stdout: AR security/selinux/built-in.a 2024-04-20T10:44:01.987 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-map.o 2024-04-20T10:44:01.998 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/extents.o 2024-04-20T10:44:02.062 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/min_addr.o 2024-04-20T10:44:02.174 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/rng.o 2024-04-20T10:44:02.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:01 smithi046 ceph-mon[21574]: pgmap v309: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 381 KiB/s wr, 606 op/s 2024-04-20T10:44:02.356 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:01 smithi042 ceph-mon[17078]: pgmap v309: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 381 KiB/s wr, 606 op/s 2024-04-20T10:44:02.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:01 smithi162 ceph-mon[21854]: pgmap v309: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 381 KiB/s wr, 606 op/s 2024-04-20T10:44:02.697 INFO:tasks.workunit.client.0.smithi042.stdout: CC arch/x86/kernel/acpi/cstate.o 2024-04-20T10:44:02.947 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/io_uring.o 2024-04-20T10:44:03.018 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/security.o 2024-04-20T10:44:03.142 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/vmalloc.o 2024-04-20T10:44:03.349 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-merge.o 2024-04-20T10:44:03.411 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/drbg.o 2024-04-20T10:44:03.553 INFO:tasks.workunit.client.0.smithi042.stdout: AR arch/x86/kernel/acpi/built-in.a 2024-04-20T10:44:03.633 INFO:tasks.workunit.client.0.smithi042.stderr:make[3]: *** [scripts/Makefile.build:480: arch/x86/kernel] Error 2 2024-04-20T10:44:03.633 INFO:tasks.workunit.client.0.smithi042.stderr:make[2]: *** [scripts/Makefile.build:480: arch/x86] Error 2 2024-04-20T10:44:03.633 INFO:tasks.workunit.client.0.smithi042.stderr:make[2]: *** Waiting for unfinished jobs.... 2024-04-20T10:44:03.643 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/lsm_audit.o 2024-04-20T10:44:03.718 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/hrtimer.o 2024-04-20T10:44:04.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:03 smithi046 ceph-mon[21574]: pgmap v310: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 554 KiB/s wr, 2.06k op/s 2024-04-20T10:44:04.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:03 smithi042 ceph-mon[17078]: pgmap v310: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 554 KiB/s wr, 2.06k op/s 2024-04-20T10:44:04.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:03 smithi162 ceph-mon[21854]: pgmap v310: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 554 KiB/s wr, 2.06k op/s 2024-04-20T10:44:04.743 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/jitterentropy.o 2024-04-20T10:44:04.894 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/jitterentropy-kcapi.o 2024-04-20T10:44:05.176 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-timeout.o 2024-04-20T10:44:05.501 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/extents_status.o 2024-04-20T10:44:05.546 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/timekeeping.o 2024-04-20T10:44:05.776 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/ghash-generic.o 2024-04-20T10:44:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:05 smithi046 ceph-mon[21574]: pgmap v311: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 270 KiB/s wr, 2.25k op/s 2024-04-20T10:44:06.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:05 smithi042 ceph-mon[17078]: pgmap v311: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 270 KiB/s wr, 2.25k op/s 2024-04-20T10:44:06.493 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:05 smithi162 ceph-mon[21854]: pgmap v311: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 270 KiB/s wr, 2.25k op/s 2024-04-20T10:44:06.611 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-lib.o 2024-04-20T10:44:06.633 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/hash_info.o 2024-04-20T10:44:06.642 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-mq.o 2024-04-20T10:44:06.644 INFO:tasks.workunit.client.0.smithi042.stdout: CC security/device_cgroup.o 2024-04-20T10:44:06.838 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/process_vm_access.o 2024-04-20T10:44:06.934 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/rsapubkey.asn1.o 2024-04-20T10:44:07.116 INFO:tasks.workunit.client.0.smithi042.stdout: CC crypto/rsaprivkey.asn1.o 2024-04-20T10:44:07.237 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/ntp.o 2024-04-20T10:44:07.267 INFO:tasks.workunit.client.0.smithi042.stdout: AR crypto/built-in.a 2024-04-20T10:44:07.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:07 smithi046 ceph-mon[21574]: osdmap e95: 12 total, 12 up, 12 in 2024-04-20T10:44:07.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:06 smithi042 ceph-mon[17078]: osdmap e95: 12 total, 12 up, 12 in 2024-04-20T10:44:07.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:06 smithi162 ceph-mon[21854]: osdmap e95: 12 total, 12 up, 12 in 2024-04-20T10:44:07.442 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/futex/core.o 2024-04-20T10:44:07.813 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/xattr.o 2024-04-20T10:44:08.075 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/jbd2/transaction.o 2024-04-20T10:44:08.114 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/file.o 2024-04-20T10:44:08.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:08 smithi046 ceph-mon[21574]: pgmap v313: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 673 KiB/s wr, 3.31k op/s 2024-04-20T10:44:08.311 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/page_alloc.o 2024-04-20T10:44:08.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:08 smithi042 ceph-mon[17078]: pgmap v313: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 673 KiB/s wr, 3.31k op/s 2024-04-20T10:44:08.376 INFO:tasks.workunit.client.0.smithi042.stdout: AR security/built-in.a 2024-04-20T10:44:08.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:08 smithi162 ceph-mon[21854]: pgmap v313: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 673 KiB/s wr, 3.31k op/s 2024-04-20T10:44:08.482 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ramfs/inode.o 2024-04-20T10:44:08.486 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/clocksource.o 2024-04-20T10:44:08.707 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/futex/syscalls.o 2024-04-20T10:44:09.079 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/nop.o 2024-04-20T10:44:09.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:09 smithi046 ceph-mon[21574]: pgmap v314: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 4.46k op/s 2024-04-20T10:44:09.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:09 smithi042 ceph-mon[17078]: pgmap v314: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 4.46k op/s 2024-04-20T10:44:09.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:09 smithi162 ceph-mon[21854]: pgmap v314: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 4.46k op/s 2024-04-20T10:44:09.831 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ramfs/file-mmu.o 2024-04-20T10:44:10.011 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/jiffies.o 2024-04-20T10:44:10.077 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/fsmap.o 2024-04-20T10:44:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:10 smithi046 ceph-mon[21574]: osdmap e96: 12 total, 12 up, 12 in 2024-04-20T10:44:10.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:10 smithi042 ceph-mon[17078]: osdmap e96: 12 total, 12 up, 12 in 2024-04-20T10:44:10.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:10 smithi162 ceph-mon[21854]: osdmap e96: 12 total, 12 up, 12 in 2024-04-20T10:44:10.604 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-mq-tag.o 2024-04-20T10:44:10.608 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/jbd2/commit.o 2024-04-20T10:44:10.614 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/timer_list.o 2024-04-20T10:44:10.618 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/fs.o 2024-04-20T10:44:10.622 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/futex/pi.o 2024-04-20T10:44:10.635 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/ramfs/built-in.a 2024-04-20T10:44:10.744 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cgroup/cgroup.o 2024-04-20T10:44:11.261 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/init-mm.o 2024-04-20T10:44:11.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:11 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:44:11.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:11 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:44:11.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:11 smithi046 ceph-mon[21574]: pgmap v316: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 3.39k op/s 2024-04-20T10:44:11.348 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/futex/requeue.o 2024-04-20T10:44:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:11 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:44:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:11 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:44:11.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:11 smithi042 ceph-mon[17078]: pgmap v316: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 3.39k op/s 2024-04-20T10:44:11.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:11 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:44:11.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:11 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:44:11.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:11 smithi162 ceph-mon[21854]: pgmap v316: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 3.39k op/s 2024-04-20T10:44:11.457 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/timeconv.o 2024-04-20T10:44:11.476 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/fsync.o 2024-04-20T10:44:11.588 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/splice.o 2024-04-20T10:44:11.778 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-stat.o 2024-04-20T10:44:12.198 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/timecounter.o 2024-04-20T10:44:12.206 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/jbd2/recovery.o 2024-04-20T10:44:12.208 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/futex/waitwake.o 2024-04-20T10:44:12.217 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/memblock.o 2024-04-20T10:44:12.295 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/alarmtimer.o 2024-04-20T10:44:12.312 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/sync.o 2024-04-20T10:44:12.498 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-mq-sysfs.o 2024-04-20T10:44:12.709 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/hash.o 2024-04-20T10:44:12.908 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/futex/built-in.a 2024-04-20T10:44:12.993 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/ialloc.o 2024-04-20T10:44:13.247 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/advise.o 2024-04-20T10:44:13.292 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/jbd2/checkpoint.o 2024-04-20T10:44:13.376 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-mq-cpumap.o 2024-04-20T10:44:13.514 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/posix-timers.o 2024-04-20T10:44:13.565 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/madvise.o 2024-04-20T10:44:13.647 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cgroup/rstat.o 2024-04-20T10:44:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:13 smithi046 ceph-mon[21574]: pgmap v317: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 4.34k op/s 2024-04-20T10:44:14.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:13 smithi042 ceph-mon[17078]: pgmap v317: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 4.34k op/s 2024-04-20T10:44:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:13 smithi162 ceph-mon[21854]: pgmap v317: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 4.34k op/s 2024-04-20T10:44:14.354 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/filetable.o 2024-04-20T10:44:14.392 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-mq-sched.o 2024-04-20T10:44:14.431 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/hugetlbfs/inode.o 2024-04-20T10:44:14.498 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/jbd2/revoke.o 2024-04-20T10:44:14.814 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cgroup/namespace.o 2024-04-20T10:44:15.097 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/posix-cpu-timers.o 2024-04-20T10:44:15.201 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/page_io.o 2024-04-20T10:44:15.269 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/indirect.o 2024-04-20T10:44:15.449 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/jbd2/journal.o 2024-04-20T10:44:15.486 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/ioctl.o 2024-04-20T10:44:15.595 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/openclose.o 2024-04-20T10:44:15.599 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cgroup/cgroup-v1.o 2024-04-20T10:44:15.749 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/hugetlbfs/built-in.a 2024-04-20T10:44:15.815 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_clock.o 2024-04-20T10:44:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:15 smithi046 ceph-mon[21574]: pgmap v318: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.8 MiB/s wr, 4.11k op/s 2024-04-20T10:44:16.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:15 smithi042 ceph-mon[17078]: pgmap v318: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.8 MiB/s wr, 4.11k op/s 2024-04-20T10:44:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:15 smithi162 ceph-mon[21854]: pgmap v318: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.8 MiB/s wr, 4.11k op/s 2024-04-20T10:44:16.236 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/posix-clock.o 2024-04-20T10:44:16.309 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/swap_state.o 2024-04-20T10:44:16.339 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/ring_buffer.o 2024-04-20T10:44:16.506 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/genhd.o 2024-04-20T10:44:16.833 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cgroup/freezer.o 2024-04-20T10:44:16.876 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/uring_cmd.o 2024-04-20T10:44:17.031 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/inline.o 2024-04-20T10:44:17.099 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/itimer.o 2024-04-20T10:44:17.522 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/swapfile.o 2024-04-20T10:44:17.749 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cgroup/legacy_freezer.o 2024-04-20T10:44:17.751 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/ioprio.o 2024-04-20T10:44:17.759 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/jbd2/built-in.a 2024-04-20T10:44:17.840 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/epoll.o 2024-04-20T10:44:18.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:17 smithi046 ceph-mon[21574]: pgmap v319: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 2.77k op/s 2024-04-20T10:44:18.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:17 smithi042 ceph-mon[17078]: pgmap v319: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 2.77k op/s 2024-04-20T10:44:18.133 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/clockevents.o 2024-04-20T10:44:18.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:17 smithi162 ceph-mon[21854]: pgmap v319: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 2.77k op/s 2024-04-20T10:44:18.347 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace.o 2024-04-20T10:44:18.401 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/bpf/core.o 2024-04-20T10:44:18.576 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cgroup/pids.o 2024-04-20T10:44:18.598 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/inode.o 2024-04-20T10:44:18.693 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/badblocks.o 2024-04-20T10:44:18.709 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/statx.o 2024-04-20T10:44:18.905 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/tick-common.o 2024-04-20T10:44:19.303 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cgroup/rdma.o 2024-04-20T10:44:19.423 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-rq-qos.o 2024-04-20T10:44:19.609 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/net.o 2024-04-20T10:44:19.661 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/swap_slots.o 2024-04-20T10:44:19.954 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/tick-broadcast.o 2024-04-20T10:44:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:19 smithi046 ceph-mon[21574]: pgmap v320: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 376 KiB/s wr, 1.25k op/s 2024-04-20T10:44:20.102 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cgroup/cpuset.o 2024-04-20T10:44:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:19 smithi042 ceph-mon[17078]: pgmap v320: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 376 KiB/s wr, 1.25k op/s 2024-04-20T10:44:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:19 smithi162 ceph-mon[21854]: pgmap v320: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 376 KiB/s wr, 1.25k op/s 2024-04-20T10:44:20.291 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/disk-events.o 2024-04-20T10:44:20.521 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/dmapool.o 2024-04-20T10:44:20.630 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/bpf/built-in.a 2024-04-20T10:44:20.678 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-ia-ranges.o 2024-04-20T10:44:20.764 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/tick-broadcast-hrtimer.o 2024-04-20T10:44:21.232 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fat/cache.o 2024-04-20T10:44:21.337 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/tick-oneshot.o 2024-04-20T10:44:21.366 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/hugetlb.o 2024-04-20T10:44:21.380 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/msg_ring.o 2024-04-20T10:44:21.553 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/early-lookup.o 2024-04-20T10:44:21.638 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_output.o 2024-04-20T10:44:21.934 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/tick-sched.o 2024-04-20T10:44:21.951 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:21 smithi162 ceph-mon[21854]: pgmap v321: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 325 KiB/s wr, 1.08k op/s 2024-04-20T10:44:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:21 smithi046 ceph-mon[21574]: pgmap v321: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 325 KiB/s wr, 1.08k op/s 2024-04-20T10:44:22.110 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:21 smithi042 ceph-mon[17078]: pgmap v321: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 325 KiB/s wr, 1.08k op/s 2024-04-20T10:44:22.197 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cgroup/misc.o 2024-04-20T10:44:22.228 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fat/dir.o 2024-04-20T10:44:22.231 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/ioctl.o 2024-04-20T10:44:22.498 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/bsg.o 2024-04-20T10:44:22.800 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/timeout.o 2024-04-20T10:44:22.909 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cgroup/debug.o 2024-04-20T10:44:23.108 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/vsyscall.o 2024-04-20T10:44:23.200 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_seq.o 2024-04-20T10:44:23.390 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-cgroup.o 2024-04-20T10:44:23.632 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fat/fatent.o 2024-04-20T10:44:23.650 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/timekeeping_debug.o 2024-04-20T10:44:23.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:44:23.686 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:44:23.716 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/cgroup/built-in.a 2024-04-20T10:44:23.920 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/sqpoll.o 2024-04-20T10:44:23.924 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_stat.o 2024-04-20T10:44:23.966 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_printk.o 2024-04-20T10:44:24.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.044+0000 7ff70ae00640 1 -- 172.21.15.42:0/988250407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7041150b0 msgr2=0x7ff7041174a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:24.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.044+0000 7ff70ae00640 1 --2- 172.21.15.42:0/988250407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7041150b0 0x7ff7041174a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009900 tx=0x7ff6fc02bc50 comp rx=0 tx=0).stop 2024-04-20T10:44:24.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.048+0000 7ff70ae00640 1 -- 172.21.15.42:0/988250407 shutdown_connections 2024-04-20T10:44:24.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.048+0000 7ff70ae00640 1 --2- 172.21.15.42:0/988250407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7041150b0 0x7ff7041174a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.048+0000 7ff70ae00640 1 --2- 172.21.15.42:0/988250407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff704109880 0x7ff704114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.048+0000 7ff70ae00640 1 --2- 172.21.15.42:0/988250407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff704108f60 0x7ff704109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.048+0000 7ff70ae00640 1 -- 172.21.15.42:0/988250407 >> 172.21.15.42:0/988250407 conn(0x7ff70406cee0 msgr2=0x7ff70406d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:24.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.048+0000 7ff70ae00640 1 -- 172.21.15.42:0/988250407 shutdown_connections 2024-04-20T10:44:24.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.048+0000 7ff70ae00640 1 -- 172.21.15.42:0/988250407 wait complete. 2024-04-20T10:44:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.049+0000 7ff70ae00640 1 Processor -- start 2024-04-20T10:44:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.049+0000 7ff70ae00640 1 -- start start 2024-04-20T10:44:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.050+0000 7ff70ae00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff704108f60 0x7ff7041a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.050+0000 7ff70ae00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff704109880 0x7ff7041a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.050+0000 7ff70ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7041150b0 0x7ff7041a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.050+0000 7ff70ae00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff70411aa90 con 0x7ff704109880 2024-04-20T10:44:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.050+0000 7ff70ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff70411a910 con 0x7ff7041150b0 2024-04-20T10:44:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.050+0000 7ff70ae00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff70411ac10 con 0x7ff704108f60 2024-04-20T10:44:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.051+0000 7ff70a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7041150b0 0x7ff7041a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.051+0000 7ff709a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff704108f60 0x7ff7041a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.051+0000 7ff70a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7041150b0 0x7ff7041a5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:34426/0 (socket says 172.21.15.42:34426) 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.051+0000 7ff70a400640 1 -- 172.21.15.42:0/2985345497 learned_addr learned my addr 172.21.15.42:0/2985345497 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.051+0000 7ff709000640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff704109880 0x7ff7041a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.051+0000 7ff70a400640 1 -- 172.21.15.42:0/2985345497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff704108f60 msgr2=0x7ff7041a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.051+0000 7ff70a400640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff704108f60 0x7ff7041a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.051+0000 7ff70a400640 1 -- 172.21.15.42:0/2985345497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff704109880 msgr2=0x7ff7041a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.051+0000 7ff70a400640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff704109880 0x7ff7041a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.051+0000 7ff70a400640 1 -- 172.21.15.42:0/2985345497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7041a5fc0 con 0x7ff7041150b0 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.051+0000 7ff709a00640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff704108f60 0x7ff7041a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.051+0000 7ff709000640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff704109880 0x7ff7041a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.052+0000 7ff70a400640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7041150b0 0x7ff7041a5890 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009850 tx=0x7ff6fc0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.052+0000 7ff702000640 1 -- 172.21.15.42:0/2985345497 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc041020 con 0x7ff7041150b0 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.052+0000 7ff702000640 1 -- 172.21.15.42:0/2985345497 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6fc005310 con 0x7ff7041150b0 2024-04-20T10:44:24.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.052+0000 7ff702000640 1 -- 172.21.15.42:0/2985345497 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc003e60 con 0x7ff7041150b0 2024-04-20T10:44:24.055 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:23 smithi046 ceph-mon[21574]: pgmap v322: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 813 KiB/s wr, 1.06k op/s 2024-04-20T10:44:24.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.054+0000 7ff70ae00640 1 -- 172.21.15.42:0/2985345497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7041a6250 con 0x7ff7041150b0 2024-04-20T10:44:24.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.054+0000 7ff70ae00640 1 -- 172.21.15.42:0/2985345497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70410a900 con 0x7ff7041150b0 2024-04-20T10:44:24.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.062+0000 7ff70f1ef740 1 Processor -- start 2024-04-20T10:44:24.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.067+0000 7ff702000640 1 -- 172.21.15.42:0/2985345497 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff6fc044020 con 0x7ff7041150b0 2024-04-20T10:44:24.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.067+0000 7ff70f1ef740 1 -- start start 2024-04-20T10:44:24.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.067+0000 7ff70f1ef740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add949380 0x559add949760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.067+0000 7ff70f1ef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949ca0 0x559add952340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.067+0000 7ff70f1ef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add952880 0x559add954c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.067+0000 7ff70f1ef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559add8af150 con 0x559add952880 2024-04-20T10:44:24.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.067+0000 7ff70f1ef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559add855370 con 0x559add949ca0 2024-04-20T10:44:24.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.067+0000 7ff70f1ef740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559add8ab110 con 0x559add949380 2024-04-20T10:44:24.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.068+0000 7ff709a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949ca0 0x559add952340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.068+0000 7ff709a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949ca0 0x559add952340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:34430/0 (socket says 172.21.15.42:34430) 2024-04-20T10:44:24.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.068+0000 7ff709a00640 1 -- 172.21.15.42:0/799543940 learned_addr learned my addr 172.21.15.42:0/799543940 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:24.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.068+0000 7ff709a00640 1 -- 172.21.15.42:0/799543940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add949380 msgr2=0x559add949760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:44:24.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.068+0000 7ff709a00640 1 --2- 172.21.15.42:0/799543940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add949380 0x559add949760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.068+0000 7ff709a00640 1 -- 172.21.15.42:0/799543940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add952880 msgr2=0x559add954c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:44:24.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.068+0000 7ff709000640 1 --2- 172.21.15.42:0/799543940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add952880 0x559add954c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.069+0000 7ff709a00640 1 --2- 172.21.15.42:0/799543940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add952880 0x559add954c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.069+0000 7ff709a00640 1 -- 172.21.15.42:0/799543940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559add68b7d0 con 0x559add949ca0 2024-04-20T10:44:24.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.069+0000 7ff709000640 1 --2- 172.21.15.42:0/799543940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add952880 0x559add954c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:44:24.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.069+0000 7ff709a00640 1 --2- 172.21.15.42:0/799543940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949ca0 0x559add952340 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800c4a0 tx=0x7ff6f800c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddc17dc29d4c9753 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:24.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.071+0000 7ff702000640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff6ec0a3de0 0x7ff6ec0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.073+0000 7ff6f6a00640 1 -- 172.21.15.42:0/799543940 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f802b070 con 0x559add949ca0 2024-04-20T10:44:24.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.073+0000 7ff6f6a00640 1 -- 172.21.15.42:0/799543940 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6f8026900 con 0x559add949ca0 2024-04-20T10:44:24.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.073+0000 7ff6f6a00640 1 -- 172.21.15.42:0/799543940 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800a9f0 con 0x559add949ca0 2024-04-20T10:44:24.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.073+0000 7ff702000640 1 -- 172.21.15.42:0/2985345497 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7ff6fc0f1dd0 con 0x7ff7041150b0 2024-04-20T10:44:24.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.073+0000 7ff709000640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff6ec0a3de0 0x7ff6ec0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.074+0000 7ff70f1ef740 1 -- 172.21.15.42:0/799543940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949ca0 msgr2=0x559add952340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:24.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.074+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/799543940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949ca0 0x559add952340 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800c4a0 tx=0x7ff6f800c8a0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.074+0000 7ff709000640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff6ec0a3de0 0x7ff6ec0a62a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0006fd0 tx=0x7ff6f0008040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:24.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.076+0000 7ff70f1ef740 1 -- 172.21.15.42:0/799543940 shutdown_connections 2024-04-20T10:44:24.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.076+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/799543940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add952880 0x559add954c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.076+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/799543940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949ca0 0x559add952340 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.076+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/799543940 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add949380 0x559add949760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.076+0000 7ff70f1ef740 1 -- 172.21.15.42:0/799543940 >> 172.21.15.42:0/799543940 conn(0x559add943ab0 msgr2=0x559add943e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:24.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.078+0000 7ff70f1ef740 1 -- 172.21.15.42:0/799543940 shutdown_connections 2024-04-20T10:44:24.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.078+0000 7ff70f1ef740 1 -- 172.21.15.42:0/799543940 wait complete. 2024-04-20T10:44:24.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.079+0000 7ff70f1ef740 1 -- start start 2024-04-20T10:44:24.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.079+0000 7ff70f1ef740 1 Processor -- start 2024-04-20T10:44:24.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.083+0000 7ff70f1ef740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add949380 0x559add94f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.083+0000 7ff70f1ef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add94fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.083+0000 7ff70f1ef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add952880 0x559add950320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.083+0000 7ff70f1ef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559add955670 con 0x559add949ca0 2024-04-20T10:44:24.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.083+0000 7ff70f1ef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559add868740 con 0x559add952880 2024-04-20T10:44:24.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.083+0000 7ff70f1ef740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559add9557f0 con 0x559add949380 2024-04-20T10:44:24.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.083+0000 7ff70f1ef740 10 client.0 _ll_register_callbacks cb 0x559add863b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:44:24.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.084+0000 7ff70a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add952880 0x559add950320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.084+0000 7ff70a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add952880 0x559add950320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:34444/0 (socket says 172.21.15.42:34444) 2024-04-20T10:44:24.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.084+0000 7ff70a400640 1 -- 172.21.15.42:0/222369925 learned_addr learned my addr 172.21.15.42:0/222369925 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:24.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.084+0000 7ff70a400640 1 -- 172.21.15.42:0/222369925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add949380 msgr2=0x559add94f8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:44:24.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.084+0000 7ff709000640 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add94fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.084+0000 7ff70a400640 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add949380 0x559add94f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.084+0000 7ff70a400640 1 -- 172.21.15.42:0/222369925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 msgr2=0x559add94fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:24.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.084+0000 7ff70a400640 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add94fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.084+0000 7ff70a400640 1 -- 172.21.15.42:0/222369925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559add5c13f0 con 0x559add952880 2024-04-20T10:44:24.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.084+0000 7ff709000640 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add94fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:44:24.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.085+0000 7ff70a400640 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add952880 0x559add950320 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0ba8b0 tx=0x7ff6fc0be6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb95205188a8e041 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.086+0000 7ff6eb400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:44:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.086+0000 7ff6eb400640 1 -- 172.21.15.42:0/222369925 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0c2290 con 0x559add952880 2024-04-20T10:44:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.087+0000 7ff6eb400640 1 -- 172.21.15.42:0/222369925 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7ff6fc0d72e0 con 0x559add952880 2024-04-20T10:44:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.087+0000 7ff70f1ef740 1 -- 172.21.15.42:0/222369925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6f8026900 con 0x559add952880 2024-04-20T10:44:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.087+0000 7ff6eb400640 1 -- 172.21.15.42:0/222369925 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff6fc0c6ac0 con 0x559add952880 2024-04-20T10:44:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.087+0000 7ff70f1ef740 10 client.26546 fetch_fsmap learned FSMap version 19 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.087+0000 7ff70f1ef740 1 -- 172.21.15.42:0/222369925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559add5810f0 con 0x559add952880 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.087+0000 7ff6eb400640 1 -- 172.21.15.42:0/222369925 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6f8026900 con 0x559add952880 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.087+0000 7ff6eb400640 1 -- 172.21.15.42:0/222369925 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7ff6fc0c66c0 con 0x559add952880 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.088+0000 7ff70f1ef740 10 client.26546 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.088+0000 7ff70f1ef740 10 client.26546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.088+0000 7ff70f1ef740 20 client.26546 populate_metadata read hostname 'smithi042' 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.088+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559add962b70 0x559add964ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.088+0000 7ff70f1ef740 4 client.26546 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.088+0000 7ff70f1ef740 1 -- 172.21.15.42:0/222369925 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559add8bcb70 con 0x559add962b70 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.088+0000 7ff709a00640 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559add962b70 0x559add964ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.089+0000 7ff709a00640 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559add962b70 0x559add964ff0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b0d52f0834d6532 server_cookie=eccb31076cd6c7b2 in_seq=0 out_seq=0 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.089+0000 7ff6eb400640 10 client.26546 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.089+0000 7ff6eb400640 1 -- 172.21.15.42:0/222369925 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559add8bcb70 con 0x559add962b70 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.089+0000 7ff6eb400640 10 client.26546 handle_command_reply: tid=0 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.090+0000 7ff70f1ef740 1 client.26546 shutdown 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.090+0000 7ff70f1ef740 1 -- 172.21.15.42:0/222369925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add952880 msgr2=0x559add950320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.090+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add952880 0x559add950320 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0ba8b0 tx=0x7ff6fc0be6a0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.090+0000 7ff70f1ef740 1 -- 172.21.15.42:0/222369925 shutdown_connections 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.090+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559add962b70 0x559add964ff0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.090+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add952880 0x559add950320 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.090+0000 7ff6eb400640 0 client.26546 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.090+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add94fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.090+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/222369925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add949380 0x559add94f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.090+0000 7ff70f1ef740 1 -- 172.21.15.42:0/222369925 >> 172.21.15.42:0/222369925 conn(0x559add943ab0 msgr2=0x559add945800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.090+0000 7ff70f1ef740 1 -- 172.21.15.42:0/222369925 shutdown_connections 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.091+0000 7ff70f1ef740 1 -- 172.21.15.42:0/222369925 wait complete. 2024-04-20T10:44:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.091+0000 7ff70f1ef740 20 client.26546 trim_cache size 0 max 16384 2024-04-20T10:44:24.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:23 smithi042 ceph-mon[17078]: pgmap v322: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 813 KiB/s wr, 1.06k op/s 2024-04-20T10:44:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.140+0000 7ff70f1ef740 1 Processor -- start 2024-04-20T10:44:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.140+0000 7ff70f1ef740 1 -- start start 2024-04-20T10:44:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.140+0000 7ff70f1ef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949380 0x559add951fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.141+0000 7ff70f1ef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add95e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.141+0000 7ff709000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add95e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.142+0000 7ff709000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add95e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33100/0 (socket says 172.21.15.42:33100) 2024-04-20T10:44:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.142+0000 7ff709a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949380 0x559add951fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.142+0000 7ff709000640 1 -- 172.21.15.42:0/2154444055 learned_addr learned my addr 172.21.15.42:0/2154444055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.142+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/2154444055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add952880 0x559add96a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.142+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2154444055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff70407e8c0 con 0x559add949ca0 2024-04-20T10:44:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.142+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2154444055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff704081120 con 0x559add949380 2024-04-20T10:44:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.142+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2154444055 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff70407ef10 con 0x559add952880 2024-04-20T10:44:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.142+0000 7ff70a400640 1 --2- 172.21.15.42:0/2154444055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add952880 0x559add96a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.143+0000 7ff709a00640 1 -- 172.21.15.42:0/2154444055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add952880 msgr2=0x559add96a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.143+0000 7ff709a00640 1 --2- 172.21.15.42:0/2154444055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add952880 0x559add96a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.143+0000 7ff709a00640 1 -- 172.21.15.42:0/2154444055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 msgr2=0x559add95e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.143+0000 7ff709a00640 1 --2- 172.21.15.42:0/2154444055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add95e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.143+0000 7ff709a00640 1 -- 172.21.15.42:0/2154444055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559add949ad0 con 0x559add949380 2024-04-20T10:44:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.143+0000 7ff709000640 1 --2- 172.21.15.42:0/2154444055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add95e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:44:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.144+0000 7ff709a00640 1 --2- 172.21.15.42:0/2154444055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949380 0x559add951fd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800c470 tx=0x7ff6f800a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb36f30bbc4432b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:24.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.144+0000 7ff6f6000640 1 -- 172.21.15.42:0/2154444055 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f802b040 con 0x559add949380 2024-04-20T10:44:24.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.144+0000 7ff6f6000640 1 -- 172.21.15.42:0/2154444055 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6f8037070 con 0x559add949380 2024-04-20T10:44:24.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.145+0000 7ff6f6000640 1 -- 172.21.15.42:0/2154444055 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f802b040 con 0x559add949380 2024-04-20T10:44:24.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.146+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2154444055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949380 msgr2=0x559add951fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:24.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.146+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/2154444055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949380 0x559add951fd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800c470 tx=0x7ff6f800a5d0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:23 smithi162 ceph-mon[21854]: pgmap v322: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 813 KiB/s wr, 1.06k op/s 2024-04-20T10:44:24.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.150+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2154444055 shutdown_connections 2024-04-20T10:44:24.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.150+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/2154444055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add952880 0x559add96a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.150+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/2154444055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add95e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.150+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/2154444055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949380 0x559add951fd0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.150+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2154444055 >> 172.21.15.42:0/2154444055 conn(0x559add943ab0 msgr2=0x559add943e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:24.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.150+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2154444055 shutdown_connections 2024-04-20T10:44:24.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.150+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2154444055 wait complete. 2024-04-20T10:44:24.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.150+0000 7ff70f1ef740 1 -- start start 2024-04-20T10:44:24.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.151+0000 7ff70f1ef740 1 Processor -- start 2024-04-20T10:44:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.154+0000 7ff70f1ef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949380 0x559add956580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.154+0000 7ff70f1ef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add956ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.154+0000 7ff70f1ef740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add952880 0x559add957000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.154+0000 7ff70f1ef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff70407f5e0 con 0x559add949ca0 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.154+0000 7ff70f1ef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff704080a50 con 0x559add949380 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.154+0000 7ff70f1ef740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff70407fcb0 con 0x559add952880 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.154+0000 7ff70f1ef740 10 client.0 _ll_register_callbacks cb 0x7ff6fc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.155+0000 7ff709000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add956ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.155+0000 7ff709a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949380 0x559add956580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.155+0000 7ff709000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add956ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33112/0 (socket says 172.21.15.42:33112) 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.155+0000 7ff709000640 1 -- 172.21.15.42:0/2137538769 learned_addr learned my addr 172.21.15.42:0/2137538769 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.155+0000 7ff70a400640 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add952880 0x559add957000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.156+0000 7ff70a400640 1 -- 172.21.15.42:0/2137538769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949380 msgr2=0x559add956580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.156+0000 7ff70a400640 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949380 0x559add956580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.156+0000 7ff70a400640 1 -- 172.21.15.42:0/2137538769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 msgr2=0x559add956ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.156+0000 7ff70a400640 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add956ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.156+0000 7ff70a400640 1 -- 172.21.15.42:0/2137538769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559add9c8270 con 0x559add952880 2024-04-20T10:44:24.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.156+0000 7ff709000640 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add956ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:44:24.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.156+0000 7ff70a400640 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add952880 0x559add957000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0c6a30 tx=0x7ff6fc003920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=543f19c0a46c5028 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:24.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.156+0000 7ff6eb400640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:44:24.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.156+0000 7ff6eb400640 1 -- 172.21.15.42:0/2137538769 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0ba180 con 0x559add952880 2024-04-20T10:44:24.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.157+0000 7ff6eb400640 1 -- 172.21.15.42:0/2137538769 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7ff6fc0d7320 con 0x559add952880 2024-04-20T10:44:24.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.157+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2137538769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6f8037070 con 0x559add952880 2024-04-20T10:44:24.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.157+0000 7ff6eb400640 1 -- 172.21.15.42:0/2137538769 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff6fc0bda20 con 0x559add952880 2024-04-20T10:44:24.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.157+0000 7ff70f1ef740 10 client.26545 fetch_fsmap learned FSMap version 19 2024-04-20T10:44:24.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.157+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2137538769 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559add9c8270 con 0x559add952880 2024-04-20T10:44:24.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.157+0000 7ff6eb400640 1 -- 172.21.15.42:0/2137538769 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6f8037070 con 0x559add952880 2024-04-20T10:44:24.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.158+0000 7ff6eb400640 1 -- 172.21.15.42:0/2137538769 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7ff6fc0bcf60 con 0x559add952880 2024-04-20T10:44:24.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.158+0000 7ff70f1ef740 10 client.26545 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:44:24.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.158+0000 7ff70f1ef740 10 client.26545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:44:24.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.158+0000 7ff70f1ef740 20 client.26545 populate_metadata read hostname 'smithi042' 2024-04-20T10:44:24.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.158+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559add962b70 0x559add9cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.158+0000 7ff70f1ef740 4 client.26545 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:44:24.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.158+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2137538769 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559add8b69e0 con 0x559add962b70 2024-04-20T10:44:24.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.158+0000 7ff709a00640 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559add962b70 0x559add9cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.163+0000 7ff709a00640 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559add962b70 0x559add9cb9c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8375c6acc4b805c0 server_cookie=a0687f0dd73b5421 in_seq=0 out_seq=0 2024-04-20T10:44:24.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.163+0000 7ff6eb400640 10 client.26545 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:24.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.163+0000 7ff6eb400640 1 -- 172.21.15.42:0/2137538769 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559add8b69e0 con 0x559add962b70 2024-04-20T10:44:24.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.163+0000 7ff6eb400640 10 client.26545 handle_command_reply: tid=0 2024-04-20T10:44:24.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.163+0000 7ff70f1ef740 1 client.26545 shutdown 2024-04-20T10:44:24.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.163+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2137538769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add952880 msgr2=0x559add957000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:24.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.163+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add952880 0x559add957000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0c6a30 tx=0x7ff6fc003920 comp rx=0 tx=0).stop 2024-04-20T10:44:24.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.164+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2137538769 shutdown_connections 2024-04-20T10:44:24.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.164+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559add962b70 0x559add9cb9c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.164+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559add952880 0x559add957000 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.164+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559add949ca0 0x559add956ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.164+0000 7ff6eb400640 0 client.26545 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:24.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.164+0000 7ff70f1ef740 1 --2- 172.21.15.42:0/2137538769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559add949380 0x559add956580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.164+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2137538769 >> 172.21.15.42:0/2137538769 conn(0x559add943ab0 msgr2=0x559add942280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:24.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.164+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2137538769 shutdown_connections 2024-04-20T10:44:24.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.164+0000 7ff70f1ef740 1 -- 172.21.15.42:0/2137538769 wait complete. 2024-04-20T10:44:24.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.164+0000 7ff70f1ef740 20 client.26545 trim_cache size 0 max 16384 2024-04-20T10:44:24.166 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:44:24.166 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:44:24.167 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:44:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:44:24.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.169+0000 7ff70ae00640 1 -- 172.21.15.42:0/2985345497 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff6ec0a3de0 msgr2=0x7ff6ec0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:24.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.169+0000 7ff70ae00640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff6ec0a3de0 0x7ff6ec0a62a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0006fd0 tx=0x7ff6f0008040 comp rx=0 tx=0).stop 2024-04-20T10:44:24.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.169+0000 7ff70ae00640 1 -- 172.21.15.42:0/2985345497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7041150b0 msgr2=0x7ff7041a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:24.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.169+0000 7ff70ae00640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7041150b0 0x7ff7041a5890 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009850 tx=0x7ff6fc0045e0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.169+0000 7ff70ae00640 1 -- 172.21.15.42:0/2985345497 shutdown_connections 2024-04-20T10:44:24.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.169+0000 7ff70ae00640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7ff6ec0a3de0 0x7ff6ec0a62a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.169+0000 7ff70ae00640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7041150b0 0x7ff7041a5890 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.169+0000 7ff70ae00640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff704109880 0x7ff7041a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.169+0000 7ff70ae00640 1 --2- 172.21.15.42:0/2985345497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff704108f60 0x7ff7041a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.169+0000 7ff70ae00640 1 -- 172.21.15.42:0/2985345497 >> 172.21.15.42:0/2985345497 conn(0x7ff70406cee0 msgr2=0x7ff704070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:24.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.170+0000 7ff70ae00640 1 -- 172.21.15.42:0/2985345497 shutdown_connections 2024-04-20T10:44:24.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.170+0000 7ff70ae00640 1 -- 172.21.15.42:0/2985345497 wait complete. 2024-04-20T10:44:24.212 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/mballoc.o 2024-04-20T10:44:24.227 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:44:24.228 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:44:24.632 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/time/namespace.o 2024-04-20T10:44:24.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.685+0000 7f4843e00640 1 -- 172.21.15.42:0/541443755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4844109900 msgr2=0x7f4844115640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:24.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.685+0000 7f4843e00640 1 --2- 172.21.15.42:0/541443755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4844109900 0x7f4844115640 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4830008440 tx=0x7f483002fd50 comp rx=0 tx=0).stop 2024-04-20T10:44:24.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.686+0000 7f4843e00640 1 -- 172.21.15.42:0/541443755 shutdown_connections 2024-04-20T10:44:24.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.686+0000 7f4843e00640 1 --2- 172.21.15.42:0/541443755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4844109900 0x7f4844115640 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.686+0000 7f4843e00640 1 --2- 172.21.15.42:0/541443755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4844108f60 0x7f48441093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.686+0000 7f4843e00640 1 --2- 172.21.15.42:0/541443755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f484410e780 0x7f484410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.686+0000 7f4843e00640 1 -- 172.21.15.42:0/541443755 >> 172.21.15.42:0/541443755 conn(0x7f484406ce70 msgr2=0x7f484406d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:24.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.694+0000 7f4843e00640 1 -- 172.21.15.42:0/541443755 shutdown_connections 2024-04-20T10:44:24.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.695+0000 7f4843e00640 1 -- 172.21.15.42:0/541443755 wait complete. 2024-04-20T10:44:24.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.696+0000 7f4843e00640 1 Processor -- start 2024-04-20T10:44:24.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.700+0000 7f4843e00640 1 -- start start 2024-04-20T10:44:24.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.701+0000 7f4843e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4844108f60 0x7f484411f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.701+0000 7f4843e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4844109900 0x7f4844118280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.701+0000 7f4843e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484410e780 0x7f48441187c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.701+0000 7f4843e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4844120f20 con 0x7f4844108f60 2024-04-20T10:44:24.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.701+0000 7f4843e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4844120da0 con 0x7f484410e780 2024-04-20T10:44:24.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.701+0000 7f4843e00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f48441210a0 con 0x7f4844109900 2024-04-20T10:44:24.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.702+0000 7f4843400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484410e780 0x7f48441187c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.702+0000 7f4843400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484410e780 0x7f48441187c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:34490/0 (socket says 172.21.15.42:34490) 2024-04-20T10:44:24.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.702+0000 7f4843400640 1 -- 172.21.15.42:0/888267317 learned_addr learned my addr 172.21.15.42:0/888267317 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:24.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.702+0000 7f4842000640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4844109900 0x7f4844118280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.702+0000 7f4843400640 1 -- 172.21.15.42:0/888267317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4844109900 msgr2=0x7f4844118280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:24.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.702+0000 7f4843400640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4844109900 0x7f4844118280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.702+0000 7f4843400640 1 -- 172.21.15.42:0/888267317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4844108f60 msgr2=0x7f484411f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:24.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.702+0000 7f4843400640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4844108f60 0x7f484411f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:24.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.702+0000 7f4843400640 1 -- 172.21.15.42:0/888267317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4844118d80 con 0x7f484410e780 2024-04-20T10:44:24.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.703+0000 7f4842000640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4844109900 0x7f4844118280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:44:24.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.705+0000 7f4843400640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484410e780 0x7f48441187c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f483003a040 tx=0x7f483000aa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:24.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.705+0000 7f483b400640 1 -- 172.21.15.42:0/888267317 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830045070 con 0x7f484410e780 2024-04-20T10:44:24.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.705+0000 7f483b400640 1 -- 172.21.15.42:0/888267317 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4830012520 con 0x7f484410e780 2024-04-20T10:44:24.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.705+0000 7f483b400640 1 -- 172.21.15.42:0/888267317 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830038950 con 0x7f484410e780 2024-04-20T10:44:24.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.705+0000 7f4843e00640 1 -- 172.21.15.42:0/888267317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4844119010 con 0x7f484410e780 2024-04-20T10:44:24.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.708+0000 7f4843e00640 1 -- 172.21.15.42:0/888267317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f484411cfc0 con 0x7f484410e780 2024-04-20T10:44:24.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.711+0000 7f4843e00640 1 -- 172.21.15.42:0/888267317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4844109f70 con 0x7f484410e780 2024-04-20T10:44:24.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.713+0000 7f483b400640 1 -- 172.21.15.42:0/888267317 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4830048070 con 0x7f484410e780 2024-04-20T10:44:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.715+0000 7f483b400640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f48280a3de0 0x7f48280a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.716+0000 7f4842a00640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f48280a3de0 0x7f48280a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.719+0000 7f483b400640 1 -- 172.21.15.42:0/888267317 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f48300f9350 con 0x7f484410e780 2024-04-20T10:44:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.719+0000 7f4842a00640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f48280a3de0 0x7f48280a62a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f483400a860 tx=0x7f4834005b60 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:24.722+0000 7f483b400640 1 -- 172.21.15.42:0/888267317 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f48300be4c0 con 0x7f484410e780 2024-04-20T10:44:24.951 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fat/file.o 2024-04-20T10:44:24.962 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-ioprio.o 2024-04-20T10:44:24.970 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/hugetlb_vmemmap.o 2024-04-20T10:44:24.978 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/pid_list.o 2024-04-20T10:44:25.191 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/fdinfo.o 2024-04-20T10:44:25.209 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/mempolicy.o 2024-04-20T10:44:25.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.235+0000 7f4843e00640 1 -- 172.21.15.42:0/888267317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f484411d9e0 con 0x7f484410e780 2024-04-20T10:44:25.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.236+0000 7f483b400640 1 -- 172.21.15.42:0/888267317 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f48300bddc0 con 0x7f484410e780 2024-04-20T10:44:25.237 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:44:25.237 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:44:25.237 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:44:25.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.244+0000 7f4843e00640 1 -- 172.21.15.42:0/888267317 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f48280a3de0 msgr2=0x7f48280a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:25.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.245+0000 7f4843e00640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f48280a3de0 0x7f48280a62a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f483400a860 tx=0x7f4834005b60 comp rx=0 tx=0).stop 2024-04-20T10:44:25.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.245+0000 7f4843e00640 1 -- 172.21.15.42:0/888267317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484410e780 msgr2=0x7f48441187c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:25.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.245+0000 7f4843e00640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484410e780 0x7f48441187c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f483003a040 tx=0x7f483000aa10 comp rx=0 tx=0).stop 2024-04-20T10:44:25.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.245+0000 7f4843e00640 1 -- 172.21.15.42:0/888267317 shutdown_connections 2024-04-20T10:44:25.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.245+0000 7f4843e00640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f48280a3de0 0x7f48280a62a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.245+0000 7f4843e00640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484410e780 0x7f48441187c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.245+0000 7f4843e00640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4844109900 0x7f4844118280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.245+0000 7f4843e00640 1 --2- 172.21.15.42:0/888267317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4844108f60 0x7f484411f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.245+0000 7f4843e00640 1 -- 172.21.15.42:0/888267317 >> 172.21.15.42:0/888267317 conn(0x7f484406ce70 msgr2=0x7f48440723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.245+0000 7f4843e00640 1 -- 172.21.15.42:0/888267317 shutdown_connections 2024-04-20T10:44:25.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.246+0000 7f4843e00640 1 -- 172.21.15.42:0/888267317 wait complete. 2024-04-20T10:44:25.300 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:25 smithi042 ceph-mon[17078]: pgmap v323: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 683 KiB/s wr, 222 op/s 2024-04-20T10:44:25.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:25 smithi046 ceph-mon[21574]: pgmap v323: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 683 KiB/s wr, 222 op/s 2024-04-20T10:44:25.307 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:44:25.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:25 smithi162 ceph-mon[21854]: pgmap v323: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 683 KiB/s wr, 222 op/s 2024-04-20T10:44:25.750 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/time/built-in.a 2024-04-20T10:44:25.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.802+0000 7f8705c00640 1 -- 172.21.15.42:0/4128640857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8700108f60 msgr2=0x7f87001093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:25.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.802+0000 7f8705c00640 1 --2- 172.21.15.42:0/4128640857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8700108f60 0x7f87001093e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009900 tx=0x7f86ec02bc40 comp rx=0 tx=0).stop 2024-04-20T10:44:25.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.807+0000 7f8705c00640 1 -- 172.21.15.42:0/4128640857 shutdown_connections 2024-04-20T10:44:25.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.807+0000 7f8705c00640 1 --2- 172.21.15.42:0/4128640857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8700109920 0x7f8700115700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.807+0000 7f8705c00640 1 --2- 172.21.15.42:0/4128640857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8700108f60 0x7f87001093e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.807+0000 7f8705c00640 1 --2- 172.21.15.42:0/4128640857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f870010e840 0x7f870010ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.807+0000 7f8705c00640 1 -- 172.21.15.42:0/4128640857 >> 172.21.15.42:0/4128640857 conn(0x7f870006cee0 msgr2=0x7f870006d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:25.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.809+0000 7f8705c00640 1 -- 172.21.15.42:0/4128640857 shutdown_connections 2024-04-20T10:44:25.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.809+0000 7f8705c00640 1 -- 172.21.15.42:0/4128640857 wait complete. 2024-04-20T10:44:25.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.811+0000 7f8705c00640 1 Processor -- start 2024-04-20T10:44:25.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.811+0000 7f8705c00640 1 -- start start 2024-04-20T10:44:25.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.812+0000 7f8705c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8700108f60 0x7f87001176b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:25.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.812+0000 7f8705c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8700109920 0x7f8700117bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:25.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.812+0000 7f86ffe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8700108f60 0x7f87001176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:25.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.812+0000 7f86ffe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8700108f60 0x7f87001176b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:49890/0 (socket says 172.21.15.42:49890) 2024-04-20T10:44:25.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.812+0000 7f86ffe00640 1 -- 172.21.15.42:0/1533834057 learned_addr learned my addr 172.21.15.42:0/1533834057 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:25.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.812+0000 7f86ff400640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8700109920 0x7f8700117bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:25.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.813+0000 7f8705c00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f870010e840 0x7f8700118130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:25.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.813+0000 7f8705c00640 1 -- 172.21.15.42:0/1533834057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f870011aa30 con 0x7f870010e840 2024-04-20T10:44:25.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.813+0000 7f8705c00640 1 -- 172.21.15.42:0/1533834057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f870011a8b0 con 0x7f8700109920 2024-04-20T10:44:25.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.813+0000 7f8705c00640 1 -- 172.21.15.42:0/1533834057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f870011abb0 con 0x7f8700108f60 2024-04-20T10:44:25.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.813+0000 7f8705200640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f870010e840 0x7f8700118130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:25.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.813+0000 7f86ffe00640 1 -- 172.21.15.42:0/1533834057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8700109920 msgr2=0x7f8700117bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:25.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.813+0000 7f86ffe00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8700109920 0x7f8700117bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.813+0000 7f86ffe00640 1 -- 172.21.15.42:0/1533834057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f870010e840 msgr2=0x7f8700118130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:25.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.813+0000 7f86ffe00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f870010e840 0x7f8700118130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.814+0000 7f86ffe00640 1 -- 172.21.15.42:0/1533834057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87001b3830 con 0x7f8700108f60 2024-04-20T10:44:25.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.814+0000 7f86ffe00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8700108f60 0x7f87001176b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f86f800ebc0 tx=0x7f86f800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:25.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.814+0000 7f86fcc00640 1 -- 172.21.15.42:0/1533834057 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8021640 con 0x7f8700108f60 2024-04-20T10:44:25.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.814+0000 7f86fcc00640 1 -- 172.21.15.42:0/1533834057 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f8021cc0 con 0x7f8700108f60 2024-04-20T10:44:25.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.814+0000 7f86fcc00640 1 -- 172.21.15.42:0/1533834057 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8029c70 con 0x7f8700108f60 2024-04-20T10:44:25.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.816+0000 7f8705c00640 1 -- 172.21.15.42:0/1533834057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87001a5050 con 0x7f8700108f60 2024-04-20T10:44:25.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.816+0000 7f8705c00640 1 -- 172.21.15.42:0/1533834057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87001a5460 con 0x7f8700108f60 2024-04-20T10:44:25.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.825+0000 7f8709e3c740 1 Processor -- start 2024-04-20T10:44:25.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.826+0000 7f8709e3c740 1 -- start start 2024-04-20T10:44:25.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.826+0000 7f8709e3c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b39060e170 0x55b39060e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:25.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.826+0000 7f86fcc00640 1 -- 172.21.15.42:0/1533834057 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f86f8030020 con 0x7f8700108f60 2024-04-20T10:44:25.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.829+0000 7f86fcc00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f86e40a3da0 0x7f86e40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:25.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.829+0000 7f86fcc00640 1 -- 172.21.15.42:0/1533834057 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f86f80e1ab0 con 0x7f8700108f60 2024-04-20T10:44:25.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.829+0000 7f8709e3c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060ea90 0x55b390617130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:25.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.829+0000 7f8709e3c740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b390617670 0x55b390619a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:25.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.829+0000 7f8709e3c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b390573050 con 0x55b39060ea90 2024-04-20T10:44:25.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.829+0000 7f8709e3c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b390519490 con 0x55b39060e170 2024-04-20T10:44:25.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.829+0000 7f8709e3c740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b39056f010 con 0x55b390617670 2024-04-20T10:44:25.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.830+0000 7f8705200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b39060e170 0x55b39060e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:25.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.830+0000 7f8705200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b39060e170 0x55b39060e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:34520/0 (socket says 172.21.15.42:34520) 2024-04-20T10:44:25.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.830+0000 7f8705200640 1 -- 172.21.15.42:0/2992701751 learned_addr learned my addr 172.21.15.42:0/2992701751 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:25.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.831+0000 7f86ff400640 1 --2- 172.21.15.42:0/2992701751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060ea90 0x55b390617130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:25.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.830+0000 7f8705200640 1 --2- 172.21.15.42:0/2992701751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b390617670 0x55b390619a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:25.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.832+0000 7f86ff400640 1 -- 172.21.15.42:0/2992701751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b390617670 msgr2=0x55b390619a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:25.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.832+0000 7f86ff400640 1 --2- 172.21.15.42:0/2992701751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b390617670 0x55b390619a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.832+0000 7f86ff400640 1 -- 172.21.15.42:0/2992701751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b39060e170 msgr2=0x55b39060e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:25.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.832+0000 7f86ff400640 1 --2- 172.21.15.42:0/2992701751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b39060e170 0x55b39060e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.832+0000 7f86ff400640 1 -- 172.21.15.42:0/2992701751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b390521070 con 0x55b39060ea90 2024-04-20T10:44:25.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.833+0000 7f86ff400640 1 --2- 172.21.15.42:0/2992701751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060ea90 0x55b390617130 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f86ec02c180 tx=0x7f86ec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a253075d6a4377 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:25.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.834+0000 7f86f5600640 1 -- 172.21.15.42:0/2992701751 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec042070 con 0x55b39060ea90 2024-04-20T10:44:25.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.834+0000 7f86f5600640 1 -- 172.21.15.42:0/2992701751 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ec02c890 con 0x55b39060ea90 2024-04-20T10:44:25.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.835+0000 7f8709e3c740 1 -- 172.21.15.42:0/2992701751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060ea90 msgr2=0x55b390617130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:25.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.835+0000 7f8709e3c740 1 --2- 172.21.15.42:0/2992701751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060ea90 0x55b390617130 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f86ec02c180 tx=0x7f86ec000bc0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.836+0000 7f8709e3c740 1 -- 172.21.15.42:0/2992701751 shutdown_connections 2024-04-20T10:44:25.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.836+0000 7f8709e3c740 1 --2- 172.21.15.42:0/2992701751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b390617670 0x55b390619a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.836+0000 7f8709e3c740 1 --2- 172.21.15.42:0/2992701751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060ea90 0x55b390617130 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.836+0000 7f8709e3c740 1 --2- 172.21.15.42:0/2992701751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b39060e170 0x55b39060e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.836+0000 7f8709e3c740 1 -- 172.21.15.42:0/2992701751 >> 172.21.15.42:0/2992701751 conn(0x55b3906088a0 msgr2=0x55b390608c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:25.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.836+0000 7f86ffe00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f86e40a3da0 0x7f86e40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:25.841 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_sched_switch.o 2024-04-20T10:44:25.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.838+0000 7f86ffe00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f86e40a3da0 0x7f86e40a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f86f80df1d0 tx=0x7f86f800ef50 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:25.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.841+0000 7f8709e3c740 1 -- 172.21.15.42:0/2992701751 shutdown_connections 2024-04-20T10:44:25.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.841+0000 7f8709e3c740 1 -- 172.21.15.42:0/2992701751 wait complete. 2024-04-20T10:44:25.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.841+0000 7f8709e3c740 1 -- start start 2024-04-20T10:44:25.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.842+0000 7f8709e3c740 1 Processor -- start 2024-04-20T10:44:25.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.843+0000 7f8709e3c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 0x55b3906146c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:25.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.843+0000 7f8709e3c740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390614c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.844+0000 7f8709e3c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b390617670 0x55b390615140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.844+0000 7f8709e3c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b39061a4a0 con 0x55b39060e170 2024-04-20T10:44:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.844+0000 7f8709e3c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b39052c860 con 0x55b390617670 2024-04-20T10:44:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.844+0000 7f8709e3c740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b39061a620 con 0x55b39060ea90 2024-04-20T10:44:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.844+0000 7f8709e3c740 10 client.0 _ll_register_callbacks cb 0x55b3904fafe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:44:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.844+0000 7f8705200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 0x55b3906146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.844+0000 7f8705200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 0x55b3906146c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33162/0 (socket says 172.21.15.42:33162) 2024-04-20T10:44:25.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.844+0000 7f8705200640 1 -- 172.21.15.42:0/2715199237 learned_addr learned my addr 172.21.15.42:0/2715199237 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:25.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.845+0000 7f8705200640 1 -- 172.21.15.42:0/2715199237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 msgr2=0x55b390614c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:44:25.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.845+0000 7f8705200640 1 --2- 172.21.15.42:0/2715199237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390614c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.846+0000 7f8705200640 1 -- 172.21.15.42:0/2715199237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b390617670 msgr2=0x55b390615140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:44:25.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.846+0000 7f8705200640 1 --2- 172.21.15.42:0/2715199237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b390617670 0x55b390615140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.846+0000 7f8705200640 1 -- 172.21.15.42:0/2715199237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b390530950 con 0x55b39060e170 2024-04-20T10:44:25.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.847+0000 7f8705200640 1 --2- 172.21.15.42:0/2715199237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 0x55b3906146c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f86f0013e90 tx=0x7f86f000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d98420ed7d3221f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:25.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.847+0000 7f86ea000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:44:25.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.848+0000 7f86ea000640 1 -- 172.21.15.42:0/2715199237 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0013110 con 0x55b39060e170 2024-04-20T10:44:25.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.848+0000 7f86ea000640 1 -- 172.21.15.42:0/2715199237 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f86f000fcc0 con 0x55b39060e170 2024-04-20T10:44:25.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.848+0000 7f8709e3c740 1 -- 172.21.15.42:0/2715199237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b390580a70 con 0x55b39060e170 2024-04-20T10:44:25.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.849+0000 7f86ea000640 1 -- 172.21.15.42:0/2715199237 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f86f0004050 con 0x55b39060e170 2024-04-20T10:44:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.850+0000 7f8709e3c740 10 client.16443 fetch_fsmap learned FSMap version 19 2024-04-20T10:44:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.851+0000 7f8709e3c740 1 -- 172.21.15.42:0/2715199237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b3904fe820 con 0x55b39060e170 2024-04-20T10:44:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.854+0000 7f86ea000640 1 -- 172.21.15.42:0/2715199237 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b390580a70 con 0x55b39060e170 2024-04-20T10:44:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.854+0000 7f86ea000640 1 -- 172.21.15.42:0/2715199237 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f86f0011440 con 0x55b39060e170 2024-04-20T10:44:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.855+0000 7f8709e3c740 10 client.16443 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:44:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.855+0000 7f8709e3c740 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:44:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.855+0000 7f8709e3c740 20 client.16443 populate_metadata read hostname 'smithi042' 2024-04-20T10:44:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.855+0000 7f8709e3c740 1 --2- 172.21.15.42:0/2715199237 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b390627870 0x55b390629ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.855+0000 7f8709e3c740 4 client.16443 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:44:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.855+0000 7f8709e3c740 1 -- 172.21.15.42:0/2715199237 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b39057a8e0 con 0x55b390627870 2024-04-20T10:44:25.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.858+0000 7f86ff400640 1 --2- 172.21.15.42:0/2715199237 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b390627870 0x55b390629ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:25.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.858+0000 7f86ff400640 1 --2- 172.21.15.42:0/2715199237 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b390627870 0x55b390629ca0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb75fbed900828a1 server_cookie=4caa8522eb6cba5b in_seq=0 out_seq=0 2024-04-20T10:44:25.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.858+0000 7f86ea000640 10 client.16443 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:25.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.875+0000 7f86ea000640 1 -- 172.21.15.42:0/2715199237 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b39057a8e0 con 0x55b390627870 2024-04-20T10:44:25.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.875+0000 7f86ea000640 10 client.16443 handle_command_reply: tid=0 2024-04-20T10:44:25.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.876+0000 7f8709e3c740 1 client.16443 shutdown 2024-04-20T10:44:25.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.882+0000 7f8709e3c740 1 -- 172.21.15.42:0/2715199237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 msgr2=0x55b3906146c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:25.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.882+0000 7f8709e3c740 1 --2- 172.21.15.42:0/2715199237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 0x55b3906146c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f86f0013e90 tx=0x7f86f000f790 comp rx=0 tx=0).stop 2024-04-20T10:44:25.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.890+0000 7f8709e3c740 1 -- 172.21.15.42:0/2715199237 shutdown_connections 2024-04-20T10:44:25.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.890+0000 7f8709e3c740 1 --2- 172.21.15.42:0/2715199237 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b390627870 0x55b390629ca0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.890+0000 7f8709e3c740 1 --2- 172.21.15.42:0/2715199237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b390617670 0x55b390615140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.890+0000 7f8709e3c740 1 --2- 172.21.15.42:0/2715199237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390614c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.895 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-iolatency.o 2024-04-20T10:44:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.890+0000 7f8709e3c740 1 --2- 172.21.15.42:0/2715199237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 0x55b3906146c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.890+0000 7f8709e3c740 1 -- 172.21.15.42:0/2715199237 >> 172.21.15.42:0/2715199237 conn(0x55b3906088a0 msgr2=0x55b39060a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.890+0000 7f86ea000640 0 client.16443 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.891+0000 7f8709e3c740 1 -- 172.21.15.42:0/2715199237 shutdown_connections 2024-04-20T10:44:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.891+0000 7f8709e3c740 1 -- 172.21.15.42:0/2715199237 wait complete. 2024-04-20T10:44:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.893+0000 7f8709e3c740 20 client.16443 trim_cache size 0 max 16384 2024-04-20T10:44:26.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:25.995+0000 7f8709e3c740 1 Processor -- start 2024-04-20T10:44:26.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.003+0000 7f8709e3c740 1 -- start start 2024-04-20T10:44:26.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.003+0000 7f8709e3c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 0x55b390616df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:26.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.003+0000 7f8709e3c740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390623420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:26.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.003+0000 7f8709e3c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b390617670 0x55b3906077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:26.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.003+0000 7f8709e3c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f87001abf60 con 0x55b39060e170 2024-04-20T10:44:26.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.003+0000 7f8709e3c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b49d0 con 0x55b390617670 2024-04-20T10:44:26.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.003+0000 7f8709e3c740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f87001ac5b0 con 0x55b39060ea90 2024-04-20T10:44:26.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.007+0000 7f86ff400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390623420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:26.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.007+0000 7f86ff400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390623420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:49918/0 (socket says 172.21.15.42:49918) 2024-04-20T10:44:26.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.007+0000 7f86ff400640 1 -- 172.21.15.42:0/16870908 learned_addr learned my addr 172.21.15.42:0/16870908 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:26.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.007+0000 7f8705200640 1 --2- 172.21.15.42:0/16870908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b390617670 0x55b3906077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:26.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.007+0000 7f8705200640 1 --2- 172.21.15.42:0/16870908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 0x55b390616df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:26.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.008+0000 7f8705200640 1 -- 172.21.15.42:0/16870908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 msgr2=0x55b390623420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:26.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.008+0000 7f8705200640 1 --2- 172.21.15.42:0/16870908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390623420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.008+0000 7f8705200640 1 -- 172.21.15.42:0/16870908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b390617670 msgr2=0x55b3906077e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:26.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.008+0000 7f8705200640 1 --2- 172.21.15.42:0/16870908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b390617670 0x55b3906077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.008+0000 7f8705200640 1 -- 172.21.15.42:0/16870908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3902853f0 con 0x55b39060e170 2024-04-20T10:44:26.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.008+0000 7f8705200640 1 --2- 172.21.15.42:0/16870908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 0x55b390616df0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f86f0011810 tx=0x7f86f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a7430c77aa8b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:26.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.009+0000 7f86f4c00640 1 -- 172.21.15.42:0/16870908 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0000e60 con 0x55b39060e170 2024-04-20T10:44:26.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.009+0000 7f86f4c00640 1 -- 172.21.15.42:0/16870908 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f0011e50 con 0x55b39060e170 2024-04-20T10:44:26.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.009+0000 7f86f4c00640 1 -- 172.21.15.42:0/16870908 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f002f070 con 0x55b39060e170 2024-04-20T10:44:26.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.010+0000 7f8709e3c740 1 -- 172.21.15.42:0/16870908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 msgr2=0x55b390616df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:26.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.010+0000 7f8709e3c740 1 --2- 172.21.15.42:0/16870908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 0x55b390616df0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f86f0011810 tx=0x7f86f0000bc0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.013+0000 7f8709e3c740 1 -- 172.21.15.42:0/16870908 shutdown_connections 2024-04-20T10:44:26.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.013+0000 7f8709e3c740 1 --2- 172.21.15.42:0/16870908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b390617670 0x55b3906077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.013+0000 7f8709e3c740 1 --2- 172.21.15.42:0/16870908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390623420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.013+0000 7f8709e3c740 1 --2- 172.21.15.42:0/16870908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b39060e170 0x55b390616df0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.013+0000 7f8709e3c740 1 -- 172.21.15.42:0/16870908 >> 172.21.15.42:0/16870908 conn(0x55b3906088a0 msgr2=0x55b3906070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.013+0000 7f8709e3c740 1 -- 172.21.15.42:0/16870908 shutdown_connections 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.013+0000 7f8709e3c740 1 -- 172.21.15.42:0/16870908 wait complete. 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.014+0000 7f8709e3c740 1 -- start start 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.015+0000 7f8709e3c740 1 Processor -- start 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.016+0000 7f8709e3c740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b39060e170 0x55b39061fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.016+0000 7f8709e3c740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390610f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.016+0000 7f8709e3c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b390617670 0x55b390611490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.016+0000 7f8709e3c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f87001acc80 con 0x55b390617670 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.016+0000 7f8709e3c740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f87001b4300 con 0x55b39060e170 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.016+0000 7f8709e3c740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f87001ad350 con 0x55b39060ea90 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.016+0000 7f86ff400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390610f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:26.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.016+0000 7f8709e3c740 10 client.0 _ll_register_callbacks cb 0x7f86f0013800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.016+0000 7f86ff400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390610f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:49926/0 (socket says 172.21.15.42:49926) 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.016+0000 7f86ff400640 1 -- 172.21.15.42:0/3253497057 learned_addr learned my addr 172.21.15.42:0/3253497057 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.017+0000 7f86ff400640 1 -- 172.21.15.42:0/3253497057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b39060e170 msgr2=0x55b39061fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.017+0000 7f86ff400640 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b39060e170 0x55b39061fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.017+0000 7f86ff400640 1 -- 172.21.15.42:0/3253497057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b390617670 msgr2=0x55b390611490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.017+0000 7f8705200640 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b390617670 0x55b390611490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.017+0000 7f86ff400640 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b390617670 0x55b390611490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.017+0000 7f86ff400640 1 -- 172.21.15.42:0/3253497057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39034f7d0 con 0x55b39060ea90 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.017+0000 7f8705200640 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b390617670 0x55b390611490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.017+0000 7f86ff400640 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390610f50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55b39060dc40 tx=0x7f86ec005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb63329a6dc19fb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.018+0000 7f86ea000640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.018+0000 7f86ea000640 1 -- 172.21.15.42:0/3253497057 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec054060 con 0x55b39060ea90 2024-04-20T10:44:26.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.018+0000 7f86ea000640 1 -- 172.21.15.42:0/3253497057 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f86ec004240 con 0x55b39060ea90 2024-04-20T10:44:26.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.019+0000 7f8709e3c740 1 -- 172.21.15.42:0/3253497057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f0011e50 con 0x55b39060ea90 2024-04-20T10:44:26.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.020+0000 7f86ea000640 1 -- 172.21.15.42:0/3253497057 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f86ec042040 con 0x55b39060ea90 2024-04-20T10:44:26.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.020+0000 7f8709e3c740 10 client.26560 fetch_fsmap learned FSMap version 19 2024-04-20T10:44:26.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.020+0000 7f8709e3c740 1 -- 172.21.15.42:0/3253497057 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b3906afa00 con 0x55b39060ea90 2024-04-20T10:44:26.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.022+0000 7f86ea000640 1 -- 172.21.15.42:0/3253497057 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f0011e50 con 0x55b39060ea90 2024-04-20T10:44:26.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.022+0000 7f86ea000640 1 -- 172.21.15.42:0/3253497057 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f86ec005380 con 0x55b39060ea90 2024-04-20T10:44:26.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.022+0000 7f8709e3c740 10 client.26560 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:44:26.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.022+0000 7f8709e3c740 10 client.26560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:44:26.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.022+0000 7f8709e3c740 20 client.26560 populate_metadata read hostname 'smithi042' 2024-04-20T10:44:26.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.022+0000 7f8709e3c740 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b3906afd20 0x55b3906b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:26.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.022+0000 7f8709e3c740 4 client.26560 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:44:26.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.022+0000 7f8709e3c740 1 -- 172.21.15.42:0/3253497057 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3904f65a0 con 0x55b3906afd20 2024-04-20T10:44:26.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.022+0000 7f8705200640 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b3906afd20 0x55b3906b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:26.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.023+0000 7f8705200640 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b3906afd20 0x55b3906b2110 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b980118c284d64 server_cookie=410f2fab13c991c3 in_seq=0 out_seq=0 2024-04-20T10:44:26.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.023+0000 7f86ea000640 10 client.26560 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:26.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.023+0000 7f86ea000640 1 -- 172.21.15.42:0/3253497057 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3904f65a0 con 0x55b3906afd20 2024-04-20T10:44:26.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.023+0000 7f86ea000640 10 client.26560 handle_command_reply: tid=0 2024-04-20T10:44:26.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.023+0000 7f8709e3c740 1 client.26560 shutdown 2024-04-20T10:44:26.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.025+0000 7f8709e3c740 1 -- 172.21.15.42:0/3253497057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 msgr2=0x55b390610f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.025+0000 7f8709e3c740 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390610f50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55b39060dc40 tx=0x7f86ec005fd0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.025+0000 7f8709e3c740 1 -- 172.21.15.42:0/3253497057 shutdown_connections 2024-04-20T10:44:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.025+0000 7f8709e3c740 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55b3906afd20 0x55b3906b2110 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.025+0000 7f8709e3c740 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b390617670 0x55b390611490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.025+0000 7f8709e3c740 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b39060ea90 0x55b390610f50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.025+0000 7f8709e3c740 1 --2- 172.21.15.42:0/3253497057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b39060e170 0x55b39061fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.025+0000 7f86ea000640 0 client.26560 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:26.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.025+0000 7f8709e3c740 1 -- 172.21.15.42:0/3253497057 >> 172.21.15.42:0/3253497057 conn(0x55b3906088a0 msgr2=0x55b390614f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:26.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.026+0000 7f8709e3c740 1 -- 172.21.15.42:0/3253497057 shutdown_connections 2024-04-20T10:44:26.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.026+0000 7f8709e3c740 1 -- 172.21.15.42:0/3253497057 wait complete. 2024-04-20T10:44:26.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.026+0000 7f8709e3c740 20 client.26560 trim_cache size 0 max 16384 2024-04-20T10:44:26.031 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:44:26.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.035+0000 7f8705c00640 1 -- 172.21.15.42:0/1533834057 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f86e40a3da0 msgr2=0x7f86e40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:26.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.035+0000 7f8705c00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f86e40a3da0 0x7f86e40a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f86f80df1d0 tx=0x7f86f800ef50 comp rx=0 tx=0).stop 2024-04-20T10:44:26.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.035+0000 7f8705c00640 1 -- 172.21.15.42:0/1533834057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8700108f60 msgr2=0x7f87001176b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:26.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.035+0000 7f8705c00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8700108f60 0x7f87001176b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f86f800ebc0 tx=0x7f86f800c6a0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.035+0000 7f8705c00640 1 -- 172.21.15.42:0/1533834057 shutdown_connections 2024-04-20T10:44:26.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.035+0000 7f8705c00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f86e40a3da0 0x7f86e40a6260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.035+0000 7f8705c00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f870010e840 0x7f8700118130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.035+0000 7f8705c00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8700109920 0x7f8700117bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.035+0000 7f8705c00640 1 --2- 172.21.15.42:0/1533834057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8700108f60 0x7f87001176b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:26.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.035+0000 7f8705c00640 1 -- 172.21.15.42:0/1533834057 >> 172.21.15.42:0/1533834057 conn(0x7f870006cee0 msgr2=0x7f870010af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:26.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.039+0000 7f8705c00640 1 -- 172.21.15.42:0/1533834057 shutdown_connections 2024-04-20T10:44:26.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:26.039+0000 7f8705c00640 1 -- 172.21.15.42:0/1533834057 wait complete. 2024-04-20T10:44:26.143 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fat/inode.o 2024-04-20T10:44:26.145 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/isofs/namei.o 2024-04-20T10:44:26.219 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/tctx.o 2024-04-20T10:44:26.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:26 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/888267317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:44:26.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:26 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/888267317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:44:26.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:26 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/888267317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:44:26.564 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/migrate.o 2024-04-20T10:44:26.968 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/isofs/inode.o 2024-04-20T10:44:27.030 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_nop.o 2024-04-20T10:44:27.082 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:44:27.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:44:27.083 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:44:27.269 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/poll.o 2024-04-20T10:44:27.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:27 smithi042 ceph-mon[17078]: pgmap v324: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 963 KiB/s wr, 200 op/s 2024-04-20T10:44:27.367 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/sparse.o 2024-04-20T10:44:27.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:27 smithi162 ceph-mon[21854]: pgmap v324: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 963 KiB/s wr, 200 op/s 2024-04-20T10:44:27.400 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-iocost.o 2024-04-20T10:44:27.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.536+0000 7f8ed1600640 1 -- 172.21.15.42:0/345008265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc10c510 msgr2=0x7f8ecc108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:27.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.536+0000 7f8ed1600640 1 --2- 172.21.15.42:0/345008265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc10c510 0x7f8ecc108f60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009900 tx=0x7f8eb402bc40 comp rx=0 tx=0).stop 2024-04-20T10:44:27.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.538+0000 7f8ed1600640 1 -- 172.21.15.42:0/345008265 shutdown_connections 2024-04-20T10:44:27.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.538+0000 7f8ed1600640 1 --2- 172.21.15.42:0/345008265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ecc116f60 0x7f8ecc119350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.538+0000 7f8ed1600640 1 --2- 172.21.15.42:0/345008265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ecc1094a0 0x7f8ecc109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.538+0000 7f8ed1600640 1 --2- 172.21.15.42:0/345008265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc10c510 0x7f8ecc108f60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.538+0000 7f8ed1600640 1 -- 172.21.15.42:0/345008265 >> 172.21.15.42:0/345008265 conn(0x7f8ecc06cee0 msgr2=0x7f8ecc06d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:27.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.538+0000 7f8ed1600640 1 -- 172.21.15.42:0/345008265 shutdown_connections 2024-04-20T10:44:27.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.538+0000 7f8ed1600640 1 -- 172.21.15.42:0/345008265 wait complete. 2024-04-20T10:44:27.540 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fat/misc.o 2024-04-20T10:44:27.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.540+0000 7f8ed1600640 1 Processor -- start 2024-04-20T10:44:27.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.540+0000 7f8ed1600640 1 -- start start 2024-04-20T10:44:27.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.541+0000 7f8ed1600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ecc1094a0 0x7f8ecc10ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.541+0000 7f8ed1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc10c510 0x7f8ecc1104b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.541+0000 7f8ed1600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ecc116f60 0x7f8ecc1a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.541+0000 7f8ed1600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc11cc40 con 0x7f8ecc116f60 2024-04-20T10:44:27.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.541+0000 7f8ed1600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc11cac0 con 0x7f8ecc10c510 2024-04-20T10:44:27.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.541+0000 7f8ed1600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc11cdc0 con 0x7f8ecc1094a0 2024-04-20T10:44:27.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.541+0000 7f8ecb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc10c510 0x7f8ecc1104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.541+0000 7f8ecbe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ecc1094a0 0x7f8ecc10ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.541+0000 7f8ecb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc10c510 0x7f8ecc1104b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:34558/0 (socket says 172.21.15.42:34558) 2024-04-20T10:44:27.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.541+0000 7f8ecb400640 1 -- 172.21.15.42:0/349784551 learned_addr learned my addr 172.21.15.42:0/349784551 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:27.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.542+0000 7f8ed0c00640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ecc116f60 0x7f8ecc1a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.542+0000 7f8ecb400640 1 -- 172.21.15.42:0/349784551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ecc1094a0 msgr2=0x7f8ecc10ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:27.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.542+0000 7f8ecb400640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ecc1094a0 0x7f8ecc10ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.542+0000 7f8ecb400640 1 -- 172.21.15.42:0/349784551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ecc116f60 msgr2=0x7f8ecc1a8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:27.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.542+0000 7f8ecb400640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ecc116f60 0x7f8ecc1a8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.542+0000 7f8ecb400640 1 -- 172.21.15.42:0/349784551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc1a9830 con 0x7f8ecc10c510 2024-04-20T10:44:27.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.542+0000 7f8ed0c00640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ecc116f60 0x7f8ecc1a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:44:27.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.543+0000 7f8ecb400640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc10c510 0x7f8ecc1104b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00db80 tx=0x7f8ebc00b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:27.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.546+0000 7f8ec8c00640 1 -- 172.21.15.42:0/349784551 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc017070 con 0x7f8ecc10c510 2024-04-20T10:44:27.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.546+0000 7f8ec8c00640 1 -- 172.21.15.42:0/349784551 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc003b70 con 0x7f8ecc10c510 2024-04-20T10:44:27.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.546+0000 7f8ec8c00640 1 -- 172.21.15.42:0/349784551 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc0106c0 con 0x7f8ecc10c510 2024-04-20T10:44:27.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.546+0000 7f8ed1600640 1 -- 172.21.15.42:0/349784551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ecc1a9b20 con 0x7f8ecc10c510 2024-04-20T10:44:27.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.546+0000 7f8ed1600640 1 -- 172.21.15.42:0/349784551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ecc116060 con 0x7f8ecc10c510 2024-04-20T10:44:27.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:27 smithi046 ceph-mon[21574]: pgmap v324: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 963 KiB/s wr, 200 op/s 2024-04-20T10:44:27.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.555+0000 7f8ed5873740 1 Processor -- start 2024-04-20T10:44:27.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.556+0000 7f8ec8c00640 1 -- 172.21.15.42:0/349784551 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8ebc004020 con 0x7f8ecc10c510 2024-04-20T10:44:27.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.556+0000 7f8ed5873740 1 -- start start 2024-04-20T10:44:27.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.557+0000 7f8ed5873740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x5577547934d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.557+0000 7f8ed5873740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x55775479c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.557+0000 7f8ed5873740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x55775479e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.557+0000 7f8ed5873740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5577546f7d20 con 0x557754793a10 2024-04-20T10:44:27.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.557+0000 7f8ed5873740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55775469e3e0 con 0x5577547930f0 2024-04-20T10:44:27.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.557+0000 7f8ed5873740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5577546f3ce0 con 0x55775479c5f0 2024-04-20T10:44:27.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.558+0000 7f8ed0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x5577547934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.558+0000 7f8ed0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x5577547934d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:34572/0 (socket says 172.21.15.42:34572) 2024-04-20T10:44:27.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.558+0000 7f8ed0c00640 1 -- 172.21.15.42:0/3378121293 learned_addr learned my addr 172.21.15.42:0/3378121293 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:27.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.558+0000 7f8ed0c00640 1 -- 172.21.15.42:0/3378121293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 msgr2=0x55775479e9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:44:27.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.558+0000 7f8ed0c00640 1 --2- 172.21.15.42:0/3378121293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x55775479e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.558+0000 7f8ed0c00640 1 -- 172.21.15.42:0/3378121293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 msgr2=0x55775479c0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:44:27.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.558+0000 7f8ed0c00640 1 --2- 172.21.15.42:0/3378121293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x55775479c0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.558+0000 7f8ed0c00640 1 -- 172.21.15.42:0/3378121293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577546bd980 con 0x5577547930f0 2024-04-20T10:44:27.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.559+0000 7f8ec8c00640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8eb00a3ef0 0x7f8eb00a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.559+0000 7f8ec8c00640 1 -- 172.21.15.42:0/349784551 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f8ebc0d0570 con 0x7f8ecc10c510 2024-04-20T10:44:27.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.560+0000 7f8ecbe00640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8eb00a3ef0 0x7f8eb00a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.561+0000 7f8ecbe00640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8eb00a3ef0 0x7f8eb00a63b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8eb402efd0 tx=0x7f8eb402cf90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:27.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.563+0000 7f8ed0c00640 1 --2- 172.21.15.42:0/3378121293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x5577547934d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009af0 tx=0x7f8ec4010370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdf4782082273186 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.563+0000 7f8ec1200640 1 -- 172.21.15.42:0/3378121293 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4010da0 con 0x5577547930f0 2024-04-20T10:44:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.563+0000 7f8ec1200640 1 -- 172.21.15.42:0/3378121293 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec401b960 con 0x5577547930f0 2024-04-20T10:44:27.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.563+0000 7f8ec1200640 1 -- 172.21.15.42:0/3378121293 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4012a70 con 0x5577547930f0 2024-04-20T10:44:27.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.565+0000 7f8ed5873740 1 -- 172.21.15.42:0/3378121293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 msgr2=0x5577547934d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:27.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.565+0000 7f8ed5873740 1 --2- 172.21.15.42:0/3378121293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x5577547934d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009af0 tx=0x7f8ec4010370 comp rx=0 tx=0).stop 2024-04-20T10:44:27.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.566+0000 7f8ed5873740 1 -- 172.21.15.42:0/3378121293 shutdown_connections 2024-04-20T10:44:27.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.566+0000 7f8ed5873740 1 --2- 172.21.15.42:0/3378121293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x55775479e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.566+0000 7f8ed5873740 1 --2- 172.21.15.42:0/3378121293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x55775479c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.566+0000 7f8ed5873740 1 --2- 172.21.15.42:0/3378121293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x5577547934d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.566+0000 7f8ed5873740 1 -- 172.21.15.42:0/3378121293 >> 172.21.15.42:0/3378121293 conn(0x55775478d820 msgr2=0x55775478dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:27.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.566+0000 7f8ed5873740 1 -- 172.21.15.42:0/3378121293 shutdown_connections 2024-04-20T10:44:27.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.566+0000 7f8ed5873740 1 -- 172.21.15.42:0/3378121293 wait complete. 2024-04-20T10:44:27.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.567+0000 7f8ed5873740 1 -- start start 2024-04-20T10:44:27.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.567+0000 7f8ed5873740 1 Processor -- start 2024-04-20T10:44:27.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.568+0000 7f8ed5873740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x557754799730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.568+0000 7f8ed5873740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557754793a10 0x557754799c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.568+0000 7f8ecbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x557754799730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.569+0000 7f8ecbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x557754799730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:34588/0 (socket says 172.21.15.42:34588) 2024-04-20T10:44:27.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.569+0000 7f8ecbe00640 1 -- 172.21.15.42:0/4270799428 learned_addr learned my addr 172.21.15.42:0/4270799428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:27.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.569+0000 7f8ed5873740 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55775479c5f0 0x55775479a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.569+0000 7f8ed5873740 1 -- 172.21.15.42:0/4270799428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55775479f3e0 con 0x55775479c5f0 2024-04-20T10:44:27.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.569+0000 7f8ed5873740 1 -- 172.21.15.42:0/4270799428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5577546b17a0 con 0x5577547930f0 2024-04-20T10:44:27.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.569+0000 7f8ed5873740 1 -- 172.21.15.42:0/4270799428 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55775479f560 con 0x557754793a10 2024-04-20T10:44:27.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.569+0000 7f8ed5873740 10 client.0 _ll_register_callbacks cb 0x5577546b3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:44:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.569+0000 7f8ecbe00640 1 -- 172.21.15.42:0/4270799428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557754793a10 msgr2=0x557754799c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:44:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.569+0000 7f8ecbe00640 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557754793a10 0x557754799c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.569+0000 7f8ecbe00640 1 -- 172.21.15.42:0/4270799428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55775479c5f0 msgr2=0x55775479a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.570+0000 7f8ed0c00640 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55775479c5f0 0x55775479a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.570+0000 7f8ecbe00640 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55775479c5f0 0x55775479a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.570+0000 7f8ecbe00640 1 -- 172.21.15.42:0/4270799428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577544d27e0 con 0x5577547930f0 2024-04-20T10:44:27.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.570+0000 7f8ed0c00640 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55775479c5f0 0x55775479a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:44:27.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.572+0000 7f8ecbe00640 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x557754799730 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009900 tx=0x7f8eb4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2004c9253450b7d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:27.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.572+0000 7f8eb9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:44:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.572+0000 7f8eb9600640 1 -- 172.21.15.42:0/4270799428 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb404f070 con 0x5577547930f0 2024-04-20T10:44:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.573+0000 7f8ed5873740 1 -- 172.21.15.42:0/4270799428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec401b960 con 0x5577547930f0 2024-04-20T10:44:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.573+0000 7f8eb9600640 1 -- 172.21.15.42:0/4270799428 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f8eb4004f80 con 0x5577547930f0 2024-04-20T10:44:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.573+0000 7f8eb9600640 1 -- 172.21.15.42:0/4270799428 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8eb40044d0 con 0x5577547930f0 2024-04-20T10:44:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.573+0000 7f8ed5873740 10 client.26576 fetch_fsmap learned FSMap version 19 2024-04-20T10:44:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.573+0000 7f8ed5873740 1 -- 172.21.15.42:0/4270799428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557754532f70 con 0x5577547930f0 2024-04-20T10:44:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.573+0000 7f8eb9600640 1 -- 172.21.15.42:0/4270799428 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec401b960 con 0x5577547930f0 2024-04-20T10:44:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.573+0000 7f8eb9600640 1 -- 172.21.15.42:0/4270799428 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f8eb4003b20 con 0x5577547930f0 2024-04-20T10:44:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.574+0000 7f8ed5873740 10 client.26576 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:44:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.574+0000 7f8ed5873740 10 client.26576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:44:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.574+0000 7f8ed5873740 20 client.26576 populate_metadata read hostname 'smithi042' 2024-04-20T10:44:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.574+0000 7f8ed5873740 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5577547acde0 0x5577547af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.574+0000 7f8ed5873740 4 client.26576 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:44:27.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.574+0000 7f8ed5873740 1 -- 172.21.15.42:0/4270799428 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557754705740 con 0x5577547acde0 2024-04-20T10:44:27.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.574+0000 7f8ecb400640 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5577547acde0 0x5577547af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.575+0000 7f8ecb400640 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5577547acde0 0x5577547af270 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5598d42872bf6b9b server_cookie=bfbb64f250d61775 in_seq=0 out_seq=0 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.575+0000 7f8eb9600640 10 client.26576 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.576+0000 7f8eb9600640 1 -- 172.21.15.42:0/4270799428 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557754705740 con 0x5577547acde0 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.576+0000 7f8eb9600640 10 client.26576 handle_command_reply: tid=0 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.576+0000 7f8ed5873740 1 client.26576 shutdown 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.576+0000 7f8ed5873740 1 -- 172.21.15.42:0/4270799428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 msgr2=0x557754799730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.576+0000 7f8ed5873740 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x557754799730 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009900 tx=0x7f8eb4009260 comp rx=0 tx=0).stop 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.576+0000 7f8ed5873740 1 -- 172.21.15.42:0/4270799428 shutdown_connections 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.576+0000 7f8ed5873740 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x5577547acde0 0x5577547af270 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.576+0000 7f8ed5873740 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55775479c5f0 0x55775479a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.576+0000 7f8ed5873740 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557754793a10 0x557754799c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.576+0000 7f8ed5873740 1 --2- 172.21.15.42:0/4270799428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x557754799730 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.576+0000 7f8ed5873740 1 -- 172.21.15.42:0/4270799428 >> 172.21.15.42:0/4270799428 conn(0x55775478d820 msgr2=0x55775478f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:27.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.578+0000 7f8eb9600640 0 client.26576 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:27.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.578+0000 7f8ed5873740 1 -- 172.21.15.42:0/4270799428 shutdown_connections 2024-04-20T10:44:27.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.578+0000 7f8ed5873740 1 -- 172.21.15.42:0/4270799428 wait complete. 2024-04-20T10:44:27.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.578+0000 7f8ed5873740 20 client.26576 trim_cache size 0 max 16384 2024-04-20T10:44:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.657+0000 7f8ed5873740 1 Processor -- start 2024-04-20T10:44:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.657+0000 7f8ed5873740 1 -- start start 2024-04-20T10:44:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.658+0000 7f8ed5873740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x55775479be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.658+0000 7f8ed5873740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x5577547a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.658+0000 7f8ed5873740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x5577547b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.658+0000 7f8ed5873740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1c3200 con 0x557754793a10 2024-04-20T10:44:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.658+0000 7f8ed5873740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1c5a60 con 0x5577547930f0 2024-04-20T10:44:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.658+0000 7f8ed5873740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1c3850 con 0x55775479c5f0 2024-04-20T10:44:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.659+0000 7f8ed0c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x5577547b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.659+0000 7f8ed0c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x5577547b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:49964/0 (socket says 172.21.15.42:49964) 2024-04-20T10:44:27.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.659+0000 7f8ed0c00640 1 -- 172.21.15.42:0/976614312 learned_addr learned my addr 172.21.15.42:0/976614312 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.659+0000 7f8ecb400640 1 --2- 172.21.15.42:0/976614312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x5577547a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.660+0000 7f8ed0c00640 1 -- 172.21.15.42:0/976614312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 msgr2=0x55775479be60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:44:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.660+0000 7f8ed0c00640 1 --2- 172.21.15.42:0/976614312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x55775479be60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.660+0000 7f8ed0c00640 1 -- 172.21.15.42:0/976614312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 msgr2=0x5577547a82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.660+0000 7f8ed0c00640 1 --2- 172.21.15.42:0/976614312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x5577547a82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.660+0000 7f8ed0c00640 1 -- 172.21.15.42:0/976614312 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577543ca170 con 0x55775479c5f0 2024-04-20T10:44:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.660+0000 7f8ecb400640 1 --2- 172.21.15.42:0/976614312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x5577547a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:44:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.660+0000 7f8ed0c00640 1 --2- 172.21.15.42:0/976614312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x5577547b4b80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8ec401a000 tx=0x7f8ec4005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45cd4d7d3c00ab54 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.661+0000 7f8ebbe00640 1 -- 172.21.15.42:0/976614312 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4029070 con 0x55775479c5f0 2024-04-20T10:44:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.661+0000 7f8ebbe00640 1 -- 172.21.15.42:0/976614312 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4005510 con 0x55775479c5f0 2024-04-20T10:44:27.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.661+0000 7f8ebbe00640 1 -- 172.21.15.42:0/976614312 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4004d90 con 0x55775479c5f0 2024-04-20T10:44:27.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.662+0000 7f8ed5873740 1 -- 172.21.15.42:0/976614312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 msgr2=0x5577547b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:27.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.662+0000 7f8ed5873740 1 --2- 172.21.15.42:0/976614312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x5577547b4b80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8ec401a000 tx=0x7f8ec4005b20 comp rx=0 tx=0).stop 2024-04-20T10:44:27.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.666+0000 7f8ed5873740 1 -- 172.21.15.42:0/976614312 shutdown_connections 2024-04-20T10:44:27.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.666+0000 7f8ed5873740 1 --2- 172.21.15.42:0/976614312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x5577547b4b80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.666+0000 7f8ed5873740 1 --2- 172.21.15.42:0/976614312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x5577547a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.666+0000 7f8ed5873740 1 --2- 172.21.15.42:0/976614312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x55775479be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.666+0000 7f8ed5873740 1 -- 172.21.15.42:0/976614312 >> 172.21.15.42:0/976614312 conn(0x55775478d820 msgr2=0x557754794a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:27.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.672+0000 7f8ed5873740 1 -- 172.21.15.42:0/976614312 shutdown_connections 2024-04-20T10:44:27.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.674+0000 7f8ed5873740 1 -- 172.21.15.42:0/976614312 wait complete. 2024-04-20T10:44:27.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.674+0000 7f8ed5873740 1 -- start start 2024-04-20T10:44:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.675+0000 7f8ed5873740 1 Processor -- start 2024-04-20T10:44:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.675+0000 7f8ed5873740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x55775479fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.676+0000 7f8ed5873740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x55775479ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.676+0000 7f8ed5873740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x55775480b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.676+0000 7f8ecbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x55775479fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.676+0000 7f8ecbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x55775479fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:34604/0 (socket says 172.21.15.42:34604) 2024-04-20T10:44:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.676+0000 7f8ecbe00640 1 -- 172.21.15.42:0/265075467 learned_addr learned my addr 172.21.15.42:0/265075467 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.676+0000 7f8ed5873740 1 -- 172.21.15.42:0/265075467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1c3f20 con 0x557754793a10 2024-04-20T10:44:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.676+0000 7f8ed5873740 1 -- 172.21.15.42:0/265075467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1c5390 con 0x5577547930f0 2024-04-20T10:44:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.676+0000 7f8ed5873740 1 -- 172.21.15.42:0/265075467 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc1c45f0 con 0x55775479c5f0 2024-04-20T10:44:27.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.676+0000 7f8ed5873740 10 client.0 _ll_register_callbacks cb 0x7f8eb4042070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:44:27.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.676+0000 7f8ed0c00640 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x55775480b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.676+0000 7f8ecb400640 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x55775479ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.677+0000 7f8ecbe00640 1 -- 172.21.15.42:0/265075467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 msgr2=0x55775480b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:27.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.677+0000 7f8ecbe00640 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x55775480b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.677+0000 7f8ecbe00640 1 -- 172.21.15.42:0/265075467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 msgr2=0x55775479ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:27.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.677+0000 7f8ecbe00640 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x55775479ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.677+0000 7f8ecbe00640 1 -- 172.21.15.42:0/265075467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775480b9e0 con 0x5577547930f0 2024-04-20T10:44:27.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.677+0000 7f8ecb400640 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x55775479ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:44:27.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.681+0000 7f8ecbe00640 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x55775479fa80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8eb404b540 tx=0x7f8eb4005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cee44a089ffc049 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:27.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.682+0000 7f8eb9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:44:27.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.682+0000 7f8eb9600640 1 -- 172.21.15.42:0/265075467 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4064070 con 0x5577547930f0 2024-04-20T10:44:27.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.682+0000 7f8eb9600640 1 -- 172.21.15.42:0/265075467 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f8eb404bda0 con 0x5577547930f0 2024-04-20T10:44:27.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.682+0000 7f8ed5873740 1 -- 172.21.15.42:0/265075467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec4005510 con 0x5577547930f0 2024-04-20T10:44:27.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.682+0000 7f8eb9600640 1 -- 172.21.15.42:0/265075467 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8eb406e050 con 0x5577547930f0 2024-04-20T10:44:27.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.683+0000 7f8ed5873740 10 client.26579 fetch_fsmap learned FSMap version 19 2024-04-20T10:44:27.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.683+0000 7f8ed5873740 1 -- 172.21.15.42:0/265075467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557754812db0 con 0x5577547930f0 2024-04-20T10:44:27.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.683+0000 7f8eb9600640 1 -- 172.21.15.42:0/265075467 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4005510 con 0x5577547930f0 2024-04-20T10:44:27.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.683+0000 7f8eb9600640 1 -- 172.21.15.42:0/265075467 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f8eb4005310 con 0x5577547930f0 2024-04-20T10:44:27.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.684+0000 7f8ed5873740 10 client.26579 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:44:27.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.684+0000 7f8ed5873740 10 client.26579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:44:27.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.684+0000 7f8ed5873740 20 client.26579 populate_metadata read hostname 'smithi042' 2024-04-20T10:44:27.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.684+0000 7f8ed5873740 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x557754813470 0x557754815890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:27.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.684+0000 7f8ed5873740 4 client.26579 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:44:27.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.684+0000 7f8ed5873740 1 -- 172.21.15.42:0/265075467 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5577546ff5b0 con 0x557754813470 2024-04-20T10:44:27.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.684+0000 7f8ecb400640 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x557754813470 0x557754815890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:27.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.685+0000 7f8ecb400640 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x557754813470 0x557754815890 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35cef613f5c01846 server_cookie=9ac35f7ab657f4d9 in_seq=0 out_seq=0 2024-04-20T10:44:27.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.685+0000 7f8eb9600640 10 client.26579 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:27.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.690+0000 7f8eb9600640 1 -- 172.21.15.42:0/265075467 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5577546ff5b0 con 0x557754813470 2024-04-20T10:44:27.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.690+0000 7f8eb9600640 10 client.26579 handle_command_reply: tid=0 2024-04-20T10:44:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.690+0000 7f8ed5873740 1 client.26579 shutdown 2024-04-20T10:44:27.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.693+0000 7f8ed5873740 1 -- 172.21.15.42:0/265075467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 msgr2=0x55775479fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:27.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.693+0000 7f8ed5873740 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x55775479fa80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8eb404b540 tx=0x7f8eb4005e10 comp rx=0 tx=0).stop 2024-04-20T10:44:27.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.693+0000 7f8ed5873740 1 -- 172.21.15.42:0/265075467 shutdown_connections 2024-04-20T10:44:27.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.693+0000 7f8ed5873740 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x557754813470 0x557754815890 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.693+0000 7f8ed5873740 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55775479c5f0 0x55775480b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.693+0000 7f8ed5873740 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557754793a10 0x55775479ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.693+0000 7f8ed5873740 1 --2- 172.21.15.42:0/265075467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577547930f0 0x55775479fa80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.693+0000 7f8eb9600640 0 client.26579 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:27.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.693+0000 7f8ed5873740 1 -- 172.21.15.42:0/265075467 >> 172.21.15.42:0/265075467 conn(0x55775478d820 msgr2=0x55775478c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:27.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.695+0000 7f8ed5873740 1 -- 172.21.15.42:0/265075467 shutdown_connections 2024-04-20T10:44:27.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.695+0000 7f8ed5873740 1 -- 172.21.15.42:0/265075467 wait complete. 2024-04-20T10:44:27.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.695+0000 7f8ed5873740 20 client.26579 trim_cache size 0 max 16384 2024-04-20T10:44:27.696 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:44:27.696 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:44:27.696 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "7046c3c8-28e3-4ab4-b6cd-8fafe5c353ae", 2024-04-20T10:44:27.696 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:44:27.697 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:44:27.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.698+0000 7f8ed1600640 1 -- 172.21.15.42:0/349784551 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8eb00a3ef0 msgr2=0x7f8eb00a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:27.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.698+0000 7f8ed1600640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8eb00a3ef0 0x7f8eb00a63b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8eb402efd0 tx=0x7f8eb402cf90 comp rx=0 tx=0).stop 2024-04-20T10:44:27.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.698+0000 7f8ed1600640 1 -- 172.21.15.42:0/349784551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc10c510 msgr2=0x7f8ecc1104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:27.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.698+0000 7f8ed1600640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc10c510 0x7f8ecc1104b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00db80 tx=0x7f8ebc00b630 comp rx=0 tx=0).stop 2024-04-20T10:44:27.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.698+0000 7f8ed1600640 1 -- 172.21.15.42:0/349784551 shutdown_connections 2024-04-20T10:44:27.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.698+0000 7f8ed1600640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8eb00a3ef0 0x7f8eb00a63b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.698+0000 7f8ed1600640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ecc116f60 0x7f8ecc1a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.698+0000 7f8ed1600640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ecc10c510 0x7f8ecc1104b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.698+0000 7f8ed1600640 1 --2- 172.21.15.42:0/349784551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ecc1094a0 0x7f8ecc10ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:27.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.698+0000 7f8ed1600640 1 -- 172.21.15.42:0/349784551 >> 172.21.15.42:0/349784551 conn(0x7f8ecc06cee0 msgr2=0x7f8ecc117340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:27.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.700+0000 7f8ed1600640 1 -- 172.21.15.42:0/349784551 shutdown_connections 2024-04-20T10:44:27.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:27.700+0000 7f8ed1600640 1 -- 172.21.15.42:0/349784551 wait complete. 2024-04-20T10:44:27.753 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/events/core.o 2024-04-20T10:44:27.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:44:27.758 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:44:28.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:28 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:44:28.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:28 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:28.066 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:28 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:28.162 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/blktrace.o 2024-04-20T10:44:28.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.283+0000 7fdc27e00640 1 -- 172.21.15.42:0/796347722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2810d0f0 msgr2=0x7fdc28119600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:28.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.283+0000 7fdc27e00640 1 --2- 172.21.15.42:0/796347722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2810d0f0 0x7fdc28119600 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdc20013150 tx=0x7fdc20033fc0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.288+0000 7fdc27e00640 1 -- 172.21.15.42:0/796347722 shutdown_connections 2024-04-20T10:44:28.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.288+0000 7fdc27e00640 1 --2- 172.21.15.42:0/796347722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2810d0f0 0x7fdc28119600 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.288+0000 7fdc27e00640 1 --2- 172.21.15.42:0/796347722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc2810c730 0x7fdc2810cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.288+0000 7fdc27e00640 1 --2- 172.21.15.42:0/796347722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc28112740 0x7fdc28112b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.288+0000 7fdc27e00640 1 -- 172.21.15.42:0/796347722 >> 172.21.15.42:0/796347722 conn(0x7fdc2806cda0 msgr2=0x7fdc2810af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:28.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.292+0000 7fdc27e00640 1 -- 172.21.15.42:0/796347722 shutdown_connections 2024-04-20T10:44:28.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.292+0000 7fdc27e00640 1 -- 172.21.15.42:0/796347722 wait complete. 2024-04-20T10:44:28.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.293+0000 7fdc27e00640 1 Processor -- start 2024-04-20T10:44:28.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.295+0000 7fdc27e00640 1 -- start start 2024-04-20T10:44:28.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.296+0000 7fdc27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2810c730 0x7fdc281a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.296+0000 7fdc27e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc2810d0f0 0x7fdc281a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.296+0000 7fdc27e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc28112740 0x7fdc2819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.296+0000 7fdc1e000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc2810d0f0 0x7fdc281a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.296+0000 7fdc1e000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc2810d0f0 0x7fdc281a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:49986/0 (socket says 172.21.15.42:49986) 2024-04-20T10:44:28.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.298+0000 7fdc1e000640 1 -- 172.21.15.42:0/1036410484 learned_addr learned my addr 172.21.15.42:0/1036410484 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:28.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.298+0000 7fdc27e00640 1 -- 172.21.15.42:0/1036410484 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdc28070270 con 0x7fdc28112740 2024-04-20T10:44:28.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.298+0000 7fdc27e00640 1 -- 172.21.15.42:0/1036410484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc280700f0 con 0x7fdc2810c730 2024-04-20T10:44:28.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.298+0000 7fdc27e00640 1 -- 172.21.15.42:0/1036410484 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fdc280703f0 con 0x7fdc2810d0f0 2024-04-20T10:44:28.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.298+0000 7fdc2d800640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc28112740 0x7fdc2819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.298+0000 7fdc1e000640 1 -- 172.21.15.42:0/1036410484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2810c730 msgr2=0x7fdc281a4830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T10:44:28.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.298+0000 7fdc1e000640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2810c730 0x7fdc281a4830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.298+0000 7fdc1e000640 1 -- 172.21.15.42:0/1036410484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc28112740 msgr2=0x7fdc2819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.298+0000 7fdc1e000640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc28112740 0x7fdc2819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.298+0000 7fdc1e000640 1 -- 172.21.15.42:0/1036410484 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2819f680 con 0x7fdc2810d0f0 2024-04-20T10:44:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.299+0000 7fdc2d800640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc28112740 0x7fdc2819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:44:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.299+0000 7fdc1e000640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc2810d0f0 0x7fdc281a4d70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c920 tx=0x7fdc1000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.299+0000 7fdc1fe00640 1 -- 172.21.15.42:0/1036410484 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10007c70 con 0x7fdc2810d0f0 2024-04-20T10:44:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.299+0000 7fdc1fe00640 1 -- 172.21.15.42:0/1036410484 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc10007e10 con 0x7fdc2810d0f0 2024-04-20T10:44:28.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.299+0000 7fdc27e00640 1 -- 172.21.15.42:0/1036410484 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2819f970 con 0x7fdc2810d0f0 2024-04-20T10:44:28.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.299+0000 7fdc1fe00640 1 -- 172.21.15.42:0/1036410484 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc100189a0 con 0x7fdc2810d0f0 2024-04-20T10:44:28.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.300+0000 7fdc27e00640 1 -- 172.21.15.42:0/1036410484 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc281ab6f0 con 0x7fdc2810d0f0 2024-04-20T10:44:28.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.310+0000 7fdc30413740 1 Processor -- start 2024-04-20T10:44:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.310+0000 7fdc30413740 1 -- start start 2024-04-20T10:44:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.311+0000 7fdc30413740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805b380 0x55d1b805b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.311+0000 7fdc30413740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805bca0 0x55d1b8064340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.311+0000 7fdc30413740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b8064880 0x55d1b8066c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.311+0000 7fdc30413740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d1b7fc1150 con 0x55d1b805bca0 2024-04-20T10:44:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.311+0000 7fdc30413740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1b7f67370 con 0x55d1b8064880 2024-04-20T10:44:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.311+0000 7fdc30413740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d1b7fbd110 con 0x55d1b805b380 2024-04-20T10:44:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.312+0000 7fdc26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805bca0 0x55d1b8064340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.313+0000 7fdc26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805bca0 0x55d1b8064340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33260/0 (socket says 172.21.15.42:33260) 2024-04-20T10:44:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.313+0000 7fdc26a00640 1 -- 172.21.15.42:0/3093188606 learned_addr learned my addr 172.21.15.42:0/3093188606 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.313+0000 7fdc1fe00640 1 -- 172.21.15.42:0/1036410484 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fdc10018b40 con 0x7fdc2810d0f0 2024-04-20T10:44:28.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.313+0000 7fdc1e000640 1 --2- 172.21.15.42:0/3093188606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b8064880 0x55d1b8066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.314+0000 7fdc2d800640 1 --2- 172.21.15.42:0/3093188606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805b380 0x55d1b805b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.314+0000 7fdc1e000640 1 -- 172.21.15.42:0/3093188606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805b380 msgr2=0x55d1b805b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:28.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.314+0000 7fdc1e000640 1 --2- 172.21.15.42:0/3093188606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805b380 0x55d1b805b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.314+0000 7fdc1e000640 1 -- 172.21.15.42:0/3093188606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805bca0 msgr2=0x55d1b8064340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:28.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.314+0000 7fdc1e000640 1 --2- 172.21.15.42:0/3093188606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805bca0 0x55d1b8064340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.314+0000 7fdc1e000640 1 -- 172.21.15.42:0/3093188606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b7d9d7d0 con 0x55d1b8064880 2024-04-20T10:44:28.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.314+0000 7fdc26a00640 1 --2- 172.21.15.42:0/3093188606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805bca0 0x55d1b8064340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:44:28.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.315+0000 7fdc1e000640 1 --2- 172.21.15.42:0/3093188606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b8064880 0x55d1b8066c70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdc100cdec0 tx=0x7fdc100d6cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=841b3e10a0a4b552 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:28.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.315+0000 7fdc1fe00640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fdc0c0a3da0 0x7fdc0c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.316+0000 7fdc1fe00640 1 -- 172.21.15.42:0/1036410484 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fdc100d07a0 con 0x7fdc2810d0f0 2024-04-20T10:44:28.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.316+0000 7fdc2d800640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fdc0c0a3da0 0x7fdc0c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.317+0000 7fdc17e00640 1 -- 172.21.15.42:0/3093188606 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc100df070 con 0x55d1b8064880 2024-04-20T10:44:28.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.317+0000 7fdc17e00640 1 -- 172.21.15.42:0/3093188606 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc100db440 con 0x55d1b8064880 2024-04-20T10:44:28.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.317+0000 7fdc17e00640 1 -- 172.21.15.42:0/3093188606 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc100d22d0 con 0x55d1b8064880 2024-04-20T10:44:28.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.317+0000 7fdc2d800640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fdc0c0a3da0 0x7fdc0c0a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdc200047e0 tx=0x7fdc20011a70 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:28.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.321+0000 7fdc30413740 1 -- 172.21.15.42:0/3093188606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b8064880 msgr2=0x55d1b8066c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:28.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.321+0000 7fdc30413740 1 --2- 172.21.15.42:0/3093188606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b8064880 0x55d1b8066c70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdc100cdec0 tx=0x7fdc100d6cd0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.321+0000 7fdc30413740 1 -- 172.21.15.42:0/3093188606 shutdown_connections 2024-04-20T10:44:28.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.321+0000 7fdc30413740 1 --2- 172.21.15.42:0/3093188606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b8064880 0x55d1b8066c70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.321+0000 7fdc30413740 1 --2- 172.21.15.42:0/3093188606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805bca0 0x55d1b8064340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.321+0000 7fdc30413740 1 --2- 172.21.15.42:0/3093188606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805b380 0x55d1b805b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.321+0000 7fdc30413740 1 -- 172.21.15.42:0/3093188606 >> 172.21.15.42:0/3093188606 conn(0x55d1b8055ab0 msgr2=0x55d1b8055e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:28.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.322+0000 7fdc30413740 1 -- 172.21.15.42:0/3093188606 shutdown_connections 2024-04-20T10:44:28.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.325+0000 7fdc30413740 1 -- 172.21.15.42:0/3093188606 wait complete. 2024-04-20T10:44:28.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.325+0000 7fdc30413740 1 -- start start 2024-04-20T10:44:28.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.326+0000 7fdc30413740 1 Processor -- start 2024-04-20T10:44:28.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.327+0000 7fdc30413740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b805b380 0x55d1b8061880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.327+0000 7fdc30413740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 0x55d1b8061dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.327+0000 7fdc30413740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b8064880 0x55d1b8062300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.327+0000 7fdc30413740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d1b8067670 con 0x55d1b8064880 2024-04-20T10:44:28.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.327+0000 7fdc30413740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1b7f7a740 con 0x55d1b805b380 2024-04-20T10:44:28.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.327+0000 7fdc30413740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d1b80677f0 con 0x55d1b805bca0 2024-04-20T10:44:28.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.327+0000 7fdc30413740 10 client.0 _ll_register_callbacks cb 0x55d1b7f75b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:44:28.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.328+0000 7fdc26a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b805b380 0x55d1b8061880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.328+0000 7fdc26a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b805b380 0x55d1b8061880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:34634/0 (socket says 172.21.15.42:34634) 2024-04-20T10:44:28.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.328+0000 7fdc26a00640 1 -- 172.21.15.42:0/2007848141 learned_addr learned my addr 172.21.15.42:0/2007848141 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.328+0000 7fdc1e000640 1 --2- 172.21.15.42:0/2007848141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 0x55d1b8061dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.328+0000 7fdc1e000640 1 -- 172.21.15.42:0/2007848141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b805b380 msgr2=0x55d1b8061880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.328+0000 7fdc1e000640 1 --2- 172.21.15.42:0/2007848141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b805b380 0x55d1b8061880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.328+0000 7fdc1e000640 1 -- 172.21.15.42:0/2007848141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b8064880 msgr2=0x55d1b8062300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:44:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.328+0000 7fdc1e000640 1 --2- 172.21.15.42:0/2007848141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b8064880 0x55d1b8062300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.328+0000 7fdc1e000640 1 -- 172.21.15.42:0/2007848141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b7cd33f0 con 0x55d1b805bca0 2024-04-20T10:44:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.329+0000 7fdc1e000640 1 --2- 172.21.15.42:0/2007848141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 0x55d1b8061dc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdc100cdf20 tx=0x7fdc100cde30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21c8fc9a1d5c7878 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.329+0000 7fdc14c00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:44:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.329+0000 7fdc14c00640 1 -- 172.21.15.42:0/2007848141 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc100df030 con 0x55d1b805bca0 2024-04-20T10:44:28.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.329+0000 7fdc14c00640 1 -- 172.21.15.42:0/2007848141 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fdc100d6790 con 0x55d1b805bca0 2024-04-20T10:44:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.332+0000 7fdc30413740 1 -- 172.21.15.42:0/2007848141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc100db440 con 0x55d1b805bca0 2024-04-20T10:44:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.332+0000 7fdc14c00640 1 -- 172.21.15.42:0/2007848141 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdc100989b0 con 0x55d1b805bca0 2024-04-20T10:44:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.332+0000 7fdc30413740 10 client.26578 fetch_fsmap learned FSMap version 19 2024-04-20T10:44:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.332+0000 7fdc30413740 1 -- 172.21.15.42:0/2007848141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d1b7c930f0 con 0x55d1b805bca0 2024-04-20T10:44:28.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.333+0000 7fdc14c00640 1 -- 172.21.15.42:0/2007848141 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc100db440 con 0x55d1b805bca0 2024-04-20T10:44:28.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.334+0000 7fdc14c00640 1 -- 172.21.15.42:0/2007848141 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fdc10098d80 con 0x55d1b805bca0 2024-04-20T10:44:28.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.338+0000 7fdc30413740 10 client.26578 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:44:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.338+0000 7fdc30413740 10 client.26578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:44:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.338+0000 7fdc30413740 20 client.26578 populate_metadata read hostname 'smithi042' 2024-04-20T10:44:28.341 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/isofs/dir.o 2024-04-20T10:44:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.338+0000 7fdc30413740 1 --2- 172.21.15.42:0/2007848141 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d1b8074ab0 0x55d1b8076ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.338+0000 7fdc30413740 4 client.26578 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:44:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.338+0000 7fdc30413740 1 -- 172.21.15.42:0/2007848141 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1b7fceb70 con 0x55d1b8074ab0 2024-04-20T10:44:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.338+0000 7fdc26a00640 1 --2- 172.21.15.42:0/2007848141 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d1b8074ab0 0x55d1b8076ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.339+0000 7fdc26a00640 1 --2- 172.21.15.42:0/2007848141 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d1b8074ab0 0x55d1b8076ea0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=964ee54dd2279845 server_cookie=1c230c28915add5e in_seq=0 out_seq=0 2024-04-20T10:44:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.339+0000 7fdc14c00640 10 client.26578 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.340+0000 7fdc14c00640 1 -- 172.21.15.42:0/2007848141 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1b7fceb70 con 0x55d1b8074ab0 2024-04-20T10:44:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.340+0000 7fdc14c00640 10 client.26578 handle_command_reply: tid=0 2024-04-20T10:44:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.341+0000 7fdc30413740 1 client.26578 shutdown 2024-04-20T10:44:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.341+0000 7fdc30413740 1 -- 172.21.15.42:0/2007848141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 msgr2=0x55d1b8061dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.341+0000 7fdc30413740 1 --2- 172.21.15.42:0/2007848141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 0x55d1b8061dc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdc100cdf20 tx=0x7fdc100cde30 comp rx=0 tx=0).stop 2024-04-20T10:44:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.341+0000 7fdc30413740 1 -- 172.21.15.42:0/2007848141 shutdown_connections 2024-04-20T10:44:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.341+0000 7fdc30413740 1 --2- 172.21.15.42:0/2007848141 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d1b8074ab0 0x55d1b8076ea0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.341+0000 7fdc30413740 1 --2- 172.21.15.42:0/2007848141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b8064880 0x55d1b8062300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.341+0000 7fdc30413740 1 --2- 172.21.15.42:0/2007848141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 0x55d1b8061dc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.341+0000 7fdc30413740 1 --2- 172.21.15.42:0/2007848141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b805b380 0x55d1b8061880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.341+0000 7fdc14c00640 0 client.26578 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.341+0000 7fdc30413740 1 -- 172.21.15.42:0/2007848141 >> 172.21.15.42:0/2007848141 conn(0x55d1b8055ab0 msgr2=0x55d1b8057800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.342+0000 7fdc30413740 1 -- 172.21.15.42:0/2007848141 shutdown_connections 2024-04-20T10:44:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.342+0000 7fdc30413740 1 -- 172.21.15.42:0/2007848141 wait complete. 2024-04-20T10:44:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.342+0000 7fdc30413740 20 client.26578 trim_cache size 0 max 16384 2024-04-20T10:44:28.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:28 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:44:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:28 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:28 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:28.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.432+0000 7fdc30413740 1 Processor -- start 2024-04-20T10:44:28.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.438+0000 7fdc30413740 1 -- start start 2024-04-20T10:44:28.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.438+0000 7fdc30413740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805b380 0x55d1b8063f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.439+0000 7fdc30413740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 0x55d1b80607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.439+0000 7fdc30413740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b8064880 0x55d1b8079ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.439+0000 7fdc30413740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0401d7d0 con 0x55d1b805b380 2024-04-20T10:44:28.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.439+0000 7fdc30413740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc04020030 con 0x55d1b8064880 2024-04-20T10:44:28.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.439+0000 7fdc30413740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0401de20 con 0x55d1b805bca0 2024-04-20T10:44:28.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.440+0000 7fdc1e000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 0x55d1b80607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.440+0000 7fdc26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805b380 0x55d1b8063f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.440+0000 7fdc1e000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 0x55d1b80607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:50012/0 (socket says 172.21.15.42:50012) 2024-04-20T10:44:28.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.440+0000 7fdc1e000640 1 -- 172.21.15.42:0/4124304615 learned_addr learned my addr 172.21.15.42:0/4124304615 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:28.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.440+0000 7fdc2d800640 1 --2- 172.21.15.42:0/4124304615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b8064880 0x55d1b8079ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.441+0000 7fdc1e000640 1 -- 172.21.15.42:0/4124304615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b8064880 msgr2=0x55d1b8079ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:28.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.441+0000 7fdc1e000640 1 --2- 172.21.15.42:0/4124304615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b8064880 0x55d1b8079ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.441+0000 7fdc1e000640 1 -- 172.21.15.42:0/4124304615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805b380 msgr2=0x55d1b8063f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:28.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.441+0000 7fdc1e000640 1 --2- 172.21.15.42:0/4124304615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805b380 0x55d1b8063f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.441+0000 7fdc1e000640 1 -- 172.21.15.42:0/4124304615 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b8066c70 con 0x55d1b805bca0 2024-04-20T10:44:28.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.441+0000 7fdc26a00640 1 --2- 172.21.15.42:0/4124304615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805b380 0x55d1b8063f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:44:28.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.441+0000 7fdc1e000640 1 --2- 172.21.15.42:0/4124304615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 0x55d1b80607c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c8f0 tx=0x7fdc100983b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd468c16f017a6b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:28.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.442+0000 7fdc17400640 1 -- 172.21.15.42:0/4124304615 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc100df070 con 0x55d1b805bca0 2024-04-20T10:44:28.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.442+0000 7fdc17400640 1 -- 172.21.15.42:0/4124304615 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc100b9020 con 0x55d1b805bca0 2024-04-20T10:44:28.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.442+0000 7fdc17400640 1 -- 172.21.15.42:0/4124304615 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1009c070 con 0x55d1b805bca0 2024-04-20T10:44:28.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.443+0000 7fdc30413740 1 -- 172.21.15.42:0/4124304615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 msgr2=0x55d1b80607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:28.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.443+0000 7fdc30413740 1 --2- 172.21.15.42:0/4124304615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 0x55d1b80607c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c8f0 tx=0x7fdc100983b0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.445+0000 7fdc30413740 1 -- 172.21.15.42:0/4124304615 shutdown_connections 2024-04-20T10:44:28.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.445+0000 7fdc30413740 1 --2- 172.21.15.42:0/4124304615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b8064880 0x55d1b8079ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.445+0000 7fdc30413740 1 --2- 172.21.15.42:0/4124304615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b805bca0 0x55d1b80607c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.445+0000 7fdc30413740 1 --2- 172.21.15.42:0/4124304615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805b380 0x55d1b8063f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.445+0000 7fdc30413740 1 -- 172.21.15.42:0/4124304615 >> 172.21.15.42:0/4124304615 conn(0x55d1b8055ab0 msgr2=0x55d1b8057790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:28.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.445+0000 7fdc30413740 1 -- 172.21.15.42:0/4124304615 shutdown_connections 2024-04-20T10:44:28.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.445+0000 7fdc30413740 1 -- 172.21.15.42:0/4124304615 wait complete. 2024-04-20T10:44:28.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.446+0000 7fdc30413740 1 -- start start 2024-04-20T10:44:28.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.451+0000 7fdc30413740 1 Processor -- start 2024-04-20T10:44:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.452+0000 7fdc30413740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805b380 0x55d1b8077f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.452+0000 7fdc30413740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b805bca0 0x55d1b80784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.452+0000 7fdc30413740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b8064880 0x55d1b80d9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.452+0000 7fdc30413740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0401e4f0 con 0x55d1b805b380 2024-04-20T10:44:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.452+0000 7fdc30413740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0401f960 con 0x55d1b805bca0 2024-04-20T10:44:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.452+0000 7fdc30413740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0401ebc0 con 0x55d1b8064880 2024-04-20T10:44:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.452+0000 7fdc30413740 10 client.0 _ll_register_callbacks cb 0x7fdc18009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:44:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.452+0000 7fdc2d800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b8064880 0x55d1b80d9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.452+0000 7fdc2d800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b8064880 0x55d1b80d9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:50020/0 (socket says 172.21.15.42:50020) 2024-04-20T10:44:28.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.452+0000 7fdc2d800640 1 -- 172.21.15.42:0/2487430770 learned_addr learned my addr 172.21.15.42:0/2487430770 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:28.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.453+0000 7fdc1e000640 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b805bca0 0x55d1b80784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.453+0000 7fdc26a00640 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805b380 0x55d1b8077f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.453+0000 7fdc2d800640 1 -- 172.21.15.42:0/2487430770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b805bca0 msgr2=0x55d1b80784a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:28.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.457+0000 7fdc2d800640 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b805bca0 0x55d1b80784a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.457+0000 7fdc2d800640 1 -- 172.21.15.42:0/2487430770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805b380 msgr2=0x55d1b8077f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:28.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.457+0000 7fdc2d800640 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805b380 0x55d1b8077f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.457+0000 7fdc2d800640 1 -- 172.21.15.42:0/2487430770 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b80da1d0 con 0x55d1b8064880 2024-04-20T10:44:28.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.457+0000 7fdc26a00640 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805b380 0x55d1b8077f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:44:28.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.457+0000 7fdc1e000640 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b805bca0 0x55d1b80784a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:44:28.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.457+0000 7fdc2d800640 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b8064880 0x55d1b80d9c90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdc20011040 tx=0x7fdc20033fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3ed4792ee3ab5d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:28.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.459+0000 7fdc14c00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:44:28.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.459+0000 7fdc14c00640 1 -- 172.21.15.42:0/2487430770 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20034e60 con 0x55d1b8064880 2024-04-20T10:44:28.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.459+0000 7fdc14c00640 1 -- 172.21.15.42:0/2487430770 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fdc2000fbe0 con 0x55d1b8064880 2024-04-20T10:44:28.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.460+0000 7fdc30413740 1 -- 172.21.15.42:0/2487430770 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc100b9020 con 0x55d1b8064880 2024-04-20T10:44:28.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.460+0000 7fdc14c00640 1 -- 172.21.15.42:0/2487430770 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdc2000fde0 con 0x55d1b8064880 2024-04-20T10:44:28.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.461+0000 7fdc30413740 10 client.26584 fetch_fsmap learned FSMap version 19 2024-04-20T10:44:28.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.461+0000 7fdc30413740 1 -- 172.21.15.42:0/2487430770 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d1b80db380 con 0x55d1b8064880 2024-04-20T10:44:28.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.461+0000 7fdc14c00640 1 -- 172.21.15.42:0/2487430770 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc100b9020 con 0x55d1b8064880 2024-04-20T10:44:28.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.461+0000 7fdc14c00640 1 -- 172.21.15.42:0/2487430770 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fdc200039f0 con 0x55d1b8064880 2024-04-20T10:44:28.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.461+0000 7fdc30413740 10 client.26584 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:44:28.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.461+0000 7fdc30413740 10 client.26584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:44:28.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.461+0000 7fdc30413740 20 client.26584 populate_metadata read hostname 'smithi042' 2024-04-20T10:44:28.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.461+0000 7fdc30413740 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d1b80dbb30 0x55d1b80ddf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:28.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.461+0000 7fdc30413740 4 client.26584 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:44:28.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.461+0000 7fdc30413740 1 -- 172.21.15.42:0/2487430770 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1b7fc89e0 con 0x55d1b80dbb30 2024-04-20T10:44:28.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.465+0000 7fdc26a00640 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d1b80dbb30 0x55d1b80ddf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:28.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.465+0000 7fdc26a00640 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d1b80dbb30 0x55d1b80ddf50 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e6b55a0d8094f server_cookie=406ce3c8580a787e in_seq=0 out_seq=0 2024-04-20T10:44:28.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.467+0000 7fdc14c00640 10 client.26584 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:28.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.467+0000 7fdc14c00640 1 -- 172.21.15.42:0/2487430770 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55d1b7fc89e0 con 0x55d1b80dbb30 2024-04-20T10:44:28.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.467+0000 7fdc14c00640 10 client.26584 handle_command_reply: tid=0 2024-04-20T10:44:28.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.467+0000 7fdc30413740 1 client.26584 shutdown 2024-04-20T10:44:28.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.476+0000 7fdc30413740 1 -- 172.21.15.42:0/2487430770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b8064880 msgr2=0x55d1b80d9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.476+0000 7fdc30413740 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b8064880 0x55d1b80d9c90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdc20011040 tx=0x7fdc20033fc0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.477+0000 7fdc30413740 1 -- 172.21.15.42:0/2487430770 shutdown_connections 2024-04-20T10:44:28.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.477+0000 7fdc30413740 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d1b80dbb30 0x55d1b80ddf50 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.477+0000 7fdc30413740 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1b8064880 0x55d1b80d9c90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.477+0000 7fdc30413740 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1b805bca0 0x55d1b80784a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.477+0000 7fdc14c00640 0 client.26584 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:28.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.477+0000 7fdc30413740 1 --2- 172.21.15.42:0/2487430770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d1b805b380 0x55d1b8077f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.477+0000 7fdc30413740 1 -- 172.21.15.42:0/2487430770 >> 172.21.15.42:0/2487430770 conn(0x55d1b8055ab0 msgr2=0x55d1b8061820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:28.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.478+0000 7fdc30413740 1 -- 172.21.15.42:0/2487430770 shutdown_connections 2024-04-20T10:44:28.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.478+0000 7fdc30413740 1 -- 172.21.15.42:0/2487430770 wait complete. 2024-04-20T10:44:28.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.478+0000 7fdc30413740 20 client.26584 trim_cache size 0 max 16384 2024-04-20T10:44:28.482 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:44:28.482 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (17787 inodes in the stack)", 2024-04-20T10:44:28.482 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:44:28.482 INFO:teuthology.orchestra.run.smithi042.stdout: "7046c3c8-28e3-4ab4-b6cd-8fafe5c353ae": { 2024-04-20T10:44:28.482 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:44:28.482 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "7046c3c8-28e3-4ab4-b6cd-8fafe5c353ae", 2024-04-20T10:44:28.482 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:44:28.482 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:44:28.483 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:44:28.483 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:44:28.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.485+0000 7fdc27e00640 1 -- 172.21.15.42:0/1036410484 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fdc0c0a3da0 msgr2=0x7fdc0c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:28.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.485+0000 7fdc27e00640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fdc0c0a3da0 0x7fdc0c0a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdc200047e0 tx=0x7fdc20011a70 comp rx=0 tx=0).stop 2024-04-20T10:44:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.485+0000 7fdc27e00640 1 -- 172.21.15.42:0/1036410484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc2810d0f0 msgr2=0x7fdc281a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.485+0000 7fdc27e00640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc2810d0f0 0x7fdc281a4d70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c920 tx=0x7fdc1000cde0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.485+0000 7fdc27e00640 1 -- 172.21.15.42:0/1036410484 shutdown_connections 2024-04-20T10:44:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.485+0000 7fdc27e00640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fdc0c0a3da0 0x7fdc0c0a6260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.485+0000 7fdc27e00640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc28112740 0x7fdc2819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.485+0000 7fdc27e00640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc2810d0f0 0x7fdc281a4d70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.485+0000 7fdc27e00640 1 --2- 172.21.15.42:0/1036410484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc2810c730 0x7fdc281a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.485+0000 7fdc27e00640 1 -- 172.21.15.42:0/1036410484 >> 172.21.15.42:0/1036410484 conn(0x7fdc2806cda0 msgr2=0x7fdc2810e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.488+0000 7fdc27e00640 1 -- 172.21.15.42:0/1036410484 shutdown_connections 2024-04-20T10:44:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:28.489+0000 7fdc27e00640 1 -- 172.21.15.42:0/1036410484 wait complete. 2024-04-20T10:44:28.542 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/mmp.o 2024-04-20T10:44:28.552 INFO:tasks.cephfs.filesystem:scrub status for tag:7046c3c8-28e3-4ab4-b6cd-8fafe5c353ae - {'path': '/', 'tag': '7046c3c8-28e3-4ab4-b6cd-8fafe5c353ae', 'options': 'recursive,force'} 2024-04-20T10:44:28.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:28 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:44:28.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:28 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:28.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:28 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:28.615 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/sparse-vmemmap.o 2024-04-20T10:44:28.956 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/cancel.o 2024-04-20T10:44:29.010 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fat/nfs.o 2024-04-20T10:44:29.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:29 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:44:29.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:29 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:44:29.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:29 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:44:29.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:29 smithi042 ceph-mon[17078]: pgmap v325: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.2 MiB/s wr, 215 op/s 2024-04-20T10:44:29.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:29 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:29.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:29 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:29.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:29 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:44:29.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:29 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:44:29.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:29 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:44:29.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:29 smithi162 ceph-mon[21854]: pgmap v325: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.2 MiB/s wr, 215 op/s 2024-04-20T10:44:29.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:29 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:29.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:29 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:29.463 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/isofs/util.o 2024-04-20T10:44:29.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:29 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:44:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:29 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:44:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:29 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:44:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:29 smithi046 ceph-mon[21574]: pgmap v325: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.2 MiB/s wr, 215 op/s 2024-04-20T10:44:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:29 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:29 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:29.912 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/move_extent.o 2024-04-20T10:44:30.089 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/mmu_notifier.o 2024-04-20T10:44:30.241 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/mq-deadline.o 2024-04-20T10:44:30.289 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fat/namei_vfat.o 2024-04-20T10:44:30.329 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/kbuf.o 2024-04-20T10:44:30.341 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_events.o 2024-04-20T10:44:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi162 to 176.2M 2024-04-20T10:44:30.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: Unable to set osd_memory_target on smithi162 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-20T10:44:30.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:30.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:30 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi162 to 176.2M 2024-04-20T10:44:30.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: Unable to set osd_memory_target on smithi162 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-20T10:44:30.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:30.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:30 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:30.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi162 to 176.2M 2024-04-20T10:44:30.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: Unable to set osd_memory_target on smithi162 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-20T10:44:30.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:30.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:30 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:30.653 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/isofs/rock.o 2024-04-20T10:44:31.345 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/slub.o 2024-04-20T10:44:31.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:31 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:31 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:31 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:31 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:31 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:31 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:31 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:31 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:31 smithi162 ceph-mon[21854]: pgmap v326: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-04-20T10:44:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:31 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-04-20T10:44:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:31 smithi162 ceph-mon[21854]: Unable to set osd_memory_target on smithi046 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-20T10:44:31.480 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/namei.o 2024-04-20T10:44:31.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:31 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:31 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:31 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:31 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:31 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:31 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:31 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:31 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:31 smithi046 ceph-mon[21574]: pgmap v326: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-04-20T10:44:31.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:31 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-04-20T10:44:31.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:31 smithi046 ceph-mon[21574]: Unable to set osd_memory_target on smithi046 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-20T10:44:31.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:31 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:31 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:31 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:31 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.604 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:31 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:31 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:31 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:31 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:31.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:31 smithi042 ceph-mon[17078]: pgmap v326: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-04-20T10:44:31.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:31 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-04-20T10:44:31.605 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:31 smithi042 ceph-mon[17078]: Unable to set osd_memory_target on smithi046 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-20T10:44:31.916 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/isofs/export.o 2024-04-20T10:44:31.941 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/rsrc.o 2024-04-20T10:44:31.961 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fat/namei_msdos.o 2024-04-20T10:44:32.052 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/kyber-iosched.o 2024-04-20T10:44:33.087 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/isofs/joliet.o 2024-04-20T10:44:33.214 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/fat/built-in.a 2024-04-20T10:44:33.268 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_export.o 2024-04-20T10:44:33.345 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/page-io.o 2024-04-20T10:44:34.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:33 smithi046 ceph-mon[21574]: pgmap v327: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.2 MiB/s wr, 675 op/s 2024-04-20T10:44:34.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:33 smithi042 ceph-mon[17078]: pgmap v327: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.2 MiB/s wr, 675 op/s 2024-04-20T10:44:34.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:33 smithi162 ceph-mon[21854]: pgmap v327: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.2 MiB/s wr, 675 op/s 2024-04-20T10:44:34.275 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/isofs/compress.o 2024-04-20T10:44:34.327 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-mq-pci.o 2024-04-20T10:44:34.525 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_event_perf.o 2024-04-20T10:44:34.591 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/rw.o 2024-04-20T10:44:34.847 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/events/ring_buffer.o 2024-04-20T10:44:35.296 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/fork.o 2024-04-20T10:44:35.464 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/readpage.o 2024-04-20T10:44:35.499 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/migrate.o 2024-04-20T10:44:35.608 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/isofs/built-in.a 2024-04-20T10:44:35.611 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-mq-virtio.o 2024-04-20T10:44:35.689 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-mq-debugfs.o 2024-04-20T10:44:35.910 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_events_filter.o 2024-04-20T10:44:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:35 smithi046 ceph-mon[21574]: pgmap v328: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 714 KiB/s wr, 732 op/s 2024-04-20T10:44:36.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:35 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:36.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:35 smithi042 ceph-mon[17078]: pgmap v328: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 714 KiB/s wr, 732 op/s 2024-04-20T10:44:36.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:35 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:36.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:35 smithi162 ceph-mon[21854]: pgmap v328: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 714 KiB/s wr, 732 op/s 2024-04-20T10:44:36.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:35 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:36.404 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/events/callchain.o 2024-04-20T10:44:36.824 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/client.o 2024-04-20T10:44:36.824 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/resize.o 2024-04-20T10:44:36.836 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/opdef.o 2024-04-20T10:44:37.005 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:36 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:37.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:36 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:37.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:36 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:37.258 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/blk-pm.o 2024-04-20T10:44:38.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:37 smithi046 ceph-mon[21574]: pgmap v329: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 728 KiB/s wr, 1.27k op/s 2024-04-20T10:44:38.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:37 smithi042 ceph-mon[17078]: pgmap v329: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 728 KiB/s wr, 1.27k op/s 2024-04-20T10:44:38.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:37 smithi162 ceph-mon[21854]: pgmap v329: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 728 KiB/s wr, 1.27k op/s 2024-04-20T10:44:38.216 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_events_trigger.o 2024-04-20T10:44:38.231 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/events/hw_breakpoint.o 2024-04-20T10:44:38.296 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/memory-tiers.o 2024-04-20T10:44:38.333 INFO:tasks.workunit.client.0.smithi042.stdout: CC block/holder.o 2024-04-20T10:44:38.501 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/page_counter.o 2024-04-20T10:44:38.614 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/notif.o 2024-04-20T10:44:39.091 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/super.o 2024-04-20T10:44:39.177 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/dir.o 2024-04-20T10:44:39.229 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/exportfs/expfs.o 2024-04-20T10:44:39.416 INFO:tasks.workunit.client.0.smithi042.stdout: AR block/built-in.a 2024-04-20T10:44:39.559 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/hugetlb_cgroup.o 2024-04-20T10:44:39.629 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/clntlock.o 2024-04-20T10:44:39.865 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_eprobe.o 2024-04-20T10:44:40.048 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/events/uprobes.o 2024-04-20T10:44:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: pgmap v330: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 522 KiB/s wr, 2.48k op/s 2024-04-20T10:44:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-04-20T10:44:40.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T10:44:40.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:44:40.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:44:40.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:39 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:40.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: pgmap v330: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 522 KiB/s wr, 2.48k op/s 2024-04-20T10:44:40.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-04-20T10:44:40.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T10:44:40.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:44:40.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:44:40.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:39 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: pgmap v330: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 522 KiB/s wr, 2.48k op/s 2024-04-20T10:44:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T10:44:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-04-20T10:44:40.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T10:44:40.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:44:40.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:44:40.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:39 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:44:40.170 INFO:tasks.workunit.client.0.smithi042.stdout: CC io_uring/io-wq.o 2024-04-20T10:44:40.284 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/exportfs/built-in.a 2024-04-20T10:44:40.375 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/file.o 2024-04-20T10:44:41.035 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/early_ioremap.o 2024-04-20T10:44:41.323 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/clntproc.o 2024-04-20T10:44:42.046 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/events/built-in.a 2024-04-20T10:44:42.048 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_kprobe.o 2024-04-20T10:44:42.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:41 smithi046 ceph-mon[21574]: pgmap v331: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 224 KiB/s wr, 2.42k op/s 2024-04-20T10:44:42.057 INFO:tasks.workunit.client.0.smithi042.stdout: AR io_uring/built-in.a 2024-04-20T10:44:42.079 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/secretmem.o 2024-04-20T10:44:42.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:41 smithi042 ceph-mon[17078]: pgmap v331: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 224 KiB/s wr, 2.42k op/s 2024-04-20T10:44:42.113 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/getroot.o 2024-04-20T10:44:42.138 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/memfd.o 2024-04-20T10:44:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:41 smithi162 ceph-mon[21854]: pgmap v331: 97 pgs: 97 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 224 KiB/s wr, 2.42k op/s 2024-04-20T10:44:42.743 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nls/nls_base.o 2024-04-20T10:44:42.824 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/unicode/built-in.a 2024-04-20T10:44:42.854 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/symlink.o 2024-04-20T10:44:43.481 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nls/nls_cp437.o 2024-04-20T10:44:43.490 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/clntxdr.o 2024-04-20T10:44:43.501 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/error_report-traces.o 2024-04-20T10:44:43.586 INFO:tasks.workunit.client.0.smithi042.stdout: CC mm/ptdump.o 2024-04-20T10:44:43.635 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/inode.o 2024-04-20T10:44:43.914 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/power-traces.o 2024-04-20T10:44:44.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:43 smithi046 ceph-mon[21574]: pgmap v332: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 913 KiB/s wr, 4.22k op/s 2024-04-20T10:44:44.100 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nls/nls_ascii.o 2024-04-20T10:44:44.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:43 smithi042 ceph-mon[17078]: pgmap v332: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 913 KiB/s wr, 4.22k op/s 2024-04-20T10:44:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:43 smithi162 ceph-mon[21854]: pgmap v332: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 913 KiB/s wr, 4.22k op/s 2024-04-20T10:44:44.394 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nls/nls_iso8859-1.o 2024-04-20T10:44:44.400 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/exec_domain.o 2024-04-20T10:44:44.421 INFO:tasks.workunit.client.0.smithi042.stdout: AR mm/built-in.a 2024-04-20T10:44:44.574 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/autofs/init.o 2024-04-20T10:44:44.596 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/super.o 2024-04-20T10:44:44.821 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/host.o 2024-04-20T10:44:44.914 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nls/nls_utf8.o 2024-04-20T10:44:45.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:44 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:44:45.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:44 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:44:45.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:44 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:44:45.210 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/svc.o 2024-04-20T10:44:45.219 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/autofs/inode.o 2024-04-20T10:44:45.422 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/rpm-traces.o 2024-04-20T10:44:45.437 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/nls/built-in.a 2024-04-20T10:44:45.492 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/autofs/root.o 2024-04-20T10:44:46.031 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/sysfs.o 2024-04-20T10:44:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:45 smithi046 ceph-mon[21574]: pgmap v333: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 836 KiB/s wr, 3.88k op/s 2024-04-20T10:44:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:45 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:44:46.072 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/9p/vfs_super.o 2024-04-20T10:44:46.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:45 smithi042 ceph-mon[17078]: pgmap v333: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 836 KiB/s wr, 3.88k op/s 2024-04-20T10:44:46.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:45 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:44:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:45 smithi162 ceph-mon[21854]: pgmap v333: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 836 KiB/s wr, 3.88k op/s 2024-04-20T10:44:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:45 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:44:46.181 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/io.o 2024-04-20T10:44:46.187 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/9p/vfs_inode.o 2024-04-20T10:44:46.353 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/autofs/symlink.o 2024-04-20T10:44:46.379 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_dynevent.o 2024-04-20T10:44:46.538 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/svclock.o 2024-04-20T10:44:46.823 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/panic.o 2024-04-20T10:44:46.914 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/hostfs/built-in.a 2024-04-20T10:44:46.957 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/debugfs/inode.o 2024-04-20T10:44:46.980 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/autofs/waitq.o 2024-04-20T10:44:46.982 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/xattr.o 2024-04-20T10:44:47.294 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_probe.o 2024-04-20T10:44:47.316 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/9p/vfs_inode_dotl.o 2024-04-20T10:44:47.373 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/direct.o 2024-04-20T10:44:47.720 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/autofs/expire.o 2024-04-20T10:44:47.799 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/trace_uprobe.o 2024-04-20T10:44:47.906 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/svcshare.o 2024-04-20T10:44:47.925 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/debugfs/file.o 2024-04-20T10:44:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:47 smithi046 ceph-mon[21574]: pgmap v334: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 941 KiB/s wr, 4.11k op/s 2024-04-20T10:44:48.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:47 smithi042 ceph-mon[17078]: pgmap v334: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 941 KiB/s wr, 4.11k op/s 2024-04-20T10:44:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:47 smithi162 ceph-mon[21854]: pgmap v334: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 941 KiB/s wr, 4.11k op/s 2024-04-20T10:44:48.348 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/9p/vfs_addr.o 2024-04-20T10:44:48.454 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/autofs/dev-ioctl.o 2024-04-20T10:44:48.766 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/xattr_hurd.o 2024-04-20T10:44:48.922 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/tracefs/inode.o 2024-04-20T10:44:49.017 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/debugfs/built-in.a 2024-04-20T10:44:49.061 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/svcproc.o 2024-04-20T10:44:49.140 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/pagelist.o 2024-04-20T10:44:49.230 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/9p/vfs_file.o 2024-04-20T10:44:49.259 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/autofs/built-in.a 2024-04-20T10:44:49.316 INFO:tasks.workunit.client.0.smithi042.stdout: CC [M] fs/efivarfs/inode.o 2024-04-20T10:44:49.340 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/9p/vfs_dir.o 2024-04-20T10:44:49.636 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/xattr_trusted.o 2024-04-20T10:44:49.664 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/trace/rethook.o 2024-04-20T10:44:49.730 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/tracefs/built-in.a 2024-04-20T10:44:49.789 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cpu.o 2024-04-20T10:44:50.046 INFO:tasks.workunit.client.0.smithi042.stdout: CC [M] fs/efivarfs/file.o 2024-04-20T10:44:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:49 smithi046 ceph-mon[21574]: pgmap v335: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 874 KiB/s wr, 3.93k op/s 2024-04-20T10:44:50.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:49 smithi042 ceph-mon[17078]: pgmap v335: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 874 KiB/s wr, 3.93k op/s 2024-04-20T10:44:50.134 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/read.o 2024-04-20T10:44:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:49 smithi162 ceph-mon[21854]: pgmap v335: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 874 KiB/s wr, 3.93k op/s 2024-04-20T10:44:50.154 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/9p/vfs_dentry.o 2024-04-20T10:44:50.336 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/svcsubs.o 2024-04-20T10:44:50.466 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/xattr_user.o 2024-04-20T10:44:50.488 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/trace/built-in.a 2024-04-20T10:44:50.551 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/fast_commit.o 2024-04-20T10:44:50.775 INFO:tasks.workunit.client.0.smithi042.stdout: CC [M] fs/efivarfs/super.o 2024-04-20T10:44:50.942 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/9p/v9fs.o 2024-04-20T10:44:51.193 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/open.o 2024-04-20T10:44:51.311 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/mon.o 2024-04-20T10:44:51.588 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/read_write.o 2024-04-20T10:44:51.613 INFO:tasks.workunit.client.0.smithi042.stdout: CC [M] fs/efivarfs/vars.o 2024-04-20T10:44:51.657 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/exit.o 2024-04-20T10:44:51.797 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/9p/fid.o 2024-04-20T10:44:51.900 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/symlink.o 2024-04-20T10:44:52.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:51 smithi046 ceph-mon[21574]: pgmap v336: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 800 KiB/s wr, 2.67k op/s 2024-04-20T10:44:52.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:51 smithi042 ceph-mon[17078]: pgmap v336: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 800 KiB/s wr, 2.67k op/s 2024-04-20T10:44:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:51 smithi162 ceph-mon[21854]: pgmap v336: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 800 KiB/s wr, 2.67k op/s 2024-04-20T10:44:52.504 INFO:tasks.workunit.client.0.smithi042.stdout: LD [M] fs/efivarfs/efivarfs.o 2024-04-20T10:44:52.563 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/orphan.o 2024-04-20T10:44:52.566 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/trace.o 2024-04-20T10:44:52.588 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/xdr.o 2024-04-20T10:44:52.640 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/9p/xattr.o 2024-04-20T10:44:52.762 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/softirq.o 2024-04-20T10:44:53.056 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/unlink.o 2024-04-20T10:44:53.364 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/resource.o 2024-04-20T10:44:53.583 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/9p/built-in.a 2024-04-20T10:44:53.607 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/clnt4xdr.o 2024-04-20T10:44:53.641 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/acl.o 2024-04-20T10:44:53.821 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/write.o 2024-04-20T10:44:53.954 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/xdr4.o 2024-04-20T10:44:54.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:53 smithi046 ceph-mon[21574]: pgmap v337: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 925 KiB/s wr, 2.68k op/s 2024-04-20T10:44:54.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:53 smithi042 ceph-mon[17078]: pgmap v337: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 925 KiB/s wr, 2.68k op/s 2024-04-20T10:44:54.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:53 smithi162 ceph-mon[21854]: pgmap v337: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 925 KiB/s wr, 2.68k op/s 2024-04-20T10:44:54.160 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ext4/xattr_security.o 2024-04-20T10:44:54.350 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/file_table.o 2024-04-20T10:44:54.657 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/sysctl.o 2024-04-20T10:44:54.672 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/namespace.o 2024-04-20T10:44:54.950 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/mount_clnt.o 2024-04-20T10:44:55.010 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/super.o 2024-04-20T10:44:55.012 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/ext4/built-in.a 2024-04-20T10:44:55.102 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/capability.o 2024-04-20T10:44:55.296 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/svc4proc.o 2024-04-20T10:44:55.340 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfstrace.o 2024-04-20T10:44:55.946 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/lockd/procfs.o 2024-04-20T10:44:56.048 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/export.o 2024-04-20T10:44:56.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:55 smithi046 ceph-mon[21574]: pgmap v338: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 237 KiB/s wr, 883 op/s 2024-04-20T10:44:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:55 smithi042 ceph-mon[17078]: pgmap v338: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 237 KiB/s wr, 883 op/s 2024-04-20T10:44:56.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:55 smithi162 ceph-mon[21854]: pgmap v338: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 237 KiB/s wr, 883 op/s 2024-04-20T10:44:56.206 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/sysfs.o 2024-04-20T10:44:56.275 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/char_dev.o 2024-04-20T10:44:56.294 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/ptrace.o 2024-04-20T10:44:56.441 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/fs_context.o 2024-04-20T10:44:56.571 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfsroot.o 2024-04-20T10:44:56.786 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/lockd/built-in.a 2024-04-20T10:44:56.864 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/stat.o 2024-04-20T10:44:57.205 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/user.o 2024-04-20T10:44:57.434 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/exec.o 2024-04-20T10:44:57.540 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/pipe.o 2024-04-20T10:44:57.566 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/signal.o 2024-04-20T10:44:57.718 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/namei.o 2024-04-20T10:44:57.784 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fcntl.o 2024-04-20T10:44:58.030 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/sysctl.o 2024-04-20T10:44:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:57 smithi046 ceph-mon[21574]: pgmap v339: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 611 KiB/s wr, 705 op/s 2024-04-20T10:44:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:57 smithi042 ceph-mon[17078]: pgmap v339: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 611 KiB/s wr, 705 op/s 2024-04-20T10:44:58.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:57 smithi162 ceph-mon[21854]: pgmap v339: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 611 KiB/s wr, 705 op/s 2024-04-20T10:44:58.355 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/sys.o 2024-04-20T10:44:58.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:44:58.553 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:44:58.875 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/ioctl.o 2024-04-20T10:44:58.897 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs2super.o 2024-04-20T10:44:58.974 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/umh.o 2024-04-20T10:44:58.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.977+0000 7f4ca0a00640 1 -- 172.21.15.42:0/1440926826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9c1150b0 msgr2=0x7f4c9c1174a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:58.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.977+0000 7f4ca0a00640 1 --2- 172.21.15.42:0/1440926826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9c1150b0 0x7f4c9c1174a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009900 tx=0x7f4c8c02bc40 comp rx=0 tx=0).stop 2024-04-20T10:44:58.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.981+0000 7f4ca0a00640 1 -- 172.21.15.42:0/1440926826 shutdown_connections 2024-04-20T10:44:58.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.981+0000 7f4ca0a00640 1 --2- 172.21.15.42:0/1440926826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9c1150b0 0x7f4c9c1174a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:58.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.981+0000 7f4ca0a00640 1 --2- 172.21.15.42:0/1440926826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c9c109a90 0x7f4c9c114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:58.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.981+0000 7f4ca0a00640 1 --2- 172.21.15.42:0/1440926826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c9c109170 0x7f4c9c109550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:58.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.981+0000 7f4ca0a00640 1 -- 172.21.15.42:0/1440926826 >> 172.21.15.42:0/1440926826 conn(0x7f4c9c06d070 msgr2=0x7f4c9c06d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:58.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.981+0000 7f4ca0a00640 1 -- 172.21.15.42:0/1440926826 shutdown_connections 2024-04-20T10:44:58.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.981+0000 7f4ca0a00640 1 -- 172.21.15.42:0/1440926826 wait complete. 2024-04-20T10:44:58.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.982+0000 7f4ca0a00640 1 Processor -- start 2024-04-20T10:44:58.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.985+0000 7f4ca0a00640 1 -- start start 2024-04-20T10:44:58.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.985+0000 7f4ca0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c9c109170 0x7f4c9c10cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:58.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.985+0000 7f4ca0a00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c9c109a90 0x7f4c9c10d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:58.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.985+0000 7f4ca0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9c1150b0 0x7f4c9c10d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:58.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.985+0000 7f4ca0a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c11abd0 con 0x7f4c9c109170 2024-04-20T10:44:58.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.986+0000 7f4ca0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c11aa50 con 0x7f4c9c1150b0 2024-04-20T10:44:58.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.986+0000 7f4ca0a00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c11ad50 con 0x7f4c9c109a90 2024-04-20T10:44:58.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.986+0000 7f4c99e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c9c109170 0x7f4c9c10cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:58.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.986+0000 7f4c9a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9c1150b0 0x7f4c9c10d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:58.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.986+0000 7f4c9a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9c1150b0 0x7f4c9c10d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40460/0 (socket says 172.21.15.42:40460) 2024-04-20T10:44:58.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.986+0000 7f4c9a800640 1 -- 172.21.15.42:0/2234933633 learned_addr learned my addr 172.21.15.42:0/2234933633 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:58.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.986+0000 7f4c99400640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c9c109a90 0x7f4c9c10d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:58.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.987+0000 7f4c99400640 1 -- 172.21.15.42:0/2234933633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9c1150b0 msgr2=0x7f4c9c10d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:58.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.987+0000 7f4c99400640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9c1150b0 0x7f4c9c10d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:58.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.987+0000 7f4c99400640 1 -- 172.21.15.42:0/2234933633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c9c109170 msgr2=0x7f4c9c10cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:58.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.988+0000 7f4c99400640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c9c109170 0x7f4c9c10cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:58.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.988+0000 7f4c99400640 1 -- 172.21.15.42:0/2234933633 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9c1a3150 con 0x7f4c9c109a90 2024-04-20T10:44:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.988+0000 7f4c99e00640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c9c109170 0x7f4c9c10cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:44:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.989+0000 7f4c99400640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c9c109a90 0x7f4c9c10d120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400e9f0 tx=0x7f4c9400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.989+0000 7f4c92a00640 1 -- 172.21.15.42:0/2234933633 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9400cd80 con 0x7f4c9c109a90 2024-04-20T10:44:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.989+0000 7f4c92a00640 1 -- 172.21.15.42:0/2234933633 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c94005020 con 0x7f4c9c109a90 2024-04-20T10:44:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.989+0000 7f4c92a00640 1 -- 172.21.15.42:0/2234933633 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c940116d0 con 0x7f4c9c109a90 2024-04-20T10:44:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.989+0000 7f4ca0a00640 1 -- 172.21.15.42:0/2234933633 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9c1a3440 con 0x7f4c9c109a90 2024-04-20T10:44:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.990+0000 7f4ca0a00640 1 -- 172.21.15.42:0/2234933633 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9c1a3940 con 0x7f4c9c109a90 2024-04-20T10:44:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.999+0000 7f4ca37d5740 1 Processor -- start 2024-04-20T10:44:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:58.999+0000 7f4c92a00640 1 -- 172.21.15.42:0/2234933633 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4c94005540 con 0x7f4c9c109a90 2024-04-20T10:44:59.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.001+0000 7f4ca37d5740 1 -- start start 2024-04-20T10:44:59.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.001+0000 7f4ca37d5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2bf1a0 0x559e3c2bf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.001+0000 7f4ca37d5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 0x559e3c2c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.001+0000 7f4ca37d5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2c86a0 0x559e3c2caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.001+0000 7f4ca37d5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559e3c225150 con 0x559e3c2bfac0 2024-04-20T10:44:59.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.001+0000 7f4ca37d5740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559e3c1cb370 con 0x559e3c2bf1a0 2024-04-20T10:44:59.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.001+0000 7f4ca37d5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559e3c221110 con 0x559e3c2c86a0 2024-04-20T10:44:59.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.002+0000 7f4c9a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2bf1a0 0x559e3c2bf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.002+0000 7f4c9a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2bf1a0 0x559e3c2bf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40470/0 (socket says 172.21.15.42:40470) 2024-04-20T10:44:59.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.002+0000 7f4c9a800640 1 -- 172.21.15.42:0/3955477923 learned_addr learned my addr 172.21.15.42:0/3955477923 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:59.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.002+0000 7f4c99e00640 1 --2- 172.21.15.42:0/3955477923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 0x559e3c2c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.003+0000 7f4c99e00640 1 -- 172.21.15.42:0/3955477923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2c86a0 msgr2=0x559e3c2caa90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:44:59.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.003+0000 7f4c99e00640 1 --2- 172.21.15.42:0/3955477923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2c86a0 0x559e3c2caa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.003+0000 7f4c99e00640 1 -- 172.21.15.42:0/3955477923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2bf1a0 msgr2=0x559e3c2bf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:59.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.003+0000 7f4c99e00640 1 --2- 172.21.15.42:0/3955477923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2bf1a0 0x559e3c2bf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.003+0000 7f4c99e00640 1 -- 172.21.15.42:0/3955477923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3c0017d0 con 0x559e3c2bfac0 2024-04-20T10:44:59.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.007+0000 7f4c92a00640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c800a3d20 0x7f4c800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.008+0000 7f4c99e00640 1 --2- 172.21.15.42:0/3955477923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 0x559e3c2c8160 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4c84005fd0 tx=0x7f4c840328f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131e1c2b6f1ebc56 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:59.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.008+0000 7f4c9a800640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c800a3d20 0x7f4c800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.008+0000 7f4c8b000640 1 -- 172.21.15.42:0/3955477923 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84039070 con 0x559e3c2bfac0 2024-04-20T10:44:59.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.009+0000 7f4c8b000640 1 -- 172.21.15.42:0/3955477923 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c84033c50 con 0x559e3c2bfac0 2024-04-20T10:44:59.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.009+0000 7f4c8b000640 1 -- 172.21.15.42:0/3955477923 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8403dd30 con 0x559e3c2bfac0 2024-04-20T10:44:59.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.009+0000 7f4c92a00640 1 -- 172.21.15.42:0/2234933633 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f4c940d0c30 con 0x7f4c9c109a90 2024-04-20T10:44:59.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.009+0000 7f4c9a800640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c800a3d20 0x7f4c800a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0096c0 tx=0x7f4c8c02e040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:59.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.010+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3955477923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 msgr2=0x559e3c2c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:59.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.010+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3955477923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 0x559e3c2c8160 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4c84005fd0 tx=0x7f4c840328f0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.012+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3955477923 shutdown_connections 2024-04-20T10:44:59.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.012+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3955477923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2c86a0 0x559e3c2caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.012+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3955477923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 0x559e3c2c8160 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.012+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3955477923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2bf1a0 0x559e3c2bf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.012+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3955477923 >> 172.21.15.42:0/3955477923 conn(0x559e3c2b98d0 msgr2=0x559e3c2b9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:59.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.012+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3955477923 shutdown_connections 2024-04-20T10:44:59.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.012+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3955477923 wait complete. 2024-04-20T10:44:59.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.012+0000 7f4ca37d5740 1 -- start start 2024-04-20T10:44:59.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.013+0000 7f4ca37d5740 1 Processor -- start 2024-04-20T10:44:59.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.015+0000 7f4ca37d5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 0x559e3c2c5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.015+0000 7f4ca37d5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bfac0 0x559e3c2c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.016+0000 7f4c99e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 0x559e3c2c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.016+0000 7f4c99e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 0x559e3c2c5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56448/0 (socket says 172.21.15.42:56448) 2024-04-20T10:44:59.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.017+0000 7f4c99e00640 1 -- 172.21.15.42:0/3052292691 learned_addr learned my addr 172.21.15.42:0/3052292691 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:59.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.017+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2c86a0 0x559e3c2c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.017+0000 7f4c99400640 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bfac0 0x559e3c2c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.017+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3052292691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559e3c2cb490 con 0x559e3c2bf1a0 2024-04-20T10:44:59.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.017+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3052292691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559e3c1de740 con 0x559e3c2c86a0 2024-04-20T10:44:59.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.017+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3052292691 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559e3c2cb610 con 0x559e3c2bfac0 2024-04-20T10:44:59.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.017+0000 7f4c9a800640 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2c86a0 0x559e3c2c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.018+0000 7f4c99400640 1 -- 172.21.15.42:0/3052292691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2c86a0 msgr2=0x559e3c2c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.018+0000 7f4c99400640 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2c86a0 0x559e3c2c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.018+0000 7f4c99400640 1 -- 172.21.15.42:0/3052292691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 msgr2=0x559e3c2c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.018+0000 7f4ca37d5740 10 client.0 _ll_register_callbacks cb 0x559e3c1d9b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:44:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.018+0000 7f4c99400640 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 0x559e3c2c5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.018+0000 7f4c99400640 1 -- 172.21.15.42:0/3052292691 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3bf373f0 con 0x559e3c2bfac0 2024-04-20T10:44:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.018+0000 7f4c9a800640 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2c86a0 0x559e3c2c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:44:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.018+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3052292691 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c84033c50 con 0x559e3c2bfac0 2024-04-20T10:44:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.018+0000 7f4c99e00640 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 0x559e3c2c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:44:59.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.018+0000 7f4c99400640 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bfac0 0x559e3c2c5c90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4c940ce350 tx=0x7f4c9409c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d18d1bf5c81c5e8c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:59.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.019+0000 7f4c7f400640 10 client.26593 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:44:59.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.019+0000 7f4c7f400640 1 -- 172.21.15.42:0/3052292691 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c940a5810 con 0x559e3c2bfac0 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.019+0000 7f4c7f400640 1 -- 172.21.15.42:0/3052292691 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f4c940992c0 con 0x559e3c2bfac0 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.019+0000 7f4c7f400640 1 -- 172.21.15.42:0/3052292691 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4c9409ceb0 con 0x559e3c2bfac0 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.019+0000 7f4ca37d5740 10 client.26593 fetch_fsmap learned FSMap version 19 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.019+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3052292691 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({fsmap=19}) -- 0x559e3bef70f0 con 0x559e3c2bfac0 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.019+0000 7f4c7f400640 1 -- 172.21.15.42:0/3052292691 <== mon.1 v2:172.21.15.162:3300/0 4 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f4c9400f040 con 0x559e3c2bfac0 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.019+0000 7f4ca37d5740 10 client.26593 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.019+0000 7f4ca37d5740 10 client.26593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.020+0000 7f4ca37d5740 20 client.26593 populate_metadata read hostname 'smithi042' 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.020+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559e3c2d8990 0x559e3c2dad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.020+0000 7f4ca37d5740 4 client.26593 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.020+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3052292691 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e3c232b70 con 0x559e3c2d8990 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.020+0000 7f4c99e00640 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559e3c2d8990 0x559e3c2dad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.021+0000 7f4c99e00640 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559e3c2d8990 0x559e3c2dad80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90bb169b92a87b73 server_cookie=8b6cebba8a1eedf8 in_seq=0 out_seq=0 2024-04-20T10:44:59.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.021+0000 7f4c7f400640 10 client.26593 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:59.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.022+0000 7f4c7f400640 1 -- 172.21.15.42:0/3052292691 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e3c232b70 con 0x559e3c2d8990 2024-04-20T10:44:59.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.022+0000 7f4c7f400640 10 client.26593 handle_command_reply: tid=0 2024-04-20T10:44:59.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.022+0000 7f4ca37d5740 1 client.26593 shutdown 2024-04-20T10:44:59.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.025+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3052292691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bfac0 msgr2=0x559e3c2c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:59.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.025+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bfac0 0x559e3c2c5c90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4c940ce350 tx=0x7f4c9409c670 comp rx=0 tx=0).stop 2024-04-20T10:44:59.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.026+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3052292691 shutdown_connections 2024-04-20T10:44:59.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.026+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559e3c2d8990 0x559e3c2dad80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.026+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2c86a0 0x559e3c2c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.026+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bfac0 0x559e3c2c5c90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.026+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3052292691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 0x559e3c2c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.026+0000 7f4c7f400640 0 client.26593 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:59.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.026+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3052292691 >> 172.21.15.42:0/3052292691 conn(0x559e3c2b98d0 msgr2=0x559e3c2bb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:59.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.029+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3052292691 shutdown_connections 2024-04-20T10:44:59.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.029+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3052292691 wait complete. 2024-04-20T10:44:59.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.029+0000 7f4ca37d5740 20 client.26593 trim_cache size 0 max 16384 2024-04-20T10:44:59.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.110+0000 7f4ca37d5740 1 Processor -- start 2024-04-20T10:44:59.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.110+0000 7f4ca37d5740 1 -- start start 2024-04-20T10:44:59.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.111+0000 7f4ca37d5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bf1a0 0x559e3c2c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.111+0000 7f4ca37d5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 0x559e3c2d43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.111+0000 7f4c99e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bf1a0 0x559e3c2c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.111+0000 7f4c99e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bf1a0 0x559e3c2c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:55848/0 (socket says 172.21.15.42:55848) 2024-04-20T10:44:59.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.112+0000 7f4c99400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 0x559e3c2d43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.112+0000 7f4c99400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 0x559e3c2d43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56454/0 (socket says 172.21.15.42:56454) 2024-04-20T10:44:59.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.112+0000 7f4c99e00640 1 -- 172.21.15.42:0/3614146409 learned_addr learned my addr 172.21.15.42:0/3614146409 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:59.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.112+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3614146409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2c86a0 0x559e3c2e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.112+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3614146409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c06dc70 con 0x559e3c2bfac0 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.112+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3614146409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1b4d70 con 0x559e3c2c86a0 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.112+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3614146409 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1aab30 con 0x559e3c2bf1a0 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.112+0000 7f4c9a800640 1 --2- 172.21.15.42:0/3614146409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2c86a0 0x559e3c2e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.113+0000 7f4c99400640 1 -- 172.21.15.42:0/3614146409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bf1a0 msgr2=0x559e3c2c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.113+0000 7f4c99400640 1 --2- 172.21.15.42:0/3614146409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bf1a0 0x559e3c2c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.113+0000 7f4c99400640 1 -- 172.21.15.42:0/3614146409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2c86a0 msgr2=0x559e3c2e05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.113+0000 7f4c99400640 1 --2- 172.21.15.42:0/3614146409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2c86a0 0x559e3c2e05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.113+0000 7f4c99400640 1 -- 172.21.15.42:0/3614146409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3c2db3b0 con 0x559e3c2bfac0 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.113+0000 7f4c9a800640 1 --2- 172.21.15.42:0/3614146409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2c86a0 0x559e3c2e05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.113+0000 7f4c99400640 1 --2- 172.21.15.42:0/3614146409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 0x559e3c2d43a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4c940990b0 tx=0x7f4c9409c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c65ba7403d3ce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.113+0000 7f4c8a600640 1 -- 172.21.15.42:0/3614146409 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c940992c0 con 0x559e3c2bfac0 2024-04-20T10:44:59.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.114+0000 7f4c8a600640 1 -- 172.21.15.42:0/3614146409 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c84033c50 con 0x559e3c2bfac0 2024-04-20T10:44:59.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.114+0000 7f4c8a600640 1 -- 172.21.15.42:0/3614146409 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9409c2c0 con 0x559e3c2bfac0 2024-04-20T10:44:59.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.114+0000 7f4c99e00640 1 --2- 172.21.15.42:0/3614146409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bf1a0 0x559e3c2c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:44:59.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.115+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3614146409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 msgr2=0x559e3c2d43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:59.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.115+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3614146409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 0x559e3c2d43a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4c940990b0 tx=0x7f4c9409c6e0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.115+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3614146409 shutdown_connections 2024-04-20T10:44:59.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.115+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3614146409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2c86a0 0x559e3c2e05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.115+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3614146409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bfac0 0x559e3c2d43a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.115+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3614146409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2bf1a0 0x559e3c2c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.115+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3614146409 >> 172.21.15.42:0/3614146409 conn(0x559e3c2b98d0 msgr2=0x559e3c2b88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:59.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.115+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3614146409 shutdown_connections 2024-04-20T10:44:59.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.115+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3614146409 wait complete. 2024-04-20T10:44:59.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.116+0000 7f4ca37d5740 1 -- start start 2024-04-20T10:44:59.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.116+0000 7f4ca37d5740 1 Processor -- start 2024-04-20T10:44:59.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.117+0000 7f4ca37d5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 0x559e3c2d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.117+0000 7f4ca37d5740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2bfac0 0x559e3c2d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.117+0000 7f4ca37d5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2c86a0 0x559e3c2cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.117+0000 7f4c99e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 0x559e3c2d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4ca37d5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1ab1e0 con 0x559e3c2bf1a0 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4c99e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 0x559e3c2d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56460/0 (socket says 172.21.15.42:56460) 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4c99e00640 1 -- 172.21.15.42:0/3742596249 learned_addr learned my addr 172.21.15.42:0/3742596249 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4c99400640 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2bfac0 0x559e3c2d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3742596249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1b46a0 con 0x559e3c2bfac0 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3742596249 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1b3900 con 0x559e3c2c86a0 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4ca37d5740 10 client.0 _ll_register_callbacks cb 0x7f4c94005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4c9a800640 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2c86a0 0x559e3c2cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4c99e00640 1 -- 172.21.15.42:0/3742596249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2c86a0 msgr2=0x559e3c2cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4c99e00640 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2c86a0 0x559e3c2cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4c99e00640 1 -- 172.21.15.42:0/3742596249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2bfac0 msgr2=0x559e3c2d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4c99e00640 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2bfac0 0x559e3c2d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4c99e00640 1 -- 172.21.15.42:0/3742596249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3c2cde40 con 0x559e3c2bf1a0 2024-04-20T10:44:59.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4c9a800640 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2c86a0 0x559e3c2cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:44:59.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.118+0000 7f4c99400640 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2bfac0 0x559e3c2d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:44:59.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.119+0000 7f4c7f400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:44:59.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.119+0000 7f4c99e00640 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 0x559e3c2d08b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400c010 tx=0x7f4c8403d7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed7bf4b7143aedc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:59.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.120+0000 7f4c7f400640 1 -- 172.21.15.42:0/3742596249 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84039040 con 0x559e3c2bf1a0 2024-04-20T10:44:59.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.120+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3742596249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c84033c50 con 0x559e3c2bf1a0 2024-04-20T10:44:59.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.120+0000 7f4c7f400640 1 -- 172.21.15.42:0/3742596249 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f4c8400bae0 con 0x559e3c2bf1a0 2024-04-20T10:44:59.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.121+0000 7f4c7f400640 1 -- 172.21.15.42:0/3742596249 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4c8405c050 con 0x559e3c2bf1a0 2024-04-20T10:44:59.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.124+0000 7f4ca37d5740 10 client.16479 fetch_fsmap learned FSMap version 19 2024-04-20T10:44:59.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.124+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3742596249 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559e3c33f660 con 0x559e3c2bf1a0 2024-04-20T10:44:59.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.126+0000 7f4c7f400640 1 -- 172.21.15.42:0/3742596249 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c84033c50 con 0x559e3c2bf1a0 2024-04-20T10:44:59.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.126+0000 7f4c7f400640 1 -- 172.21.15.42:0/3742596249 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f4c84030040 con 0x559e3c2bf1a0 2024-04-20T10:44:59.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.129+0000 7f4ca37d5740 10 client.16479 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:44:59.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.129+0000 7f4ca37d5740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:44:59.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.129+0000 7f4ca37d5740 20 client.16479 populate_metadata read hostname 'smithi042' 2024-04-20T10:44:59.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.129+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559e3c33fab0 0x559e3c341ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.129+0000 7f4ca37d5740 4 client.16479 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:44:59.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.129+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3742596249 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e3c22c9e0 con 0x559e3c33fab0 2024-04-20T10:44:59.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.129+0000 7f4c99400640 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559e3c33fab0 0x559e3c341ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.131+0000 7f4c7f400640 10 client.16479 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:59.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.131+0000 7f4c99400640 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559e3c33fab0 0x559e3c341ed0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7754a7cc5057e943 server_cookie=cbe33ed6b8db4f21 in_seq=0 out_seq=0 2024-04-20T10:44:59.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.131+0000 7f4c7f400640 1 -- 172.21.15.42:0/3742596249 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559e3c22c9e0 con 0x559e3c33fab0 2024-04-20T10:44:59.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.131+0000 7f4c7f400640 10 client.16479 handle_command_reply: tid=0 2024-04-20T10:44:59.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.131+0000 7f4ca37d5740 1 client.16479 shutdown 2024-04-20T10:44:59.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.132+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3742596249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 msgr2=0x559e3c2d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:59.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.132+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 0x559e3c2d08b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400c010 tx=0x7f4c8403d7f0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.135+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3742596249 shutdown_connections 2024-04-20T10:44:59.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.135+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x559e3c33fab0 0x559e3c341ed0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.135+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559e3c2c86a0 0x559e3c2cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.135+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e3c2bfac0 0x559e3c2d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.135+0000 7f4ca37d5740 1 --2- 172.21.15.42:0/3742596249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559e3c2bf1a0 0x559e3c2d08b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.135+0000 7f4c7f400640 0 client.16479 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:44:59.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.135+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3742596249 >> 172.21.15.42:0/3742596249 conn(0x559e3c2b98d0 msgr2=0x559e3c2c5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:59.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.135+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3742596249 shutdown_connections 2024-04-20T10:44:59.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.135+0000 7f4ca37d5740 1 -- 172.21.15.42:0/3742596249 wait complete. 2024-04-20T10:44:59.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.135+0000 7f4ca37d5740 20 client.16479 trim_cache size 0 max 16384 2024-04-20T10:44:59.140 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:44:59.140 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:44:59.140 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:44:59.140 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:44:59.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.140+0000 7f4ca0a00640 1 -- 172.21.15.42:0/2234933633 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c800a3d20 msgr2=0x7f4c800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:59.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.140+0000 7f4ca0a00640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c800a3d20 0x7f4c800a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0096c0 tx=0x7f4c8c02e040 comp rx=0 tx=0).stop 2024-04-20T10:44:59.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.140+0000 7f4ca0a00640 1 -- 172.21.15.42:0/2234933633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c9c109a90 msgr2=0x7f4c9c10d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.140+0000 7f4ca0a00640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c9c109a90 0x7f4c9c10d120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400e9f0 tx=0x7f4c9400eeb0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.140+0000 7f4ca0a00640 1 -- 172.21.15.42:0/2234933633 shutdown_connections 2024-04-20T10:44:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.140+0000 7f4ca0a00640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4c800a3d20 0x7f4c800a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.140+0000 7f4ca0a00640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c9c1150b0 0x7f4c9c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.140+0000 7f4ca0a00640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c9c109a90 0x7f4c9c10d120 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.140+0000 7f4ca0a00640 1 --2- 172.21.15.42:0/2234933633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c9c109170 0x7f4c9c10cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.140+0000 7f4ca0a00640 1 -- 172.21.15.42:0/2234933633 >> 172.21.15.42:0/2234933633 conn(0x7f4c9c06d070 msgr2=0x7f4c9c071e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:59.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.145+0000 7f4ca0a00640 1 -- 172.21.15.42:0/2234933633 shutdown_connections 2024-04-20T10:44:59.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.145+0000 7f4ca0a00640 1 -- 172.21.15.42:0/2234933633 wait complete. 2024-04-20T10:44:59.185 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:44:59.186 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:44:59.217 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/workqueue.o 2024-04-20T10:44:59.289 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/proc.o 2024-04-20T10:44:59.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.753+0000 7f4643800640 1 -- 172.21.15.42:0/3652373937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463c109190 msgr2=0x7f463c109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:59.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.753+0000 7f4643800640 1 --2- 172.21.15.42:0/3652373937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463c109190 0x7f463c109570 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4624009930 tx=0x7f462402bc30 comp rx=0 tx=0).stop 2024-04-20T10:44:59.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.753+0000 7f4643800640 1 -- 172.21.15.42:0/3652373937 shutdown_connections 2024-04-20T10:44:59.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.753+0000 7f4643800640 1 --2- 172.21.15.42:0/3652373937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c1150b0 0x7f463c1174a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.753+0000 7f4643800640 1 --2- 172.21.15.42:0/3652373937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f463c109ab0 0x7f463c114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.753+0000 7f4643800640 1 --2- 172.21.15.42:0/3652373937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463c109190 0x7f463c109570 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.753+0000 7f4643800640 1 -- 172.21.15.42:0/3652373937 >> 172.21.15.42:0/3652373937 conn(0x7f463c06d080 msgr2=0x7f463c06d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:44:59.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.753+0000 7f4643800640 1 -- 172.21.15.42:0/3652373937 shutdown_connections 2024-04-20T10:44:59.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.753+0000 7f4643800640 1 -- 172.21.15.42:0/3652373937 wait complete. 2024-04-20T10:44:59.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.756+0000 7f4643800640 1 Processor -- start 2024-04-20T10:44:59.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.759+0000 7f4643800640 1 -- start start 2024-04-20T10:44:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.759+0000 7f4643800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f463c109190 0x7f463c1a3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.759+0000 7f4643800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c109ab0 0x7f463c1a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.759+0000 7f4643800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463c1150b0 0x7f463c1a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.760+0000 7f4643800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f463c11ab50 con 0x7f463c109ab0 2024-04-20T10:44:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.760+0000 7f4643800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f463c11a9d0 con 0x7f463c1150b0 2024-04-20T10:44:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.760+0000 7f4643800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f463c11acd0 con 0x7f463c109190 2024-04-20T10:44:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.760+0000 7f4641800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463c1150b0 0x7f463c1a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.760+0000 7f4641800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463c1150b0 0x7f463c1a7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40528/0 (socket says 172.21.15.42:40528) 2024-04-20T10:44:59.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.760+0000 7f4641800640 1 -- 172.21.15.42:0/3894239284 learned_addr learned my addr 172.21.15.42:0/3894239284 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:44:59.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.761+0000 7f4641800640 1 -- 172.21.15.42:0/3894239284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f463c109190 msgr2=0x7f463c1a3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:44:59.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.761+0000 7f4641800640 1 --2- 172.21.15.42:0/3894239284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f463c109190 0x7f463c1a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.761+0000 7f4641800640 1 -- 172.21.15.42:0/3894239284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c109ab0 msgr2=0x7f463c1a37b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:44:59.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.761+0000 7f4641800640 1 --2- 172.21.15.42:0/3894239284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c109ab0 0x7f463c1a37b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:44:59.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.761+0000 7f4641800640 1 -- 172.21.15.42:0/3894239284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c1a8260 con 0x7f463c1150b0 2024-04-20T10:44:59.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.761+0000 7f4641800640 1 --2- 172.21.15.42:0/3894239284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463c1150b0 0x7f463c1a7b30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f463400d950 tx=0x7f463400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:59.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.763+0000 7f463a000640 1 -- 172.21.15.42:0/3894239284 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634017070 con 0x7f463c1150b0 2024-04-20T10:44:59.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.763+0000 7f463a000640 1 -- 172.21.15.42:0/3894239284 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4634009c30 con 0x7f463c1150b0 2024-04-20T10:44:59.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.763+0000 7f463a000640 1 -- 172.21.15.42:0/3894239284 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634003e40 con 0x7f463c1150b0 2024-04-20T10:44:59.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.763+0000 7f4643800640 1 -- 172.21.15.42:0/3894239284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c1a8550 con 0x7f463c1150b0 2024-04-20T10:44:59.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.763+0000 7f4643800640 1 -- 172.21.15.42:0/3894239284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c1afde0 con 0x7f463c1150b0 2024-04-20T10:44:59.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.770+0000 7f4643800640 1 -- 172.21.15.42:0/3894239284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c068d80 con 0x7f463c1150b0 2024-04-20T10:44:59.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.778+0000 7f463a000640 1 -- 172.21.15.42:0/3894239284 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4634010460 con 0x7f463c1150b0 2024-04-20T10:44:59.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.780+0000 7f463a000640 1 --2- 172.21.15.42:0/3894239284 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f46200a3e70 0x7f46200a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:44:59.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.783+0000 7f463a000640 1 -- 172.21.15.42:0/3894239284 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f46340cfe50 con 0x7f463c1150b0 2024-04-20T10:44:59.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.784+0000 7f4640e00640 1 --2- 172.21.15.42:0/3894239284 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f46200a3e70 0x7f46200a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:44:59.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.785+0000 7f4640e00640 1 --2- 172.21.15.42:0/3894239284 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f46200a3e70 0x7f46200a6330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f46240096c0 tx=0x7f4624000bc0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:44:59.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:44:59.786+0000 7f463a000640 1 -- 172.21.15.42:0/3894239284 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f463409c4c0 con 0x7f463c1150b0 2024-04-20T10:45:00.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:44:59 smithi046 ceph-mon[21574]: pgmap v340: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.1 MiB/s wr, 454 op/s 2024-04-20T10:45:00.060 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/task_work.o 2024-04-20T10:45:00.060 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/pid.o 2024-04-20T10:45:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:44:59 smithi042 ceph-mon[17078]: pgmap v340: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.1 MiB/s wr, 454 op/s 2024-04-20T10:45:00.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:44:59 smithi162 ceph-mon[21854]: pgmap v340: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.1 MiB/s wr, 454 op/s 2024-04-20T10:45:00.292 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/extable.o 2024-04-20T10:45:00.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.449+0000 7f4643800640 1 -- 172.21.15.42:0/3894239284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f463c10dd30 con 0x7f463c1150b0 2024-04-20T10:45:00.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.450+0000 7f463a000640 1 -- 172.21.15.42:0/3894239284 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f4634009de0 con 0x7f463c1150b0 2024-04-20T10:45:00.453 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:45:00.454 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:45:00.454 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:45:00.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.454+0000 7f4643800640 1 -- 172.21.15.42:0/3894239284 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f46200a3e70 msgr2=0x7f46200a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:00.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.454+0000 7f4643800640 1 --2- 172.21.15.42:0/3894239284 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f46200a3e70 0x7f46200a6330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f46240096c0 tx=0x7f4624000bc0 comp rx=0 tx=0).stop 2024-04-20T10:45:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.454+0000 7f4643800640 1 -- 172.21.15.42:0/3894239284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463c1150b0 msgr2=0x7f463c1a7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.454+0000 7f4643800640 1 --2- 172.21.15.42:0/3894239284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463c1150b0 0x7f463c1a7b30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f463400d950 tx=0x7f463400de10 comp rx=0 tx=0).stop 2024-04-20T10:45:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.455+0000 7f4643800640 1 -- 172.21.15.42:0/3894239284 shutdown_connections 2024-04-20T10:45:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.455+0000 7f4643800640 1 --2- 172.21.15.42:0/3894239284 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f46200a3e70 0x7f46200a6330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.455+0000 7f4643800640 1 --2- 172.21.15.42:0/3894239284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463c1150b0 0x7f463c1a7b30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.455+0000 7f4643800640 1 --2- 172.21.15.42:0/3894239284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c109ab0 0x7f463c1a37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.455+0000 7f4643800640 1 --2- 172.21.15.42:0/3894239284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f463c109190 0x7f463c1a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.455+0000 7f4643800640 1 -- 172.21.15.42:0/3894239284 >> 172.21.15.42:0/3894239284 conn(0x7f463c06d080 msgr2=0x7f463c070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:00.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.456+0000 7f4643800640 1 -- 172.21.15.42:0/3894239284 shutdown_connections 2024-04-20T10:45:00.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.456+0000 7f4643800640 1 -- 172.21.15.42:0/3894239284 wait complete. 2024-04-20T10:45:00.487 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:45:00.538 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/params.o 2024-04-20T10:45:00.551 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/kthread.o 2024-04-20T10:45:00.767 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/readdir.o 2024-04-20T10:45:00.777 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:00 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3894239284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:45:00.799 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs2xdr.o 2024-04-20T10:45:00.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.981+0000 7fda5a400640 1 -- 172.21.15.42:0/2395385231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda5411b740 msgr2=0x7fda5411db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:00.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.981+0000 7fda5a400640 1 --2- 172.21.15.42:0/2395385231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda5411b740 0x7fda5411db30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fda44009930 tx=0x7fda4402bc50 comp rx=0 tx=0).stop 2024-04-20T10:45:00.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.981+0000 7fda5a400640 1 -- 172.21.15.42:0/2395385231 shutdown_connections 2024-04-20T10:45:00.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.981+0000 7fda5a400640 1 --2- 172.21.15.42:0/2395385231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda5411b740 0x7fda5411db30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:00.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.981+0000 7fda5a400640 1 --2- 172.21.15.42:0/2395385231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda5410e7e0 0x7fda5410ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:00.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.981+0000 7fda5a400640 1 --2- 172.21.15.42:0/2395385231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda541107b0 0x7fda5410e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:00.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.981+0000 7fda5a400640 1 -- 172.21.15.42:0/2395385231 >> 172.21.15.42:0/2395385231 conn(0x7fda5406d070 msgr2=0x7fda5406d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:00.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.982+0000 7fda5a400640 1 -- 172.21.15.42:0/2395385231 shutdown_connections 2024-04-20T10:45:00.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.982+0000 7fda5a400640 1 -- 172.21.15.42:0/2395385231 wait complete. 2024-04-20T10:45:00.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.983+0000 7fda5a400640 1 Processor -- start 2024-04-20T10:45:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.988+0000 7fda5a400640 1 -- start start 2024-04-20T10:45:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.988+0000 7fda5a400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda5410e7e0 0x7fda54118d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.988+0000 7fda5a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda541107b0 0x7fda54111dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.988+0000 7fda5a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5411b740 0x7fda54112310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.988+0000 7fda5a400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fda54121190 con 0x7fda5411b740 2024-04-20T10:45:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.988+0000 7fda5a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda54121010 con 0x7fda541107b0 2024-04-20T10:45:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.988+0000 7fda5a400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fda54121310 con 0x7fda5410e7e0 2024-04-20T10:45:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.989+0000 7fda52a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda541107b0 0x7fda54111dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.989+0000 7fda52a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda541107b0 0x7fda54111dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40552/0 (socket says 172.21.15.42:40552) 2024-04-20T10:45:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.989+0000 7fda52a00640 1 -- 172.21.15.42:0/4116509823 learned_addr learned my addr 172.21.15.42:0/4116509823 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.989+0000 7fda53e00640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5411b740 0x7fda54112310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:00.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.989+0000 7fda53400640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda5410e7e0 0x7fda54118d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:00.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.989+0000 7fda52a00640 1 -- 172.21.15.42:0/4116509823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda5410e7e0 msgr2=0x7fda54118d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:00.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.989+0000 7fda52a00640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda5410e7e0 0x7fda54118d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:00.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.989+0000 7fda52a00640 1 -- 172.21.15.42:0/4116509823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5411b740 msgr2=0x7fda54112310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:00.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.989+0000 7fda52a00640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5411b740 0x7fda54112310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:00.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.989+0000 7fda52a00640 1 -- 172.21.15.42:0/4116509823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda541128d0 con 0x7fda541107b0 2024-04-20T10:45:00.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.990+0000 7fda52a00640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda541107b0 0x7fda54111dd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e9f0 tx=0x7fda4000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:00.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.990+0000 7fda4be00640 1 -- 172.21.15.42:0/4116509823 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4000cd80 con 0x7fda541107b0 2024-04-20T10:45:00.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.990+0000 7fda4be00640 1 -- 172.21.15.42:0/4116509823 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda40010910 con 0x7fda541107b0 2024-04-20T10:45:00.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.990+0000 7fda4be00640 1 -- 172.21.15.42:0/4116509823 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40018a00 con 0x7fda541107b0 2024-04-20T10:45:00.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.991+0000 7fda5a400640 1 -- 172.21.15.42:0/4116509823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda54112bc0 con 0x7fda541107b0 2024-04-20T10:45:00.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:00.991+0000 7fda5a400640 1 -- 172.21.15.42:0/4116509823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda54116b50 con 0x7fda541107b0 2024-04-20T10:45:01.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.000+0000 7fda5d164740 1 Processor -- start 2024-04-20T10:45:01.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.002+0000 7fda5d164740 1 -- start start 2024-04-20T10:45:01.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.002+0000 7fda5d164740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a1fb0 0x55eca33a2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.002+0000 7fda5d164740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a28d0 0x55eca33aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.002+0000 7fda5d164740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33ab4b0 0x55eca33ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.002+0000 7fda5d164740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55eca3307050 con 0x55eca33ab4b0 2024-04-20T10:45:01.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.002+0000 7fda5d164740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eca32ad490 con 0x55eca33a1fb0 2024-04-20T10:45:01.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.002+0000 7fda5d164740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55eca3303010 con 0x55eca33a28d0 2024-04-20T10:45:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.003+0000 7fda53e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a1fb0 0x55eca33a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.003+0000 7fda53e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a1fb0 0x55eca33a2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40568/0 (socket says 172.21.15.42:40568) 2024-04-20T10:45:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.003+0000 7fda53e00640 1 -- 172.21.15.42:0/93732650 learned_addr learned my addr 172.21.15.42:0/93732650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.003+0000 7fda53400640 1 --2- 172.21.15.42:0/93732650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a28d0 0x55eca33aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.003+0000 7fda53e00640 1 -- 172.21.15.42:0/93732650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a28d0 msgr2=0x55eca33aaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.003+0000 7fda53e00640 1 --2- 172.21.15.42:0/93732650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a28d0 0x55eca33aaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.003+0000 7fda53e00640 1 -- 172.21.15.42:0/93732650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33ab4b0 msgr2=0x55eca33ad8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:45:01.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.003+0000 7fda53e00640 1 --2- 172.21.15.42:0/93732650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33ab4b0 0x55eca33ad8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.003+0000 7fda53e00640 1 -- 172.21.15.42:0/93732650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eca32b5070 con 0x55eca33a1fb0 2024-04-20T10:45:01.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.003+0000 7fda53400640 1 --2- 172.21.15.42:0/93732650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a28d0 0x55eca33aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:01.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.004+0000 7fda53e00640 1 --2- 172.21.15.42:0/93732650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a1fb0 0x55eca33a2390 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fda44009900 tx=0x7fda44009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44be09291d9fedf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:01.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.004+0000 7fda4be00640 1 -- 172.21.15.42:0/4116509823 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fda40010ac0 con 0x7fda541107b0 2024-04-20T10:45:01.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.007+0000 7fda48c00640 1 -- 172.21.15.42:0/93732650 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44042070 con 0x55eca33a1fb0 2024-04-20T10:45:01.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.007+0000 7fda48c00640 1 -- 172.21.15.42:0/93732650 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda44004240 con 0x55eca33a1fb0 2024-04-20T10:45:01.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.007+0000 7fda48c00640 1 -- 172.21.15.42:0/93732650 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44004ee0 con 0x55eca33a1fb0 2024-04-20T10:45:01.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.007+0000 7fda4be00640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fda380a3ef0 0x7fda380a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.007+0000 7fda4be00640 1 -- 172.21.15.42:0/4116509823 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fda400d0b10 con 0x7fda541107b0 2024-04-20T10:45:01.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.007+0000 7fda53400640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fda380a3ef0 0x7fda380a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:01.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.008+0000 7fda5d164740 1 -- 172.21.15.42:0/93732650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a1fb0 msgr2=0x55eca33a2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:01.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.008+0000 7fda5d164740 1 --2- 172.21.15.42:0/93732650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a1fb0 0x55eca33a2390 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fda44009900 tx=0x7fda44009610 comp rx=0 tx=0).stop 2024-04-20T10:45:01.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.009+0000 7fda53400640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fda380a3ef0 0x7fda380a63b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009fd0 tx=0x7fda4c00d040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:01.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.013+0000 7fda5d164740 1 -- 172.21.15.42:0/93732650 shutdown_connections 2024-04-20T10:45:01.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.013+0000 7fda5d164740 1 --2- 172.21.15.42:0/93732650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33ab4b0 0x55eca33ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.013+0000 7fda5d164740 1 --2- 172.21.15.42:0/93732650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a28d0 0x55eca33aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.013+0000 7fda5d164740 1 --2- 172.21.15.42:0/93732650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a1fb0 0x55eca33a2390 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.013+0000 7fda5d164740 1 -- 172.21.15.42:0/93732650 >> 172.21.15.42:0/93732650 conn(0x55eca339c6e0 msgr2=0x55eca339cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:01.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.013+0000 7fda5d164740 1 -- 172.21.15.42:0/93732650 shutdown_connections 2024-04-20T10:45:01.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.013+0000 7fda5d164740 1 -- 172.21.15.42:0/93732650 wait complete. 2024-04-20T10:45:01.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.014+0000 7fda5d164740 1 -- start start 2024-04-20T10:45:01.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.014+0000 7fda5d164740 1 Processor -- start 2024-04-20T10:45:01.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.015+0000 7fda5d164740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a1fb0 0x55eca33a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.015+0000 7fda5d164740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a28d0 0x55eca33a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.015+0000 7fda5d164740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33ab4b0 0x55eca33a9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.015+0000 7fda5d164740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55eca33ae240 con 0x55eca33ab4b0 2024-04-20T10:45:01.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.015+0000 7fda5d164740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eca32c0860 con 0x55eca33a28d0 2024-04-20T10:45:01.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.015+0000 7fda5d164740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55eca33ae3c0 con 0x55eca33a1fb0 2024-04-20T10:45:01.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.015+0000 7fda5d164740 10 client.0 _ll_register_callbacks cb 0x55eca328efe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:45:01.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.015+0000 7fda53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33ab4b0 0x55eca33a9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:01.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.016+0000 7fda53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33ab4b0 0x55eca33a9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56488/0 (socket says 172.21.15.42:56488) 2024-04-20T10:45:01.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.016+0000 7fda53e00640 1 -- 172.21.15.42:0/3061061139 learned_addr learned my addr 172.21.15.42:0/3061061139 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:01.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.016+0000 7fda52a00640 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a28d0 0x55eca33a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:01.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.016+0000 7fda53e00640 1 -- 172.21.15.42:0/3061061139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a1fb0 msgr2=0x55eca33a85b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:45:01.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.016+0000 7fda53e00640 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a1fb0 0x55eca33a85b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.016+0000 7fda53e00640 1 -- 172.21.15.42:0/3061061139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a28d0 msgr2=0x55eca33a8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:01.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.016+0000 7fda53e00640 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a28d0 0x55eca33a8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.016+0000 7fda53e00640 1 -- 172.21.15.42:0/3061061139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eca32c4950 con 0x55eca33ab4b0 2024-04-20T10:45:01.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.016+0000 7fda52a00640 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a28d0 0x55eca33a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:01.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.017+0000 7fda53e00640 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33ab4b0 0x55eca33a9030 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fda4402c260 tx=0x7fda4402c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425507feafacd5b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:01.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.017+0000 7fda3d600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:45:01.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.017+0000 7fda3d600640 1 -- 172.21.15.42:0/3061061139 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44042040 con 0x55eca33ab4b0 2024-04-20T10:45:01.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.017+0000 7fda3d600640 1 -- 172.21.15.42:0/3061061139 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fda44004ee0 con 0x55eca33ab4b0 2024-04-20T10:45:01.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.018+0000 7fda5d164740 1 -- 172.21.15.42:0/3061061139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda44004240 con 0x55eca33ab4b0 2024-04-20T10:45:01.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.018+0000 7fda3d600640 1 -- 172.21.15.42:0/3061061139 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fda440050e0 con 0x55eca33ab4b0 2024-04-20T10:45:01.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.023+0000 7fda5d164740 10 client.16488 fetch_fsmap learned FSMap version 19 2024-04-20T10:45:01.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.023+0000 7fda5d164740 1 -- 172.21.15.42:0/3061061139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55eca3292820 con 0x55eca33ab4b0 2024-04-20T10:45:01.022 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/sys_ni.o 2024-04-20T10:45:01.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.024+0000 7fda3d600640 1 -- 172.21.15.42:0/3061061139 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda44004240 con 0x55eca33ab4b0 2024-04-20T10:45:01.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.024+0000 7fda3d600640 1 -- 172.21.15.42:0/3061061139 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fda44047b60 con 0x55eca33ab4b0 2024-04-20T10:45:01.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.025+0000 7fda5d164740 10 client.16488 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:45:01.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.025+0000 7fda5d164740 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:45:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.025+0000 7fda5d164740 20 client.16488 populate_metadata read hostname 'smithi042' 2024-04-20T10:45:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.025+0000 7fda5d164740 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55eca33bb6d0 0x55eca33bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.025+0000 7fda5d164740 4 client.16488 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:45:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.025+0000 7fda5d164740 1 -- 172.21.15.42:0/3061061139 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eca3314a70 con 0x55eca33bb6d0 2024-04-20T10:45:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.025+0000 7fda53400640 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55eca33bb6d0 0x55eca33bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:01.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.027+0000 7fda53400640 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55eca33bb6d0 0x55eca33bdb30 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238916096be7a159 server_cookie=1ad3436b5c91dcbe in_seq=0 out_seq=0 2024-04-20T10:45:01.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.029+0000 7fda3d600640 10 client.16488 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.029+0000 7fda3d600640 1 -- 172.21.15.42:0/3061061139 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eca3314a70 con 0x55eca33bb6d0 2024-04-20T10:45:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.029+0000 7fda3d600640 10 client.16488 handle_command_reply: tid=0 2024-04-20T10:45:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.029+0000 7fda5d164740 1 client.16488 shutdown 2024-04-20T10:45:01.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.029+0000 7fda5d164740 1 -- 172.21.15.42:0/3061061139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33ab4b0 msgr2=0x55eca33a9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.029+0000 7fda5d164740 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33ab4b0 0x55eca33a9030 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fda4402c260 tx=0x7fda4402c8a0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.029+0000 7fda5d164740 1 -- 172.21.15.42:0/3061061139 shutdown_connections 2024-04-20T10:45:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.029+0000 7fda5d164740 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55eca33bb6d0 0x55eca33bdb30 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.029+0000 7fda5d164740 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33ab4b0 0x55eca33a9030 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.029+0000 7fda5d164740 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a28d0 0x55eca33a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.029+0000 7fda5d164740 1 --2- 172.21.15.42:0/3061061139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a1fb0 0x55eca33a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.029+0000 7fda3d600640 0 client.16488 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:01.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.030+0000 7fda5d164740 1 -- 172.21.15.42:0/3061061139 >> 172.21.15.42:0/3061061139 conn(0x55eca339c6e0 msgr2=0x55eca339e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.030+0000 7fda5d164740 1 -- 172.21.15.42:0/3061061139 shutdown_connections 2024-04-20T10:45:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.030+0000 7fda5d164740 1 -- 172.21.15.42:0/3061061139 wait complete. 2024-04-20T10:45:01.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.031+0000 7fda5d164740 20 client.16488 trim_cache size 0 max 16384 2024-04-20T10:45:01.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:00 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3894239284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:45:01.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.089+0000 7fda5d164740 1 Processor -- start 2024-04-20T10:45:01.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.093+0000 7fda5d164740 1 -- start start 2024-04-20T10:45:01.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.093+0000 7fda5d164740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a1fb0 0x55eca33aace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.093+0000 7fda5d164740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33a28d0 0x55eca33b7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.093+0000 7fda5d164740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33ab4b0 0x55eca33a7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.093+0000 7fda5d164740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fda541c3370 con 0x55eca33a28d0 2024-04-20T10:45:01.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.093+0000 7fda5d164740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda541c5bd0 con 0x55eca33a1fb0 2024-04-20T10:45:01.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.093+0000 7fda5d164740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fda541c39c0 con 0x55eca33ab4b0 2024-04-20T10:45:01.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.094+0000 7fda53e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33ab4b0 0x55eca33a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:01.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.094+0000 7fda53e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33ab4b0 0x55eca33a7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:55926/0 (socket says 172.21.15.42:55926) 2024-04-20T10:45:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.094+0000 7fda53e00640 1 -- 172.21.15.42:0/3028453397 learned_addr learned my addr 172.21.15.42:0/3028453397 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.094+0000 7fda52a00640 1 --2- 172.21.15.42:0/3028453397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33a28d0 0x55eca33b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.094+0000 7fda53400640 1 --2- 172.21.15.42:0/3028453397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a1fb0 0x55eca33aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.094+0000 7fda53e00640 1 -- 172.21.15.42:0/3028453397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a1fb0 msgr2=0x55eca33aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.094+0000 7fda53e00640 1 --2- 172.21.15.42:0/3028453397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a1fb0 0x55eca33aace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.094+0000 7fda53e00640 1 -- 172.21.15.42:0/3028453397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33a28d0 msgr2=0x55eca33b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.094+0000 7fda53e00640 1 --2- 172.21.15.42:0/3028453397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33a28d0 0x55eca33b7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.094+0000 7fda53e00640 1 -- 172.21.15.42:0/3028453397 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eca30193f0 con 0x55eca33ab4b0 2024-04-20T10:45:01.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.094+0000 7fda52a00640 1 --2- 172.21.15.42:0/3028453397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33a28d0 0x55eca33b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:01.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.098+0000 7fda53e00640 1 --2- 172.21.15.42:0/3028453397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33ab4b0 0x55eca33a7730 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fda44000c00 tx=0x7fda440347d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e2047ab1a028f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:01.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.098+0000 7fda3fe00640 1 -- 172.21.15.42:0/3028453397 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4405f070 con 0x55eca33ab4b0 2024-04-20T10:45:01.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.098+0000 7fda3fe00640 1 -- 172.21.15.42:0/3028453397 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda44009780 con 0x55eca33ab4b0 2024-04-20T10:45:01.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.098+0000 7fda3fe00640 1 -- 172.21.15.42:0/3028453397 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44042070 con 0x55eca33ab4b0 2024-04-20T10:45:01.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.099+0000 7fda5d164740 1 -- 172.21.15.42:0/3028453397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33ab4b0 msgr2=0x55eca33a7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.099+0000 7fda5d164740 1 --2- 172.21.15.42:0/3028453397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33ab4b0 0x55eca33a7730 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fda44000c00 tx=0x7fda440347d0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.100+0000 7fda5d164740 1 -- 172.21.15.42:0/3028453397 shutdown_connections 2024-04-20T10:45:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.100+0000 7fda5d164740 1 --2- 172.21.15.42:0/3028453397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33ab4b0 0x55eca33a7730 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.100+0000 7fda5d164740 1 --2- 172.21.15.42:0/3028453397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33a28d0 0x55eca33b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.100+0000 7fda5d164740 1 --2- 172.21.15.42:0/3028453397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33a1fb0 0x55eca33aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.100+0000 7fda5d164740 1 -- 172.21.15.42:0/3028453397 >> 172.21.15.42:0/3028453397 conn(0x55eca339c6e0 msgr2=0x55eca339db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.100+0000 7fda5d164740 1 -- 172.21.15.42:0/3028453397 shutdown_connections 2024-04-20T10:45:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.100+0000 7fda5d164740 1 -- 172.21.15.42:0/3028453397 wait complete. 2024-04-20T10:45:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.100+0000 7fda5d164740 1 -- start start 2024-04-20T10:45:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.101+0000 7fda5d164740 1 Processor -- start 2024-04-20T10:45:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.103+0000 7fda5d164740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a1fb0 0x55eca33b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.103+0000 7fda5d164740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33a28d0 0x55eca33b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.103+0000 7fda53400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a1fb0 0x55eca33b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.103+0000 7fda53400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a1fb0 0x55eca33b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:55936/0 (socket says 172.21.15.42:55936) 2024-04-20T10:45:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.103+0000 7fda5d164740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33ab4b0 0x55eca33a4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.103+0000 7fda53400640 1 -- 172.21.15.42:0/3132637565 learned_addr learned my addr 172.21.15.42:0/3132637565 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.103+0000 7fda52a00640 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33a28d0 0x55eca33b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:01.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.104+0000 7fda5d164740 1 -- 172.21.15.42:0/3132637565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fda541c4090 con 0x55eca33a28d0 2024-04-20T10:45:01.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.104+0000 7fda5d164740 1 -- 172.21.15.42:0/3132637565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda541c5500 con 0x55eca33ab4b0 2024-04-20T10:45:01.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.104+0000 7fda5d164740 1 -- 172.21.15.42:0/3132637565 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fda541c4760 con 0x55eca33a1fb0 2024-04-20T10:45:01.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.104+0000 7fda5d164740 10 client.0 _ll_register_callbacks cb 0x7fda44005d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:45:01.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.104+0000 7fda53400640 1 -- 172.21.15.42:0/3132637565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33ab4b0 msgr2=0x55eca33a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:01.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.104+0000 7fda53400640 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33ab4b0 0x55eca33a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.104+0000 7fda53400640 1 -- 172.21.15.42:0/3132637565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33a28d0 msgr2=0x55eca33b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:01.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.104+0000 7fda53400640 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33a28d0 0x55eca33b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.104+0000 7fda53400640 1 -- 172.21.15.42:0/3132637565 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eca30e37d0 con 0x55eca33a1fb0 2024-04-20T10:45:01.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.104+0000 7fda52a00640 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33a28d0 0x55eca33b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:01.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.105+0000 7fda53400640 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a1fb0 0x55eca33b3640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00a750 tx=0x7fda4c004a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fedbe8bab3acce9d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:01.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.105+0000 7fda3d600640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:45:01.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.105+0000 7fda3d600640 1 -- 172.21.15.42:0/3132637565 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c01e040 con 0x55eca33a1fb0 2024-04-20T10:45:01.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.105+0000 7fda3d600640 1 -- 172.21.15.42:0/3132637565 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fda4c027560 con 0x55eca33a1fb0 2024-04-20T10:45:01.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.105+0000 7fda5d164740 1 -- 172.21.15.42:0/3132637565 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda44009780 con 0x55eca33a1fb0 2024-04-20T10:45:01.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.109+0000 7fda3d600640 1 -- 172.21.15.42:0/3132637565 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fda4c027760 con 0x55eca33a1fb0 2024-04-20T10:45:01.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.112+0000 7fda5d164740 10 client.26614 fetch_fsmap learned FSMap version 19 2024-04-20T10:45:01.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.112+0000 7fda5d164740 1 -- 172.21.15.42:0/3132637565 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55eca34214d0 con 0x55eca33a1fb0 2024-04-20T10:45:01.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.112+0000 7fda3d600640 1 -- 172.21.15.42:0/3132637565 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda44009780 con 0x55eca33a1fb0 2024-04-20T10:45:01.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.113+0000 7fda3d600640 1 -- 172.21.15.42:0/3132637565 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fda4c037070 con 0x55eca33a1fb0 2024-04-20T10:45:01.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.113+0000 7fda5d164740 10 client.26614 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:45:01.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.113+0000 7fda5d164740 10 client.26614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:45:01.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.113+0000 7fda5d164740 20 client.26614 populate_metadata read hostname 'smithi042' 2024-04-20T10:45:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.113+0000 7fda5d164740 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55eca3421c50 0x55eca34240d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.113+0000 7fda5d164740 4 client.26614 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:45:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.113+0000 7fda5d164740 1 -- 172.21.15.42:0/3132637565 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eca330e8e0 con 0x55eca3421c50 2024-04-20T10:45:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.113+0000 7fda52a00640 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55eca3421c50 0x55eca34240d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:01.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.114+0000 7fda52a00640 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55eca3421c50 0x55eca34240d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83687b55f352fb32 server_cookie=2c05dc783983d324 in_seq=0 out_seq=0 2024-04-20T10:45:01.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.114+0000 7fda3d600640 10 client.26614 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:01.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.117+0000 7fda3d600640 1 -- 172.21.15.42:0/3132637565 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eca330e8e0 con 0x55eca3421c50 2024-04-20T10:45:01.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.117+0000 7fda3d600640 10 client.26614 handle_command_reply: tid=0 2024-04-20T10:45:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.117+0000 7fda5d164740 1 client.26614 shutdown 2024-04-20T10:45:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.118+0000 7fda5d164740 1 -- 172.21.15.42:0/3132637565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a1fb0 msgr2=0x55eca33b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.118+0000 7fda5d164740 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a1fb0 0x55eca33b3640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00a750 tx=0x7fda4c004a80 comp rx=0 tx=0).stop 2024-04-20T10:45:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.118+0000 7fda5d164740 1 -- 172.21.15.42:0/3132637565 shutdown_connections 2024-04-20T10:45:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.118+0000 7fda5d164740 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55eca3421c50 0x55eca34240d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.118+0000 7fda5d164740 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eca33ab4b0 0x55eca33a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.118+0000 7fda5d164740 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eca33a28d0 0x55eca33b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.118+0000 7fda3d600640 0 client.26614 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.118+0000 7fda5d164740 1 --2- 172.21.15.42:0/3132637565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eca33a1fb0 0x55eca33b3640 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.118+0000 7fda5d164740 1 -- 172.21.15.42:0/3132637565 >> 172.21.15.42:0/3132637565 conn(0x55eca339c6e0 msgr2=0x55eca33a8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:01.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.120+0000 7fda5d164740 1 -- 172.21.15.42:0/3132637565 shutdown_connections 2024-04-20T10:45:01.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.120+0000 7fda5d164740 1 -- 172.21.15.42:0/3132637565 wait complete. 2024-04-20T10:45:01.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.120+0000 7fda5d164740 20 client.26614 trim_cache size 0 max 16384 2024-04-20T10:45:01.120 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:45:01.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.124+0000 7fda5a400640 1 -- 172.21.15.42:0/4116509823 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fda380a3ef0 msgr2=0x7fda380a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.124+0000 7fda5a400640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fda380a3ef0 0x7fda380a63b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009fd0 tx=0x7fda4c00d040 comp rx=0 tx=0).stop 2024-04-20T10:45:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.124+0000 7fda5a400640 1 -- 172.21.15.42:0/4116509823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda541107b0 msgr2=0x7fda54111dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.124+0000 7fda5a400640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda541107b0 0x7fda54111dd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e9f0 tx=0x7fda4000eeb0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.124+0000 7fda5a400640 1 -- 172.21.15.42:0/4116509823 shutdown_connections 2024-04-20T10:45:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.124+0000 7fda5a400640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fda380a3ef0 0x7fda380a63b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.124+0000 7fda5a400640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5411b740 0x7fda54112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.124+0000 7fda5a400640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda541107b0 0x7fda54111dd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.124+0000 7fda5a400640 1 --2- 172.21.15.42:0/4116509823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda5410e7e0 0x7fda54118d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.124+0000 7fda5a400640 1 -- 172.21.15.42:0/4116509823 >> 172.21.15.42:0/4116509823 conn(0x7fda5406d070 msgr2=0x7fda54072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.124+0000 7fda5a400640 1 -- 172.21.15.42:0/4116509823 shutdown_connections 2024-04-20T10:45:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:01.124+0000 7fda5a400640 1 -- 172.21.15.42:0/4116509823 wait complete. 2024-04-20T10:45:01.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:00 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3894239284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:45:01.500 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/select.o 2024-04-20T10:45:01.552 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs3super.o 2024-04-20T10:45:01.578 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/dcache.o 2024-04-20T10:45:01.737 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/nsproxy.o 2024-04-20T10:45:01.870 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs3client.o 2024-04-20T10:45:01.937 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/inode.o 2024-04-20T10:45:01.982 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs3proc.o 2024-04-20T10:45:02.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:01 smithi046 ceph-mon[21574]: pgmap v341: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-04-20T10:45:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:01 smithi042 ceph-mon[17078]: pgmap v341: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-04-20T10:45:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:01 smithi162 ceph-mon[21854]: pgmap v341: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-04-20T10:45:02.180 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T10:45:02.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:45:02.180 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T10:45:02.697 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/attr.o 2024-04-20T10:45:02.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.708+0000 7f43eb200640 1 -- 172.21.15.42:0/3012535637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e410a620 msgr2=0x7f43e4108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:02.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.708+0000 7f43eb200640 1 --2- 172.21.15.42:0/3012535637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e410a620 0x7f43e4108f60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009950 tx=0x7f43d002bcd0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.712+0000 7f43eb200640 1 -- 172.21.15.42:0/3012535637 shutdown_connections 2024-04-20T10:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.712+0000 7f43eb200640 1 --2- 172.21.15.42:0/3012535637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43e4117000 0x7f43e41193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.712+0000 7f43eb200640 1 --2- 172.21.15.42:0/3012535637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e41094a0 0x7f43e4109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.712+0000 7f43eb200640 1 --2- 172.21.15.42:0/3012535637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e410a620 0x7f43e4108f60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.712+0000 7f43eb200640 1 -- 172.21.15.42:0/3012535637 >> 172.21.15.42:0/3012535637 conn(0x7f43e406d3a0 msgr2=0x7f43e4072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:02.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.716+0000 7f43eb200640 1 -- 172.21.15.42:0/3012535637 shutdown_connections 2024-04-20T10:45:02.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.716+0000 7f43eb200640 1 -- 172.21.15.42:0/3012535637 wait complete. 2024-04-20T10:45:02.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.717+0000 7f43eb200640 1 Processor -- start 2024-04-20T10:45:02.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.719+0000 7f43eb200640 1 -- start start 2024-04-20T10:45:02.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.719+0000 7f43eb200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e41094a0 0x7f43e41a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.719+0000 7f43eb200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43e410a620 0x7f43e41a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.720+0000 7f43eb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e4117000 0x7f43e41abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.720+0000 7f43e3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e41094a0 0x7f43e41a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.720+0000 7f43eb200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41139a0 con 0x7f43e41094a0 2024-04-20T10:45:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.720+0000 7f43eb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4113820 con 0x7f43e4117000 2024-04-20T10:45:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.720+0000 7f43eb200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4113b20 con 0x7f43e410a620 2024-04-20T10:45:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.720+0000 7f43e3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e41094a0 0x7f43e41a75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56530/0 (socket says 172.21.15.42:56530) 2024-04-20T10:45:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.720+0000 7f43e3e00640 1 -- 172.21.15.42:0/2808363408 learned_addr learned my addr 172.21.15.42:0/2808363408 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:02.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.720+0000 7f43e3400640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43e410a620 0x7f43e41a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.720+0000 7f43e9200640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e4117000 0x7f43e41abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.721+0000 7f43e3e00640 1 -- 172.21.15.42:0/2808363408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43e410a620 msgr2=0x7f43e41a7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:02.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.721+0000 7f43e3e00640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43e410a620 0x7f43e41a7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.721+0000 7f43e3e00640 1 -- 172.21.15.42:0/2808363408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e4117000 msgr2=0x7f43e41abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:02.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.721+0000 7f43e3e00640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e4117000 0x7f43e41abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.721+0000 7f43e3e00640 1 -- 172.21.15.42:0/2808363408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e41ac540 con 0x7f43e41094a0 2024-04-20T10:45:02.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.721+0000 7f43e3e00640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e41094a0 0x7f43e41a75a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f43d0000c00 tx=0x7f43d00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:02.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.722+0000 7f43e0c00640 1 -- 172.21.15.42:0/2808363408 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0042020 con 0x7f43e41094a0 2024-04-20T10:45:02.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.722+0000 7f43e0c00640 1 -- 172.21.15.42:0/2808363408 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d00044a0 con 0x7f43e41094a0 2024-04-20T10:45:02.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.722+0000 7f43eb200640 1 -- 172.21.15.42:0/2808363408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e41ac7d0 con 0x7f43e41094a0 2024-04-20T10:45:02.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.722+0000 7f43e0c00640 1 -- 172.21.15.42:0/2808363408 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d00039c0 con 0x7f43e41094a0 2024-04-20T10:45:02.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.723+0000 7f43eb200640 1 -- 172.21.15.42:0/2808363408 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e41b40c0 con 0x7f43e41094a0 2024-04-20T10:45:02.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.729+0000 7f43e0c00640 1 -- 172.21.15.42:0/2808363408 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f43d0045020 con 0x7f43e41094a0 2024-04-20T10:45:02.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.732+0000 7f43e0c00640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f43c80a3de0 0x7f43c80a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.732+0000 7f43e0c00640 1 -- 172.21.15.42:0/2808363408 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f43d00f2800 con 0x7f43e41094a0 2024-04-20T10:45:02.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.734+0000 7f43edfb4740 1 Processor -- start 2024-04-20T10:45:02.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.734+0000 7f43edfb4740 1 -- start start 2024-04-20T10:45:02.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.734+0000 7f43edfb4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7ab0f0 0x55c4de7ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.734+0000 7f43edfb4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7aba10 0x55c4de7b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.734+0000 7f43edfb4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7b45f0 0x55c4de7b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.734+0000 7f43edfb4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c4de70fd20 con 0x55c4de7ab0f0 2024-04-20T10:45:02.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.734+0000 7f43edfb4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c4de6b63e0 con 0x55c4de7aba10 2024-04-20T10:45:02.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.734+0000 7f43e3400640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f43c80a3de0 0x7f43c80a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.734+0000 7f43edfb4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c4de70bce0 con 0x55c4de7b45f0 2024-04-20T10:45:02.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.735+0000 7f43e9200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7b45f0 0x55c4de7b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.735+0000 7f43e9200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7b45f0 0x55c4de7b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:55968/0 (socket says 172.21.15.42:55968) 2024-04-20T10:45:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.735+0000 7f43e9200640 1 -- 172.21.15.42:0/2947544150 learned_addr learned my addr 172.21.15.42:0/2947544150 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.735+0000 7f43e9200640 1 -- 172.21.15.42:0/2947544150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7aba10 msgr2=0x55c4de7b40b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:45:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.735+0000 7f43e9200640 1 --2- 172.21.15.42:0/2947544150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7aba10 0x55c4de7b40b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.735+0000 7f43e9200640 1 -- 172.21.15.42:0/2947544150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7ab0f0 msgr2=0x55c4de7ab4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:45:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.735+0000 7f43e9200640 1 --2- 172.21.15.42:0/2947544150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7ab0f0 0x55c4de7ab4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.735+0000 7f43e9200640 1 -- 172.21.15.42:0/2947544150 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4de6d5980 con 0x55c4de7b45f0 2024-04-20T10:45:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.735+0000 7f43e3400640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f43c80a3de0 0x7f43c80a62a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f43e410c6c0 tx=0x7f43dc009680 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.736+0000 7f43e9200640 1 --2- 172.21.15.42:0/2947544150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7b45f0 0x55c4de7b69e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009990 tx=0x7f43d4009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df9b1e550362cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.738+0000 7f43d9600640 1 -- 172.21.15.42:0/2947544150 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4017070 con 0x55c4de7b45f0 2024-04-20T10:45:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.738+0000 7f43d9600640 1 -- 172.21.15.42:0/2947544150 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d4013470 con 0x55c4de7b45f0 2024-04-20T10:45:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.738+0000 7f43d9600640 1 -- 172.21.15.42:0/2947544150 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4012610 con 0x55c4de7b45f0 2024-04-20T10:45:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.739+0000 7f43edfb4740 1 -- 172.21.15.42:0/2947544150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7b45f0 msgr2=0x55c4de7b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.739+0000 7f43edfb4740 1 --2- 172.21.15.42:0/2947544150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7b45f0 0x55c4de7b69e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009990 tx=0x7f43d4009e50 comp rx=0 tx=0).stop 2024-04-20T10:45:02.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.740+0000 7f43edfb4740 1 -- 172.21.15.42:0/2947544150 shutdown_connections 2024-04-20T10:45:02.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.740+0000 7f43edfb4740 1 --2- 172.21.15.42:0/2947544150 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7b45f0 0x55c4de7b69e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.740+0000 7f43edfb4740 1 --2- 172.21.15.42:0/2947544150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7aba10 0x55c4de7b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.740+0000 7f43edfb4740 1 --2- 172.21.15.42:0/2947544150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7ab0f0 0x55c4de7ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.740+0000 7f43edfb4740 1 -- 172.21.15.42:0/2947544150 >> 172.21.15.42:0/2947544150 conn(0x55c4de7a5820 msgr2=0x55c4de7a5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:02.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.741+0000 7f43edfb4740 1 -- 172.21.15.42:0/2947544150 shutdown_connections 2024-04-20T10:45:02.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.741+0000 7f43edfb4740 1 -- 172.21.15.42:0/2947544150 wait complete. 2024-04-20T10:45:02.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.741+0000 7f43edfb4740 1 -- start start 2024-04-20T10:45:02.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.742+0000 7f43edfb4740 1 Processor -- start 2024-04-20T10:45:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.746+0000 7f43edfb4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7ab0f0 0x55c4de7b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.746+0000 7f43edfb4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7aba10 0x55c4de7b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.746+0000 7f43edfb4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7b45f0 0x55c4de7b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.746+0000 7f43edfb4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c4de7b73e0 con 0x55c4de7b45f0 2024-04-20T10:45:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.746+0000 7f43edfb4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c4de6c97a0 con 0x55c4de7ab0f0 2024-04-20T10:45:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.746+0000 7f43edfb4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c4de7b7560 con 0x55c4de7aba10 2024-04-20T10:45:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.746+0000 7f43edfb4740 10 client.0 _ll_register_callbacks cb 0x55c4de6cbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:45:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.747+0000 7f43e3400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7aba10 0x55c4de7b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.747+0000 7f43e3400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7aba10 0x55c4de7b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:55970/0 (socket says 172.21.15.42:55970) 2024-04-20T10:45:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.747+0000 7f43e3400640 1 -- 172.21.15.42:0/1885532277 learned_addr learned my addr 172.21.15.42:0/1885532277 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.747+0000 7f43e3e00640 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7ab0f0 0x55c4de7b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.747+0000 7f43e9200640 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7b45f0 0x55c4de7b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.747+0000 7f43e3400640 1 -- 172.21.15.42:0/1885532277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7ab0f0 msgr2=0x55c4de7b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:02.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.747+0000 7f43e3400640 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7ab0f0 0x55c4de7b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.747+0000 7f43e3400640 1 -- 172.21.15.42:0/1885532277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7b45f0 msgr2=0x55c4de7b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:02.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.747+0000 7f43e3400640 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7b45f0 0x55c4de7b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.747+0000 7f43e3400640 1 -- 172.21.15.42:0/1885532277 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4de4ea7e0 con 0x55c4de7aba10 2024-04-20T10:45:02.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.748+0000 7f43e3400640 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7aba10 0x55c4de7b1ca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f43dc012750 tx=0x7f43dc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28137c910e60e152 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:02.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.749+0000 7f43ce000640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:45:02.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.751+0000 7f43ce000640 1 -- 172.21.15.42:0/1885532277 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc023070 con 0x55c4de7aba10 2024-04-20T10:45:02.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.751+0000 7f43ce000640 1 -- 172.21.15.42:0/1885532277 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f43dc0119b0 con 0x55c4de7aba10 2024-04-20T10:45:02.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.751+0000 7f43edfb4740 1 -- 172.21.15.42:0/1885532277 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d4013470 con 0x55c4de7aba10 2024-04-20T10:45:02.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.759+0000 7f43ce000640 1 -- 172.21.15.42:0/1885532277 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f43dc02f050 con 0x55c4de7aba10 2024-04-20T10:45:02.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.759+0000 7f43edfb4740 10 client.26623 fetch_fsmap learned FSMap version 19 2024-04-20T10:45:02.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.759+0000 7f43edfb4740 1 -- 172.21.15.42:0/1885532277 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c4de54af70 con 0x55c4de7aba10 2024-04-20T10:45:02.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.760+0000 7f43ce000640 1 -- 172.21.15.42:0/1885532277 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d4013470 con 0x55c4de7aba10 2024-04-20T10:45:02.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.760+0000 7f43ce000640 1 -- 172.21.15.42:0/1885532277 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f43dc00f040 con 0x55c4de7aba10 2024-04-20T10:45:02.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.760+0000 7f43edfb4740 10 client.26623 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:45:02.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.760+0000 7f43edfb4740 10 client.26623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:45:02.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.760+0000 7f43edfb4740 20 client.26623 populate_metadata read hostname 'smithi042' 2024-04-20T10:45:02.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.760+0000 7f43edfb4740 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c4de7c4ab0 0x55c4de7c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.760+0000 7f43edfb4740 4 client.26623 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:45:02.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.760+0000 7f43edfb4740 1 -- 172.21.15.42:0/1885532277 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4de71d740 con 0x55c4de7c4ab0 2024-04-20T10:45:02.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.761+0000 7f43e3e00640 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c4de7c4ab0 0x55c4de7c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.762+0000 7f43e3e00640 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c4de7c4ab0 0x55c4de7c6ec0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b996267f7ee8d1 server_cookie=dcdbbfb8caa48983 in_seq=0 out_seq=0 2024-04-20T10:45:02.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.762+0000 7f43ce000640 10 client.26623 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:02.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.764+0000 7f43ce000640 1 -- 172.21.15.42:0/1885532277 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4de71d740 con 0x55c4de7c4ab0 2024-04-20T10:45:02.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.764+0000 7f43ce000640 10 client.26623 handle_command_reply: tid=0 2024-04-20T10:45:02.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.765+0000 7f43edfb4740 1 client.26623 shutdown 2024-04-20T10:45:02.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.765+0000 7f43edfb4740 1 -- 172.21.15.42:0/1885532277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7aba10 msgr2=0x55c4de7b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:02.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.765+0000 7f43edfb4740 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7aba10 0x55c4de7b1ca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f43dc012750 tx=0x7f43dc004a40 comp rx=0 tx=0).stop 2024-04-20T10:45:02.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.765+0000 7f43edfb4740 1 -- 172.21.15.42:0/1885532277 shutdown_connections 2024-04-20T10:45:02.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.765+0000 7f43edfb4740 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c4de7c4ab0 0x55c4de7c6ec0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.765+0000 7f43edfb4740 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7b45f0 0x55c4de7b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.765+0000 7f43edfb4740 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7aba10 0x55c4de7b1ca0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.765+0000 7f43edfb4740 1 --2- 172.21.15.42:0/1885532277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7ab0f0 0x55c4de7b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.765+0000 7f43ce000640 0 client.26623 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:02.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.765+0000 7f43edfb4740 1 -- 172.21.15.42:0/1885532277 >> 172.21.15.42:0/1885532277 conn(0x55c4de7a5820 msgr2=0x55c4de7a7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:02.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.765+0000 7f43edfb4740 1 -- 172.21.15.42:0/1885532277 shutdown_connections 2024-04-20T10:45:02.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.766+0000 7f43edfb4740 1 -- 172.21.15.42:0/1885532277 wait complete. 2024-04-20T10:45:02.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.766+0000 7f43edfb4740 20 client.26623 trim_cache size 0 max 16384 2024-04-20T10:45:02.778 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/bad_inode.o 2024-04-20T10:45:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.807+0000 7f43edfb4740 1 Processor -- start 2024-04-20T10:45:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.807+0000 7f43edfb4740 1 -- start start 2024-04-20T10:45:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.807+0000 7f43edfb4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7ab0f0 0x55c4de7b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.808+0000 7f43edfb4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7aba10 0x55c4de7a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.808+0000 7f43edfb4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7b45f0 0x55c4de7b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.808+0000 7f43edfb4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41ba960 con 0x55c4de7aba10 2024-04-20T10:45:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.808+0000 7f43edfb4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41bd1c0 con 0x55c4de7b45f0 2024-04-20T10:45:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.808+0000 7f43edfb4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41bafb0 con 0x55c4de7ab0f0 2024-04-20T10:45:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.808+0000 7f43e9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7b45f0 0x55c4de7b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.808+0000 7f43e9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7b45f0 0x55c4de7b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40636/0 (socket says 172.21.15.42:40636) 2024-04-20T10:45:02.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.808+0000 7f43e9200640 1 -- 172.21.15.42:0/200242390 learned_addr learned my addr 172.21.15.42:0/200242390 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.808+0000 7f43e3e00640 1 --2- 172.21.15.42:0/200242390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7ab0f0 0x55c4de7b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.809+0000 7f43e3400640 1 --2- 172.21.15.42:0/200242390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7aba10 0x55c4de7a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.809+0000 7f43e9200640 1 -- 172.21.15.42:0/200242390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7ab0f0 msgr2=0x55c4de7b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.809+0000 7f43e9200640 1 --2- 172.21.15.42:0/200242390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7ab0f0 0x55c4de7b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.809+0000 7f43e9200640 1 -- 172.21.15.42:0/200242390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7aba10 msgr2=0x55c4de7a74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.809+0000 7f43e9200640 1 --2- 172.21.15.42:0/200242390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7aba10 0x55c4de7a74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.809+0000 7f43e9200640 1 -- 172.21.15.42:0/200242390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4de7c1270 con 0x55c4de7b45f0 2024-04-20T10:45:02.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.810+0000 7f43e9200640 1 --2- 172.21.15.42:0/200242390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7b45f0 0x55c4de7b2040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f43d4006fd0 tx=0x7f43d4009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23043630c6f322bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:02.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.814+0000 7f43d8c00640 1 -- 172.21.15.42:0/200242390 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4017040 con 0x55c4de7b45f0 2024-04-20T10:45:02.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.814+0000 7f43d8c00640 1 -- 172.21.15.42:0/200242390 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d4004500 con 0x55c4de7b45f0 2024-04-20T10:45:02.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.814+0000 7f43d8c00640 1 -- 172.21.15.42:0/200242390 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4003910 con 0x55c4de7b45f0 2024-04-20T10:45:02.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.815+0000 7f43edfb4740 1 -- 172.21.15.42:0/200242390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7b45f0 msgr2=0x55c4de7b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:02.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.815+0000 7f43edfb4740 1 --2- 172.21.15.42:0/200242390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7b45f0 0x55c4de7b2040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f43d4006fd0 tx=0x7f43d4009460 comp rx=0 tx=0).stop 2024-04-20T10:45:02.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.819+0000 7f43edfb4740 1 -- 172.21.15.42:0/200242390 shutdown_connections 2024-04-20T10:45:02.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.819+0000 7f43edfb4740 1 --2- 172.21.15.42:0/200242390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7b45f0 0x55c4de7b2040 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.819+0000 7f43edfb4740 1 --2- 172.21.15.42:0/200242390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7aba10 0x55c4de7a74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.819+0000 7f43edfb4740 1 --2- 172.21.15.42:0/200242390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7ab0f0 0x55c4de7b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.819+0000 7f43edfb4740 1 -- 172.21.15.42:0/200242390 >> 172.21.15.42:0/200242390 conn(0x55c4de7a5820 msgr2=0x55c4de7a4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:02.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.819+0000 7f43edfb4740 1 -- 172.21.15.42:0/200242390 shutdown_connections 2024-04-20T10:45:02.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.819+0000 7f43edfb4740 1 -- 172.21.15.42:0/200242390 wait complete. 2024-04-20T10:45:02.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.820+0000 7f43edfb4740 1 -- start start 2024-04-20T10:45:02.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.820+0000 7f43edfb4740 1 Processor -- start 2024-04-20T10:45:02.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.821+0000 7f43edfb4740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7ab0f0 0x55c4de7aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.821+0000 7f43edfb4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7aba10 0x55c4de7af100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.822+0000 7f43edfb4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7b45f0 0x55c4de83b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.822+0000 7f43edfb4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41bb680 con 0x55c4de7b45f0 2024-04-20T10:45:02.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.822+0000 7f43edfb4740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41bcaf0 con 0x55c4de7ab0f0 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.822+0000 7f43edfb4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41bbd50 con 0x55c4de7aba10 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.822+0000 7f43edfb4740 10 client.0 _ll_register_callbacks cb 0x7f43dc016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.822+0000 7f43e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7ab0f0 0x55c4de7aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.822+0000 7f43e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7ab0f0 0x55c4de7aebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40648/0 (socket says 172.21.15.42:40648) 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.822+0000 7f43e3e00640 1 -- 172.21.15.42:0/218705867 learned_addr learned my addr 172.21.15.42:0/218705867 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.822+0000 7f43e9200640 1 --2- 172.21.15.42:0/218705867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7b45f0 0x55c4de83b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.823+0000 7f43e9200640 1 -- 172.21.15.42:0/218705867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7aba10 msgr2=0x55c4de7af100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.823+0000 7f43e9200640 1 --2- 172.21.15.42:0/218705867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7aba10 0x55c4de7af100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.823+0000 7f43e9200640 1 -- 172.21.15.42:0/218705867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7ab0f0 msgr2=0x55c4de7aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.823+0000 7f43e9200640 1 --2- 172.21.15.42:0/218705867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7ab0f0 0x55c4de7aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.823+0000 7f43e9200640 1 -- 172.21.15.42:0/218705867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4de3e2170 con 0x55c4de7b45f0 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.824+0000 7f43e9200640 1 --2- 172.21.15.42:0/218705867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7b45f0 0x55c4de83b010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f43d400e5a0 tx=0x7f43d400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a844498e1563a6d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:02.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.824+0000 7f43ce000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:45:02.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.825+0000 7f43ce000640 1 -- 172.21.15.42:0/218705867 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d402a070 con 0x55c4de7b45f0 2024-04-20T10:45:02.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.825+0000 7f43ce000640 1 -- 172.21.15.42:0/218705867 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f43d401c450 con 0x55c4de7b45f0 2024-04-20T10:45:02.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.825+0000 7f43edfb4740 1 -- 172.21.15.42:0/218705867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d4004500 con 0x55c4de7b45f0 2024-04-20T10:45:02.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.828+0000 7f43ce000640 1 -- 172.21.15.42:0/218705867 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f43d4017040 con 0x55c4de7b45f0 2024-04-20T10:45:02.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.830+0000 7f43edfb4740 10 client.16509 fetch_fsmap learned FSMap version 19 2024-04-20T10:45:02.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.830+0000 7f43edfb4740 1 -- 172.21.15.42:0/218705867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c4de83c630 con 0x55c4de7b45f0 2024-04-20T10:45:02.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.830+0000 7f43ce000640 1 -- 172.21.15.42:0/218705867 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d4004500 con 0x55c4de7b45f0 2024-04-20T10:45:02.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.831+0000 7f43ce000640 1 -- 172.21.15.42:0/218705867 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f43d402d030 con 0x55c4de7b45f0 2024-04-20T10:45:02.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.831+0000 7f43edfb4740 10 client.16509 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:45:02.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.831+0000 7f43edfb4740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:45:02.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.831+0000 7f43edfb4740 20 client.16509 populate_metadata read hostname 'smithi042' 2024-04-20T10:45:02.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.831+0000 7f43edfb4740 1 --2- 172.21.15.42:0/218705867 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c4de83c970 0x55c4de83ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:02.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.831+0000 7f43edfb4740 4 client.16509 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T10:45:02.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.831+0000 7f43edfb4740 1 -- 172.21.15.42:0/218705867 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c4de7175b0 con 0x55c4de83c970 2024-04-20T10:45:02.832 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/notifier.o 2024-04-20T10:45:02.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.831+0000 7f43e3e00640 1 --2- 172.21.15.42:0/218705867 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c4de83c970 0x55c4de83ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:02.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.832+0000 7f43ce000640 10 client.16509 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:02.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.833+0000 7f43e3e00640 1 --2- 172.21.15.42:0/218705867 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c4de83c970 0x55c4de83ed60 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db48be859cac79b server_cookie=1eb0b71accc81ff0 in_seq=0 out_seq=0 2024-04-20T10:45:02.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.837+0000 7f43ce000640 1 -- 172.21.15.42:0/218705867 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c4de7175b0 con 0x55c4de83c970 2024-04-20T10:45:02.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.837+0000 7f43ce000640 10 client.16509 handle_command_reply: tid=0 2024-04-20T10:45:02.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.837+0000 7f43edfb4740 1 client.16509 shutdown 2024-04-20T10:45:02.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.837+0000 7f43edfb4740 1 -- 172.21.15.42:0/218705867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7b45f0 msgr2=0x55c4de83b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:02.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.837+0000 7f43edfb4740 1 --2- 172.21.15.42:0/218705867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7b45f0 0x55c4de83b010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f43d400e5a0 tx=0x7f43d400ec90 comp rx=0 tx=0).stop 2024-04-20T10:45:02.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.837+0000 7f43edfb4740 1 -- 172.21.15.42:0/218705867 shutdown_connections 2024-04-20T10:45:02.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.837+0000 7f43edfb4740 1 --2- 172.21.15.42:0/218705867 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55c4de83c970 0x55c4de83ed60 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.837+0000 7f43ce000640 0 client.16509 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:02.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.837+0000 7f43edfb4740 1 --2- 172.21.15.42:0/218705867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c4de7b45f0 0x55c4de83b010 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.838+0000 7f43edfb4740 1 --2- 172.21.15.42:0/218705867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c4de7aba10 0x55c4de7af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.838+0000 7f43edfb4740 1 --2- 172.21.15.42:0/218705867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4de7ab0f0 0x55c4de7aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.838+0000 7f43edfb4740 1 -- 172.21.15.42:0/218705867 >> 172.21.15.42:0/218705867 conn(0x55c4de7a5820 msgr2=0x55c4de7a4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.847+0000 7f43edfb4740 1 -- 172.21.15.42:0/218705867 shutdown_connections 2024-04-20T10:45:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.847+0000 7f43edfb4740 1 -- 172.21.15.42:0/218705867 wait complete. 2024-04-20T10:45:02.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.847+0000 7f43edfb4740 20 client.16509 trim_cache size 0 max 16384 2024-04-20T10:45:02.847 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:45:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "return_code": 0, 2024-04-20T10:45:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_tag": "0826a9ac-d90c-4443-910b-eb74a0811906", 2024-04-20T10:45:02.847 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": "asynchronous" 2024-04-20T10:45:02.847 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:45:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.851+0000 7f43eb200640 1 -- 172.21.15.42:0/2808363408 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f43c80a3de0 msgr2=0x7f43c80a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:02.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.851+0000 7f43eb200640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f43c80a3de0 0x7f43c80a62a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f43e410c6c0 tx=0x7f43dc009680 comp rx=0 tx=0).stop 2024-04-20T10:45:02.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.851+0000 7f43eb200640 1 -- 172.21.15.42:0/2808363408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e41094a0 msgr2=0x7f43e41a75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:02.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.851+0000 7f43eb200640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e41094a0 0x7f43e41a75a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f43d0000c00 tx=0x7f43d00043b0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.851+0000 7f43eb200640 1 -- 172.21.15.42:0/2808363408 shutdown_connections 2024-04-20T10:45:02.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.851+0000 7f43eb200640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f43c80a3de0 0x7f43c80a62a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.851+0000 7f43eb200640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e4117000 0x7f43e41abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.851+0000 7f43eb200640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43e410a620 0x7f43e41a7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.851+0000 7f43eb200640 1 --2- 172.21.15.42:0/2808363408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e41094a0 0x7f43e41a75a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:02.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.851+0000 7f43eb200640 1 -- 172.21.15.42:0/2808363408 >> 172.21.15.42:0/2808363408 conn(0x7f43e406d3a0 msgr2=0x7f43e4072d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:02.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.851+0000 7f43eb200640 1 -- 172.21.15.42:0/2808363408 shutdown_connections 2024-04-20T10:45:02.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:02.851+0000 7f43eb200640 1 -- 172.21.15.42:0/2808363408 wait complete. 2024-04-20T10:45:02.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:45:02.904 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:45:03.193 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/file.o 2024-04-20T10:45:03.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.331+0000 7fe381800640 1 -- 172.21.15.42:0/1143294452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe37c101770 msgr2=0x7fe37c105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:03.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.331+0000 7fe381800640 1 --2- 172.21.15.42:0/1143294452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe37c101770 0x7fe37c105e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe368009900 tx=0x7fe36802bcc0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.334+0000 7fe381800640 1 -- 172.21.15.42:0/1143294452 shutdown_connections 2024-04-20T10:45:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.334+0000 7fe381800640 1 --2- 172.21.15.42:0/1143294452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe37c106350 0x7fe37c106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.334+0000 7fe381800640 1 --2- 172.21.15.42:0/1143294452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe37c101770 0x7fe37c105e10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.334+0000 7fe381800640 1 --2- 172.21.15.42:0/1143294452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37c100e50 0x7fe37c101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.334+0000 7fe381800640 1 -- 172.21.15.42:0/1143294452 >> 172.21.15.42:0/1143294452 conn(0x7fe37c077fc0 msgr2=0x7fe37c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.334+0000 7fe381800640 1 -- 172.21.15.42:0/1143294452 shutdown_connections 2024-04-20T10:45:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.334+0000 7fe381800640 1 -- 172.21.15.42:0/1143294452 wait complete. 2024-04-20T10:45:03.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.336+0000 7fe381800640 1 Processor -- start 2024-04-20T10:45:03.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.340+0000 7fe381800640 1 -- start start 2024-04-20T10:45:03.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.340+0000 7fe381800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37c100e50 0x7fe37c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.340+0000 7fe381800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe37c101770 0x7fe37c19f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.340+0000 7fe381800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe37c106350 0x7fe37c1a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.340+0000 7fe381800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c118940 con 0x7fe37c100e50 2024-04-20T10:45:03.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.340+0000 7fe381800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1187c0 con 0x7fe37c106350 2024-04-20T10:45:03.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.340+0000 7fe381800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c118ac0 con 0x7fe37c101770 2024-04-20T10:45:03.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.341+0000 7fe37a000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe37c101770 0x7fe37c19f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.341+0000 7fe37b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe37c106350 0x7fe37c1a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.341+0000 7fe37b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe37c106350 0x7fe37c1a36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40654/0 (socket says 172.21.15.42:40654) 2024-04-20T10:45:03.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.341+0000 7fe37a000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe37c101770 0x7fe37c19f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:56000/0 (socket says 172.21.15.42:56000) 2024-04-20T10:45:03.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.341+0000 7fe37b400640 1 -- 172.21.15.42:0/25015241 learned_addr learned my addr 172.21.15.42:0/25015241 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:03.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.341+0000 7fe37aa00640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37c100e50 0x7fe37c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.341+0000 7fe37b400640 1 -- 172.21.15.42:0/25015241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe37c101770 msgr2=0x7fe37c19f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:03.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.341+0000 7fe37b400640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe37c101770 0x7fe37c19f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.341+0000 7fe37b400640 1 -- 172.21.15.42:0/25015241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37c100e50 msgr2=0x7fe37c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:03.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.341+0000 7fe37b400640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37c100e50 0x7fe37c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.341+0000 7fe37b400640 1 -- 172.21.15.42:0/25015241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c1a3e10 con 0x7fe37c106350 2024-04-20T10:45:03.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.341+0000 7fe37aa00640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37c100e50 0x7fe37c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:45:03.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.344+0000 7fe37b400640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe37c106350 0x7fe37c1a36e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe37400ea60 tx=0x7fe37400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:03.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.344+0000 7fe373400640 1 -- 172.21.15.42:0/25015241 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37400ce40 con 0x7fe37c106350 2024-04-20T10:45:03.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.344+0000 7fe373400640 1 -- 172.21.15.42:0/25015241 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe374005020 con 0x7fe37c106350 2024-04-20T10:45:03.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.344+0000 7fe373400640 1 -- 172.21.15.42:0/25015241 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374010690 con 0x7fe37c106350 2024-04-20T10:45:03.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.344+0000 7fe381800640 1 -- 172.21.15.42:0/25015241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c1a4100 con 0x7fe37c106350 2024-04-20T10:45:03.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.345+0000 7fe381800640 1 -- 172.21.15.42:0/25015241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1073e0 con 0x7fe37c106350 2024-04-20T10:45:03.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.349+0000 7fe373400640 1 -- 172.21.15.42:0/25015241 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe374005520 con 0x7fe37c106350 2024-04-20T10:45:03.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.352+0000 7fe373400640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe3600a3da0 0x7fe3600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.352+0000 7fe37aa00640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe3600a3da0 0x7fe3600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.356+0000 7fe373400640 1 -- 172.21.15.42:0/25015241 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fe3740d0cc0 con 0x7fe37c106350 2024-04-20T10:45:03.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.357+0000 7fe37aa00640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe3600a3da0 0x7fe3600a6260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005d00 tx=0x7fe36c009550 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:03.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.361+0000 7fe38442e740 1 Processor -- start 2024-04-20T10:45:03.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.365+0000 7fe38442e740 1 -- start start 2024-04-20T10:45:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.365+0000 7fe38442e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349c1a0 0x55d58349c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.365+0000 7fe38442e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349cac0 0x55d5834a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.365+0000 7fe38442e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d5834a56a0 0x55d5834a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.365+0000 7fe38442e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d583402150 con 0x55d5834a56a0 2024-04-20T10:45:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.365+0000 7fe38442e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d5833a8370 con 0x55d58349c1a0 2024-04-20T10:45:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.365+0000 7fe38442e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d5833fe110 con 0x55d58349cac0 2024-04-20T10:45:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.366+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349c1a0 0x55d58349c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.366+0000 7fe37a000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349cac0 0x55d5834a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.366+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349c1a0 0x55d58349c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40658/0 (socket says 172.21.15.42:40658) 2024-04-20T10:45:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.366+0000 7fe37aa00640 1 -- 172.21.15.42:0/2440930934 learned_addr learned my addr 172.21.15.42:0/2440930934 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.366+0000 7fe37a000640 1 -- 172.21.15.42:0/2440930934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349c1a0 msgr2=0x55d58349c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.366+0000 7fe37a000640 1 --2- 172.21.15.42:0/2440930934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349c1a0 0x55d58349c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.366+0000 7fe37a000640 1 -- 172.21.15.42:0/2440930934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d5834a56a0 msgr2=0x55d5834a7a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:45:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.366+0000 7fe37a000640 1 --2- 172.21.15.42:0/2440930934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d5834a56a0 0x55d5834a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.366+0000 7fe37a000640 1 -- 172.21.15.42:0/2440930934 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5831de7d0 con 0x55d58349cac0 2024-04-20T10:45:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.367+0000 7fe37a000640 1 --2- 172.21.15.42:0/2440930934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349cac0 0x55d5834a5160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe368009850 tx=0x7fe368005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b89580ebdc32d2bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.368+0000 7fe367e00640 1 -- 172.21.15.42:0/2440930934 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368004240 con 0x55d58349cac0 2024-04-20T10:45:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.368+0000 7fe367e00640 1 -- 172.21.15.42:0/2440930934 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe36802cdf0 con 0x55d58349cac0 2024-04-20T10:45:03.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.369+0000 7fe367e00640 1 -- 172.21.15.42:0/2440930934 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368004240 con 0x55d58349cac0 2024-04-20T10:45:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.369+0000 7fe38442e740 1 -- 172.21.15.42:0/2440930934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349cac0 msgr2=0x55d5834a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.369+0000 7fe38442e740 1 --2- 172.21.15.42:0/2440930934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349cac0 0x55d5834a5160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe368009850 tx=0x7fe368005d50 comp rx=0 tx=0).stop 2024-04-20T10:45:03.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.370+0000 7fe38442e740 1 -- 172.21.15.42:0/2440930934 shutdown_connections 2024-04-20T10:45:03.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.370+0000 7fe38442e740 1 --2- 172.21.15.42:0/2440930934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d5834a56a0 0x55d5834a7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.370+0000 7fe38442e740 1 --2- 172.21.15.42:0/2440930934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349cac0 0x55d5834a5160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.370+0000 7fe38442e740 1 --2- 172.21.15.42:0/2440930934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349c1a0 0x55d58349c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.370+0000 7fe38442e740 1 -- 172.21.15.42:0/2440930934 >> 172.21.15.42:0/2440930934 conn(0x55d5834968d0 msgr2=0x55d583496cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:03.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.371+0000 7fe38442e740 1 -- 172.21.15.42:0/2440930934 shutdown_connections 2024-04-20T10:45:03.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.371+0000 7fe38442e740 1 -- 172.21.15.42:0/2440930934 wait complete. 2024-04-20T10:45:03.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.371+0000 7fe38442e740 1 -- start start 2024-04-20T10:45:03.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.372+0000 7fe38442e740 1 Processor -- start 2024-04-20T10:45:03.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.373+0000 7fe38442e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349c1a0 0x55d5834a2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.373+0000 7fe38442e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349cac0 0x55d5834a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.374+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349c1a0 0x55d5834a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.374+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349c1a0 0x55d5834a2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56590/0 (socket says 172.21.15.42:56590) 2024-04-20T10:45:03.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.374+0000 7fe37a000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349cac0 0x55d5834a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.374+0000 7fe37a000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349cac0 0x55d5834a2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:56008/0 (socket says 172.21.15.42:56008) 2024-04-20T10:45:03.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.374+0000 7fe37aa00640 1 -- 172.21.15.42:0/659838958 learned_addr learned my addr 172.21.15.42:0/659838958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:03.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.374+0000 7fe38442e740 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5834a56a0 0x55d5834a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.375+0000 7fe37b400640 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5834a56a0 0x55d5834a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.375+0000 7fe38442e740 1 -- 172.21.15.42:0/659838958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d5834a8480 con 0x55d58349c1a0 2024-04-20T10:45:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.375+0000 7fe38442e740 1 -- 172.21.15.42:0/659838958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d5833bb740 con 0x55d5834a56a0 2024-04-20T10:45:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.375+0000 7fe38442e740 1 -- 172.21.15.42:0/659838958 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d5834a8600 con 0x55d58349cac0 2024-04-20T10:45:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.375+0000 7fe38442e740 10 client.0 _ll_register_callbacks cb 0x55d5833b6b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:45:03.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.376+0000 7fe37b400640 1 -- 172.21.15.42:0/659838958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349cac0 msgr2=0x55d5834a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:03.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.376+0000 7fe37b400640 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349cac0 0x55d5834a2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.376+0000 7fe37b400640 1 -- 172.21.15.42:0/659838958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349c1a0 msgr2=0x55d5834a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:03.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.376+0000 7fe37b400640 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349c1a0 0x55d5834a2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.376+0000 7fe37b400640 1 -- 172.21.15.42:0/659838958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5831143f0 con 0x55d5834a56a0 2024-04-20T10:45:03.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.376+0000 7fe37aa00640 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349c1a0 0x55d5834a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:03.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.377+0000 7fe37b400640 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5834a56a0 0x55d5834a30d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe3740ce3e0 tx=0x7fe37409d3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31448616ce6d3d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:03.378 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/ksysfs.o 2024-04-20T10:45:03.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.378+0000 7fe364c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:45:03.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.378+0000 7fe364c00640 1 -- 172.21.15.42:0/659838958 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374087040 con 0x55d5834a56a0 2024-04-20T10:45:03.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.378+0000 7fe364c00640 1 -- 172.21.15.42:0/659838958 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fe3740992a0 con 0x55d5834a56a0 2024-04-20T10:45:03.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.378+0000 7fe38442e740 1 -- 172.21.15.42:0/659838958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36802cdf0 con 0x55d5834a56a0 2024-04-20T10:45:03.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.379+0000 7fe364c00640 1 -- 172.21.15.42:0/659838958 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe3740a58a0 con 0x55d5834a56a0 2024-04-20T10:45:03.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.379+0000 7fe38442e740 10 client.26657 fetch_fsmap learned FSMap version 19 2024-04-20T10:45:03.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.379+0000 7fe38442e740 1 -- 172.21.15.42:0/659838958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d5830d40f0 con 0x55d5834a56a0 2024-04-20T10:45:03.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.379+0000 7fe364c00640 1 -- 172.21.15.42:0/659838958 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe36802cdf0 con 0x55d5834a56a0 2024-04-20T10:45:03.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.380+0000 7fe364c00640 1 -- 172.21.15.42:0/659838958 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fe3740a5410 con 0x55d5834a56a0 2024-04-20T10:45:03.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.380+0000 7fe38442e740 10 client.26657 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:45:03.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.380+0000 7fe38442e740 10 client.26657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:45:03.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.380+0000 7fe38442e740 20 client.26657 populate_metadata read hostname 'smithi042' 2024-04-20T10:45:03.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.380+0000 7fe38442e740 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d5834b57d0 0x55d5834b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.380+0000 7fe38442e740 4 client.26657 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:45:03.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.380+0000 7fe38442e740 1 -- 172.21.15.42:0/659838958 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d58340fb70 con 0x55d5834b57d0 2024-04-20T10:45:03.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.381+0000 7fe37aa00640 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d5834b57d0 0x55d5834b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.384+0000 7fe37aa00640 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d5834b57d0 0x55d5834b7bc0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=207fa389b59546e1 server_cookie=533ef160c5b7c507 in_seq=0 out_seq=0 2024-04-20T10:45:03.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.386+0000 7fe364c00640 10 client.26657 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:03.386 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs3xdr.o 2024-04-20T10:45:03.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.388+0000 7fe364c00640 1 -- 172.21.15.42:0/659838958 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d58340fb70 con 0x55d5834b57d0 2024-04-20T10:45:03.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.388+0000 7fe364c00640 10 client.26657 handle_command_reply: tid=0 2024-04-20T10:45:03.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.388+0000 7fe38442e740 1 client.26657 shutdown 2024-04-20T10:45:03.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.389+0000 7fe38442e740 1 -- 172.21.15.42:0/659838958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5834a56a0 msgr2=0x55d5834a30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:03.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.389+0000 7fe38442e740 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5834a56a0 0x55d5834a30d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe3740ce3e0 tx=0x7fe37409d3f0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.390+0000 7fe38442e740 1 -- 172.21.15.42:0/659838958 shutdown_connections 2024-04-20T10:45:03.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.390+0000 7fe38442e740 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d5834b57d0 0x55d5834b7bc0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.390+0000 7fe364c00640 0 client.26657 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:03.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.390+0000 7fe38442e740 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5834a56a0 0x55d5834a30d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.390+0000 7fe38442e740 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349cac0 0x55d5834a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.390+0000 7fe38442e740 1 --2- 172.21.15.42:0/659838958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349c1a0 0x55d5834a2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.391+0000 7fe38442e740 1 -- 172.21.15.42:0/659838958 >> 172.21.15.42:0/659838958 conn(0x55d5834968d0 msgr2=0x55d583498620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:03.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.391+0000 7fe38442e740 1 -- 172.21.15.42:0/659838958 shutdown_connections 2024-04-20T10:45:03.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.392+0000 7fe38442e740 1 -- 172.21.15.42:0/659838958 wait complete. 2024-04-20T10:45:03.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.392+0000 7fe38442e740 20 client.26657 trim_cache size 0 max 16384 2024-04-20T10:45:03.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.477+0000 7fe38442e740 1 Processor -- start 2024-04-20T10:45:03.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.477+0000 7fe38442e740 1 -- start start 2024-04-20T10:45:03.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.479+0000 7fe38442e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349c1a0 0x55d5834b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.479+0000 7fe38442e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349cac0 0x55d583495030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.479+0000 7fe38442e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d5834a56a0 0x55d5834a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.479+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349c1a0 0x55d5834b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.479+0000 7fe37aa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349c1a0 0x55d5834b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:56018/0 (socket says 172.21.15.42:56018) 2024-04-20T10:45:03.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.479+0000 7fe37a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349cac0 0x55d583495030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.479+0000 7fe37a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349cac0 0x55d583495030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:40674/0 (socket says 172.21.15.42:40674) 2024-04-20T10:45:03.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.480+0000 7fe37aa00640 1 -- 172.21.15.42:0/4032867963 learned_addr learned my addr 172.21.15.42:0/4032867963 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:03.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.480+0000 7fe38442e740 1 -- 172.21.15.42:0/4032867963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0684f0 con 0x55d5834a56a0 2024-04-20T10:45:03.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.480+0000 7fe38442e740 1 -- 172.21.15.42:0/4032867963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1ac1f0 con 0x55d58349cac0 2024-04-20T10:45:03.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.480+0000 7fe38442e740 1 -- 172.21.15.42:0/4032867963 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10f410 con 0x55d58349c1a0 2024-04-20T10:45:03.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.480+0000 7fe37aa00640 1 -- 172.21.15.42:0/4032867963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349cac0 msgr2=0x55d583495030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:03.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.480+0000 7fe37aa00640 1 --2- 172.21.15.42:0/4032867963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349cac0 0x55d583495030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.480+0000 7fe37aa00640 1 -- 172.21.15.42:0/4032867963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d5834a56a0 msgr2=0x55d5834a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:03.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.480+0000 7fe37aa00640 1 --2- 172.21.15.42:0/4032867963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d5834a56a0 0x55d5834a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.480+0000 7fe37aa00640 1 -- 172.21.15.42:0/4032867963 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d583495ee0 con 0x55d58349c1a0 2024-04-20T10:45:03.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.480+0000 7fe37a000640 1 --2- 172.21.15.42:0/4032867963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349cac0 0x55d583495030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:03.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.482+0000 7fe37aa00640 1 --2- 172.21.15.42:0/4032867963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349c1a0 0x55d5834b1390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55d5834b83e0 tx=0x7fe36c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7585e7b77e7ef94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:03.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.482+0000 7fe367400640 1 -- 172.21.15.42:0/4032867963 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c004020 con 0x55d58349c1a0 2024-04-20T10:45:03.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.482+0000 7fe367400640 1 -- 172.21.15.42:0/4032867963 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe36c017040 con 0x55d58349c1a0 2024-04-20T10:45:03.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.483+0000 7fe367400640 1 -- 172.21.15.42:0/4032867963 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c004020 con 0x55d58349c1a0 2024-04-20T10:45:03.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.484+0000 7fe38442e740 1 -- 172.21.15.42:0/4032867963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349c1a0 msgr2=0x55d5834b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:03.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.484+0000 7fe38442e740 1 --2- 172.21.15.42:0/4032867963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349c1a0 0x55d5834b1390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55d5834b83e0 tx=0x7fe36c014000 comp rx=0 tx=0).stop 2024-04-20T10:45:03.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.484+0000 7fe38442e740 1 -- 172.21.15.42:0/4032867963 shutdown_connections 2024-04-20T10:45:03.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.484+0000 7fe38442e740 1 --2- 172.21.15.42:0/4032867963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d5834a56a0 0x55d5834a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.484+0000 7fe38442e740 1 --2- 172.21.15.42:0/4032867963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349cac0 0x55d583495030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.484+0000 7fe38442e740 1 --2- 172.21.15.42:0/4032867963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d58349c1a0 0x55d5834b1390 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.484+0000 7fe38442e740 1 -- 172.21.15.42:0/4032867963 >> 172.21.15.42:0/4032867963 conn(0x55d5834968d0 msgr2=0x55d5834a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:03.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.487+0000 7fe38442e740 1 -- 172.21.15.42:0/4032867963 shutdown_connections 2024-04-20T10:45:03.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.487+0000 7fe38442e740 1 -- 172.21.15.42:0/4032867963 wait complete. 2024-04-20T10:45:03.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.488+0000 7fe38442e740 1 -- start start 2024-04-20T10:45:03.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.488+0000 7fe38442e740 1 Processor -- start 2024-04-20T10:45:03.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.492+0000 7fe38442e740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349c1a0 0x55d5834ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.492+0000 7fe38442e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349cac0 0x55d58349eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.492+0000 7fe38442e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d5834a56a0 0x55d58349f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.492+0000 7fe38442e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10fab0 con 0x55d58349cac0 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.492+0000 7fe38442e740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1abb20 con 0x55d58349c1a0 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.492+0000 7fe38442e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c110180 con 0x55d5834a56a0 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.492+0000 7fe38442e740 10 client.0 _ll_register_callbacks cb 0x7fe374005810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.492+0000 7fe37b400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d5834a56a0 0x55d58349f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.493+0000 7fe37b400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d5834a56a0 0x55d58349f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:56022/0 (socket says 172.21.15.42:56022) 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.493+0000 7fe37b400640 1 -- 172.21.15.42:0/613807472 learned_addr learned my addr 172.21.15.42:0/613807472 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.493+0000 7fe37a000640 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349cac0 0x55d58349eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.493+0000 7fe37aa00640 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349c1a0 0x55d5834ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.493+0000 7fe37b400640 1 -- 172.21.15.42:0/613807472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349c1a0 msgr2=0x55d5834ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.493+0000 7fe37b400640 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349c1a0 0x55d5834ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.493+0000 7fe37b400640 1 -- 172.21.15.42:0/613807472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349cac0 msgr2=0x55d58349eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.493+0000 7fe37b400640 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349cac0 0x55d58349eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.493+0000 7fe37b400640 1 -- 172.21.15.42:0/613807472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d58349c8f0 con 0x55d5834a56a0 2024-04-20T10:45:03.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.493+0000 7fe37a000640 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349cac0 0x55d58349eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:45:03.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.494+0000 7fe37aa00640 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349c1a0 0x55d5834ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:45:03.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.494+0000 7fe37b400640 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d5834a56a0 0x55d58349f400 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe37400c760 tx=0x7fe374005490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a712d37bedac4e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:03.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.494+0000 7fe364c00640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:45:03.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.495+0000 7fe364c00640 1 -- 172.21.15.42:0/613807472 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374087040 con 0x55d5834a56a0 2024-04-20T10:45:03.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.495+0000 7fe364c00640 1 -- 172.21.15.42:0/613807472 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fe3740992a0 con 0x55d5834a56a0 2024-04-20T10:45:03.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.495+0000 7fe38442e740 1 -- 172.21.15.42:0/613807472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36c017040 con 0x55d5834a56a0 2024-04-20T10:45:03.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.496+0000 7fe364c00640 1 -- 172.21.15.42:0/613807472 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe3740bd110 con 0x55d5834a56a0 2024-04-20T10:45:03.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.496+0000 7fe38442e740 10 client.26644 fetch_fsmap learned FSMap version 19 2024-04-20T10:45:03.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.496+0000 7fe38442e740 1 -- 172.21.15.42:0/613807472 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d58351b8a0 con 0x55d5834a56a0 2024-04-20T10:45:03.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.496+0000 7fe364c00640 1 -- 172.21.15.42:0/613807472 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe36c017040 con 0x55d5834a56a0 2024-04-20T10:45:03.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.497+0000 7fe364c00640 1 -- 172.21.15.42:0/613807472 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7fe37409baf0 con 0x55d5834a56a0 2024-04-20T10:45:03.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.499+0000 7fe38442e740 10 client.26644 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:45:03.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.499+0000 7fe38442e740 10 client.26644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:45:03.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.499+0000 7fe38442e740 20 client.26644 populate_metadata read hostname 'smithi042' 2024-04-20T10:45:03.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.499+0000 7fe38442e740 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d5834b57d0 0x55d58351dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:03.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.499+0000 7fe38442e740 4 client.26644 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:45:03.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.499+0000 7fe38442e740 1 -- 172.21.15.42:0/613807472 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d5834099e0 con 0x55d5834b57d0 2024-04-20T10:45:03.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.499+0000 7fe37aa00640 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d5834b57d0 0x55d58351dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:03.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.500+0000 7fe37aa00640 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d5834b57d0 0x55d58351dff0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f93aec6b3466ef server_cookie=8734ca5818cfdbab in_seq=0 out_seq=0 2024-04-20T10:45:03.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.500+0000 7fe364c00640 10 client.26644 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:03.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.503+0000 7fe364c00640 1 -- 172.21.15.42:0/613807472 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55d5834099e0 con 0x55d5834b57d0 2024-04-20T10:45:03.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.503+0000 7fe364c00640 10 client.26644 handle_command_reply: tid=0 2024-04-20T10:45:03.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.503+0000 7fe38442e740 1 client.26644 shutdown 2024-04-20T10:45:03.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.503+0000 7fe38442e740 1 -- 172.21.15.42:0/613807472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d5834a56a0 msgr2=0x55d58349f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:03.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.503+0000 7fe38442e740 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d5834a56a0 0x55d58349f400 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe37400c760 tx=0x7fe374005490 comp rx=0 tx=0).stop 2024-04-20T10:45:03.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.503+0000 7fe38442e740 1 -- 172.21.15.42:0/613807472 shutdown_connections 2024-04-20T10:45:03.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.503+0000 7fe38442e740 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d5834b57d0 0x55d58351dff0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.503+0000 7fe38442e740 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d5834a56a0 0x55d58349f400 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.503+0000 7fe38442e740 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d58349cac0 0x55d58349eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.503+0000 7fe38442e740 1 --2- 172.21.15.42:0/613807472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d58349c1a0 0x55d5834ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.503+0000 7fe38442e740 1 -- 172.21.15.42:0/613807472 >> 172.21.15.42:0/613807472 conn(0x55d5834968d0 msgr2=0x55d583497a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:03.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.503+0000 7fe364c00640 0 client.26644 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:03.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.504+0000 7fe38442e740 1 -- 172.21.15.42:0/613807472 shutdown_connections 2024-04-20T10:45:03.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.504+0000 7fe38442e740 1 -- 172.21.15.42:0/613807472 wait complete. 2024-04-20T10:45:03.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.504+0000 7fe38442e740 20 client.26644 trim_cache size 0 max 16384 2024-04-20T10:45:03.505 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:45:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "scrub active (16789 inodes in the stack)", 2024-04-20T10:45:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": { 2024-04-20T10:45:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "0826a9ac-d90c-4443-910b-eb74a0811906": { 2024-04-20T10:45:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "path": "/", 2024-04-20T10:45:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tag": "0826a9ac-d90c-4443-910b-eb74a0811906", 2024-04-20T10:45:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: "options": "recursive,force" 2024-04-20T10:45:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:03.505 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:03.506 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:45:03.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.506+0000 7fe381800640 1 -- 172.21.15.42:0/25015241 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe3600a3da0 msgr2=0x7fe3600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:03.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.506+0000 7fe381800640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe3600a3da0 0x7fe3600a6260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005d00 tx=0x7fe36c009550 comp rx=0 tx=0).stop 2024-04-20T10:45:03.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.507+0000 7fe381800640 1 -- 172.21.15.42:0/25015241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe37c106350 msgr2=0x7fe37c1a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.507+0000 7fe381800640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe37c106350 0x7fe37c1a36e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe37400ea60 tx=0x7fe37400ef20 comp rx=0 tx=0).stop 2024-04-20T10:45:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.507+0000 7fe381800640 1 -- 172.21.15.42:0/25015241 shutdown_connections 2024-04-20T10:45:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.507+0000 7fe381800640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fe3600a3da0 0x7fe3600a6260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.507+0000 7fe381800640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe37c106350 0x7fe37c1a36e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.507+0000 7fe381800640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe37c101770 0x7fe37c19f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.507+0000 7fe381800640 1 --2- 172.21.15.42:0/25015241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe37c100e50 0x7fe37c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.507+0000 7fe381800640 1 -- 172.21.15.42:0/25015241 >> 172.21.15.42:0/25015241 conn(0x7fe37c077fc0 msgr2=0x7fe37c103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:03.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.508+0000 7fe381800640 1 -- 172.21.15.42:0/25015241 shutdown_connections 2024-04-20T10:45:03.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:03.508+0000 7fe381800640 1 -- 172.21.15.42:0/25015241 wait complete. 2024-04-20T10:45:03.550 INFO:tasks.cephfs.filesystem:scrub status for tag:0826a9ac-d90c-4443-910b-eb74a0811906 - {'path': '/', 'tag': '0826a9ac-d90c-4443-910b-eb74a0811906', 'options': 'recursive,force'} 2024-04-20T10:45:03.583 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs3acl.o 2024-04-20T10:45:03.600 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/cred.o 2024-04-20T10:45:03.806 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4proc.o 2024-04-20T10:45:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:03 smithi042 ceph-mon[17078]: pgmap v342: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-04-20T10:45:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:03 smithi042 ceph-mon[17078]: scrub queued for path: / 2024-04-20T10:45:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:03 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:45:03.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:03 smithi042 ceph-mon[17078]: scrub summary: active paths [/] 2024-04-20T10:45:03.955 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/filesystems.o 2024-04-20T10:45:03.988 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/namespace.o 2024-04-20T10:45:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:03 smithi046 ceph-mon[21574]: pgmap v342: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-04-20T10:45:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:03 smithi046 ceph-mon[21574]: scrub queued for path: / 2024-04-20T10:45:04.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:03 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:45:04.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:03 smithi046 ceph-mon[21574]: scrub summary: active paths [/] 2024-04-20T10:45:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:03 smithi162 ceph-mon[21854]: pgmap v342: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-04-20T10:45:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:03 smithi162 ceph-mon[21854]: scrub queued for path: / 2024-04-20T10:45:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:03 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:45:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:03 smithi162 ceph-mon[21854]: scrub summary: active paths [/] 2024-04-20T10:45:04.315 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4xdr.o 2024-04-20T10:45:05.042 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/reboot.o 2024-04-20T10:45:05.121 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/seq_file.o 2024-04-20T10:45:05.132 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/async.o 2024-04-20T10:45:05.276 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4state.o 2024-04-20T10:45:05.864 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/xattr.o 2024-04-20T10:45:05.876 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/libfs.o 2024-04-20T10:45:06.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:05 smithi042 ceph-mon[17078]: pgmap v343: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-20T10:45:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:05 smithi162 ceph-mon[21854]: pgmap v343: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-20T10:45:06.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:05 smithi046 ceph-mon[21574]: pgmap v343: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-20T10:45:06.331 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fs-writeback.o 2024-04-20T10:45:06.498 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/range.o 2024-04-20T10:45:06.686 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/smpboot.o 2024-04-20T10:45:07.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:06 smithi042 ceph-mon[17078]: osdmap e97: 12 total, 12 up, 12 in 2024-04-20T10:45:07.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:06 smithi162 ceph-mon[21854]: osdmap e97: 12 total, 12 up, 12 in 2024-04-20T10:45:07.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:06 smithi046 ceph-mon[21574]: osdmap e97: 12 total, 12 up, 12 in 2024-04-20T10:45:07.358 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/pnode.o 2024-04-20T10:45:07.491 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/splice.o 2024-04-20T10:45:07.610 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/ucount.o 2024-04-20T10:45:07.652 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4renewd.o 2024-04-20T10:45:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:07 smithi042 ceph-mon[17078]: pgmap v345: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.0 MiB/s wr, 357 op/s 2024-04-20T10:45:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:07 smithi162 ceph-mon[21854]: pgmap v345: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.0 MiB/s wr, 357 op/s 2024-04-20T10:45:08.215 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4super.o 2024-04-20T10:45:08.239 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4file.o 2024-04-20T10:45:08.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:07 smithi046 ceph-mon[21574]: pgmap v345: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.0 MiB/s wr, 357 op/s 2024-04-20T10:45:08.470 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/sync.o 2024-04-20T10:45:08.575 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/regset.o 2024-04-20T10:45:08.966 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/delegation.o 2024-04-20T10:45:09.108 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/utimes.o 2024-04-20T10:45:09.376 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/ksyms_common.o 2024-04-20T10:45:09.396 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4idmap.o 2024-04-20T10:45:09.655 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/d_path.o 2024-04-20T10:45:09.692 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/groups.o 2024-04-20T10:45:09.941 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/stack.o 2024-04-20T10:45:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:09 smithi042 ceph-mon[17078]: pgmap v346: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 358 KiB/s wr, 786 op/s 2024-04-20T10:45:10.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:09 smithi042 ceph-mon[17078]: osdmap e98: 12 total, 12 up, 12 in 2024-04-20T10:45:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:09 smithi162 ceph-mon[21854]: pgmap v346: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 358 KiB/s wr, 786 op/s 2024-04-20T10:45:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:09 smithi162 ceph-mon[21854]: osdmap e98: 12 total, 12 up, 12 in 2024-04-20T10:45:10.300 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/kcmp.o 2024-04-20T10:45:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:09 smithi046 ceph-mon[21574]: pgmap v346: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 358 KiB/s wr, 786 op/s 2024-04-20T10:45:10.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:09 smithi046 ceph-mon[21574]: osdmap e98: 12 total, 12 up, 12 in 2024-04-20T10:45:10.308 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/freezer.o 2024-04-20T10:45:10.393 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fs_struct.o 2024-04-20T10:45:10.802 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/profile.o 2024-04-20T10:45:10.813 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/callback.o 2024-04-20T10:45:10.871 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/statfs.o 2024-04-20T10:45:11.322 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fs_pin.o 2024-04-20T10:45:11.482 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nsfs.o 2024-04-20T10:45:11.488 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/callback_xdr.o 2024-04-20T10:45:11.541 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/callback_proc.o 2024-04-20T10:45:11.573 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fs_types.o 2024-04-20T10:45:12.018 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/stacktrace.o 2024-04-20T10:45:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:11 smithi042 ceph-mon[17078]: pgmap v348: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 172 KiB/s wr, 947 op/s 2024-04-20T10:45:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:11 smithi162 ceph-mon[21854]: pgmap v348: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 172 KiB/s wr, 947 op/s 2024-04-20T10:45:12.161 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/dma.o 2024-04-20T10:45:12.167 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fs_context.o 2024-04-20T10:45:12.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:11 smithi046 ceph-mon[21574]: pgmap v348: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 172 KiB/s wr, 947 op/s 2024-04-20T10:45:12.336 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fs_parser.o 2024-04-20T10:45:12.359 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fsopen.o 2024-04-20T10:45:12.530 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/smp.o 2024-04-20T10:45:13.042 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4namespace.o 2024-04-20T10:45:13.094 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4getroot.o 2024-04-20T10:45:13.334 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/init.o 2024-04-20T10:45:13.343 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/uid16.o 2024-04-20T10:45:13.419 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4client.o 2024-04-20T10:45:13.580 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/kernel_read_file.o 2024-04-20T10:45:13.615 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/mnt_idmapping.o 2024-04-20T10:45:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:13 smithi162 ceph-mon[21854]: pgmap v349: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 343 KiB/s wr, 3.28k op/s 2024-04-20T10:45:14.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:13 smithi046 ceph-mon[21574]: pgmap v349: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 343 KiB/s wr, 3.28k op/s 2024-04-20T10:45:14.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:13 smithi042 ceph-mon[17078]: pgmap v349: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 343 KiB/s wr, 3.28k op/s 2024-04-20T10:45:14.465 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/remap_range.o 2024-04-20T10:45:14.489 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4session.o 2024-04-20T10:45:14.553 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/dns_resolve.o 2024-04-20T10:45:14.633 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/kallsyms.o 2024-04-20T10:45:14.826 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4trace.o 2024-04-20T10:45:14.830 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/buffer.o 2024-04-20T10:45:14.950 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/nfs/nfs4sysctl.o 2024-04-20T10:45:15.042 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/mpage.o 2024-04-20T10:45:15.944 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/proc_namespace.o 2024-04-20T10:45:15.961 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/acct.o 2024-04-20T10:45:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:15 smithi162 ceph-mon[21854]: pgmap v350: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 315 KiB/s wr, 3.28k op/s 2024-04-20T10:45:16.268 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/direct-io.o 2024-04-20T10:45:16.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:15 smithi046 ceph-mon[21574]: pgmap v350: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 315 KiB/s wr, 3.28k op/s 2024-04-20T10:45:16.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:15 smithi042 ceph-mon[17078]: pgmap v350: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 315 KiB/s wr, 3.28k op/s 2024-04-20T10:45:16.435 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/crash_core.o 2024-04-20T10:45:16.506 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/kexec_core.o 2024-04-20T10:45:16.573 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/kexec.o 2024-04-20T10:45:17.041 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/eventpoll.o 2024-04-20T10:45:17.473 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/compat.o 2024-04-20T10:45:17.729 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/utsname.o 2024-04-20T10:45:17.916 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/pid_namespace.o 2024-04-20T10:45:18.149 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/anon_inodes.o 2024-04-20T10:45:18.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:17 smithi162 ceph-mon[21854]: pgmap v351: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 514 KiB/s wr, 3.45k op/s 2024-04-20T10:45:18.162 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/stop_machine.o 2024-04-20T10:45:18.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:17 smithi046 ceph-mon[21574]: pgmap v351: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 514 KiB/s wr, 3.45k op/s 2024-04-20T10:45:18.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:17 smithi042 ceph-mon[17078]: pgmap v351: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 514 KiB/s wr, 3.45k op/s 2024-04-20T10:45:18.450 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/audit.o 2024-04-20T10:45:18.457 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/signalfd.o 2024-04-20T10:45:18.527 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/nfs/built-in.a 2024-04-20T10:45:18.653 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/timerfd.o 2024-04-20T10:45:18.659 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/auditfilter.o 2024-04-20T10:45:19.092 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/eventfd.o 2024-04-20T10:45:19.114 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/auditsc.o 2024-04-20T10:45:19.328 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/aio.o 2024-04-20T10:45:19.612 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/audit_watch.o 2024-04-20T10:45:19.623 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/locks.o 2024-04-20T10:45:19.913 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/binfmt_misc.o 2024-04-20T10:45:20.142 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/audit_fsnotify.o 2024-04-20T10:45:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:19 smithi162 ceph-mon[21854]: pgmap v352: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.6 MiB/s wr, 4.59k op/s 2024-04-20T10:45:20.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:19 smithi046 ceph-mon[21574]: pgmap v352: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.6 MiB/s wr, 4.59k op/s 2024-04-20T10:45:20.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:19 smithi042 ceph-mon[17078]: pgmap v352: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.6 MiB/s wr, 4.59k op/s 2024-04-20T10:45:20.472 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/binfmt_script.o 2024-04-20T10:45:20.480 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/binfmt_elf.o 2024-04-20T10:45:20.642 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/compat_binfmt_elf.o 2024-04-20T10:45:20.933 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/audit_tree.o 2024-04-20T10:45:21.046 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/mbcache.o 2024-04-20T10:45:21.068 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/posix_acl.o 2024-04-20T10:45:21.156 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/kprobes.o 2024-04-20T10:45:21.170 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/seccomp.o 2024-04-20T10:45:21.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:20 smithi046 ceph-mon[21574]: scrub summary: idle+waiting paths [/] 2024-04-20T10:45:21.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:20 smithi046 ceph-mon[21574]: scrub summary: idle 2024-04-20T10:45:21.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:20 smithi042 ceph-mon[17078]: scrub summary: idle+waiting paths [/] 2024-04-20T10:45:21.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:20 smithi042 ceph-mon[17078]: scrub summary: idle 2024-04-20T10:45:21.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:20 smithi162 ceph-mon[21854]: scrub summary: idle+waiting paths [/] 2024-04-20T10:45:21.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:20 smithi162 ceph-mon[21854]: scrub summary: idle 2024-04-20T10:45:21.597 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/relay.o 2024-04-20T10:45:21.860 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/utsname_sysctl.o 2024-04-20T10:45:21.891 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/delayacct.o 2024-04-20T10:45:22.139 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/coredump.o 2024-04-20T10:45:22.168 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/drop_caches.o 2024-04-20T10:45:22.203 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/taskstats.o 2024-04-20T10:45:22.241 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/sysctls.o 2024-04-20T10:45:22.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:21 smithi046 ceph-mon[21574]: pgmap v353: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 3.96k op/s 2024-04-20T10:45:22.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:21 smithi042 ceph-mon[17078]: pgmap v353: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 3.96k op/s 2024-04-20T10:45:22.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:21 smithi162 ceph-mon[21854]: pgmap v353: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 3.96k op/s 2024-04-20T10:45:22.533 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/tsacct.o 2024-04-20T10:45:22.576 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/tracepoint.o 2024-04-20T10:45:22.686 INFO:tasks.workunit.client.0.smithi042.stdout: CC fs/fhandle.o 2024-04-20T10:45:22.763 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/irq_work.o 2024-04-20T10:45:22.913 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/static_call.o 2024-04-20T10:45:22.930 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/static_call_inline.o 2024-04-20T10:45:23.241 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/crash_dump.o 2024-04-20T10:45:23.322 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/jump_label.o 2024-04-20T10:45:23.360 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/context_tracking.o 2024-04-20T10:45:23.428 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/iomem.o 2024-04-20T10:45:23.489 INFO:tasks.workunit.client.0.smithi042.stdout: CC kernel/rseq.o 2024-04-20T10:45:23.607 INFO:tasks.workunit.client.0.smithi042.stdout: AR fs/built-in.a 2024-04-20T10:45:24.222 INFO:tasks.workunit.client.0.smithi042.stdout: AR kernel/built-in.a 2024-04-20T10:45:24.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:23 smithi046 ceph-mon[21574]: pgmap v354: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.4 MiB/s wr, 4.67k op/s 2024-04-20T10:45:24.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:23 smithi042 ceph-mon[17078]: pgmap v354: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.4 MiB/s wr, 4.67k op/s 2024-04-20T10:45:24.361 INFO:tasks.workunit.client.0.smithi042.stderr:make[1]: *** [/home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/Makefile:2034: .] Error 2 2024-04-20T10:45:24.362 INFO:tasks.workunit.client.0.smithi042.stderr:make: *** [Makefile:234: __sub-make] Error 2 2024-04-20T10:45:24.364 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T10:45:24.364 INFO:tasks.workunit:Stopping ['kernel_untar_build.sh'] on client.0... 2024-04-20T10:45:24.364 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-20T10:45:24.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:23 smithi162 ceph-mon[21854]: pgmap v354: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.4 MiB/s wr, 4.67k op/s 2024-04-20T10:45:25.484 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test kernel_untar_build.sh) on smithi042 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2024-04-20T10:45:25.753 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9e1e488ef85640df8d29cacccc3dcc74 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test kernel_untar_build.sh) on smithi042 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2024-04-20T10:45:25.756 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-20T10:45:25.766 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-20T10:45:25.767 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T10:45:25.767 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-04-20T10:45:25.767 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-20T10:45:25.874 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-20T10:45:25.972 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-20T10:45:26.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.179+0000 7f254fe00640 1 -- 172.21.15.42:0/1026348252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2550104b90 msgr2=0x7f2550108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:26.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.179+0000 7f254fe00640 1 --2- 172.21.15.42:0/1026348252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2550104b90 0x7f2550108bd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f253c009900 tx=0x7f253c02bc40 comp rx=0 tx=0).stop 2024-04-20T10:45:26.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.179+0000 7f254fe00640 1 -- 172.21.15.42:0/1026348252 shutdown_connections 2024-04-20T10:45:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.179+0000 7f254fe00640 1 --2- 172.21.15.42:0/1026348252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25501092b0 0x7f2550110e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.179+0000 7f254fe00640 1 --2- 172.21.15.42:0/1026348252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2550104b90 0x7f2550108bd0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.179+0000 7f254fe00640 1 --2- 172.21.15.42:0/1026348252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25501041e0 0x7f25501045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.179+0000 7f254fe00640 1 -- 172.21.15.42:0/1026348252 >> 172.21.15.42:0/1026348252 conn(0x7f25500ffa80 msgr2=0x7f2550101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.179+0000 7f254fe00640 1 -- 172.21.15.42:0/1026348252 shutdown_connections 2024-04-20T10:45:26.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.179+0000 7f254fe00640 1 -- 172.21.15.42:0/1026348252 wait complete. 2024-04-20T10:45:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.180+0000 7f254fe00640 1 Processor -- start 2024-04-20T10:45:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.180+0000 7f254fe00640 1 -- start start 2024-04-20T10:45:26.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.181+0000 7f254fe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25501041e0 0x7f25501a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:26.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.181+0000 7f254fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25501092b0 0x7f25501a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:26.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.181+0000 7f254fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255019a740 0x7f255019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:26.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.181+0000 7f254fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2550114320 con 0x7f25501092b0 2024-04-20T10:45:26.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.181+0000 7f254fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25501141a0 con 0x7f255019a740 2024-04-20T10:45:26.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.181+0000 7f254fe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f25501144a0 con 0x7f25501041e0 2024-04-20T10:45:26.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.181+0000 7f254e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25501092b0 0x7f25501a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:26.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.181+0000 7f254ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25501041e0 0x7f25501a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:26.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.181+0000 7f254ea00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25501041e0 0x7f25501a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:41328/0 (socket says 172.21.15.42:41328) 2024-04-20T10:45:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.181+0000 7f254f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255019a740 0x7f255019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.181+0000 7f254e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25501092b0 0x7f25501a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49772/0 (socket says 172.21.15.42:49772) 2024-04-20T10:45:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.181+0000 7f254ea00640 1 -- 172.21.15.42:0/4039768585 learned_addr learned my addr 172.21.15.42:0/4039768585 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f254ea00640 1 -- 172.21.15.42:0/4039768585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255019a740 msgr2=0x7f255019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f254ea00640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255019a740 0x7f255019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f254ea00640 1 -- 172.21.15.42:0/4039768585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25501092b0 msgr2=0x7f25501a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f254ea00640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25501092b0 0x7f25501a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f254ea00640 1 -- 172.21.15.42:0/4039768585 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255019b340 con 0x7f25501041e0 2024-04-20T10:45:26.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f254f400640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255019a740 0x7f255019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:26.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f254e000640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25501092b0 0x7f25501a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:26.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f254ea00640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25501041e0 0x7f25501a0680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f254000e9f0 tx=0x7f254000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:26.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f2547400640 1 -- 172.21.15.42:0/4039768585 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254000cd80 con 0x7f25501041e0 2024-04-20T10:45:26.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f2547400640 1 -- 172.21.15.42:0/4039768585 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2540010910 con 0x7f25501041e0 2024-04-20T10:45:26.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f254fe00640 1 -- 172.21.15.42:0/4039768585 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25501a7420 con 0x7f25501041e0 2024-04-20T10:45:26.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f2547400640 1 -- 172.21.15.42:0/4039768585 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540018990 con 0x7f25501041e0 2024-04-20T10:45:26.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.182+0000 7f254fe00640 1 -- 172.21.15.42:0/4039768585 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25501a77d0 con 0x7f25501041e0 2024-04-20T10:45:26.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.184+0000 7f254fe00640 1 -- 172.21.15.42:0/4039768585 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25501056b0 con 0x7f25501041e0 2024-04-20T10:45:26.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.186+0000 7f2547400640 1 -- 172.21.15.42:0/4039768585 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2540010ac0 con 0x7f25501041e0 2024-04-20T10:45:26.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.187+0000 7f2547400640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f25340a3d10 0x7f25340a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:26.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.187+0000 7f2547400640 1 -- 172.21.15.42:0/4039768585 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f25400d0d30 con 0x7f25501041e0 2024-04-20T10:45:26.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.187+0000 7f254e000640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f25340a3d10 0x7f25340a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:26.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.188+0000 7f254e000640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f25340a3d10 0x7f25340a61d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f253c009230 tx=0x7f253c005fd0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:26.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.189+0000 7f2547400640 1 -- 172.21.15.42:0/4039768585 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f254009ccf0 con 0x7f25501041e0 2024-04-20T10:45:26.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:25 smithi046 ceph-mon[21574]: pgmap v355: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.3 MiB/s wr, 3.18k op/s 2024-04-20T10:45:26.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:25 smithi042 ceph-mon[17078]: pgmap v355: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.3 MiB/s wr, 3.18k op/s 2024-04-20T10:45:26.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:25 smithi162 ceph-mon[21854]: pgmap v355: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.3 MiB/s wr, 3.18k op/s 2024-04-20T10:45:26.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.424+0000 7f254fe00640 1 -- 172.21.15.42:0/4039768585 --> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f255010c550 con 0x7f25340a3d10 2024-04-20T10:45:26.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.438+0000 7f2547400640 1 -- 172.21.15.42:0/4039768585 <== mgr.15024 v2:172.21.15.42:6800/170978916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f253c004a70 con 0x7f25340a3d10 2024-04-20T10:45:26.438 INFO:teuthology.orchestra.run.smithi042.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-20T00:00:00", "created": "2024-04-20T10:34:08", "first": "2024-04-20T10:35:00", "last": "2024-04-20T10:45:00", "last_pruned": "2024-04-20T10:45:00", "created_count": 11, "pruned_count": 5, "active": true} 2024-04-20T10:45:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.441+0000 7f254fe00640 1 -- 172.21.15.42:0/4039768585 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f25340a3d10 msgr2=0x7f25340a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.441+0000 7f254fe00640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f25340a3d10 0x7f25340a61d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f253c009230 tx=0x7f253c005fd0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.441+0000 7f254fe00640 1 -- 172.21.15.42:0/4039768585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25501041e0 msgr2=0x7f25501a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:26.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.441+0000 7f254fe00640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25501041e0 0x7f25501a0680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f254000e9f0 tx=0x7f254000eeb0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.441+0000 7f254fe00640 1 -- 172.21.15.42:0/4039768585 shutdown_connections 2024-04-20T10:45:26.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.441+0000 7f254fe00640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f25340a3d10 0x7f25340a61d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.441+0000 7f254fe00640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f255019a740 0x7f255019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.441+0000 7f254fe00640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25501092b0 0x7f25501a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.441+0000 7f254fe00640 1 --2- 172.21.15.42:0/4039768585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25501041e0 0x7f25501a0680 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.441+0000 7f254fe00640 1 -- 172.21.15.42:0/4039768585 >> 172.21.15.42:0/4039768585 conn(0x7f25500ffa80 msgr2=0x7f2550100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:26.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.441+0000 7f254fe00640 1 -- 172.21.15.42:0/4039768585 shutdown_connections 2024-04-20T10:45:26.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.441+0000 7f254fe00640 1 -- 172.21.15.42:0/4039768585 wait complete. 2024-04-20T10:45:26.459 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-20T10:45:26.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.653+0000 7f38eee00640 1 -- 172.21.15.42:0/3731000156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e8104bc0 msgr2=0x7f38e8108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:26.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.653+0000 7f38eee00640 1 --2- 172.21.15.42:0/3731000156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e8104bc0 0x7f38e8108c00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009900 tx=0x7f38dc02bc30 comp rx=0 tx=0).stop 2024-04-20T10:45:26.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.654+0000 7f38eee00640 1 -- 172.21.15.42:0/3731000156 shutdown_connections 2024-04-20T10:45:26.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.654+0000 7f38eee00640 1 --2- 172.21.15.42:0/3731000156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e81092e0 0x7f38e8110eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.654+0000 7f38eee00640 1 --2- 172.21.15.42:0/3731000156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e8104bc0 0x7f38e8108c00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.654+0000 7f38eee00640 1 --2- 172.21.15.42:0/3731000156 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e8104210 0x7f38e81045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.654+0000 7f38eee00640 1 -- 172.21.15.42:0/3731000156 >> 172.21.15.42:0/3731000156 conn(0x7f38e80ffa70 msgr2=0x7f38e8101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:26.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.654+0000 7f38eee00640 1 -- 172.21.15.42:0/3731000156 shutdown_connections 2024-04-20T10:45:26.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.654+0000 7f38eee00640 1 -- 172.21.15.42:0/3731000156 wait complete. 2024-04-20T10:45:26.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.655+0000 7f38eee00640 1 Processor -- start 2024-04-20T10:45:26.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.655+0000 7f38eee00640 1 -- start start 2024-04-20T10:45:26.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.655+0000 7f38eee00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e8104210 0x7f38e81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:26.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.655+0000 7f38eee00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e81092e0 0x7f38e81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:26.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.655+0000 7f38eee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e819a750 0x7f38e819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:26.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.655+0000 7f38eee00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f38e8114370 con 0x7f38e8104210 2024-04-20T10:45:26.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.655+0000 7f38eee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38e81141f0 con 0x7f38e819a750 2024-04-20T10:45:26.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.655+0000 7f38eee00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f38e81144f0 con 0x7f38e81092e0 2024-04-20T10:45:26.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.656+0000 7f38ed000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e81092e0 0x7f38e81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:26.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.656+0000 7f38ee400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e819a750 0x7f38e819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:26.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.656+0000 7f38ed000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e81092e0 0x7f38e81a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:41358/0 (socket says 172.21.15.42:41358) 2024-04-20T10:45:26.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.656+0000 7f38ed000640 1 -- 172.21.15.42:0/1540354090 learned_addr learned my addr 172.21.15.42:0/1540354090 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:26.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.656+0000 7f38eda00640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e8104210 0x7f38e81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:26.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.656+0000 7f38ee400640 1 -- 172.21.15.42:0/1540354090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e81092e0 msgr2=0x7f38e81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:26.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.656+0000 7f38ee400640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e81092e0 0x7f38e81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.656+0000 7f38ee400640 1 -- 172.21.15.42:0/1540354090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e8104210 msgr2=0x7f38e81a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:26.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.656+0000 7f38ee400640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e8104210 0x7f38e81a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.656+0000 7f38ee400640 1 -- 172.21.15.42:0/1540354090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e819b350 con 0x7f38e819a750 2024-04-20T10:45:26.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.656+0000 7f38ed000640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e81092e0 0x7f38e81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:26.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.657+0000 7f38ee400640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e819a750 0x7f38e819abd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009fd0 tx=0x7f38e000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:26.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.657+0000 7f38e6000640 1 -- 172.21.15.42:0/1540354090 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e000ce20 con 0x7f38e819a750 2024-04-20T10:45:26.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.657+0000 7f38eee00640 1 -- 172.21.15.42:0/1540354090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e81a73e0 con 0x7f38e819a750 2024-04-20T10:45:26.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.657+0000 7f38e6000640 1 -- 172.21.15.42:0/1540354090 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e0005020 con 0x7f38e819a750 2024-04-20T10:45:26.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.657+0000 7f38e6000640 1 -- 172.21.15.42:0/1540354090 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e0010720 con 0x7f38e819a750 2024-04-20T10:45:26.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.657+0000 7f38eee00640 1 -- 172.21.15.42:0/1540354090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e81a7790 con 0x7f38e819a750 2024-04-20T10:45:26.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.658+0000 7f38eee00640 1 -- 172.21.15.42:0/1540354090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e81056e0 con 0x7f38e819a750 2024-04-20T10:45:26.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.660+0000 7f38e6000640 1 -- 172.21.15.42:0/1540354090 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f38e0005540 con 0x7f38e819a750 2024-04-20T10:45:26.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.662+0000 7f38e6000640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f38d00a3d10 0x7f38d00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:26.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.662+0000 7f38e6000640 1 -- 172.21.15.42:0/1540354090 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f38e00d0e70 con 0x7f38e819a750 2024-04-20T10:45:26.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.662+0000 7f38eda00640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f38d00a3d10 0x7f38d00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.663+0000 7f38eda00640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f38d00a3d10 0x7f38d00a61d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009bd0 tx=0x7f38d4005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.664+0000 7f38e6000640 1 -- 172.21.15.42:0/1540354090 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f38e009d480 con 0x7f38e819a750 2024-04-20T10:45:26.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.903+0000 7f38eee00640 1 -- 172.21.15.42:0/1540354090 --> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f38e8104bc0 con 0x7f38d00a3d10 2024-04-20T10:45:26.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.917+0000 7f38e6000640 1 -- 172.21.15.42:0/1540354090 <== mgr.15024 v2:172.21.15.42:6800/170978916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f38e8104bc0 con 0x7f38d00a3d10 2024-04-20T10:45:26.917 INFO:teuthology.orchestra.run.smithi042.stdout:/ 1m 6m3h 2024-04-20T10:45:26.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.919+0000 7f38eee00640 1 -- 172.21.15.42:0/1540354090 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f38d00a3d10 msgr2=0x7f38d00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:26.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.920+0000 7f38eee00640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f38d00a3d10 0x7f38d00a61d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009bd0 tx=0x7f38d4005f90 comp rx=0 tx=0).stop 2024-04-20T10:45:26.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.920+0000 7f38eee00640 1 -- 172.21.15.42:0/1540354090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e819a750 msgr2=0x7f38e819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:26.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.920+0000 7f38eee00640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e819a750 0x7f38e819abd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009fd0 tx=0x7f38e000ef10 comp rx=0 tx=0).stop 2024-04-20T10:45:26.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.920+0000 7f38eee00640 1 -- 172.21.15.42:0/1540354090 shutdown_connections 2024-04-20T10:45:26.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.920+0000 7f38eee00640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f38d00a3d10 0x7f38d00a61d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.920+0000 7f38eee00640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e819a750 0x7f38e819abd0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.920+0000 7f38eee00640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e81092e0 0x7f38e81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.920+0000 7f38eee00640 1 --2- 172.21.15.42:0/1540354090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e8104210 0x7f38e81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.920+0000 7f38eee00640 1 -- 172.21.15.42:0/1540354090 >> 172.21.15.42:0/1540354090 conn(0x7f38e80ffa70 msgr2=0x7f38e8100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.920+0000 7f38eee00640 1 -- 172.21.15.42:0/1540354090 shutdown_connections 2024-04-20T10:45:26.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:26.920+0000 7f38eee00640 1 -- 172.21.15.42:0/1540354090 wait complete. 2024-04-20T10:45:26.941 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-20T10:45:26.952 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-20T10:45:28.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:27 smithi046 ceph-mon[21574]: from='client.26650 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:45:28.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:27 smithi046 ceph-mon[21574]: pgmap v356: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 2.95k op/s 2024-04-20T10:45:28.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:27 smithi046 ceph-mon[21574]: from='client.26672 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:45:28.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:27 smithi042 ceph-mon[17078]: from='client.26650 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:45:28.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:27 smithi042 ceph-mon[17078]: pgmap v356: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 2.95k op/s 2024-04-20T10:45:28.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:27 smithi042 ceph-mon[17078]: from='client.26672 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:45:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:27 smithi162 ceph-mon[21854]: from='client.26650 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:45:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:27 smithi162 ceph-mon[21854]: pgmap v356: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 2.95k op/s 2024-04-20T10:45:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:27 smithi162 ceph-mon[21854]: from='client.26672 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T10:45:30.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:29 smithi046 ceph-mon[21574]: pgmap v357: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 2.28k op/s 2024-04-20T10:45:30.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:29 smithi042 ceph-mon[17078]: pgmap v357: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 2.28k op/s 2024-04-20T10:45:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:29 smithi162 ceph-mon[21854]: pgmap v357: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 2.28k op/s 2024-04-20T10:45:32.306 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:31 smithi046 ceph-mon[21574]: pgmap v358: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 264 KiB/s wr, 914 op/s 2024-04-20T10:45:32.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:31 smithi042 ceph-mon[17078]: pgmap v358: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 264 KiB/s wr, 914 op/s 2024-04-20T10:45:32.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:31 smithi162 ceph-mon[21854]: pgmap v358: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 264 KiB/s wr, 914 op/s 2024-04-20T10:45:33.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T10:45:33.552 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T10:45:33.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.777+0000 7f1efd800640 1 -- 172.21.15.42:0/4070664146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef8108500 msgr2=0x7f1ef8108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.777+0000 7f1efd800640 1 --2- 172.21.15.42:0/4070664146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef8108500 0x7f1ef8108960 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009900 tx=0x7f1ee002bcd0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.778+0000 7f1efd800640 1 -- 172.21.15.42:0/4070664146 shutdown_connections 2024-04-20T10:45:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.778+0000 7f1efd800640 1 --2- 172.21.15.42:0/4070664146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ef8108ea0 0x7f1ef81152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.778+0000 7f1efd800640 1 --2- 172.21.15.42:0/4070664146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef8108500 0x7f1ef8108960 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.778+0000 7f1efd800640 1 --2- 172.21.15.42:0/4070664146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef810d3d0 0x7f1ef810d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.778+0000 7f1efd800640 1 -- 172.21.15.42:0/4070664146 >> 172.21.15.42:0/4070664146 conn(0x7f1ef8077d80 msgr2=0x7f1ef8106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.778+0000 7f1efd800640 1 -- 172.21.15.42:0/4070664146 shutdown_connections 2024-04-20T10:45:33.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.778+0000 7f1efd800640 1 -- 172.21.15.42:0/4070664146 wait complete. 2024-04-20T10:45:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.779+0000 7f1efd800640 1 Processor -- start 2024-04-20T10:45:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.779+0000 7f1efd800640 1 -- start start 2024-04-20T10:45:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.779+0000 7f1efd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ef8108500 0x7f1ef81a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.779+0000 7f1efd800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef8108ea0 0x7f1ef81a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.779+0000 7f1efd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef810d3d0 0x7f1ef819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.779+0000 7f1efd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8118840 con 0x7f1ef8108500 2024-04-20T10:45:33.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.779+0000 7f1efd800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81186c0 con 0x7f1ef810d3d0 2024-04-20T10:45:33.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.779+0000 7f1efd800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81189c0 con 0x7f1ef8108ea0 2024-04-20T10:45:33.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.779+0000 7f1efce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef810d3d0 0x7f1ef819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.779+0000 7f1ef7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ef8108500 0x7f1ef81a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.779+0000 7f1efce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef810d3d0 0x7f1ef819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59624/0 (socket says 172.21.15.42:59624) 2024-04-20T10:45:33.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.779+0000 7f1efce00640 1 -- 172.21.15.42:0/160823198 learned_addr learned my addr 172.21.15.42:0/160823198 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:33.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.780+0000 7f1eefe00640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef8108ea0 0x7f1ef81a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.780+0000 7f1efce00640 1 -- 172.21.15.42:0/160823198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef8108ea0 msgr2=0x7f1ef81a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:33.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.780+0000 7f1efce00640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef8108ea0 0x7f1ef81a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.780+0000 7f1efce00640 1 -- 172.21.15.42:0/160823198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ef8108500 msgr2=0x7f1ef81a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.780+0000 7f1efce00640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ef8108500 0x7f1ef81a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.780+0000 7f1efce00640 1 -- 172.21.15.42:0/160823198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef819f3e0 con 0x7f1ef810d3d0 2024-04-20T10:45:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.780+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ef8108500 0x7f1ef81a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.780+0000 7f1efce00640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef810d3d0 0x7f1ef819eb20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000ea60 tx=0x7f1ef000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.780+0000 7f1ef5600640 1 -- 172.21.15.42:0/160823198 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000ce40 con 0x7f1ef810d3d0 2024-04-20T10:45:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.780+0000 7f1ef5600640 1 -- 172.21.15.42:0/160823198 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef0005020 con 0x7f1ef810d3d0 2024-04-20T10:45:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.780+0000 7f1efd800640 1 -- 172.21.15.42:0/160823198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef819f6d0 con 0x7f1ef810d3d0 2024-04-20T10:45:33.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.781+0000 7f1ef5600640 1 -- 172.21.15.42:0/160823198 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0010690 con 0x7f1ef810d3d0 2024-04-20T10:45:33.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.781+0000 7f1efd800640 1 -- 172.21.15.42:0/160823198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef81ab9e0 con 0x7f1ef810d3d0 2024-04-20T10:45:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.784+0000 7f1f01bd8740 1 Processor -- start 2024-04-20T10:45:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.784+0000 7f1ef5600640 1 -- 172.21.15.42:0/160823198 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1ef0005520 con 0x7f1ef810d3d0 2024-04-20T10:45:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.784+0000 7f1f01bd8740 1 -- start start 2024-04-20T10:45:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1f01bd8740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf9375760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1f01bd8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 0x55bcf937e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1f01bd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf937e880 0x55bcf9380c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1f01bd8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bcf92db150 con 0x55bcf9375ca0 2024-04-20T10:45:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1f01bd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bcf9281370 con 0x55bcf937e880 2024-04-20T10:45:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1f01bd8740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55bcf92d7110 con 0x55bcf9375380 2024-04-20T10:45:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1ef7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 0x55bcf937e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf9375760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1ef7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 0x55bcf937e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43382/0 (socket says 172.21.15.42:43382) 2024-04-20T10:45:33.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1ef7e00640 1 -- 172.21.15.42:0/792537137 learned_addr learned my addr 172.21.15.42:0/792537137 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:33.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1eefe00640 1 --2- 172.21.15.42:0/792537137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf937e880 0x55bcf9380c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1ef7e00640 1 -- 172.21.15.42:0/792537137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 msgr2=0x55bcf9375760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:33.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/792537137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf9375760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1ef7e00640 1 -- 172.21.15.42:0/792537137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf937e880 msgr2=0x55bcf9380c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:33.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/792537137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf937e880 0x55bcf9380c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.785+0000 7f1ef7e00640 1 -- 172.21.15.42:0/792537137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcf90b77d0 con 0x55bcf9375ca0 2024-04-20T10:45:33.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.786+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/792537137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 0x55bcf937e340 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80114f0 tx=0x7f1ee80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8ac1266cf347fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:33.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.786+0000 7f1ef5600640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f1edc0a3da0 0x7f1edc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.786+0000 7f1ef5600640 1 -- 172.21.15.42:0/160823198 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f1ef00d0cc0 con 0x7f1ef810d3d0 2024-04-20T10:45:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.786+0000 7f1eefe00640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f1edc0a3da0 0x7f1edc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.786+0000 7f1eed600640 1 -- 172.21.15.42:0/792537137 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee801e070 con 0x55bcf9375ca0 2024-04-20T10:45:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.786+0000 7f1eed600640 1 -- 172.21.15.42:0/792537137 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee8011250 con 0x55bcf9375ca0 2024-04-20T10:45:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.786+0000 7f1eed600640 1 -- 172.21.15.42:0/792537137 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8019990 con 0x55bcf9375ca0 2024-04-20T10:45:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.787+0000 7f1f01bd8740 1 -- 172.21.15.42:0/792537137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 msgr2=0x55bcf937e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.787+0000 7f1eefe00640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f1edc0a3da0 0x7f1edc0a6260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0005ab0 tx=0x7f1ee0045040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.787+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/792537137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 0x55bcf937e340 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80114f0 tx=0x7f1ee80119b0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.787+0000 7f1f01bd8740 1 -- 172.21.15.42:0/792537137 shutdown_connections 2024-04-20T10:45:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.787+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/792537137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf937e880 0x55bcf9380c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.787+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/792537137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 0x55bcf937e340 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.787+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/792537137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf9375760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.787+0000 7f1f01bd8740 1 -- 172.21.15.42:0/792537137 >> 172.21.15.42:0/792537137 conn(0x55bcf936fab0 msgr2=0x55bcf936fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.787+0000 7f1f01bd8740 1 -- 172.21.15.42:0/792537137 shutdown_connections 2024-04-20T10:45:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.787+0000 7f1f01bd8740 1 -- 172.21.15.42:0/792537137 wait complete. 2024-04-20T10:45:33.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.788+0000 7f1f01bd8740 1 -- start start 2024-04-20T10:45:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.788+0000 7f1f01bd8740 1 Processor -- start 2024-04-20T10:45:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1f01bd8740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf937b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1f01bd8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 0x55bcf937be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1f01bd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf937e880 0x55bcf937c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1f01bd8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bcf9381670 con 0x55bcf9375ca0 2024-04-20T10:45:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1f01bd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bcf9294740 con 0x55bcf937e880 2024-04-20T10:45:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1f01bd8740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55bcf93817f0 con 0x55bcf9375380 2024-04-20T10:45:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1f01bd8740 10 client.0 _ll_register_callbacks cb 0x55bcf928fb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:45:33.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 0x55bcf937be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 0x55bcf937be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43392/0 (socket says 172.21.15.42:43392) 2024-04-20T10:45:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1eefe00640 1 -- 172.21.15.42:0/4292409368 learned_addr learned my addr 172.21.15.42:0/4292409368 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1ef7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf937b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1efce00640 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf937e880 0x55bcf937c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1efce00640 1 -- 172.21.15.42:0/4292409368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 msgr2=0x55bcf937b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1efce00640 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf937b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1efce00640 1 -- 172.21.15.42:0/4292409368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 msgr2=0x55bcf937be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:33.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1efce00640 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 0x55bcf937be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.789+0000 7f1efce00640 1 -- 172.21.15.42:0/4292409368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcf8fed3f0 con 0x55bcf937e880 2024-04-20T10:45:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.790+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf937b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.790+0000 7f1efce00640 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf937e880 0x55bcf937c3a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00ce380 tx=0x7f1ef009cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b9691f566392c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.790+0000 7f1ee6000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:45:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.790+0000 7f1ee6000640 1 -- 172.21.15.42:0/4292409368 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00bd070 con 0x55bcf937e880 2024-04-20T10:45:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.790+0000 7f1ee6000640 1 -- 172.21.15.42:0/4292409368 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f1ef00a53b0 con 0x55bcf937e880 2024-04-20T10:45:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.790+0000 7f1f01bd8740 1 -- 172.21.15.42:0/4292409368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ee8011250 con 0x55bcf937e880 2024-04-20T10:45:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.790+0000 7f1ee6000640 1 -- 172.21.15.42:0/4292409368 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1ef00bd210 con 0x55bcf937e880 2024-04-20T10:45:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.790+0000 7f1f01bd8740 10 client.26684 fetch_fsmap learned FSMap version 19 2024-04-20T10:45:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.790+0000 7f1f01bd8740 1 -- 172.21.15.42:0/4292409368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bcf8fad0f0 con 0x55bcf937e880 2024-04-20T10:45:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.790+0000 7f1ee6000640 1 -- 172.21.15.42:0/4292409368 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee8011250 con 0x55bcf937e880 2024-04-20T10:45:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.791+0000 7f1ee6000640 1 -- 172.21.15.42:0/4292409368 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f1ef00cead0 con 0x55bcf937e880 2024-04-20T10:45:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.791+0000 7f1f01bd8740 10 client.26684 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:45:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.791+0000 7f1f01bd8740 10 client.26684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:45:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.791+0000 7f1f01bd8740 20 client.26684 populate_metadata read hostname 'smithi042' 2024-04-20T10:45:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.791+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bcf938ec00 0x55bcf9391040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.791+0000 7f1f01bd8740 4 client.26684 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:45:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.791+0000 7f1f01bd8740 1 -- 172.21.15.42:0/4292409368 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcf92e8b70 con 0x55bcf938ec00 2024-04-20T10:45:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.791+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bcf938ec00 0x55bcf9391040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.792+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bcf938ec00 0x55bcf9391040 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa106acf8f6c190 server_cookie=f1471d5b2297d6ee in_seq=0 out_seq=0 2024-04-20T10:45:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.792+0000 7f1ee6000640 10 client.26684 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1ee6000640 1 -- 172.21.15.42:0/4292409368 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bcf92e8b70 con 0x55bcf938ec00 2024-04-20T10:45:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1ee6000640 10 client.26684 handle_command_reply: tid=0 2024-04-20T10:45:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1f01bd8740 1 client.26684 shutdown 2024-04-20T10:45:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1f01bd8740 1 -- 172.21.15.42:0/4292409368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf937e880 msgr2=0x55bcf937c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf937e880 0x55bcf937c3a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00ce380 tx=0x7f1ef009cbd0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1f01bd8740 1 -- 172.21.15.42:0/4292409368 shutdown_connections 2024-04-20T10:45:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bcf938ec00 0x55bcf9391040 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf937e880 0x55bcf937c3a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf9375ca0 0x55bcf937be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/4292409368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf937b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1f01bd8740 1 -- 172.21.15.42:0/4292409368 >> 172.21.15.42:0/4292409368 conn(0x55bcf936fab0 msgr2=0x55bcf9371800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1ee6000640 0 client.26684 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1f01bd8740 1 -- 172.21.15.42:0/4292409368 shutdown_connections 2024-04-20T10:45:33.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1f01bd8740 1 -- 172.21.15.42:0/4292409368 wait complete. 2024-04-20T10:45:33.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.793+0000 7f1f01bd8740 20 client.26684 trim_cache size 0 max 16384 2024-04-20T10:45:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1f01bd8740 1 Processor -- start 2024-04-20T10:45:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1f01bd8740 1 -- start start 2024-04-20T10:45:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1f01bd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375380 0x55bcf938a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1f01bd8740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375ca0 0x55bcf937ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1f01bd8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf937e880 0x55bcf936e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1f01bd8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81b2220 con 0x55bcf937e880 2024-04-20T10:45:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1f01bd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81b4a80 con 0x55bcf9375380 2024-04-20T10:45:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1f01bd8740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81b2870 con 0x55bcf9375ca0 2024-04-20T10:45:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1efce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf937e880 0x55bcf936e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1ef7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375380 0x55bcf938a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1ef7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375380 0x55bcf938a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59650/0 (socket says 172.21.15.42:59650) 2024-04-20T10:45:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1ef7e00640 1 -- 172.21.15.42:0/3078106608 learned_addr learned my addr 172.21.15.42:0/3078106608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:33.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.813+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375ca0 0x55bcf937ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.814+0000 7f1ef7e00640 1 -- 172.21.15.42:0/3078106608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375ca0 msgr2=0x55bcf937ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.814+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/3078106608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375ca0 0x55bcf937ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.814+0000 7f1ef7e00640 1 -- 172.21.15.42:0/3078106608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf937e880 msgr2=0x55bcf936e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.814+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/3078106608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf937e880 0x55bcf936e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.814+0000 7f1ef7e00640 1 -- 172.21.15.42:0/3078106608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcf937a8d0 con 0x55bcf9375380 2024-04-20T10:45:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.814+0000 7f1efce00640 1 --2- 172.21.15.42:0/3078106608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf937e880 0x55bcf936e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:45:33.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.814+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/3078106608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375380 0x55bcf938a530 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8019540 tx=0x7f1ee8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91e1c1b4b163bcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:33.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.814+0000 7f1eecc00640 1 -- 172.21.15.42:0/3078106608 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee801e040 con 0x55bcf9375380 2024-04-20T10:45:33.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.814+0000 7f1eecc00640 1 -- 172.21.15.42:0/3078106608 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee80050e0 con 0x55bcf9375380 2024-04-20T10:45:33.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.814+0000 7f1eecc00640 1 -- 172.21.15.42:0/3078106608 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8003850 con 0x55bcf9375380 2024-04-20T10:45:33.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.815+0000 7f1f01bd8740 1 -- 172.21.15.42:0/3078106608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375380 msgr2=0x55bcf938a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:33.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.815+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/3078106608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375380 0x55bcf938a530 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8019540 tx=0x7f1ee8014000 comp rx=0 tx=0).stop 2024-04-20T10:45:33.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.815+0000 7f1f01bd8740 1 -- 172.21.15.42:0/3078106608 shutdown_connections 2024-04-20T10:45:33.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.815+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/3078106608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf937e880 0x55bcf936e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.815+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/3078106608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375ca0 0x55bcf937ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.815+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/3078106608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375380 0x55bcf938a530 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.815+0000 7f1f01bd8740 1 -- 172.21.15.42:0/3078106608 >> 172.21.15.42:0/3078106608 conn(0x55bcf936fab0 msgr2=0x55bcf936e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:33.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.815+0000 7f1f01bd8740 1 -- 172.21.15.42:0/3078106608 shutdown_connections 2024-04-20T10:45:33.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.815+0000 7f1f01bd8740 1 -- 172.21.15.42:0/3078106608 wait complete. 2024-04-20T10:45:33.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.816+0000 7f1f01bd8740 1 -- start start 2024-04-20T10:45:33.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.816+0000 7f1f01bd8740 1 Processor -- start 2024-04-20T10:45:33.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.816+0000 7f1f01bd8740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf9382820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.816+0000 7f1f01bd8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375ca0 0x55bcf9382d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.816+0000 7f1f01bd8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf937e880 0x55bcf93f4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.816+0000 7f1f01bd8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81b2f40 con 0x55bcf937e880 2024-04-20T10:45:33.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.816+0000 7f1f01bd8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81b43b0 con 0x55bcf9375ca0 2024-04-20T10:45:33.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.816+0000 7f1f01bd8740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef81b3610 con 0x55bcf9375380 2024-04-20T10:45:33.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.816+0000 7f1f01bd8740 10 client.0 _ll_register_callbacks cb 0x7f1ef009d2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:45:33.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.816+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375ca0 0x55bcf9382d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.817+0000 7f1efce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf937e880 0x55bcf93f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.817+0000 7f1ef7e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf9382820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.817+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375ca0 0x55bcf9382d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59656/0 (socket says 172.21.15.42:59656) 2024-04-20T10:45:33.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.817+0000 7f1eefe00640 1 -- 172.21.15.42:0/308944065 learned_addr learned my addr 172.21.15.42:0/308944065 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:33.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.817+0000 7f1ef7e00640 1 -- 172.21.15.42:0/308944065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375ca0 msgr2=0x55bcf9382d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:33.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.817+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/308944065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375ca0 0x55bcf9382d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.817+0000 7f1ef7e00640 1 -- 172.21.15.42:0/308944065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf937e880 msgr2=0x55bcf93f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:33.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.817+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/308944065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf937e880 0x55bcf93f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.817+0000 7f1ef7e00640 1 -- 172.21.15.42:0/308944065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcf93f46c0 con 0x55bcf9375380 2024-04-20T10:45:33.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.817+0000 7f1ef7e00640 1 --2- 172.21.15.42:0/308944065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf9382820 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80058c0 tx=0x7f1ee8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea6e63f941151394 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:33.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.817+0000 7f1ee6000640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-04-20T10:45:33.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.818+0000 7f1ee6000640 1 -- 172.21.15.42:0/308944065 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee801e070 con 0x55bcf9375380 2024-04-20T10:45:33.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.818+0000 7f1f01bd8740 1 -- 172.21.15.42:0/308944065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ee80050e0 con 0x55bcf9375380 2024-04-20T10:45:33.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.818+0000 7f1ee6000640 1 -- 172.21.15.42:0/308944065 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f1ee8011150 con 0x55bcf9375380 2024-04-20T10:45:33.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.818+0000 7f1ee6000640 1 -- 172.21.15.42:0/308944065 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1ee8039050 con 0x55bcf9375380 2024-04-20T10:45:33.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.818+0000 7f1f01bd8740 10 client.26674 fetch_fsmap learned FSMap version 19 2024-04-20T10:45:33.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.818+0000 7f1f01bd8740 1 -- 172.21.15.42:0/308944065 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bcf93f5850 con 0x55bcf9375380 2024-04-20T10:45:33.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.818+0000 7f1ee6000640 1 -- 172.21.15.42:0/308944065 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee80050e0 con 0x55bcf9375380 2024-04-20T10:45:33.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.819+0000 7f1ee6000640 1 -- 172.21.15.42:0/308944065 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f1ee8003680 con 0x55bcf9375380 2024-04-20T10:45:33.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.819+0000 7f1f01bd8740 10 client.26674 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:45:33.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.819+0000 7f1f01bd8740 10 client.26674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:45:33.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.819+0000 7f1f01bd8740 20 client.26674 populate_metadata read hostname 'smithi042' 2024-04-20T10:45:33.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.819+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/308944065 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bcf93f5d20 0x55bcf93f8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:33.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.819+0000 7f1f01bd8740 4 client.26674 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T10:45:33.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.819+0000 7f1f01bd8740 1 -- 172.21.15.42:0/308944065 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bcf92e29e0 con 0x55bcf93f5d20 2024-04-20T10:45:33.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.819+0000 7f1eefe00640 1 --2- 172.21.15.42:0/308944065 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bcf93f5d20 0x55bcf93f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:33.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1eefe00640 1 --2- 172.21.15.42:0/308944065 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bcf93f5d20 0x55bcf93f8110 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf3e4bd348d28ab5 server_cookie=27889fd4d7489cc6 in_seq=0 out_seq=0 2024-04-20T10:45:33.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1ee6000640 10 client.26674 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:33.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1ee6000640 1 -- 172.21.15.42:0/308944065 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bcf92e29e0 con 0x55bcf93f5d20 2024-04-20T10:45:33.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1ee6000640 10 client.26674 handle_command_reply: tid=0 2024-04-20T10:45:33.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1f01bd8740 1 client.26674 shutdown 2024-04-20T10:45:33.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1f01bd8740 1 -- 172.21.15.42:0/308944065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 msgr2=0x55bcf9382820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:33.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/308944065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf9382820 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80058c0 tx=0x7f1ee8009310 comp rx=0 tx=0).stop 2024-04-20T10:45:33.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1f01bd8740 1 -- 172.21.15.42:0/308944065 shutdown_connections 2024-04-20T10:45:33.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/308944065 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55bcf93f5d20 0x55bcf93f8110 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/308944065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bcf937e880 0x55bcf93f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/308944065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bcf9375ca0 0x55bcf9382d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1f01bd8740 1 --2- 172.21.15.42:0/308944065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bcf9375380 0x55bcf9382820 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1ee6000640 0 client.26674 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:33.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1f01bd8740 1 -- 172.21.15.42:0/308944065 >> 172.21.15.42:0/308944065 conn(0x55bcf936fab0 msgr2=0x55bcf936eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:33.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1f01bd8740 1 -- 172.21.15.42:0/308944065 shutdown_connections 2024-04-20T10:45:33.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.820+0000 7f1f01bd8740 1 -- 172.21.15.42:0/308944065 wait complete. 2024-04-20T10:45:33.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1f01bd8740 20 client.26674 trim_cache size 0 max 16384 2024-04-20T10:45:33.829 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:45:33.829 INFO:teuthology.orchestra.run.smithi042.stdout: "status": "no active scrubs running", 2024-04-20T10:45:33.830 INFO:teuthology.orchestra.run.smithi042.stdout: "scrubs": {} 2024-04-20T10:45:33.830 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:45:33.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1efd800640 1 -- 172.21.15.42:0/160823198 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f1edc0a3da0 msgr2=0x7f1edc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:33.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1efd800640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f1edc0a3da0 0x7f1edc0a6260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0005ab0 tx=0x7f1ee0045040 comp rx=0 tx=0).stop 2024-04-20T10:45:33.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1efd800640 1 -- 172.21.15.42:0/160823198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef810d3d0 msgr2=0x7f1ef819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:33.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1efd800640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef810d3d0 0x7f1ef819eb20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000ea60 tx=0x7f1ef000ef20 comp rx=0 tx=0).stop 2024-04-20T10:45:33.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1efd800640 1 -- 172.21.15.42:0/160823198 shutdown_connections 2024-04-20T10:45:33.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1efd800640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f1edc0a3da0 0x7f1edc0a6260 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1efd800640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef810d3d0 0x7f1ef819eb20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1efd800640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef8108ea0 0x7f1ef81a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1efd800640 1 --2- 172.21.15.42:0/160823198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ef8108500 0x7f1ef81a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:33.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1efd800640 1 -- 172.21.15.42:0/160823198 >> 172.21.15.42:0/160823198 conn(0x7f1ef8077d80 msgr2=0x7f1ef80783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:33.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1efd800640 1 -- 172.21.15.42:0/160823198 shutdown_connections 2024-04-20T10:45:33.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:33.821+0000 7f1efd800640 1 -- 172.21.15.42:0/160823198 wait complete. 2024-04-20T10:45:33.839 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T10:45:33.840 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:45:34.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.045+0000 7f46ef400640 1 -- 172.21.15.42:0/3324306141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46e8104d60 msgr2=0x7f46e8108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.045+0000 7f46ef400640 1 --2- 172.21.15.42:0/3324306141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46e8104d60 0x7f46e8108da0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009860 tx=0x7f46d002bb70 comp rx=0 tx=0).stop 2024-04-20T10:45:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.045+0000 7f46ef400640 1 -- 172.21.15.42:0/3324306141 shutdown_connections 2024-04-20T10:45:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.045+0000 7f46ef400640 1 --2- 172.21.15.42:0/3324306141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46e8109480 0x7f46e8111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.045+0000 7f46ef400640 1 --2- 172.21.15.42:0/3324306141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46e8104d60 0x7f46e8108da0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.045+0000 7f46ef400640 1 --2- 172.21.15.42:0/3324306141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46e81043b0 0x7f46e8104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.045+0000 7f46ef400640 1 -- 172.21.15.42:0/3324306141 >> 172.21.15.42:0/3324306141 conn(0x7f46e80ffc50 msgr2=0x7f46e8102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.045+0000 7f46ef400640 1 -- 172.21.15.42:0/3324306141 shutdown_connections 2024-04-20T10:45:34.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.045+0000 7f46ef400640 1 -- 172.21.15.42:0/3324306141 wait complete. 2024-04-20T10:45:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.047+0000 7f46ef400640 1 Processor -- start 2024-04-20T10:45:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.047+0000 7f46ef400640 1 -- start start 2024-04-20T10:45:34.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.047+0000 7f46ef400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46e81043b0 0x7f46e81a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.047+0000 7f46ef400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46e8109480 0x7f46e81a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.047+0000 7f46ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46e819a8b0 0x7f46e819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.047+0000 7f46ef400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8114510 con 0x7f46e81043b0 2024-04-20T10:45:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.047+0000 7f46ef400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8114390 con 0x7f46e819a8b0 2024-04-20T10:45:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.047+0000 7f46ef400640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8114690 con 0x7f46e8109480 2024-04-20T10:45:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.048+0000 7f46eca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46e81043b0 0x7f46e81a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.048+0000 7f46dfe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46e8109480 0x7f46e81a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.048+0000 7f46eca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46e81043b0 0x7f46e81a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43424/0 (socket says 172.21.15.42:43424) 2024-04-20T10:45:34.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.048+0000 7f46eca00640 1 -- 172.21.15.42:0/882943752 learned_addr learned my addr 172.21.15.42:0/882943752 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:34.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.048+0000 7f46ed400640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46e819a8b0 0x7f46e819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.048+0000 7f46dfe00640 1 -- 172.21.15.42:0/882943752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46e819a8b0 msgr2=0x7f46e819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.048+0000 7f46dfe00640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46e819a8b0 0x7f46e819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.048+0000 7f46dfe00640 1 -- 172.21.15.42:0/882943752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46e81043b0 msgr2=0x7f46e81a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.048+0000 7f46dfe00640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46e81043b0 0x7f46e81a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.048+0000 7f46dfe00640 1 -- 172.21.15.42:0/882943752 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e819b3d0 con 0x7f46e8109480 2024-04-20T10:45:34.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.048+0000 7f46eca00640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46e81043b0 0x7f46e81a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:34.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.048+0000 7f46dfe00640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46e8109480 0x7f46e81a0c40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f46d002c0b0 tx=0x7f46d00045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:34.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.049+0000 7f46e6000640 1 -- 172.21.15.42:0/882943752 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0041020 con 0x7f46e8109480 2024-04-20T10:45:34.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.049+0000 7f46e6000640 1 -- 172.21.15.42:0/882943752 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d002c9f0 con 0x7f46e8109480 2024-04-20T10:45:34.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.049+0000 7f46ef400640 1 -- 172.21.15.42:0/882943752 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e819b660 con 0x7f46e8109480 2024-04-20T10:45:34.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.049+0000 7f46e6000640 1 -- 172.21.15.42:0/882943752 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0039040 con 0x7f46e8109480 2024-04-20T10:45:34.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.049+0000 7f46ef400640 1 -- 172.21.15.42:0/882943752 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e81a78b0 con 0x7f46e8109480 2024-04-20T10:45:34.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.050+0000 7f46ef400640 1 -- 172.21.15.42:0/882943752 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e80691d0 con 0x7f46e8109480 2024-04-20T10:45:34.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.052+0000 7f46e6000640 1 -- 172.21.15.42:0/882943752 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f46d0044020 con 0x7f46e8109480 2024-04-20T10:45:34.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.054+0000 7f46e6000640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f46cc0a3da0 0x7f46cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.054+0000 7f46e6000640 1 -- 172.21.15.42:0/882943752 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f46d00f21b0 con 0x7f46e8109480 2024-04-20T10:45:34.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.054+0000 7f46eca00640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f46cc0a3da0 0x7f46cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.055+0000 7f46eca00640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f46cc0a3da0 0x7f46cc0a6260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f46e819b9f0 tx=0x7f46d8009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:34.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.056+0000 7f46e6000640 1 -- 172.21.15.42:0/882943752 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f46d004b020 con 0x7f46e8109480 2024-04-20T10:45:34.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:33 smithi046 ceph-mon[21574]: pgmap v359: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 283 KiB/s wr, 917 op/s 2024-04-20T10:45:34.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.306+0000 7f46ef400640 1 -- 172.21.15.42:0/882943752 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46e8104d60 con 0x7f46e8109480 2024-04-20T10:45:34.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.307+0000 7f46e6000640 1 -- 172.21.15.42:0/882943752 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f46d00be950 con 0x7f46e8109480 2024-04-20T10:45:34.308 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:45:34.309 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:45:34.309 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:45:34.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.310+0000 7f46ef400640 1 -- 172.21.15.42:0/882943752 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f46cc0a3da0 msgr2=0x7f46cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.310+0000 7f46ef400640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f46cc0a3da0 0x7f46cc0a6260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f46e819b9f0 tx=0x7f46d8009500 comp rx=0 tx=0).stop 2024-04-20T10:45:34.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.310+0000 7f46ef400640 1 -- 172.21.15.42:0/882943752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46e8109480 msgr2=0x7f46e81a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.310+0000 7f46ef400640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46e8109480 0x7f46e81a0c40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f46d002c0b0 tx=0x7f46d00045b0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.310+0000 7f46ef400640 1 -- 172.21.15.42:0/882943752 shutdown_connections 2024-04-20T10:45:34.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.310+0000 7f46ef400640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f46cc0a3da0 0x7f46cc0a6260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.310+0000 7f46ef400640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46e819a8b0 0x7f46e819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.310+0000 7f46ef400640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46e8109480 0x7f46e81a0c40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.310+0000 7f46ef400640 1 --2- 172.21.15.42:0/882943752 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46e81043b0 0x7f46e81a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.310+0000 7f46ef400640 1 -- 172.21.15.42:0/882943752 >> 172.21.15.42:0/882943752 conn(0x7f46e80ffc50 msgr2=0x7f46e8100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:34.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.311+0000 7f46ef400640 1 -- 172.21.15.42:0/882943752 shutdown_connections 2024-04-20T10:45:34.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.311+0000 7f46ef400640 1 -- 172.21.15.42:0/882943752 wait complete. 2024-04-20T10:45:34.329 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:33 smithi042 ceph-mon[17078]: pgmap v359: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 283 KiB/s wr, 917 op/s 2024-04-20T10:45:34.333 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T10:45:34.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:33 smithi162 ceph-mon[21854]: pgmap v359: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 283 KiB/s wr, 917 op/s 2024-04-20T10:45:34.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.552+0000 7f5e68200640 1 -- 172.21.15.42:0/2309205817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60074c30 msgr2=0x7f5e6010e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.552+0000 7f5e68200640 1 --2- 172.21.15.42:0/2309205817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60074c30 0x7f5e6010e300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009860 tx=0x7f5e4c02bc10 comp rx=0 tx=0).stop 2024-04-20T10:45:34.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.552+0000 7f5e68200640 1 -- 172.21.15.42:0/2309205817 shutdown_connections 2024-04-20T10:45:34.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.553+0000 7f5e68200640 1 --2- 172.21.15.42:0/2309205817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6010e840 0x7f5e60110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.553+0000 7f5e68200640 1 --2- 172.21.15.42:0/2309205817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60074c30 0x7f5e6010e300 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.553+0000 7f5e68200640 1 --2- 172.21.15.42:0/2309205817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e60074310 0x7f5e600746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.553+0000 7f5e68200640 1 -- 172.21.15.42:0/2309205817 >> 172.21.15.42:0/2309205817 conn(0x7f5e600fda70 msgr2=0x7f5e600ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:34.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.553+0000 7f5e68200640 1 -- 172.21.15.42:0/2309205817 shutdown_connections 2024-04-20T10:45:34.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.553+0000 7f5e68200640 1 -- 172.21.15.42:0/2309205817 wait complete. 2024-04-20T10:45:34.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e68200640 1 Processor -- start 2024-04-20T10:45:34.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e68200640 1 -- start start 2024-04-20T10:45:34.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e68200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e60074310 0x7f5e601a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e68200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60074c30 0x7f5e601a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e68200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6010e840 0x7f5e6019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e68200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60114550 con 0x7f5e60074c30 2024-04-20T10:45:34.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e68200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601143d0 con 0x7f5e6010e840 2024-04-20T10:45:34.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e68200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601146d0 con 0x7f5e60074310 2024-04-20T10:45:34.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e64e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60074c30 0x7f5e601a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e64e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60074c30 0x7f5e601a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43436/0 (socket says 172.21.15.42:43436) 2024-04-20T10:45:34.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e64e00640 1 -- 172.21.15.42:0/302579340 learned_addr learned my addr 172.21.15.42:0/302579340 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:34.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e65800640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e60074310 0x7f5e601a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.554+0000 7f5e66200640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6010e840 0x7f5e6019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.555+0000 7f5e64e00640 1 -- 172.21.15.42:0/302579340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e60074310 msgr2=0x7f5e601a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.555+0000 7f5e64e00640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e60074310 0x7f5e601a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.555+0000 7f5e64e00640 1 -- 172.21.15.42:0/302579340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6010e840 msgr2=0x7f5e6019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.555+0000 7f5e64e00640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6010e840 0x7f5e6019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.555+0000 7f5e64e00640 1 -- 172.21.15.42:0/302579340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6019b0b0 con 0x7f5e60074c30 2024-04-20T10:45:34.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.555+0000 7f5e65800640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e60074310 0x7f5e601a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:34.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.555+0000 7f5e64e00640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60074c30 0x7f5e601a0cc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c037040 tx=0x7f5e4c0344c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:34.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.555+0000 7f5e5e000640 1 -- 172.21.15.42:0/302579340 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c041020 con 0x7f5e60074c30 2024-04-20T10:45:34.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.555+0000 7f5e5e000640 1 -- 172.21.15.42:0/302579340 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e4c02ca90 con 0x7f5e60074c30 2024-04-20T10:45:34.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.555+0000 7f5e68200640 1 -- 172.21.15.42:0/302579340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6019b340 con 0x7f5e60074c30 2024-04-20T10:45:34.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.555+0000 7f5e5e000640 1 -- 172.21.15.42:0/302579340 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c039040 con 0x7f5e60074c30 2024-04-20T10:45:34.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.556+0000 7f5e68200640 1 -- 172.21.15.42:0/302579340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e601a7640 con 0x7f5e60074c30 2024-04-20T10:45:34.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.560+0000 7f5e6afe1740 1 Processor -- start 2024-04-20T10:45:34.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.560+0000 7f5e6afe1740 1 -- start start 2024-04-20T10:45:34.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.560+0000 7f5e5e000640 1 -- 172.21.15.42:0/302579340 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5e4c044020 con 0x7f5e60074c30 2024-04-20T10:45:34.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.560+0000 7f5e6afe1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.560+0000 7f5e6afe1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f18d0 0x55d65d4f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.560+0000 7f5e6afe1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.560+0000 7f5e6afe1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d65d456050 con 0x55d65d4fa4b0 2024-04-20T10:45:34.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.560+0000 7f5e6afe1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d65d3fc490 con 0x55d65d4f18d0 2024-04-20T10:45:34.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.560+0000 7f5e6afe1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d65d452010 con 0x55d65d4f0fb0 2024-04-20T10:45:34.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.561+0000 7f5e64e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.561+0000 7f5e66200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.561+0000 7f5e64e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4fc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43446/0 (socket says 172.21.15.42:43446) 2024-04-20T10:45:34.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.561+0000 7f5e64e00640 1 -- 172.21.15.42:0/820898508 learned_addr learned my addr 172.21.15.42:0/820898508 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:34.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.561+0000 7f5e65800640 1 --2- 172.21.15.42:0/820898508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f18d0 0x55d65d4f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.561+0000 7f5e66200640 1 -- 172.21.15.42:0/820898508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f18d0 msgr2=0x55d65d4f9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:34.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.561+0000 7f5e66200640 1 --2- 172.21.15.42:0/820898508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f18d0 0x55d65d4f9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.561+0000 7f5e66200640 1 -- 172.21.15.42:0/820898508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 msgr2=0x55d65d4fc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:34.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.561+0000 7f5e66200640 1 --2- 172.21.15.42:0/820898508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4fc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.561+0000 7f5e66200640 1 -- 172.21.15.42:0/820898508 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65d404070 con 0x55d65d4f0fb0 2024-04-20T10:45:34.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.561+0000 7f5e66200640 1 --2- 172.21.15.42:0/820898508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f1390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009b80 tx=0x7f5e58011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6b4afe6c641dcf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:34.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.562+0000 7f5e52a00640 1 -- 172.21.15.42:0/820898508 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58004050 con 0x55d65d4f0fb0 2024-04-20T10:45:34.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.562+0000 7f5e52a00640 1 -- 172.21.15.42:0/820898508 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e58005500 con 0x55d65d4f0fb0 2024-04-20T10:45:34.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.562+0000 7f5e52a00640 1 -- 172.21.15.42:0/820898508 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58004d00 con 0x55d65d4f0fb0 2024-04-20T10:45:34.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.562+0000 7f5e5e000640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5e480a3d20 0x7f5e480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.562+0000 7f5e5e000640 1 -- 172.21.15.42:0/302579340 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f5e4c0f1e00 con 0x7f5e60074c30 2024-04-20T10:45:34.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.562+0000 7f5e65800640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5e480a3d20 0x7f5e480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.562+0000 7f5e6afe1740 1 -- 172.21.15.42:0/820898508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f0fb0 msgr2=0x55d65d4f1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.562+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/820898508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f1390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009b80 tx=0x7f5e58011810 comp rx=0 tx=0).stop 2024-04-20T10:45:34.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.563+0000 7f5e6afe1740 1 -- 172.21.15.42:0/820898508 shutdown_connections 2024-04-20T10:45:34.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.563+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/820898508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4fc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.563+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/820898508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f18d0 0x55d65d4f9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.563+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/820898508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f1390 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.563+0000 7f5e6afe1740 1 -- 172.21.15.42:0/820898508 >> 172.21.15.42:0/820898508 conn(0x55d65d4eb6e0 msgr2=0x55d65d4ebac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:34.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.563+0000 7f5e6afe1740 1 -- 172.21.15.42:0/820898508 shutdown_connections 2024-04-20T10:45:34.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.563+0000 7f5e6afe1740 1 -- 172.21.15.42:0/820898508 wait complete. 2024-04-20T10:45:34.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.563+0000 7f5e65800640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5e480a3d20 0x7f5e480a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55d65d4f0a80 tx=0x7f5e54010040 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:34.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.563+0000 7f5e6afe1740 1 -- start start 2024-04-20T10:45:34.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.563+0000 7f5e6afe1740 1 Processor -- start 2024-04-20T10:45:34.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.564+0000 7f5e6afe1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.564+0000 7f5e6afe1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f18d0 0x55d65d4f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.564+0000 7f5e6afe1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.564+0000 7f5e6afe1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d65d4fd2a0 con 0x55d65d4fa4b0 2024-04-20T10:45:34.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.564+0000 7f5e6afe1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d65d40f860 con 0x55d65d4f0fb0 2024-04-20T10:45:34.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.564+0000 7f5e6afe1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d65d4fd420 con 0x55d65d4f18d0 2024-04-20T10:45:34.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.564+0000 7f5e6afe1740 10 client.0 _ll_register_callbacks cb 0x55d65d3ddfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:45:34.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.564+0000 7f5e65800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.564+0000 7f5e66200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.564+0000 7f5e66200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4f7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43462/0 (socket says 172.21.15.42:43462) 2024-04-20T10:45:34.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.564+0000 7f5e64e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f18d0 0x55d65d4f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.564+0000 7f5e66200640 1 -- 172.21.15.42:0/2395628437 learned_addr learned my addr 172.21.15.42:0/2395628437 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:34.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e66200640 1 -- 172.21.15.42:0/2395628437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f18d0 msgr2=0x55d65d4f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:34.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e66200640 1 --2- 172.21.15.42:0/2395628437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f18d0 0x55d65d4f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e66200640 1 -- 172.21.15.42:0/2395628437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 msgr2=0x55d65d4f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:34.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e66200640 1 --2- 172.21.15.42:0/2395628437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e66200640 1 -- 172.21.15.42:0/2395628437 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65d413950 con 0x55d65d4fa4b0 2024-04-20T10:45:34.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e66200640 1 --2- 172.21.15.42:0/2395628437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4f7f70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5e58011d10 tx=0x7f5e58004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7f35d941574dfc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:34.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e47400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-20T10:45:34.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e47400640 1 -- 172.21.15.42:0/2395628437 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58029070 con 0x55d65d4fa4b0 2024-04-20T10:45:34.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e47400640 1 -- 172.21.15.42:0/2395628437 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f5e58003a00 con 0x55d65d4fa4b0 2024-04-20T10:45:34.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e6afe1740 1 -- 172.21.15.42:0/2395628437 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e58005500 con 0x55d65d4fa4b0 2024-04-20T10:45:34.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e47400640 1 -- 172.21.15.42:0/2395628437 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5e58035050 con 0x55d65d4fa4b0 2024-04-20T10:45:34.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e6afe1740 10 client.16563 fetch_fsmap learned FSMap version 19 2024-04-20T10:45:34.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.565+0000 7f5e6afe1740 1 -- 172.21.15.42:0/2395628437 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d65d3e1820 con 0x55d65d4fa4b0 2024-04-20T10:45:34.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.566+0000 7f5e47400640 1 -- 172.21.15.42:0/2395628437 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e58005500 con 0x55d65d4fa4b0 2024-04-20T10:45:34.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.566+0000 7f5e47400640 1 -- 172.21.15.42:0/2395628437 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f5e58005120 con 0x55d65d4fa4b0 2024-04-20T10:45:34.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.566+0000 7f5e6afe1740 10 client.16563 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:45:34.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.566+0000 7f5e6afe1740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:45:34.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.566+0000 7f5e6afe1740 20 client.16563 populate_metadata read hostname 'smithi042' 2024-04-20T10:45:34.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.566+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/2395628437 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d65d50a3e0 0x55d65d50c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.566+0000 7f5e6afe1740 4 client.16563 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:45:34.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.566+0000 7f5e6afe1740 1 -- 172.21.15.42:0/2395628437 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d65d463a70 con 0x55d65d50a3e0 2024-04-20T10:45:34.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.567+0000 7f5e65800640 1 --2- 172.21.15.42:0/2395628437 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d65d50a3e0 0x55d65d50c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.567+0000 7f5e65800640 1 --2- 172.21.15.42:0/2395628437 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d65d50a3e0 0x55d65d50c7d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36682bd20889dbcf server_cookie=bdb0a08b37389a97 in_seq=0 out_seq=0 2024-04-20T10:45:34.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.567+0000 7f5e47400640 10 client.16563 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:34.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e47400640 1 -- 172.21.15.42:0/2395628437 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d65d463a70 con 0x55d65d50a3e0 2024-04-20T10:45:34.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e47400640 10 client.16563 handle_command_reply: tid=0 2024-04-20T10:45:34.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e6afe1740 1 client.16563 shutdown 2024-04-20T10:45:34.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e6afe1740 1 -- 172.21.15.42:0/2395628437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 msgr2=0x55d65d4f7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/2395628437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4f7f70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5e58011d10 tx=0x7f5e58004280 comp rx=0 tx=0).stop 2024-04-20T10:45:34.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e6afe1740 1 -- 172.21.15.42:0/2395628437 shutdown_connections 2024-04-20T10:45:34.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/2395628437 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d65d50a3e0 0x55d65d50c7d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/2395628437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4f7f70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/2395628437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f18d0 0x55d65d4f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/2395628437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e47400640 0 client.16563 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:34.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e6afe1740 1 -- 172.21.15.42:0/2395628437 >> 172.21.15.42:0/2395628437 conn(0x55d65d4eb6e0 msgr2=0x55d65d4ed430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:34.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e6afe1740 1 -- 172.21.15.42:0/2395628437 shutdown_connections 2024-04-20T10:45:34.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e6afe1740 1 -- 172.21.15.42:0/2395628437 wait complete. 2024-04-20T10:45:34.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.568+0000 7f5e6afe1740 20 client.16563 trim_cache size 0 max 16384 2024-04-20T10:45:34.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e6afe1740 1 Processor -- start 2024-04-20T10:45:34.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e6afe1740 1 -- start start 2024-04-20T10:45:34.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e6afe1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e6afe1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f18d0 0x55d65d506190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e6afe1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e6afe1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601ae040 con 0x55d65d4fa4b0 2024-04-20T10:45:34.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e6afe1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b08a0 con 0x55d65d4f0fb0 2024-04-20T10:45:34.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e6afe1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601ae690 con 0x55d65d4f18d0 2024-04-20T10:45:34.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e66200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e66200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43464/0 (socket says 172.21.15.42:43464) 2024-04-20T10:45:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e64e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f18d0 0x55d65d506190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e66200640 1 -- 172.21.15.42:0/4148304886 learned_addr learned my addr 172.21.15.42:0/4148304886 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.583+0000 7f5e65800640 1 --2- 172.21.15.42:0/4148304886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.584+0000 7f5e66200640 1 -- 172.21.15.42:0/4148304886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f18d0 msgr2=0x55d65d506190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.584+0000 7f5e66200640 1 --2- 172.21.15.42:0/4148304886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f18d0 0x55d65d506190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.584+0000 7f5e66200640 1 -- 172.21.15.42:0/4148304886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 msgr2=0x55d65d4f9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.584+0000 7f5e66200640 1 --2- 172.21.15.42:0/4148304886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.584+0000 7f5e66200640 1 -- 172.21.15.42:0/4148304886 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65d1683f0 con 0x55d65d4fa4b0 2024-04-20T10:45:34.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.584+0000 7f5e66200640 1 --2- 172.21.15.42:0/4148304886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4f6650 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5e58011850 tx=0x7f5e580236f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10501609326d4bfc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:34.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.584+0000 7f5e52000640 1 -- 172.21.15.42:0/4148304886 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58029040 con 0x55d65d4fa4b0 2024-04-20T10:45:34.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.584+0000 7f5e52000640 1 -- 172.21.15.42:0/4148304886 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e580054a0 con 0x55d65d4fa4b0 2024-04-20T10:45:34.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.584+0000 7f5e52000640 1 -- 172.21.15.42:0/4148304886 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5802e440 con 0x55d65d4fa4b0 2024-04-20T10:45:34.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.585+0000 7f5e6afe1740 1 -- 172.21.15.42:0/4148304886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 msgr2=0x55d65d4f6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.585+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/4148304886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4f6650 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5e58011850 tx=0x7f5e580236f0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.585+0000 7f5e6afe1740 1 -- 172.21.15.42:0/4148304886 shutdown_connections 2024-04-20T10:45:34.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.585+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/4148304886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4fa4b0 0x55d65d4f6650 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.585+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/4148304886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4f18d0 0x55d65d506190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.585+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/4148304886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4f9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.585+0000 7f5e6afe1740 1 -- 172.21.15.42:0/4148304886 >> 172.21.15.42:0/4148304886 conn(0x55d65d4eb6e0 msgr2=0x55d65d4e9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:34.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.585+0000 7f5e6afe1740 1 -- 172.21.15.42:0/4148304886 shutdown_connections 2024-04-20T10:45:34.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.585+0000 7f5e6afe1740 1 -- 172.21.15.42:0/4148304886 wait complete. 2024-04-20T10:45:34.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.585+0000 7f5e6afe1740 1 -- start start 2024-04-20T10:45:34.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e6afe1740 1 Processor -- start 2024-04-20T10:45:34.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e6afe1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e6afe1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4f18d0 0x55d65d4fe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e6afe1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4fa4b0 0x55d65d5020b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e6afe1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601aed60 con 0x55d65d4f18d0 2024-04-20T10:45:34.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e6afe1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b01d0 con 0x55d65d4f0fb0 2024-04-20T10:45:34.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e6afe1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601af430 con 0x55d65d4fa4b0 2024-04-20T10:45:34.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e6afe1740 10 client.0 _ll_register_callbacks cb 0x7f5e58009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:45:34.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e64e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4f18d0 0x55d65d4fe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e65800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4fdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e65800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4fdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59710/0 (socket says 172.21.15.42:59710) 2024-04-20T10:45:34.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e65800640 1 -- 172.21.15.42:0/986386199 learned_addr learned my addr 172.21.15.42:0/986386199 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:34.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.586+0000 7f5e66200640 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4fa4b0 0x55d65d5020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.587+0000 7f5e65800640 1 -- 172.21.15.42:0/986386199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4fa4b0 msgr2=0x55d65d5020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:34.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.587+0000 7f5e65800640 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4fa4b0 0x55d65d5020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.587+0000 7f5e65800640 1 -- 172.21.15.42:0/986386199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4f18d0 msgr2=0x55d65d4fe1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:45:34.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.587+0000 7f5e65800640 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4f18d0 0x55d65d4fe1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.587+0000 7f5e65800640 1 -- 172.21.15.42:0/986386199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65d2327d0 con 0x55d65d4f0fb0 2024-04-20T10:45:34.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.587+0000 7f5e64e00640 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4f18d0 0x55d65d4fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:34.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.587+0000 7f5e66200640 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4fa4b0 0x55d65d5020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:34.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.587+0000 7f5e65800640 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4fdca0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55d65d50cf80 tx=0x7f5e54016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8d907f3bdfd2fb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:34.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.587+0000 7f5e47400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-20T10:45:34.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.587+0000 7f5e47400640 1 -- 172.21.15.42:0/986386199 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54004b20 con 0x55d65d4f0fb0 2024-04-20T10:45:34.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.587+0000 7f5e6afe1740 1 -- 172.21.15.42:0/986386199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e580054a0 con 0x55d65d4f0fb0 2024-04-20T10:45:34.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.587+0000 7f5e47400640 1 -- 172.21.15.42:0/986386199 <== mon.2 v2:172.21.15.46:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f5e54004cc0 con 0x55d65d4f0fb0 2024-04-20T10:45:34.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.588+0000 7f5e47400640 1 -- 172.21.15.42:0/986386199 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5e54003f50 con 0x55d65d4f0fb0 2024-04-20T10:45:34.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.588+0000 7f5e6afe1740 10 client.26714 fetch_fsmap learned FSMap version 19 2024-04-20T10:45:34.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.588+0000 7f5e6afe1740 1 -- 172.21.15.42:0/986386199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d65d570160 con 0x55d65d4f0fb0 2024-04-20T10:45:34.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.588+0000 7f5e47400640 1 -- 172.21.15.42:0/986386199 <== mon.2 v2:172.21.15.46:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e580054a0 con 0x55d65d4f0fb0 2024-04-20T10:45:34.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.588+0000 7f5e47400640 1 -- 172.21.15.42:0/986386199 <== mon.2 v2:172.21.15.46:3300/0 5 ==== fsmap(e 19) ==== 8323+0+0 (secure 0 0 0) 0x7f5e5400b040 con 0x55d65d4f0fb0 2024-04-20T10:45:34.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.588+0000 7f5e6afe1740 10 client.26714 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:45:34.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.588+0000 7f5e6afe1740 10 client.26714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T10:45:34.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.588+0000 7f5e6afe1740 20 client.26714 populate_metadata read hostname 'smithi042' 2024-04-20T10:45:34.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.588+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d65d570910 0x55d65d572d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:34.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.588+0000 7f5e6afe1740 4 client.26714 mds_command: new command op to 24584 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T10:45:34.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.588+0000 7f5e6afe1740 1 -- 172.21.15.42:0/986386199 --> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d65d45d8e0 con 0x55d65d570910 2024-04-20T10:45:34.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e64e00640 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d65d570910 0x55d65d572d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:34.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e64e00640 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d65d570910 0x55d65d572d30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c8852e481e5779 server_cookie=9c81aeb799e71545 in_seq=0 out_seq=0 2024-04-20T10:45:34.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e47400640 10 client.26714 ms_handle_connect on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:34.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e47400640 1 -- 172.21.15.42:0/986386199 <== mds.0 v2:172.21.15.46:6838/2026912868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d65d45d8e0 con 0x55d65d570910 2024-04-20T10:45:34.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e47400640 10 client.26714 handle_command_reply: tid=0 2024-04-20T10:45:34.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e6afe1740 1 client.26714 shutdown 2024-04-20T10:45:34.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e6afe1740 1 -- 172.21.15.42:0/986386199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 msgr2=0x55d65d4fdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4fdca0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55d65d50cf80 tx=0x7f5e54016000 comp rx=0 tx=0).stop 2024-04-20T10:45:34.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e6afe1740 1 -- 172.21.15.42:0/986386199 shutdown_connections 2024-04-20T10:45:34.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.46:6838/2026912868,v1:172.21.15.46:6839/2026912868] conn(0x55d65d570910 0x55d65d572d30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d65d4fa4b0 0x55d65d5020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d65d4f18d0 0x55d65d4fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e6afe1740 1 --2- 172.21.15.42:0/986386199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d65d4f0fb0 0x55d65d4fdca0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e47400640 0 client.26714 ms_handle_reset on v2:172.21.15.46:6838/2026912868 2024-04-20T10:45:34.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.589+0000 7f5e6afe1740 1 -- 172.21.15.42:0/986386199 >> 172.21.15.42:0/986386199 conn(0x55d65d4eb6e0 msgr2=0x55d65d4e9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:34.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.590+0000 7f5e6afe1740 1 -- 172.21.15.42:0/986386199 shutdown_connections 2024-04-20T10:45:34.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.590+0000 7f5e6afe1740 1 -- 172.21.15.42:0/986386199 wait complete. 2024-04-20T10:45:34.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.590+0000 7f5e6afe1740 20 client.26714 trim_cache size 0 max 16384 2024-04-20T10:45:34.602 INFO:teuthology.orchestra.run.smithi042.stdout:[] 2024-04-20T10:45:34.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.590+0000 7f5e68200640 1 -- 172.21.15.42:0/302579340 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5e480a3d20 msgr2=0x7f5e480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.590+0000 7f5e68200640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5e480a3d20 0x7f5e480a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55d65d4f0a80 tx=0x7f5e54010040 comp rx=0 tx=0).stop 2024-04-20T10:45:34.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.590+0000 7f5e68200640 1 -- 172.21.15.42:0/302579340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60074c30 msgr2=0x7f5e601a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.590+0000 7f5e68200640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60074c30 0x7f5e601a0cc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c037040 tx=0x7f5e4c0344c0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.590+0000 7f5e68200640 1 -- 172.21.15.42:0/302579340 shutdown_connections 2024-04-20T10:45:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.590+0000 7f5e68200640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5e480a3d20 0x7f5e480a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.590+0000 7f5e68200640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6010e840 0x7f5e6019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.590+0000 7f5e68200640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60074c30 0x7f5e601a0cc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.591+0000 7f5e68200640 1 --2- 172.21.15.42:0/302579340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e60074310 0x7f5e601a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:34.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.591+0000 7f5e68200640 1 -- 172.21.15.42:0/302579340 >> 172.21.15.42:0/302579340 conn(0x7f5e600fda70 msgr2=0x7f5e600ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:34.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.591+0000 7f5e68200640 1 -- 172.21.15.42:0/302579340 shutdown_connections 2024-04-20T10:45:34.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:34.591+0000 7f5e68200640 1 -- 172.21.15.42:0/302579340 wait complete. 2024-04-20T10:45:35.205 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:34 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/882943752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:45:35.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:34 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/882943752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:45:35.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:34 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/882943752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:45:35.608 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-20T10:45:35.609 INFO:tasks.fwd_scrub:done joining 2024-04-20T10:45:35.609 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-20T10:45:35.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:45:35.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.839+0000 7f5c6b200640 1 -- 172.21.15.42:0/1938564982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6c076db0 msgr2=0x7f5c6c1126c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:35.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.839+0000 7f5c6b200640 1 --2- 172.21.15.42:0/1938564982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6c076db0 0x7f5c6c1126c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009860 tx=0x7f5c5c02bb70 comp rx=0 tx=0).stop 2024-04-20T10:45:35.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.840+0000 7f5c6b200640 1 -- 172.21.15.42:0/1938564982 shutdown_connections 2024-04-20T10:45:35.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.840+0000 7f5c6b200640 1 --2- 172.21.15.42:0/1938564982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c6c112c00 0x7f5c6c114ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:35.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.840+0000 7f5c6b200640 1 --2- 172.21.15.42:0/1938564982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6c076db0 0x7f5c6c1126c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:35.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.840+0000 7f5c6b200640 1 --2- 172.21.15.42:0/1938564982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c076490 0x7f5c6c076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:35.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.840+0000 7f5c6b200640 1 -- 172.21.15.42:0/1938564982 >> 172.21.15.42:0/1938564982 conn(0x7f5c6c0ff9f0 msgr2=0x7f5c6c101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:35.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.840+0000 7f5c6b200640 1 -- 172.21.15.42:0/1938564982 shutdown_connections 2024-04-20T10:45:35.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.840+0000 7f5c6b200640 1 -- 172.21.15.42:0/1938564982 wait complete. 2024-04-20T10:45:35.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.841+0000 7f5c6b200640 1 Processor -- start 2024-04-20T10:45:35.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.841+0000 7f5c6b200640 1 -- start start 2024-04-20T10:45:35.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.841+0000 7f5c6b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c076490 0x7f5c6c1a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:35.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c6b200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c6c076db0 0x7f5c6c1a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:35.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c6b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6c112c00 0x7f5c6c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:35.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c6b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c118760 con 0x7f5c6c112c00 2024-04-20T10:45:35.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c6b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c1185e0 con 0x7f5c6c076490 2024-04-20T10:45:35.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c6b200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c1188e0 con 0x7f5c6c076db0 2024-04-20T10:45:35.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c69e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c076490 0x7f5c6c1a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:35.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c69e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c076490 0x7f5c6c1a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59726/0 (socket says 172.21.15.42:59726) 2024-04-20T10:45:35.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c6a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6c112c00 0x7f5c6c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:35.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c69e00640 1 -- 172.21.15.42:0/1043760710 learned_addr learned my addr 172.21.15.42:0/1043760710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:35.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c69400640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c6c076db0 0x7f5c6c1a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:35.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c69e00640 1 -- 172.21.15.42:0/1043760710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c6c076db0 msgr2=0x7f5c6c1a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:35.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c69e00640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c6c076db0 0x7f5c6c1a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:35.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c69e00640 1 -- 172.21.15.42:0/1043760710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6c112c00 msgr2=0x7f5c6c19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:35.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c69e00640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6c112c00 0x7f5c6c19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:35.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c69e00640 1 -- 172.21.15.42:0/1043760710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c19f220 con 0x7f5c6c076490 2024-04-20T10:45:35.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.842+0000 7f5c6a800640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6c112c00 0x7f5c6c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:35.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.843+0000 7f5c69e00640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c076490 0x7f5c6c1a48f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400e9c0 tx=0x7f5c5400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:35.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.843+0000 7f5c62a00640 1 -- 172.21.15.42:0/1043760710 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5400cde0 con 0x7f5c6c076490 2024-04-20T10:45:35.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.843+0000 7f5c6b200640 1 -- 172.21.15.42:0/1043760710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6c19f510 con 0x7f5c6c076490 2024-04-20T10:45:35.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.843+0000 7f5c62a00640 1 -- 172.21.15.42:0/1043760710 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c540035f0 con 0x7f5c6c076490 2024-04-20T10:45:35.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.843+0000 7f5c62a00640 1 -- 172.21.15.42:0/1043760710 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54010650 con 0x7f5c6c076490 2024-04-20T10:45:35.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.843+0000 7f5c6b200640 1 -- 172.21.15.42:0/1043760710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6c1ab7b0 con 0x7f5c6c076490 2024-04-20T10:45:35.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.844+0000 7f5c6b200640 1 -- 172.21.15.42:0/1043760710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6c0690f0 con 0x7f5c6c076490 2024-04-20T10:45:35.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.848+0000 7f5c62a00640 1 -- 172.21.15.42:0/1043760710 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5c540037a0 con 0x7f5c6c076490 2024-04-20T10:45:35.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.849+0000 7f5c62a00640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5c500a3d20 0x7f5c500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:35.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.849+0000 7f5c62a00640 1 -- 172.21.15.42:0/1043760710 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f5c540d0cb0 con 0x7f5c6c076490 2024-04-20T10:45:35.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.849+0000 7f5c69400640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5c500a3d20 0x7f5c500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:35.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.850+0000 7f5c62a00640 1 -- 172.21.15.42:0/1043760710 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5c5409d350 con 0x7f5c6c076490 2024-04-20T10:45:35.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:35.850+0000 7f5c69400640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5c500a3d20 0x7f5c500a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c02c180 tx=0x7f5c5c0056d0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.066+0000 7f5c6b200640 1 -- 172.21.15.42:0/1043760710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c6c19ffd0 con 0x7f5c6c076490 2024-04-20T10:45:36.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.068+0000 7f5c62a00640 1 -- 172.21.15.42:0/1043760710 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+23190 (secure 0 0 0) 0x7f5c5409cc50 con 0x7f5c6c076490 2024-04-20T10:45:36.069 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:45:36.070 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":98,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:45:09.043030+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/230562472":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/1321607472":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/3804122415":"2024-04-21T10:34:00.299359+0000","172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/3610568749":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/869471821":"2024-04-21T10:34:00.299359+0000","172.21.15.42:6800/3610568749":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:45:36.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.071+0000 7f5c6b200640 1 -- 172.21.15.42:0/1043760710 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5c500a3d20 msgr2=0x7f5c500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:36.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.071+0000 7f5c6b200640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5c500a3d20 0x7f5c500a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c02c180 tx=0x7f5c5c0056d0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.071+0000 7f5c6b200640 1 -- 172.21.15.42:0/1043760710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c076490 msgr2=0x7f5c6c1a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.071+0000 7f5c6b200640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c076490 0x7f5c6c1a48f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400e9c0 tx=0x7f5c5400ee80 comp rx=0 tx=0).stop 2024-04-20T10:45:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.071+0000 7f5c6b200640 1 -- 172.21.15.42:0/1043760710 shutdown_connections 2024-04-20T10:45:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.071+0000 7f5c6b200640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f5c500a3d20 0x7f5c500a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.071+0000 7f5c6b200640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c6c112c00 0x7f5c6c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.071+0000 7f5c6b200640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c6c076db0 0x7f5c6c1a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.071+0000 7f5c6b200640 1 --2- 172.21.15.42:0/1043760710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c6c076490 0x7f5c6c1a48f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.071+0000 7f5c6b200640 1 -- 172.21.15.42:0/1043760710 >> 172.21.15.42:0/1043760710 conn(0x7f5c6c0ff9f0 msgr2=0x7f5c6c114220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:36.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.072+0000 7f5c6b200640 1 -- 172.21.15.42:0/1043760710 shutdown_connections 2024-04-20T10:45:36.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.072+0000 7f5c6b200640 1 -- 172.21.15.42:0/1043760710 wait complete. 2024-04-20T10:45:36.093 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:26:16.537072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:31:33.523692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:31:34.019519+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:45:36.094 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:45:36.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:35 smithi046 ceph-mon[21574]: pgmap v360: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 164 KiB/s wr, 69 op/s 2024-04-20T10:45:36.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.318+0000 7faf53200640 1 -- 172.21.15.42:0/3670895850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf5410e560 msgr2=0x7faf541109b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.318+0000 7faf53200640 1 --2- 172.21.15.42:0/3670895850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf5410e560 0x7faf541109b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009900 tx=0x7faf4c02bcd0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.319+0000 7faf53200640 1 -- 172.21.15.42:0/3670895850 shutdown_connections 2024-04-20T10:45:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.319+0000 7faf53200640 1 --2- 172.21.15.42:0/3670895850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf5410e560 0x7faf541109b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.319+0000 7faf53200640 1 --2- 172.21.15.42:0/3670895850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf54074cb0 0x7faf5410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.319+0000 7faf53200640 1 --2- 172.21.15.42:0/3670895850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf54074390 0x7faf54074770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.319+0000 7faf53200640 1 -- 172.21.15.42:0/3670895850 >> 172.21.15.42:0/3670895850 conn(0x7faf540fd830 msgr2=0x7faf540ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:36.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.319+0000 7faf53200640 1 -- 172.21.15.42:0/3670895850 shutdown_connections 2024-04-20T10:45:36.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.319+0000 7faf53200640 1 -- 172.21.15.42:0/3670895850 wait complete. 2024-04-20T10:45:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.320+0000 7faf53200640 1 Processor -- start 2024-04-20T10:45:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.320+0000 7faf53200640 1 -- start start 2024-04-20T10:45:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.320+0000 7faf53200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf54074390 0x7faf541a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.320+0000 7faf53200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf54074cb0 0x7faf541a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.320+0000 7faf53200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf5410e560 0x7faf5419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.320+0000 7faf53200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faf541142e0 con 0x7faf5410e560 2024-04-20T10:45:36.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.320+0000 7faf53200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf54114160 con 0x7faf54074cb0 2024-04-20T10:45:36.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.320+0000 7faf53200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7faf54114460 con 0x7faf54074390 2024-04-20T10:45:36.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.321+0000 7faf51e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf54074390 0x7faf541a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:36.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.321+0000 7faf49400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf54074cb0 0x7faf541a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:36.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.321+0000 7faf52800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf5410e560 0x7faf5419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:36.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.321+0000 7faf51e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf54074390 0x7faf541a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:51700/0 (socket says 172.21.15.42:51700) 2024-04-20T10:45:36.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.321+0000 7faf51e00640 1 -- 172.21.15.42:0/990503374 learned_addr learned my addr 172.21.15.42:0/990503374 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:36.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.321+0000 7faf51e00640 1 -- 172.21.15.42:0/990503374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf54074cb0 msgr2=0x7faf541a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:36.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.321+0000 7faf51e00640 1 --2- 172.21.15.42:0/990503374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf54074cb0 0x7faf541a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.321+0000 7faf51e00640 1 -- 172.21.15.42:0/990503374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf5410e560 msgr2=0x7faf5419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:36.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.321+0000 7faf51e00640 1 --2- 172.21.15.42:0/990503374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf5410e560 0x7faf5419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.321+0000 7faf51e00640 1 -- 172.21.15.42:0/990503374 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5419af00 con 0x7faf54074390 2024-04-20T10:45:36.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.321+0000 7faf51e00640 1 --2- 172.21.15.42:0/990503374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf54074390 0x7faf541a0580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf4400ebc0 tx=0x7faf4400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:36.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.322+0000 7faf4b400640 1 -- 172.21.15.42:0/990503374 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44010630 con 0x7faf54074390 2024-04-20T10:45:36.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.322+0000 7faf53200640 1 -- 172.21.15.42:0/990503374 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5419b1f0 con 0x7faf54074390 2024-04-20T10:45:36.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.322+0000 7faf4b400640 1 -- 172.21.15.42:0/990503374 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf44010cb0 con 0x7faf54074390 2024-04-20T10:45:36.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.322+0000 7faf4b400640 1 -- 172.21.15.42:0/990503374 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44018bd0 con 0x7faf54074390 2024-04-20T10:45:36.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.322+0000 7faf53200640 1 -- 172.21.15.42:0/990503374 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf541a7500 con 0x7faf54074390 2024-04-20T10:45:36.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.323+0000 7faf53200640 1 -- 172.21.15.42:0/990503374 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf54075a40 con 0x7faf54074390 2024-04-20T10:45:36.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.325+0000 7faf4b400640 1 -- 172.21.15.42:0/990503374 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7faf440107d0 con 0x7faf54074390 2024-04-20T10:45:36.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.326+0000 7faf4b400640 1 --2- 172.21.15.42:0/990503374 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7faf380a3d20 0x7faf380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:36.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.327+0000 7faf4b400640 1 -- 172.21.15.42:0/990503374 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7faf440d0e80 con 0x7faf54074390 2024-04-20T10:45:36.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.327+0000 7faf49400640 1 --2- 172.21.15.42:0/990503374 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7faf380a3d20 0x7faf380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:36.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.327+0000 7faf49400640 1 --2- 172.21.15.42:0/990503374 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7faf380a3d20 0x7faf380a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf5419be90 tx=0x7faf3c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:36.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.331+0000 7faf4b400640 1 -- 172.21.15.42:0/990503374 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7faf4409ce40 con 0x7faf54074390 2024-04-20T10:45:36.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:35 smithi042 ceph-mon[17078]: pgmap v360: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 164 KiB/s wr, 69 op/s 2024-04-20T10:45:36.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:35 smithi162 ceph-mon[21854]: pgmap v360: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 164 KiB/s wr, 69 op/s 2024-04-20T10:45:36.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.540+0000 7faf53200640 1 -- 172.21.15.42:0/990503374 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf541096e0 con 0x7faf54074390 2024-04-20T10:45:36.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.541+0000 7faf4b400640 1 -- 172.21.15.42:0/990503374 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7faf440a1140 con 0x7faf54074390 2024-04-20T10:45:36.542 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-20T10:45:36.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.544+0000 7faf53200640 1 -- 172.21.15.42:0/990503374 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7faf380a3d20 msgr2=0x7faf380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:36.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.544+0000 7faf53200640 1 --2- 172.21.15.42:0/990503374 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7faf380a3d20 0x7faf380a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf5419be90 tx=0x7faf3c005f90 comp rx=0 tx=0).stop 2024-04-20T10:45:36.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.544+0000 7faf53200640 1 -- 172.21.15.42:0/990503374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf54074390 msgr2=0x7faf541a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:36.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.544+0000 7faf53200640 1 --2- 172.21.15.42:0/990503374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf54074390 0x7faf541a0580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf4400ebc0 tx=0x7faf4400c6a0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.544+0000 7faf53200640 1 -- 172.21.15.42:0/990503374 shutdown_connections 2024-04-20T10:45:36.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.544+0000 7faf53200640 1 --2- 172.21.15.42:0/990503374 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7faf380a3d20 0x7faf380a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.544+0000 7faf53200640 1 --2- 172.21.15.42:0/990503374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf5410e560 0x7faf5419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.544+0000 7faf53200640 1 --2- 172.21.15.42:0/990503374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf54074cb0 0x7faf541a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.544+0000 7faf53200640 1 --2- 172.21.15.42:0/990503374 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf54074390 0x7faf541a0580 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.544+0000 7faf53200640 1 -- 172.21.15.42:0/990503374 >> 172.21.15.42:0/990503374 conn(0x7faf540fd830 msgr2=0x7faf540fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:36.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.544+0000 7faf53200640 1 -- 172.21.15.42:0/990503374 shutdown_connections 2024-04-20T10:45:36.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.545+0000 7faf53200640 1 -- 172.21.15.42:0/990503374 wait complete. 2024-04-20T10:45:36.566 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:45:36.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.789+0000 7f8fb9c00640 1 -- 172.21.15.42:0/39480017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fb4102b50 msgr2=0x7f8fb4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:36.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.789+0000 7f8fb9c00640 1 --2- 172.21.15.42:0/39480017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fb4102b50 0x7f8fb4106b90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009be0 tx=0x7f8fa402bef0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.789+0000 7f8fb9c00640 1 -- 172.21.15.42:0/39480017 shutdown_connections 2024-04-20T10:45:36.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.789+0000 7f8fb9c00640 1 --2- 172.21.15.42:0/39480017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb4107270 0x7f8fb410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.789+0000 7f8fb9c00640 1 --2- 172.21.15.42:0/39480017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fb4102b50 0x7f8fb4106b90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.789+0000 7f8fb9c00640 1 --2- 172.21.15.42:0/39480017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fb41021a0 0x7f8fb4102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.789+0000 7f8fb9c00640 1 -- 172.21.15.42:0/39480017 >> 172.21.15.42:0/39480017 conn(0x7f8fb40793b0 msgr2=0x7f8fb40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:36.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.789+0000 7f8fb9c00640 1 -- 172.21.15.42:0/39480017 shutdown_connections 2024-04-20T10:45:36.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.789+0000 7f8fb9c00640 1 -- 172.21.15.42:0/39480017 wait complete. 2024-04-20T10:45:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.790+0000 7f8fb9c00640 1 Processor -- start 2024-04-20T10:45:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.790+0000 7f8fb9c00640 1 -- start start 2024-04-20T10:45:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb9c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fb41021a0 0x7f8fb419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:36.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb4107270 0x7f8fb419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:36.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fb4198660 0x7f8fb4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:36.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb9c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb41122e0 con 0x7f8fb4107270 2024-04-20T10:45:36.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb4112160 con 0x7f8fb4198660 2024-04-20T10:45:36.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb9c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb4112460 con 0x7f8fb41021a0 2024-04-20T10:45:36.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb4107270 0x7f8fb419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:36.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb4107270 0x7f8fb419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43538/0 (socket says 172.21.15.42:43538) 2024-04-20T10:45:36.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb2000640 1 -- 172.21.15.42:0/402704261 learned_addr learned my addr 172.21.15.42:0/402704261 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:36.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb2a00640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fb41021a0 0x7f8fb419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:36.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb3400640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fb4198660 0x7f8fb4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:36.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb2000640 1 -- 172.21.15.42:0/402704261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fb41021a0 msgr2=0x7f8fb419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:36.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.791+0000 7f8fb2000640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fb41021a0 0x7f8fb419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.792+0000 7f8fb2000640 1 -- 172.21.15.42:0/402704261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fb4198660 msgr2=0x7f8fb4198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:36.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.792+0000 7f8fb2000640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fb4198660 0x7f8fb4198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:36.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.792+0000 7f8fb2000640 1 -- 172.21.15.42:0/402704261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb4199260 con 0x7f8fb4107270 2024-04-20T10:45:36.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.792+0000 7f8fb3400640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fb4198660 0x7f8fb4198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:36.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.792+0000 7f8fb2000640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb4107270 0x7f8fb419eae0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4000f80 tx=0x7f8fa4033730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:36.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.792+0000 7f8fab400640 1 -- 172.21.15.42:0/402704261 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4043020 con 0x7f8fb4107270 2024-04-20T10:45:36.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.792+0000 7f8fb9c00640 1 -- 172.21.15.42:0/402704261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb41994f0 con 0x7f8fb4107270 2024-04-20T10:45:36.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.792+0000 7f8fab400640 1 -- 172.21.15.42:0/402704261 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa402ca60 con 0x7f8fb4107270 2024-04-20T10:45:36.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.792+0000 7f8fab400640 1 -- 172.21.15.42:0/402704261 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa403e6b0 con 0x7f8fb4107270 2024-04-20T10:45:36.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.792+0000 7f8fb9c00640 1 -- 172.21.15.42:0/402704261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb41a5690 con 0x7f8fb4107270 2024-04-20T10:45:36.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.794+0000 7f8fb9c00640 1 -- 172.21.15.42:0/402704261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb4103670 con 0x7f8fb4107270 2024-04-20T10:45:36.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.796+0000 7f8fab400640 1 -- 172.21.15.42:0/402704261 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f8fa4046020 con 0x7f8fb4107270 2024-04-20T10:45:36.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.797+0000 7f8fab400640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8f980a3d20 0x7f8f980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:36.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.797+0000 7f8fab400640 1 -- 172.21.15.42:0/402704261 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f8fa40f2ce0 con 0x7f8fb4107270 2024-04-20T10:45:36.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.797+0000 7f8fb2a00640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8f980a3d20 0x7f8f980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:36.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.798+0000 7f8fb2a00640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8f980a3d20 0x7f8f980a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009bd0 tx=0x7f8f9c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:36.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:36.802+0000 7f8fab400640 1 -- 172.21.15.42:0/402704261 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8fa404d020 con 0x7f8fb4107270 2024-04-20T10:45:37.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.011+0000 7f8fb9c00640 1 -- 172.21.15.42:0/402704261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8fb4102b50 con 0x7f8fb4107270 2024-04-20T10:45:37.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.012+0000 7f8fab400640 1 -- 172.21.15.42:0/402704261 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7f8fa40bf400 con 0x7f8fb4107270 2024-04-20T10:45:37.012 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-20T10:45:37.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.014+0000 7f8fb9c00640 1 -- 172.21.15.42:0/402704261 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8f980a3d20 msgr2=0x7f8f980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.014+0000 7f8fb9c00640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8f980a3d20 0x7f8f980a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009bd0 tx=0x7f8f9c005f90 comp rx=0 tx=0).stop 2024-04-20T10:45:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.014+0000 7f8fb9c00640 1 -- 172.21.15.42:0/402704261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb4107270 msgr2=0x7f8fb419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.014+0000 7f8fb9c00640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb4107270 0x7f8fb419eae0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4000f80 tx=0x7f8fa4033730 comp rx=0 tx=0).stop 2024-04-20T10:45:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.015+0000 7f8fb9c00640 1 -- 172.21.15.42:0/402704261 shutdown_connections 2024-04-20T10:45:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.015+0000 7f8fb9c00640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f8f980a3d20 0x7f8f980a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.015+0000 7f8fb9c00640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fb4198660 0x7f8fb4198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.015+0000 7f8fb9c00640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fb4107270 0x7f8fb419eae0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.015+0000 7f8fb9c00640 1 --2- 172.21.15.42:0/402704261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fb41021a0 0x7f8fb419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.015+0000 7f8fb9c00640 1 -- 172.21.15.42:0/402704261 >> 172.21.15.42:0/402704261 conn(0x7f8fb40793b0 msgr2=0x7f8fb40fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:37.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.015+0000 7f8fb9c00640 1 -- 172.21.15.42:0/402704261 shutdown_connections 2024-04-20T10:45:37.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.015+0000 7f8fb9c00640 1 -- 172.21.15.42:0/402704261 wait complete. 2024-04-20T10:45:37.035 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:45:37.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.242+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1254705277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8ec101f70 msgr2=0x7fb8ec102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:37.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.242+0000 7fb8f0c00640 1 --2- 172.21.15.42:0/1254705277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8ec101f70 0x7fb8ec102350 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009ca0 tx=0x7fb8dc02c100 comp rx=0 tx=0).stop 2024-04-20T10:45:37.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.242+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1254705277 shutdown_connections 2024-04-20T10:45:37.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.242+0000 7fb8f0c00640 1 --2- 172.21.15.42:0/1254705277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec107040 0x7fb8ec10ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.242+0000 7fb8f0c00640 1 --2- 172.21.15.42:0/1254705277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8ec102920 0x7fb8ec106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.242+0000 7fb8f0c00640 1 --2- 172.21.15.42:0/1254705277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8ec101f70 0x7fb8ec102350 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.242+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1254705277 >> 172.21.15.42:0/1254705277 conn(0x7fb8ec0fd830 msgr2=0x7fb8ec0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:37.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.243+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1254705277 shutdown_connections 2024-04-20T10:45:37.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.243+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1254705277 wait complete. 2024-04-20T10:45:37.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.244+0000 7fb8f0c00640 1 Processor -- start 2024-04-20T10:45:37.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.244+0000 7fb8f0c00640 1 -- start start 2024-04-20T10:45:37.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.244+0000 7fb8f0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8ec101f70 0x7fb8ec108c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:37.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.244+0000 7fb8f0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8ec107040 0x7fb8ec109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:37.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.244+0000 7fb8f0c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec1096b0 0x7fb8ec1aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:37.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.244+0000 7fb8f0c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec1120f0 con 0x7fb8ec101f70 2024-04-20T10:45:37.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.244+0000 7fb8f0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec111f70 con 0x7fb8ec107040 2024-04-20T10:45:37.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.244+0000 7fb8f0c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec112270 con 0x7fb8ec1096b0 2024-04-20T10:45:37.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.244+0000 7fb8eb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8ec101f70 0x7fb8ec108c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:37.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.244+0000 7fb8eb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8ec101f70 0x7fb8ec108c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43560/0 (socket says 172.21.15.42:43560) 2024-04-20T10:45:37.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.244+0000 7fb8eb400640 1 -- 172.21.15.42:0/1762176653 learned_addr learned my addr 172.21.15.42:0/1762176653 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:37.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.245+0000 7fb8e2a00640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8ec107040 0x7fb8ec109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:37.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.245+0000 7fb8ebe00640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec1096b0 0x7fb8ec1aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:37.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.245+0000 7fb8eb400640 1 -- 172.21.15.42:0/1762176653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec1096b0 msgr2=0x7fb8ec1aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:37.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.245+0000 7fb8eb400640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec1096b0 0x7fb8ec1aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.245+0000 7fb8eb400640 1 -- 172.21.15.42:0/1762176653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8ec107040 msgr2=0x7fb8ec109170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:37.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.245+0000 7fb8eb400640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8ec107040 0x7fb8ec109170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.245+0000 7fb8eb400640 1 -- 172.21.15.42:0/1762176653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ec1ab2d0 con 0x7fb8ec101f70 2024-04-20T10:45:37.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.245+0000 7fb8eb400640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8ec101f70 0x7fb8ec108c30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005e30 tx=0x7fb8dc032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:37.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.245+0000 7fb8e8c00640 1 -- 172.21.15.42:0/1762176653 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc039040 con 0x7fb8ec101f70 2024-04-20T10:45:37.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.246+0000 7fb8e8c00640 1 -- 172.21.15.42:0/1762176653 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8dc044070 con 0x7fb8ec101f70 2024-04-20T10:45:37.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.246+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1762176653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ec1ab560 con 0x7fb8ec101f70 2024-04-20T10:45:37.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.246+0000 7fb8e8c00640 1 -- 172.21.15.42:0/1762176653 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0042a0 con 0x7fb8ec101f70 2024-04-20T10:45:37.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.246+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1762176653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ec1aba70 con 0x7fb8ec101f70 2024-04-20T10:45:37.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.247+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1762176653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8ec103440 con 0x7fb8ec101f70 2024-04-20T10:45:37.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.249+0000 7fb8e8c00640 1 -- 172.21.15.42:0/1762176653 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb8dc003970 con 0x7fb8ec101f70 2024-04-20T10:45:37.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.250+0000 7fb8e8c00640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb8d00a3da0 0x7fb8d00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:37.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.251+0000 7fb8e8c00640 1 -- 172.21.15.42:0/1762176653 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fb8dc0f2fb0 con 0x7fb8ec101f70 2024-04-20T10:45:37.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.251+0000 7fb8e2a00640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb8d00a3da0 0x7fb8d00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:37.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.252+0000 7fb8e2a00640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb8d00a3da0 0x7fb8d00a6260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40095f0 tx=0x7fb8d4005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:37.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.253+0000 7fb8e8c00640 1 -- 172.21.15.42:0/1762176653 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb8dc04e020 con 0x7fb8ec101f70 2024-04-20T10:45:37.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:36 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1043760710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:45:37.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:36 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/990503374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:45:37.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:36 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1043760710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:45:37.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:36 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/990503374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:45:37.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:36 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1043760710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:45:37.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:36 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/990503374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:45:37.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.461+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1762176653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb8ec102950 con 0x7fb8ec101f70 2024-04-20T10:45:37.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.462+0000 7fb8e8c00640 1 -- 172.21.15.42:0/1762176653 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7fb8dc0bf6d0 con 0x7fb8ec101f70 2024-04-20T10:45:37.463 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-20T10:45:37.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.464+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1762176653 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb8d00a3da0 msgr2=0x7fb8d00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:37.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.464+0000 7fb8f0c00640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb8d00a3da0 0x7fb8d00a6260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40095f0 tx=0x7fb8d4005f90 comp rx=0 tx=0).stop 2024-04-20T10:45:37.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.464+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1762176653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8ec101f70 msgr2=0x7fb8ec108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:37.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.464+0000 7fb8f0c00640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8ec101f70 0x7fb8ec108c30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005e30 tx=0x7fb8dc032f40 comp rx=0 tx=0).stop 2024-04-20T10:45:37.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.465+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1762176653 shutdown_connections 2024-04-20T10:45:37.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.465+0000 7fb8f0c00640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fb8d00a3da0 0x7fb8d00a6260 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.465+0000 7fb8f0c00640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8ec1096b0 0x7fb8ec1aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.465+0000 7fb8f0c00640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8ec107040 0x7fb8ec109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.465+0000 7fb8f0c00640 1 --2- 172.21.15.42:0/1762176653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8ec101f70 0x7fb8ec108c30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.465+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1762176653 >> 172.21.15.42:0/1762176653 conn(0x7fb8ec0fd830 msgr2=0x7fb8ec0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:37.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.465+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1762176653 shutdown_connections 2024-04-20T10:45:37.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.465+0000 7fb8f0c00640 1 -- 172.21.15.42:0/1762176653 wait complete. 2024-04-20T10:45:37.486 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-20T10:45:37.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.698+0000 7f06bb600640 1 -- 172.21.15.42:0/4251999341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b4108f90 msgr2=0x7f06b41154a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:37.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.698+0000 7f06bb600640 1 --2- 172.21.15.42:0/4251999341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b4108f90 0x7f06b41154a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009900 tx=0x7f06a402bc40 comp rx=0 tx=0).stop 2024-04-20T10:45:37.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.699+0000 7f06bb600640 1 -- 172.21.15.42:0/4251999341 shutdown_connections 2024-04-20T10:45:37.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.699+0000 7f06bb600640 1 --2- 172.21.15.42:0/4251999341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b4108f90 0x7f06b41154a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.699+0000 7f06bb600640 1 --2- 172.21.15.42:0/4251999341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06b41085d0 0x7f06b4108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.699+0000 7f06bb600640 1 --2- 172.21.15.42:0/4251999341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b410e670 0x7f06b410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.699+0000 7f06bb600640 1 -- 172.21.15.42:0/4251999341 >> 172.21.15.42:0/4251999341 conn(0x7f06b4076580 msgr2=0x7f06b40789a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:37.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.699+0000 7f06bb600640 1 -- 172.21.15.42:0/4251999341 shutdown_connections 2024-04-20T10:45:37.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.699+0000 7f06bb600640 1 -- 172.21.15.42:0/4251999341 wait complete. 2024-04-20T10:45:37.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.700+0000 7f06bb600640 1 Processor -- start 2024-04-20T10:45:37.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.700+0000 7f06bb600640 1 -- start start 2024-04-20T10:45:37.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.700+0000 7f06bb600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b41085d0 0x7f06b419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:37.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.700+0000 7f06bb600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4108f90 0x7f06b419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:37.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.700+0000 7f06bb600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06b410e670 0x7f06b41a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:37.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.700+0000 7f06bb600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f06b4118a00 con 0x7f06b410e670 2024-04-20T10:45:37.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.700+0000 7f06bb600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f06b4118880 con 0x7f06b41085d0 2024-04-20T10:45:37.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.700+0000 7f06bb600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f06b4118b80 con 0x7f06b4108f90 2024-04-20T10:45:37.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06b410e670 0x7f06b41a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:37.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b41085d0 0x7f06b419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:37.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4108f90 0x7f06b419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:37.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06b410e670 0x7f06b41a36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43576/0 (socket says 172.21.15.42:43576) 2024-04-20T10:45:37.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b9600640 1 -- 172.21.15.42:0/3688765759 learned_addr learned my addr 172.21.15.42:0/3688765759 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:45:37.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b41085d0 0x7f06b419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:59800/0 (socket says 172.21.15.42:59800) 2024-04-20T10:45:37.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4108f90 0x7f06b419f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:51756/0 (socket says 172.21.15.42:51756) 2024-04-20T10:45:37.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b9600640 1 -- 172.21.15.42:0/3688765759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4108f90 msgr2=0x7f06b419f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:37.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b9600640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4108f90 0x7f06b419f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b9600640 1 -- 172.21.15.42:0/3688765759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b41085d0 msgr2=0x7f06b419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:37.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b9600640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b41085d0 0x7f06b419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:37.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b9600640 1 -- 172.21.15.42:0/3688765759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b41a3dd0 con 0x7f06b410e670 2024-04-20T10:45:37.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.701+0000 7f06b8c00640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b41085d0 0x7f06b419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:45:37.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.702+0000 7f06b9600640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06b410e670 0x7f06b41a36a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f06a402c180 tx=0x7f06a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:37.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.702+0000 7f06b1600640 1 -- 172.21.15.42:0/3688765759 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4041070 con 0x7f06b410e670 2024-04-20T10:45:37.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.702+0000 7f06bb600640 1 -- 172.21.15.42:0/3688765759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b41a4060 con 0x7f06b410e670 2024-04-20T10:45:37.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.702+0000 7f06b1600640 1 -- 172.21.15.42:0/3688765759 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06a402ccd0 con 0x7f06b410e670 2024-04-20T10:45:37.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.702+0000 7f06b1600640 1 -- 172.21.15.42:0/3688765759 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4003d00 con 0x7f06b410e670 2024-04-20T10:45:37.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.702+0000 7f06bb600640 1 -- 172.21.15.42:0/3688765759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b41ab950 con 0x7f06b410e670 2024-04-20T10:45:37.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.703+0000 7f06bb600640 1 -- 172.21.15.42:0/3688765759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b4069240 con 0x7f06b410e670 2024-04-20T10:45:37.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.705+0000 7f06b1600640 1 -- 172.21.15.42:0/3688765759 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f06a4044070 con 0x7f06b410e670 2024-04-20T10:45:37.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.707+0000 7f06b1600640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f06980a3d20 0x7f06980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:45:37.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.707+0000 7f06b1600640 1 -- 172.21.15.42:0/3688765759 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f06a40f2560 con 0x7f06b410e670 2024-04-20T10:45:37.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.707+0000 7f06b8c00640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f06980a3d20 0x7f06980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:45:37.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.708+0000 7f06b8c00640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f06980a3d20 0x7f06980a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f06b4109710 tx=0x7f06ac009500 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:45:37.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.712+0000 7f06b1600640 1 -- 172.21.15.42:0/3688765759 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f06a40be630 con 0x7f06b410e670 2024-04-20T10:45:37.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.966+0000 7f06bb600640 1 -- 172.21.15.42:0/3688765759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f06b41abb20 con 0x7f06b410e670 2024-04-20T10:45:37.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:37.984+0000 7f06b1600640 1 -- 172.21.15.42:0/3688765759 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+746058 (secure 0 0 0) 0x7f06a40c2930 con 0x7f06b410e670 2024-04-20T10:45:37.988 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:45:37.989 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:45:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 27, 2024-04-20T10:45:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "active_gid": 15024, 2024-04-20T10:45:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "active_name": "x", 2024-04-20T10:45:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "active_addrs": { 2024-04-20T10:45:37.990 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-04-20T10:45:37.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:37.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-04-20T10:45:37.990 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-04-20T10:45:37.990 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 170978916 2024-04-20T10:45:37.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:37.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:37.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-04-20T10:45:37.991 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-04-20T10:45:37.991 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 170978916 2024-04-20T10:45:37.991 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:37.991 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-20T10:45:37.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:37.991 INFO:teuthology.orchestra.run.smithi042.stdout: "active_addr": "172.21.15.42:6801/170978916", 2024-04-20T10:45:37.991 INFO:teuthology.orchestra.run.smithi042.stdout: "active_change": "2024-04-20T10:34:00.299732+0000", 2024-04-20T10:45:37.991 INFO:teuthology.orchestra.run.smithi042.stdout: "active_mgr_features": 4540701547738038271, 2024-04-20T10:45:37.991 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-04-20T10:45:37.992 INFO:teuthology.orchestra.run.smithi042.stdout: "standbys": [ 2024-04-20T10:45:37.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:37.992 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 25133, 2024-04-20T10:45:37.992 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "y", 2024-04-20T10:45:37.992 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_features": 4540701547738038271, 2024-04-20T10:45:37.992 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-04-20T10:45:37.992 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:37.992 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-04-20T10:45:37.993 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:37.993 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:37.993 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:37.993 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:37.993 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:37.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:37.993 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:37.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:37.994 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:37.994 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:37.994 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:37.994 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:37.994 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T10:45:37.994 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:37.994 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:37.994 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:37.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:37.995 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:37.995 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:37.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:37.995 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:37.995 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:37.995 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:37.995 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:37.995 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:37.996 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:37.996 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:37.996 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:37.996 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:37.996 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:37.996 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:37.996 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:37.996 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:37.997 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:37.997 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:37.997 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:37.997 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:37.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:37.997 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:37.997 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:37.997 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:37.997 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:37.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:37.998 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:37.998 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:37.998 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:37.998 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:37.998 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:37.998 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:37.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:37.999 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:37.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:37.999 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:37.999 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:37.999 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:37.999 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:37.999 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:37.999 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:37.999 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.000 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.000 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.000 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.000 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.000 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.000 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.000 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.000 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.000 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.001 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.001 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.001 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.001 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.001 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.001 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.001 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.001 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.002 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.002 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.002 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.002 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.002 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.002 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.002 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.002 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-04-20T10:45:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-04-20T10:45:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.003 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.004 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.004 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.004 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-04-20T10:45:38.004 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.004 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.004 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-04-20T10:45:38.004 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-04-20T10:45:38.005 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.005 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.005 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-04-20T10:45:38.005 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.005 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.005 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.005 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-04-20T10:45:38.006 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.006 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.006 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.006 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-04-20T10:45:38.006 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-04-20T10:45:38.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.006 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.006 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.007 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.007 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.007 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.007 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.007 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-04-20T10:45:38.007 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.007 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.007 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.008 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-04-20T10:45:38.008 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-04-20T10:45:38.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.008 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.008 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.008 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.008 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.008 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.009 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-04-20T10:45:38.009 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.009 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.009 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-04-20T10:45:38.009 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-04-20T10:45:38.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.009 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.010 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-04-20T10:45:38.010 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.010 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.010 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.010 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-04-20T10:45:38.010 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.010 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.011 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.011 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-04-20T10:45:38.011 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-04-20T10:45:38.011 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.011 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.011 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.011 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.011 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.012 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.012 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.012 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-04-20T10:45:38.012 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.012 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.012 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.012 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-04-20T10:45:38.013 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-04-20T10:45:38.013 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.013 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.013 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.013 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.014 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.014 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.014 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T10:45:38.014 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.014 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.014 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.014 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-04-20T10:45:38.014 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-04-20T10:45:38.015 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.015 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.015 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.015 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.015 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.015 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.015 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.015 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-04-20T10:45:38.015 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.016 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.016 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.016 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.016 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-04-20T10:45:38.016 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.017 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.017 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.017 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-04-20T10:45:38.017 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-04-20T10:45:38.017 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.017 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.017 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.017 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.018 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.018 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.018 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T10:45:38.018 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.018 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.018 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.018 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-04-20T10:45:38.019 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-04-20T10:45:38.019 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.019 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.019 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.019 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-04-20T10:45:38.019 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.019 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.019 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.019 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T10:45:38.020 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T10:45:38.020 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.020 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.020 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-04-20T10:45:38.020 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-04-20T10:45:38.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:38.020 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.021 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.021 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:38.021 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-04-20T10:45:38.021 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-04-20T10:45:38.021 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.021 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T10:45:38.021 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T10:45:38.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.022 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.022 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-04-20T10:45:38.022 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-04-20T10:45:38.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:38.022 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.022 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-04-20T10:45:38.023 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-20T10:45:38.023 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-04-20T10:45:38.023 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.023 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T10:45:38.023 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.023 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.023 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.023 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-04-20T10:45:38.024 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-04-20T10:45:38.024 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.024 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.024 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.024 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T10:45:38.024 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.024 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.024 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.024 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T10:45:38.025 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T10:45:38.025 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.025 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.025 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-04-20T10:45:38.025 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-04-20T10:45:38.025 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:38.025 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.026 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-04-20T10:45:38.026 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-04-20T10:45:38.026 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-04-20T10:45:38.026 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.026 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-04-20T10:45:38.026 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T10:45:38.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.027 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.027 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-04-20T10:45:38.027 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-04-20T10:45:38.027 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.027 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.027 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.027 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-04-20T10:45:38.028 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.028 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.028 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.028 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T10:45:38.028 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T10:45:38.028 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.028 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.029 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-04-20T10:45:38.029 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-04-20T10:45:38.029 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:38.029 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.029 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.029 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:38.029 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-04-20T10:45:38.029 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-04-20T10:45:38.029 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.030 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T10:45:38.030 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T10:45:38.030 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.030 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.030 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.030 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.030 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.031 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.031 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.031 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.031 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.031 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.031 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.031 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.032 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.032 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.032 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.032 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.032 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.032 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.032 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.032 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.033 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.033 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.033 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.033 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.033 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.033 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.033 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.034 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.034 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.034 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.034 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.034 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.034 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.034 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.034 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.035 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.035 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.035 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.035 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.035 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.035 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.035 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.036 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.036 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.036 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.036 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.036 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.036 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.036 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.036 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.036 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.037 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.037 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.037 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.037 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.037 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.037 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.037 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.038 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.038 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.038 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.038 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.038 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.038 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-04-20T10:45:38.038 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-04-20T10:45:38.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:38.039 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.039 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:38.039 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.039 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.039 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.039 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T10:45:38.039 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.040 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.040 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-04-20T10:45:38.040 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-04-20T10:45:38.040 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.040 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.040 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.040 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-04-20T10:45:38.041 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.041 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.041 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.041 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-04-20T10:45:38.041 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-04-20T10:45:38.041 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-04-20T10:45:38.041 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-04-20T10:45:38.041 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-04-20T10:45:38.041 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.042 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-04-20T10:45:38.042 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.042 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.042 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.042 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-04-20T10:45:38.042 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-04-20T10:45:38.042 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.042 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.043 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.043 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.043 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.043 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.043 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T10:45:38.043 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.043 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.044 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.044 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:38.044 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:38.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.044 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.044 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.044 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:38.044 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.045 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.045 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.045 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T10:45:38.045 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.045 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.045 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-04-20T10:45:38.046 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-04-20T10:45:38.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.046 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.046 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-20T10:45:38.046 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-20T10:45:38.046 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.046 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.046 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T10:45:38.047 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T10:45:38.047 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.047 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.047 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-04-20T10:45:38.047 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-04-20T10:45:38.047 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:38.047 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.047 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.049 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.049 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.049 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.049 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.050 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T10:45:38.050 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.050 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.050 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.050 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-04-20T10:45:38.051 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.051 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.051 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.051 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-04-20T10:45:38.051 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-04-20T10:45:38.051 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:38.051 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.051 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.051 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-04-20T10:45:38.052 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.052 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.052 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.052 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T10:45:38.052 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.052 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.052 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.053 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-04-20T10:45:38.053 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-04-20T10:45:38.053 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.053 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.053 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.053 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-04-20T10:45:38.053 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.053 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.053 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.054 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T10:45:38.054 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.054 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.054 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-04-20T10:45:38.054 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-04-20T10:45:38.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.055 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.055 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.055 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-04-20T10:45:38.055 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.055 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.055 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.055 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T10:45:38.055 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.056 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.056 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-04-20T10:45:38.056 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-04-20T10:45:38.056 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.056 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.056 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.056 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.056 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.057 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.057 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.057 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T10:45:38.057 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T10:45:38.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.057 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.057 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-04-20T10:45:38.057 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-04-20T10:45:38.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.058 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.058 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.058 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:38.058 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.058 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.058 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.058 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T10:45:38.059 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.059 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.059 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.059 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-04-20T10:45:38.059 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T10:45:38.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:38.059 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.060 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-04-20T10:45:38.060 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.060 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.060 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.060 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T10:45:38.060 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.060 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.060 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.061 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-04-20T10:45:38.061 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-04-20T10:45:38.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.061 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.061 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.061 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.061 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.061 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.062 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-04-20T10:45:38.062 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-04-20T10:45:38.062 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-04-20T10:45:38.062 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.062 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T10:45:38.062 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.062 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.062 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.063 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-04-20T10:45:38.063 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-04-20T10:45:38.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.063 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.063 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.063 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.063 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.063 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.064 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T10:45:38.064 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.064 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.064 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-04-20T10:45:38.064 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-04-20T10:45:38.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.065 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.065 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.065 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.065 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.065 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.065 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.065 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T10:45:38.065 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.065 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.066 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.066 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-04-20T10:45:38.066 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-04-20T10:45:38.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.066 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.066 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.066 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.067 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.067 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.067 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T10:45:38.067 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.067 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.067 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-04-20T10:45:38.068 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-04-20T10:45:38.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.068 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.068 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.068 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T10:45:38.068 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.068 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.068 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.068 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-20T10:45:38.069 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.069 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.069 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.069 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-04-20T10:45:38.069 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-04-20T10:45:38.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.069 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.070 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T10:45:38.070 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.070 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.070 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.070 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-04-20T10:45:38.070 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.070 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.070 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.071 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-04-20T10:45:38.071 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-04-20T10:45:38.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.071 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.071 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T10:45:38.071 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.072 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.072 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.072 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-04-20T10:45:38.072 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.072 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.072 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-04-20T10:45:38.072 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-04-20T10:45:38.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.073 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.073 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T10:45:38.073 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.073 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.073 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.073 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-20T10:45:38.074 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.074 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.074 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-04-20T10:45:38.074 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-04-20T10:45:38.074 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.074 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.075 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T10:45:38.075 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.075 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.075 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.075 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-04-20T10:45:38.075 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.076 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.076 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-04-20T10:45:38.076 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-04-20T10:45:38.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.076 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.076 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T10:45:38.077 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.077 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.077 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.077 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-04-20T10:45:38.077 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.077 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.077 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-04-20T10:45:38.078 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-04-20T10:45:38.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.078 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.078 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.078 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T10:45:38.078 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.079 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.079 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.079 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-04-20T10:45:38.079 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.079 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.079 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-04-20T10:45:38.079 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-04-20T10:45:38.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.080 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.080 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.080 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T10:45:38.080 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.080 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.080 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.080 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-04-20T10:45:38.081 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.081 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.081 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-04-20T10:45:38.081 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-04-20T10:45:38.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.081 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.082 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T10:45:38.082 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.082 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.082 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.082 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-04-20T10:45:38.082 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.083 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.083 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-04-20T10:45:38.083 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-04-20T10:45:38.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.083 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.083 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T10:45:38.084 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.084 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.084 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.084 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-04-20T10:45:38.084 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.085 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.085 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.085 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-04-20T10:45:38.085 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-04-20T10:45:38.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.085 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.086 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T10:45:38.086 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.086 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.086 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.086 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-20T10:45:38.086 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.086 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.087 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-04-20T10:45:38.087 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-04-20T10:45:38.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.087 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.087 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T10:45:38.087 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.088 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.088 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.088 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-04-20T10:45:38.088 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.088 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.088 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-04-20T10:45:38.089 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-04-20T10:45:38.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.089 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.089 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.089 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T10:45:38.089 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.089 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.089 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.090 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-20T10:45:38.090 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.090 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.090 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.090 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-04-20T10:45:38.090 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-04-20T10:45:38.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.090 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.091 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.091 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T10:45:38.091 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.091 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.091 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.091 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-04-20T10:45:38.091 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.092 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.092 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_samba": { 2024-04-20T10:45:38.092 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_samba", 2024-04-20T10:45:38.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.092 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.092 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T10:45:38.092 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.093 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.093 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.093 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Samba/SMB container image", 2024-04-20T10:45:38.093 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.093 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.093 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-04-20T10:45:38.094 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-04-20T10:45:38.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.094 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.094 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T10:45:38.094 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.094 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.094 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.094 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-04-20T10:45:38.095 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.095 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.095 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-04-20T10:45:38.095 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-04-20T10:45:38.095 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.095 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.095 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.096 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.096 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.096 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.096 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.096 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T10:45:38.096 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.096 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.096 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.097 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-04-20T10:45:38.097 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-04-20T10:45:38.097 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.097 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.097 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:38.097 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.098 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.098 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.098 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T10:45:38.098 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.098 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.098 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.098 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-04-20T10:45:38.098 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T10:45:38.099 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.099 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.099 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.099 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-04-20T10:45:38.099 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.099 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.099 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.099 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T10:45:38.100 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.100 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.100 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.100 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-04-20T10:45:38.100 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-04-20T10:45:38.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.100 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.101 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-04-20T10:45:38.101 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.101 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.101 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.101 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T10:45:38.101 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.101 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.102 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-04-20T10:45:38.102 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-04-20T10:45:38.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.102 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.102 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-04-20T10:45:38.102 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.102 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.103 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.103 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-04-20T10:45:38.103 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.103 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.103 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.103 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-04-20T10:45:38.103 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-04-20T10:45:38.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.104 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.104 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.104 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.104 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.104 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.104 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T10:45:38.104 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.105 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.105 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-04-20T10:45:38.105 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-04-20T10:45:38.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.105 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.105 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:38.106 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.106 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.106 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.106 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-04-20T10:45:38.106 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.106 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.106 INFO:teuthology.orchestra.run.smithi042.stdout: "grafana_dashboards_path": { 2024-04-20T10:45:38.107 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "grafana_dashboards_path", 2024-04-20T10:45:38.107 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.107 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.107 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.107 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T10:45:38.107 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.107 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.107 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.108 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T10:45:38.108 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.108 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.108 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.108 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-04-20T10:45:38.108 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-04-20T10:45:38.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.108 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.109 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:38.109 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.109 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.109 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.109 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-04-20T10:45:38.109 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.110 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.110 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-04-20T10:45:38.110 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-04-20T10:45:38.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.110 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.110 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.111 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.111 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.111 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.111 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T10:45:38.111 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.111 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.112 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-04-20T10:45:38.112 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-04-20T10:45:38.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.112 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.112 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.112 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.112 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.112 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.113 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T10:45:38.113 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.113 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.113 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.113 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.113 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.114 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.114 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.114 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.114 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.114 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.114 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.114 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.115 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.115 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.115 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.115 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.115 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.115 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.115 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.116 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-04-20T10:45:38.116 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-04-20T10:45:38.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.116 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.122 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.123 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.123 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.123 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.123 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T10:45:38.123 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.123 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.124 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.124 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.124 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.124 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.124 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.124 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.124 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.125 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T10:45:38.125 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.125 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.125 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.125 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.125 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.126 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.126 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.126 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.126 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.126 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.126 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.126 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.127 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.127 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.127 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.127 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.127 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.127 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.127 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.127 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.128 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.128 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.128 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.128 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.128 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.128 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.128 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.129 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.129 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.129 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.129 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T10:45:38.129 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T10:45:38.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.130 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.130 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.130 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.130 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.130 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.130 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T10:45:38.130 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.131 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.131 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T10:45:38.131 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T10:45:38.131 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.131 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.131 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.131 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-04-20T10:45:38.132 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.132 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.132 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.132 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T10:45:38.132 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.132 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.132 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.132 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-04-20T10:45:38.133 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-04-20T10:45:38.133 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.133 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.133 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.133 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.133 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.133 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.133 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.133 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-04-20T10:45:38.134 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.134 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.134 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.134 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-04-20T10:45:38.134 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-04-20T10:45:38.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.134 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.135 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.135 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.135 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.135 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.135 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T10:45:38.135 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.135 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.136 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-04-20T10:45:38.136 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-04-20T10:45:38.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.136 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.136 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.136 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.137 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.137 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.137 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-04-20T10:45:38.137 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.137 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.137 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.137 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-04-20T10:45:38.137 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-04-20T10:45:38.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.138 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.138 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-04-20T10:45:38.138 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.138 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.138 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.138 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-04-20T10:45:38.139 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-04-20T10:45:38.139 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.139 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T10:45:38.139 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.139 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.139 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-04-20T10:45:38.139 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-04-20T10:45:38.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.140 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.140 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-04-20T10:45:38.140 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.140 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.140 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.140 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T10:45:38.140 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.141 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.141 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.141 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-04-20T10:45:38.141 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-04-20T10:45:38.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.141 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.141 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T10:45:38.141 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.142 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.142 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.142 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T10:45:38.142 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.142 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.142 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.142 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-04-20T10:45:38.143 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-04-20T10:45:38.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.143 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.143 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.143 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.143 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.143 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.144 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T10:45:38.144 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.144 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.144 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-04-20T10:45:38.144 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-04-20T10:45:38.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.144 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.145 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.145 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.145 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.145 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.145 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T10:45:38.145 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.145 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.146 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.146 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-04-20T10:45:38.146 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-04-20T10:45:38.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.146 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.146 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.147 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.147 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.147 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.147 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T10:45:38.147 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.147 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.147 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.147 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-04-20T10:45:38.148 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-04-20T10:45:38.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.148 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.148 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.148 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.148 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.148 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.148 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T10:45:38.149 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.149 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.149 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.149 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-04-20T10:45:38.149 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-04-20T10:45:38.149 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.149 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.150 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.150 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.150 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.150 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.150 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T10:45:38.150 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.151 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.151 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-04-20T10:45:38.151 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-04-20T10:45:38.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.151 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.151 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-04-20T10:45:38.151 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.152 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.152 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.152 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-04-20T10:45:38.152 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.152 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.152 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-04-20T10:45:38.153 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-04-20T10:45:38.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.153 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.153 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.153 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.153 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.153 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.153 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T10:45:38.154 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.154 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.154 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.154 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-04-20T10:45:38.154 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-04-20T10:45:38.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.154 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.155 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.155 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.155 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.155 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.155 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T10:45:38.155 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.155 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.155 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.156 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-04-20T10:45:38.156 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-04-20T10:45:38.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.156 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.156 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.156 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.156 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.157 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.157 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T10:45:38.157 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.157 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.157 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-04-20T10:45:38.157 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-04-20T10:45:38.157 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.158 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.158 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.158 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.158 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.158 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.158 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T10:45:38.158 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.158 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.159 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.159 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-04-20T10:45:38.159 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-04-20T10:45:38.159 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.159 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.159 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.159 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T10:45:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-04-20T10:45:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-04-20T10:45:38.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.161 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.161 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.161 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.161 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.161 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.161 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T10:45:38.161 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.162 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.162 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.162 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.162 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-04-20T10:45:38.162 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.163 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.163 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.163 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.163 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.163 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.163 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.163 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.163 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.164 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.164 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.164 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.164 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.164 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.164 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.164 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.164 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.164 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.165 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.165 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.165 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.165 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.165 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.165 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.166 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.166 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.166 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.166 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.166 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.166 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.166 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.167 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.167 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.167 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.167 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.167 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.167 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.167 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.167 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.168 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.169 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.169 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.169 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.169 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.169 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.169 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.169 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.170 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.170 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.170 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.170 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.170 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.170 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.170 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.170 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.171 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-04-20T10:45:38.171 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-04-20T10:45:38.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.171 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.171 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.171 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-04-20T10:45:38.171 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.171 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.172 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.172 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T10:45:38.172 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.172 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.172 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.172 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-04-20T10:45:38.172 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-04-20T10:45:38.172 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.173 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.173 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-04-20T10:45:38.173 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.173 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.173 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.173 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T10:45:38.173 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.173 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.174 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.174 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.174 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-04-20T10:45:38.174 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.174 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.175 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.175 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T10:45:38.175 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T10:45:38.175 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.175 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.175 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.175 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.175 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.175 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.176 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.176 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.176 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.176 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.176 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.176 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T10:45:38.176 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T10:45:38.177 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.177 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.177 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.177 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.177 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.177 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.177 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.177 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.177 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.178 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.178 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T10:45:38.178 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T10:45:38.178 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.178 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.178 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.179 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.179 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.179 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.179 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.179 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.179 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.179 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.179 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.180 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-04-20T10:45:38.180 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T10:45:38.180 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.180 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.180 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.180 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.180 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.180 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.180 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.181 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.181 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.181 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.181 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.181 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T10:45:38.181 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T10:45:38.181 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.181 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.182 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.182 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.182 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.182 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.182 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.182 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.182 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.183 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T10:45:38.183 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T10:45:38.183 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.183 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.183 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.183 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.183 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.183 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.184 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.184 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.184 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.184 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.184 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T10:45:38.184 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T10:45:38.185 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.185 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.185 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.185 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.185 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.185 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.185 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.185 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.185 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.186 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.186 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.186 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T10:45:38.186 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T10:45:38.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.186 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.187 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.187 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.187 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.187 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.187 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.187 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.187 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.187 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.187 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.188 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T10:45:38.188 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T10:45:38.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.188 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.188 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.188 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.188 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.189 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.189 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.189 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.189 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.189 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.189 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T10:45:38.190 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T10:45:38.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.190 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.190 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.190 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.190 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.190 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.191 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.191 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.191 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.191 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.191 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T10:45:38.191 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T10:45:38.192 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.192 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.192 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.192 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.192 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.192 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.192 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.192 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.193 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.193 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.193 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.193 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T10:45:38.193 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T10:45:38.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.193 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.194 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.194 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.194 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.194 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.194 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.194 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.194 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.194 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.195 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.195 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T10:45:38.195 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T10:45:38.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.195 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.195 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.196 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.196 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.196 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.196 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.196 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.196 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.196 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.197 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T10:45:38.197 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T10:45:38.197 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.197 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.197 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.197 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.197 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.197 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.198 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.198 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.198 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.198 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.198 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.198 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T10:45:38.198 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T10:45:38.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.199 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.199 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-20T10:45:38.199 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.199 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.199 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.199 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.199 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.200 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.200 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T10:45:38.200 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T10:45:38.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.200 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.200 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.201 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.201 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.201 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.201 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.201 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.201 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.201 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.201 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.202 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-04-20T10:45:38.202 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-04-20T10:45:38.202 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.202 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.202 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.202 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.202 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.203 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.203 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.203 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.203 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.203 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.203 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.203 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T10:45:38.203 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T10:45:38.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.204 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.204 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-20T10:45:38.204 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.204 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.204 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.204 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.204 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.205 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.205 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T10:45:38.205 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T10:45:38.205 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.205 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.205 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.205 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.206 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.206 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.206 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.206 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.206 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.206 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.206 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.207 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T10:45:38.207 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T10:45:38.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.207 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.207 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.207 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.207 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.207 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.208 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.208 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.208 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.208 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.208 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.208 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T10:45:38.208 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T10:45:38.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.209 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.209 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.209 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.209 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.209 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.209 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.209 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.209 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.210 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.210 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T10:45:38.210 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T10:45:38.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.210 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.210 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.210 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.211 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.211 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.211 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.211 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.211 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.211 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.211 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.212 INFO:teuthology.orchestra.run.smithi042.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T10:45:38.212 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T10:45:38.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.212 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.212 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "[]", 2024-04-20T10:45:38.212 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.212 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.212 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.213 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.213 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.213 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.213 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.213 INFO:teuthology.orchestra.run.smithi042.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T10:45:38.213 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T10:45:38.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.214 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.214 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.214 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "{}", 2024-04-20T10:45:38.214 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.214 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.214 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.214 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.214 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.215 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.215 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T10:45:38.215 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T10:45:38.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.215 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.215 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.216 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.216 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.216 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.216 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.216 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.216 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.217 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T10:45:38.217 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T10:45:38.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.217 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.217 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.217 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.217 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.217 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.217 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.217 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.218 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.218 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.218 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.218 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T10:45:38.218 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T10:45:38.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.218 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.219 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.219 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.219 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.219 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.219 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.219 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.219 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.219 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.220 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.220 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T10:45:38.220 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T10:45:38.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.220 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.220 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.220 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.220 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.221 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.221 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.221 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.221 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.221 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.221 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.221 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T10:45:38.221 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T10:45:38.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.222 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.222 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.222 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.222 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.222 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.222 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.222 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.222 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.223 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.223 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.223 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T10:45:38.223 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T10:45:38.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.223 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.223 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.223 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.224 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.224 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.224 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.224 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.224 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.224 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.224 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.224 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T10:45:38.225 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T10:45:38.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.225 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.225 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.225 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.225 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.225 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.225 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.225 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.226 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.226 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.226 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.226 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T10:45:38.226 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T10:45:38.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.226 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.227 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.227 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.227 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.227 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.227 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.227 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.227 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.227 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.228 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T10:45:38.228 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T10:45:38.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.228 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.228 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.228 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.228 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.229 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.229 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.229 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.229 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.229 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.229 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.229 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T10:45:38.230 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T10:45:38.230 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.230 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.230 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.230 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.230 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.230 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.230 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.230 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.231 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.231 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.231 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.231 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T10:45:38.231 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T10:45:38.231 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.231 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.231 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.232 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T10:45:38.232 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.232 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.232 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.232 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.232 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.232 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.233 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T10:45:38.233 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T10:45:38.233 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.233 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.233 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.233 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.233 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.234 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.234 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.234 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.234 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.234 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.234 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T10:45:38.234 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T10:45:38.235 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.235 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.235 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.235 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-04-20T10:45:38.235 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.235 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.235 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.235 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.236 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.236 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.236 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.236 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T10:45:38.236 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T10:45:38.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.236 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.237 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-04-20T10:45:38.237 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.237 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.237 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.237 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.237 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.237 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.237 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.238 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T10:45:38.238 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T10:45:38.238 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.238 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.238 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.238 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.238 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.238 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.239 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.239 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.239 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.239 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.239 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T10:45:38.239 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T10:45:38.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.240 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.240 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.240 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-20T10:45:38.240 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.240 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.240 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.240 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.240 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.240 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.241 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.241 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T10:45:38.241 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T10:45:38.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.241 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.241 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.241 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.242 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.242 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.242 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.242 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.242 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.242 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.242 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T10:45:38.243 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T10:45:38.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.243 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.243 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.243 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.243 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.243 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.243 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.244 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.244 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.244 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.244 INFO:teuthology.orchestra.run.smithi042.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T10:45:38.244 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T10:45:38.244 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.244 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.244 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.245 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.245 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.245 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.245 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.245 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.245 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.245 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.245 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.246 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T10:45:38.246 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T10:45:38.246 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.246 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.246 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.246 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:38.246 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.246 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.246 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.247 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.247 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.247 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.247 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.247 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T10:45:38.247 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T10:45:38.247 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.248 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.248 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.248 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.248 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.248 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.248 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.248 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.249 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.249 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T10:45:38.249 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T10:45:38.249 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.249 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.249 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.249 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-20T10:45:38.250 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.250 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.250 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.250 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.250 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.250 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.250 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-04-20T10:45:38.251 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-04-20T10:45:38.251 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.251 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.251 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.251 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.251 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.251 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.251 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.251 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.252 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.252 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.252 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.252 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-04-20T10:45:38.252 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-04-20T10:45:38.252 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.252 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.253 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.253 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.253 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.253 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.253 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.253 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.253 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.254 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-04-20T10:45:38.254 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-04-20T10:45:38.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.254 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.254 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.254 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.254 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-04-20T10:45:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-04-20T10:45:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-04-20T10:45:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.256 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.256 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.256 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-04-20T10:45:38.256 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.256 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.256 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.256 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.256 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.257 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.257 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.257 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-04-20T10:45:38.257 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-04-20T10:45:38.257 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.257 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.257 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.257 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.258 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.258 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.258 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.258 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.258 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.258 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.258 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.258 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.259 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.259 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.259 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.259 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.259 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.259 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.259 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.260 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.260 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.260 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.260 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.260 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.260 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.260 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.260 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.260 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.261 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.261 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.261 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.261 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.261 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.262 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.262 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.262 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.262 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.262 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.262 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.262 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.263 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.263 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.263 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.263 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.263 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.263 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.264 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.264 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.264 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.264 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.264 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.264 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.264 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.264 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.264 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.265 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.265 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.265 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.265 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.265 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.265 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.266 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.266 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.266 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.266 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.266 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.266 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.267 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-04-20T10:45:38.267 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-04-20T10:45:38.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.267 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.267 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.267 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.267 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.267 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.268 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-04-20T10:45:38.268 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.268 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.268 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-04-20T10:45:38.268 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T10:45:38.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.268 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.269 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.269 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.269 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.269 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.269 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.269 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.269 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.270 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.270 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-20T10:45:38.270 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-20T10:45:38.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.270 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.270 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.270 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-04-20T10:45:38.270 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.271 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.271 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.271 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.271 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.271 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.271 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-20T10:45:38.272 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-20T10:45:38.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.272 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.272 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-04-20T10:45:38.272 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.272 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.272 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.272 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.273 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.273 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.273 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-04-20T10:45:38.273 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-04-20T10:45:38.273 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.273 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.274 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.274 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.274 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.274 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.274 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.274 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.274 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.275 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-04-20T10:45:38.275 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-04-20T10:45:38.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.275 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.275 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-04-20T10:45:38.275 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.275 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.276 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.276 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.276 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.276 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.276 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-04-20T10:45:38.276 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-04-20T10:45:38.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.277 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.277 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-04-20T10:45:38.277 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.277 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.277 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.277 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.277 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-04-20T10:45:38.278 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.278 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.278 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.278 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.278 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.278 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-04-20T10:45:38.278 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-04-20T10:45:38.278 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.279 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.279 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.279 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-04-20T10:45:38.279 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-04-20T10:45:38.279 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-04-20T10:45:38.279 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.279 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.279 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.280 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.280 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-04-20T10:45:38.280 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-04-20T10:45:38.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.280 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.280 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.281 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.281 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.281 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.281 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.281 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.281 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.281 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.282 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-04-20T10:45:38.282 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.282 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.282 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.282 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-04-20T10:45:38.282 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-04-20T10:45:38.283 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.283 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.283 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.283 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.283 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.283 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.283 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-04-20T10:45:38.283 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.284 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.284 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.284 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.284 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.284 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.284 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.284 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.284 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.285 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.285 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.285 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.285 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.285 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.285 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.285 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.285 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.285 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.286 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.286 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.286 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.286 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.286 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.286 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.286 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.287 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.287 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.287 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.287 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.287 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.287 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.287 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.288 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.288 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.288 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.288 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.288 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.288 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.288 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.288 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.289 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.289 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.289 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.289 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.289 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.289 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.289 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.289 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.290 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.290 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.290 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.290 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.290 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.290 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.290 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.291 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.291 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.291 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.291 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.291 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.291 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.292 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.292 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.292 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-04-20T10:45:38.292 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-04-20T10:45:38.292 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.292 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.293 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.293 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-04-20T10:45:38.293 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.293 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.293 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.293 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T10:45:38.293 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.293 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.293 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.294 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-04-20T10:45:38.294 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-04-20T10:45:38.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.294 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.294 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-04-20T10:45:38.294 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.295 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.295 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.295 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T10:45:38.295 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.295 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.295 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.295 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-04-20T10:45:38.295 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-04-20T10:45:38.296 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.296 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.296 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-04-20T10:45:38.296 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.296 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.296 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.296 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-04-20T10:45:38.297 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.297 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.297 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-04-20T10:45:38.297 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-04-20T10:45:38.297 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.297 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.297 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.298 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-04-20T10:45:38.298 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.298 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.298 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.298 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T10:45:38.298 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.298 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.298 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.299 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-04-20T10:45:38.299 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-04-20T10:45:38.299 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.299 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.299 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.299 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.299 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.300 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.300 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.300 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T10:45:38.300 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.300 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.300 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:38.300 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:38.301 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.301 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.301 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.301 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:38.301 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.301 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.301 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.301 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T10:45:38.302 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.302 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.302 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.302 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-04-20T10:45:38.302 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-04-20T10:45:38.302 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.302 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.302 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.303 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-04-20T10:45:38.303 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.303 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.303 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.303 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T10:45:38.303 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.304 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.304 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.304 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-04-20T10:45:38.304 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.304 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.304 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.305 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.305 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.305 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.305 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.305 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.305 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.305 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.306 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.306 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.306 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.306 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.306 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.306 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.306 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.306 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.307 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.307 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.307 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.307 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.307 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.307 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.307 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.308 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.308 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.308 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.308 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.308 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.308 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.308 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.308 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.308 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.308 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.309 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.309 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.309 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.309 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.309 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.309 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.310 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.310 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.310 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.310 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.310 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.310 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.310 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.310 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.311 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.311 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.311 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.311 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.311 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.311 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.311 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.311 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.312 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.312 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.312 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.312 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.312 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.312 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.312 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.312 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.313 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.313 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-04-20T10:45:38.313 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-04-20T10:45:38.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.313 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.313 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-04-20T10:45:38.313 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.314 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.314 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.314 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.314 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.314 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.314 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.314 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-04-20T10:45:38.315 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-04-20T10:45:38.315 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.315 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.315 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.315 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-04-20T10:45:38.315 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.315 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.315 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.315 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.316 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.316 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.316 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.316 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:38.316 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:38.316 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.316 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.317 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:38.317 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.317 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.317 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.317 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.317 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.317 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.318 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.318 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-04-20T10:45:38.318 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-04-20T10:45:38.319 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-04-20T10:45:38.319 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.319 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-04-20T10:45:38.319 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-04-20T10:45:38.319 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.319 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.319 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.319 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-04-20T10:45:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T10:45:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.320 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-04-20T10:45:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-04-20T10:45:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-04-20T10:45:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.321 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T10:45:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-04-20T10:45:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-04-20T10:45:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.322 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-04-20T10:45:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.323 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-04-20T10:45:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-04-20T10:45:38.324 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T10:45:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.325 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.326 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.327 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.328 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.328 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.328 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.328 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.328 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.328 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.328 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.328 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.329 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.329 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.329 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.329 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.329 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.329 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.329 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.330 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.330 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.330 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.330 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.330 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.330 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.330 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.330 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.331 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.331 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.331 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.331 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.331 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.331 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.331 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.331 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.332 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.332 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.332 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.332 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.332 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.332 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.332 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.333 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.333 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.333 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.333 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.333 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.333 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.333 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.333 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-04-20T10:45:38.334 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-04-20T10:45:38.334 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.334 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.334 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.334 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.334 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.334 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.335 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-04-20T10:45:38.335 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.335 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.335 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.335 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-04-20T10:45:38.335 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-04-20T10:45:38.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.335 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.336 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.336 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-04-20T10:45:38.336 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.336 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.336 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.336 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-04-20T10:45:38.336 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.336 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.336 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.337 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-04-20T10:45:38.337 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-04-20T10:45:38.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.337 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.337 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.337 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-04-20T10:45:38.338 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.338 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.338 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.338 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T10:45:38.338 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.338 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.338 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.338 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-04-20T10:45:38.339 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-04-20T10:45:38.339 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.339 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.339 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-20T10:45:38.339 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-20T10:45:38.339 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-04-20T10:45:38.339 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.339 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T10:45:38.340 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.340 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.340 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.340 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-04-20T10:45:38.340 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-04-20T10:45:38.340 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.340 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.341 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.341 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.341 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.341 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.341 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.341 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-04-20T10:45:38.341 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.341 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.341 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.342 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-04-20T10:45:38.342 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-04-20T10:45:38.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.342 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.342 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-04-20T10:45:38.342 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.343 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.343 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.343 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T10:45:38.343 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.343 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.343 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.343 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.343 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.344 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-04-20T10:45:38.344 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.344 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.344 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.344 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.344 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.344 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.344 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.344 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.345 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.345 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.345 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.345 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.345 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.345 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.345 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.345 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.346 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.346 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.346 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.346 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.346 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.346 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.346 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.346 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.347 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.347 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.347 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.347 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.347 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.347 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.347 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.347 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.348 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.348 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.348 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.348 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.348 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.348 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.349 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.349 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.349 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.349 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.349 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.349 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.349 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.349 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.350 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.350 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.350 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.350 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.350 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.350 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.350 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.351 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.351 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.351 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.351 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.351 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.351 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.351 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.351 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.352 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.352 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.352 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-04-20T10:45:38.352 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.353 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.353 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.353 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.353 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.353 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.353 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.353 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.353 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.354 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.354 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.354 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.354 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.354 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.354 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.354 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.354 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.355 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.355 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.355 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.355 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.355 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.355 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.355 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.356 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.356 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.356 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.356 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.356 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.356 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.356 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.356 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.357 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.357 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.357 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.357 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.357 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.357 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.358 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.358 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.358 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.358 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.358 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.358 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.358 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.358 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.359 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.359 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.359 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.359 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.359 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.359 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.359 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.359 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.360 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.360 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.360 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.360 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.360 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.360 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.360 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.360 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.361 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.361 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.361 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.361 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-04-20T10:45:38.361 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.362 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.362 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.362 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-04-20T10:45:38.362 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-04-20T10:45:38.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.362 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.362 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-04-20T10:45:38.363 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.363 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.363 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.363 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-20T10:45:38.363 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.363 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.363 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.363 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-04-20T10:45:38.364 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-04-20T10:45:38.364 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.364 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.364 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.364 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.364 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-04-20T10:45:38.364 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.364 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.364 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-20T10:45:38.365 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.365 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.365 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.365 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.365 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.365 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.365 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.365 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.366 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.366 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.366 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.366 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.366 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.366 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.366 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.366 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.367 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.367 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.367 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.367 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.367 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.367 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.367 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.368 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.368 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.368 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.368 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.368 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.368 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.368 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.368 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.369 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.369 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.369 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.369 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.369 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.369 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.369 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.369 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.370 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.370 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.370 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.370 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.370 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.370 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.370 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.370 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.370 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.371 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.371 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.371 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.371 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.371 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.371 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.371 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.372 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.372 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.372 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.372 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.372 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.372 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.372 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.372 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.372 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.373 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.373 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.373 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.373 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.373 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.373 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.374 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-04-20T10:45:38.374 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.374 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.374 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.374 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-04-20T10:45:38.374 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-04-20T10:45:38.374 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.374 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.375 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.375 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-04-20T10:45:38.375 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.375 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.375 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.375 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-04-20T10:45:38.375 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T10:45:38.375 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.375 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.376 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.376 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.376 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.376 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.376 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.376 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.376 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.377 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.377 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.377 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.377 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.377 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.377 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.377 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.377 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.377 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.378 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.378 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.378 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.378 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.378 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.378 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.378 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.379 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.379 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.379 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.379 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.379 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.379 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.379 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.380 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.380 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.380 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.380 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.380 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.380 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.381 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.381 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.381 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.381 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.381 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.381 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.381 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.381 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.381 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.382 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.382 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.382 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.382 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.382 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.382 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.383 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.383 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.383 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.383 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.383 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.383 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.383 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.383 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.383 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.384 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.384 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.384 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-04-20T10:45:38.384 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-04-20T10:45:38.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.384 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.385 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.385 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.385 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.385 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.385 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-04-20T10:45:38.385 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T10:45:38.385 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.385 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.386 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-04-20T10:45:38.386 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-04-20T10:45:38.386 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.386 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.386 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.386 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-04-20T10:45:38.386 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.386 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.387 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.387 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-04-20T10:45:38.387 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.387 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.387 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.387 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-04-20T10:45:38.387 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-04-20T10:45:38.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.388 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.388 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-04-20T10:45:38.388 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.388 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.388 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.388 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T10:45:38.388 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.389 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.389 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.389 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-04-20T10:45:38.389 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-04-20T10:45:38.389 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.389 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.389 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.389 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.390 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.390 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.390 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.390 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-04-20T10:45:38.390 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.390 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.390 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.390 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-04-20T10:45:38.391 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-04-20T10:45:38.391 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.391 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.391 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.391 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-04-20T10:45:38.391 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.391 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.391 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.391 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T10:45:38.392 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T10:45:38.392 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.392 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.392 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-04-20T10:45:38.393 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.393 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.393 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.393 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.393 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.393 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.393 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.394 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.394 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.394 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.394 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.394 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.394 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.394 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.394 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.395 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.395 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.395 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.395 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.395 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.395 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.395 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.395 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.395 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.396 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.396 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.396 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.396 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.396 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.396 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.397 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.397 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.397 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.397 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.397 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.398 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.398 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.398 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.398 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.398 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.398 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.398 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.398 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.398 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.399 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.399 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.399 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.399 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.399 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.399 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.399 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.400 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.400 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.400 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.400 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.400 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.400 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.400 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.400 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.400 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.400 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.401 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.401 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-04-20T10:45:38.401 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.401 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.402 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.402 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.402 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.402 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.402 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.402 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.403 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.403 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.403 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.403 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.403 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.403 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.403 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.403 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.403 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.404 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.404 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.404 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.404 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.404 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.404 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.405 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.405 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.405 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.405 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.405 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.405 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.405 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.405 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.406 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.406 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.406 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.406 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.406 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.407 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.407 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.407 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.407 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.407 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.407 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.407 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.407 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.407 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.408 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.408 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.408 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.408 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.408 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.408 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.409 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.409 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.409 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.409 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.409 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.409 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.409 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.409 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.409 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.410 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.410 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-04-20T10:45:38.410 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.410 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.411 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.411 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.411 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.411 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.411 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.411 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.411 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.412 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.412 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.412 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.412 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.412 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.412 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.412 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.412 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.412 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.413 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.413 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.413 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.413 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.413 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.413 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.413 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.414 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.414 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.414 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.414 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.414 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.414 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.414 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.414 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.415 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.415 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.415 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.415 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.415 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.415 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.415 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.416 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.416 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.416 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.416 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.416 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.416 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.416 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.416 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.416 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.417 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.417 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.417 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.417 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.417 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.418 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.418 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.418 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.418 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.418 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.418 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.418 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.418 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.419 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.419 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.419 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-04-20T10:45:38.419 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.419 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.419 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.419 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-04-20T10:45:38.420 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-04-20T10:45:38.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.420 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.420 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.420 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.420 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.420 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.420 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.420 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T10:45:38.421 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.421 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.421 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.421 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.421 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.421 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.421 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.422 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.422 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.422 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.422 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.422 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.422 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.422 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.422 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.423 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.423 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.423 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.423 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.423 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.423 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.423 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.423 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.424 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.424 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.424 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.424 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.424 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.424 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.424 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.424 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.424 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.425 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.425 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.425 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.425 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.425 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.425 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.426 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.426 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.426 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.426 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.426 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.426 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.426 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.426 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.427 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.427 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.427 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.427 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.427 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.427 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.428 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.428 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.428 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.428 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.428 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.428 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.428 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.429 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.429 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.429 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.429 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.429 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-04-20T10:45:38.429 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-04-20T10:45:38.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.429 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.430 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.430 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.430 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.430 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.430 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.430 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-04-20T10:45:38.430 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-04-20T10:45:38.430 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-04-20T10:45:38.431 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.431 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-04-20T10:45:38.431 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.431 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.431 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-04-20T10:45:38.432 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.432 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.432 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.432 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.432 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.433 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.433 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.433 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.433 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.433 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.433 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.433 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.433 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.434 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.434 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.434 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.434 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.434 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.434 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.435 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.435 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.435 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.435 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.435 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.435 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.435 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.435 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.435 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.436 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.436 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.436 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.436 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.436 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.436 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.436 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.437 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.437 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.437 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.437 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.437 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.437 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.437 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.438 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.438 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.438 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.438 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.438 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.438 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.438 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.439 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.439 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.439 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.439 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.439 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.439 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.439 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.439 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.439 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.440 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.440 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.440 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.440 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.440 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.440 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.441 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-04-20T10:45:38.441 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.441 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.441 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.441 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.441 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.441 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.442 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.442 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.442 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.442 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.442 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.442 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.442 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.442 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.443 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.443 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.443 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.443 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.443 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.443 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.443 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.443 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.444 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.444 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.444 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.444 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.444 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.444 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.444 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.444 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.445 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.445 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.445 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.445 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.445 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.446 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.446 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.446 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.446 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.446 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.446 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.446 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.446 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.447 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.447 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.447 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.447 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.447 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.447 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.448 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.448 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.448 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.448 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.448 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.448 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.448 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.448 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.449 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.449 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.449 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.449 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.449 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.449 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-04-20T10:45:38.450 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.450 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.450 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.450 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.450 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.450 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.450 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.450 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.451 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.451 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.451 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.451 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.451 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.451 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.451 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.451 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.451 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.452 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.452 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.452 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.452 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.452 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.452 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.453 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.453 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.453 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.453 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.453 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.453 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.453 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.454 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.454 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.454 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.454 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.454 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.455 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.455 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.455 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.455 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.455 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.455 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.455 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.455 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.456 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.456 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.456 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.456 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.456 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.456 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.456 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.456 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.457 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.457 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.457 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.457 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.457 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.457 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.457 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.457 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.458 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.458 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.458 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.458 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.458 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:38.458 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:38.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.458 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.459 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.459 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:38.459 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.459 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.459 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.459 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.459 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.460 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.460 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-04-20T10:45:38.460 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-04-20T10:45:38.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:38.460 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.460 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-04-20T10:45:38.461 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-04-20T10:45:38.461 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.461 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.461 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-04-20T10:45:38.461 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T10:45:38.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.461 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.461 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.461 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.462 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-04-20T10:45:38.462 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.462 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.462 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.462 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-04-20T10:45:38.462 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-04-20T10:45:38.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.463 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.463 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.463 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.463 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.463 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.463 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T10:45:38.463 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.464 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.464 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-04-20T10:45:38.464 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-04-20T10:45:38.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.464 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.464 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.464 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.465 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.465 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.465 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.465 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.465 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.465 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.465 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.466 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.466 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.466 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.466 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.466 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.466 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.466 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.467 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.467 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.467 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.467 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.467 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.467 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.467 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.467 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.467 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.468 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.468 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.468 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.468 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.468 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.468 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.468 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.469 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.469 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.469 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.469 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.469 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.469 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.469 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.469 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.470 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.470 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.470 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.470 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.470 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.470 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.471 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.471 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.471 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.471 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.471 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.471 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.471 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.471 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.471 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.472 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.472 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.472 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.472 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.472 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.472 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.473 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.473 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.473 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.473 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.473 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.473 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.473 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.473 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.473 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.474 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-04-20T10:45:38.474 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-04-20T10:45:38.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.474 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.474 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-04-20T10:45:38.474 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.474 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.475 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.475 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-04-20T10:45:38.475 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.475 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.475 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:38.475 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:38.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.476 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.476 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-20T10:45:38.476 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.476 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.476 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.476 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-04-20T10:45:38.476 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.477 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.477 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.477 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.477 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.477 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-04-20T10:45:38.477 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.477 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.478 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.478 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-04-20T10:45:38.478 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-04-20T10:45:38.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.478 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.478 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.478 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.478 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.479 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.479 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.479 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.479 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.479 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.479 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-04-20T10:45:38.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-04-20T10:45:38.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.480 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.480 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.480 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.480 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.480 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.480 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.480 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T10:45:38.480 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T10:45:38.481 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.481 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.481 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.481 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.481 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.482 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.482 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.482 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.482 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.482 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.482 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.482 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.482 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.482 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.483 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.483 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.483 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.483 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.483 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.483 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.483 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.483 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.484 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.484 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.484 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.484 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.484 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.484 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.485 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.485 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.485 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.485 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.485 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.485 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.485 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.485 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.486 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.486 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.486 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.486 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.486 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.486 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.486 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.486 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.486 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.487 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.487 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.487 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.487 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.487 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.487 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.487 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.487 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.488 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.488 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.488 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.488 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.488 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.488 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.488 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.489 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.489 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.489 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-04-20T10:45:38.489 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-04-20T10:45:38.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.489 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.489 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.489 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.490 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.490 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.490 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.490 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.490 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.490 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.491 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T10:45:38.491 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T10:45:38.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.491 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.491 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.491 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-04-20T10:45:38.491 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.491 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.491 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.492 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.492 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.492 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.492 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.492 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-04-20T10:45:38.492 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-04-20T10:45:38.492 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:38.492 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.493 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.493 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-04-20T10:45:38.493 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.493 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.493 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.493 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.493 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.493 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.494 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.494 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-20T10:45:38.494 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-20T10:45:38.494 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.494 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.494 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.494 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-04-20T10:45:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T10:45:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.495 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-20T10:45:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-20T10:45:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-04-20T10:45:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.496 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T10:45:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-04-20T10:45:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-04-20T10:45:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.497 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-04-20T10:45:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.498 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-04-20T10:45:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-04-20T10:45:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-04-20T10:45:38.499 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-04-20T10:45:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-04-20T10:45:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.500 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-04-20T10:45:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-04-20T10:45:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-04-20T10:45:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-04-20T10:45:38.501 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-04-20T10:45:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-04-20T10:45:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.503 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.504 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.506 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.507 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.508 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.510 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-04-20T10:45:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-04-20T10:45:38.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.512 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-04-20T10:45:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T10:45:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.513 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-04-20T10:45:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-04-20T10:45:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.515 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-04-20T10:45:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-04-20T10:45:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-04-20T10:45:38.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.518 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-04-20T10:45:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-04-20T10:45:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.519 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.520 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.521 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.522 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.523 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.524 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.526 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.527 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.528 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-20T10:45:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-20T10:45:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.529 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-20T10:45:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-20T10:45:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.531 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-04-20T10:45:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.534 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.536 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.538 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.539 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.541 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-04-20T10:45:38.542 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.543 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.544 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.546 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.547 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.548 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.550 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-04-20T10:45:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-04-20T10:45:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.551 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-04-20T10:45:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T10:45:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.552 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-04-20T10:45:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.553 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.554 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.556 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.557 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.558 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.560 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-04-20T10:45:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-04-20T10:45:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.562 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-04-20T10:45:38.563 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-04-20T10:45:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-04-20T10:45:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.564 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-04-20T10:45:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-04-20T10:45:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.565 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.566 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-04-20T10:45:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-04-20T10:45:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.567 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-04-20T10:45:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-04-20T10:45:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.569 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.570 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-04-20T10:45:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-04-20T10:45:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.571 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-04-20T10:45:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-04-20T10:45:38.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.573 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-04-20T10:45:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-04-20T10:45:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-04-20T10:45:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-04-20T10:45:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-20T10:45:38.576 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-04-20T10:45:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-04-20T10:45:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-04-20T10:45:38.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.578 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-04-20T10:45:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-04-20T10:45:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.579 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.580 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-04-20T10:45:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-04-20T10:45:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.581 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.582 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-04-20T10:45:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-04-20T10:45:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-04-20T10:45:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T10:45:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-04-20T10:45:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-04-20T10:45:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.585 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-04-20T10:45:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.587 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.588 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.589 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.589 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.589 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.589 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.589 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.589 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.590 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.590 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.590 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.590 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.590 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.590 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.591 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.591 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.591 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.591 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.591 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.591 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.591 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.592 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.592 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.592 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.592 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.592 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.592 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.592 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.592 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.592 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.593 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.593 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.593 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.593 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.593 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.594 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.594 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.594 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.594 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.594 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.594 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.594 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.595 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.595 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-04-20T10:45:38.595 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.595 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.595 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.595 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.595 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.596 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.596 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.596 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.596 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.596 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.596 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.596 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.596 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.596 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.597 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.597 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.597 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.597 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.597 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.597 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.597 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.597 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.598 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.598 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.598 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.598 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.598 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.598 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.599 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.599 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.599 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.599 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.599 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.599 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.600 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.600 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.600 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.600 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.600 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.600 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.600 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.601 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.601 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.601 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.601 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.601 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.601 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.601 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.601 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.601 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.602 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.602 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.602 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.602 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.602 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.602 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.603 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.603 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.603 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.603 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.603 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.603 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.604 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.604 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-04-20T10:45:38.604 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.604 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.604 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.604 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.604 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.605 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.605 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.605 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.605 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.605 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.605 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.605 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.605 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.606 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.606 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.606 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.606 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.606 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.606 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.606 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.606 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.607 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.607 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.607 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.607 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.607 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.607 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.608 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.608 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.608 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.608 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.608 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.608 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.609 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.609 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.609 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.609 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.609 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.609 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.610 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.610 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.610 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.610 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.610 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.610 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.610 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.610 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.610 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.611 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.611 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.611 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.611 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.611 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.612 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.612 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.612 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.612 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.612 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.612 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.613 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.613 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-04-20T10:45:38.613 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.613 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.613 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.613 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-04-20T10:45:38.613 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-04-20T10:45:38.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.614 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.614 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T10:45:38.614 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.614 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.614 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.614 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.615 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.615 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.615 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:38.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:38.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.615 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.616 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-04-20T10:45:38.616 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.616 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.616 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.616 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.616 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.616 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.616 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.617 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.617 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.617 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.617 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.617 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.617 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.618 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.618 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.618 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.618 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.618 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.618 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.618 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.618 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.619 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.619 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.619 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.619 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.619 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.619 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.620 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.620 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.620 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.620 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.620 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.620 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.620 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.620 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.620 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.621 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.621 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.621 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.621 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.621 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.621 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.622 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.622 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.622 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.622 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.622 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.622 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.622 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.622 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.623 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.623 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.623 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.623 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.623 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.623 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.623 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.624 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.624 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.624 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.624 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.624 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.624 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.624 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.625 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-04-20T10:45:38.625 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.625 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.625 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.626 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-04-20T10:45:38.626 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-04-20T10:45:38.626 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.626 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.626 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.626 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.626 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.626 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.627 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.627 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T10:45:38.627 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.627 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.627 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-04-20T10:45:38.627 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-04-20T10:45:38.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.628 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.628 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.628 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.628 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.628 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.628 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T10:45:38.629 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.629 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.629 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-04-20T10:45:38.629 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-04-20T10:45:38.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.629 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.630 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.630 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.630 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.630 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.630 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T10:45:38.630 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.631 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.631 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-04-20T10:45:38.631 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-04-20T10:45:38.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.631 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.631 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.632 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.632 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.632 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.632 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T10:45:38.632 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.632 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.632 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-04-20T10:45:38.633 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-04-20T10:45:38.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.633 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.633 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.633 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.633 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.633 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.634 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T10:45:38.634 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.634 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.634 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-04-20T10:45:38.634 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-04-20T10:45:38.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.634 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.635 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.635 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.635 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.635 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.635 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.635 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.635 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.636 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.636 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-04-20T10:45:38.636 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-04-20T10:45:38.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.636 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.636 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.636 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.637 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.637 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.637 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.637 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.637 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.637 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-04-20T10:45:38.637 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-04-20T10:45:38.638 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.638 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.638 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T10:45:38.638 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.638 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.638 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.638 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.639 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.639 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.639 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.639 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-04-20T10:45:38.639 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-04-20T10:45:38.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.639 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.640 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.640 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.640 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.640 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.640 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.640 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.640 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.641 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:38.641 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:38.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.641 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.641 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-04-20T10:45:38.641 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-04-20T10:45:38.641 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.642 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.642 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.642 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.642 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.642 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-04-20T10:45:38.642 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-04-20T10:45:38.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.643 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.643 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-04-20T10:45:38.643 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.643 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.643 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.643 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.643 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.644 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.644 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-04-20T10:45:38.644 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-04-20T10:45:38.644 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.644 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.644 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.644 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-04-20T10:45:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-04-20T10:45:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.658 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-04-20T10:45:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-04-20T10:45:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.659 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.660 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-04-20T10:45:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-04-20T10:45:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.661 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-04-20T10:45:38.662 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-04-20T10:45:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T10:45:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.663 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-04-20T10:45:38.664 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.665 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.665 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.665 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.665 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.665 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.665 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.666 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.666 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.666 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.666 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.666 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.666 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.666 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.666 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.666 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.667 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.667 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.667 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.667 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.667 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.667 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.667 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.668 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.668 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.668 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.668 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.668 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.668 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.668 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.669 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.669 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.669 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.669 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.669 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.669 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.669 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.670 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.670 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.670 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.670 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.670 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.670 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.670 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.670 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.671 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.671 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.671 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.671 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.671 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.671 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.671 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.672 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.672 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.672 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.672 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.672 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.672 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.673 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.673 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.673 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-04-20T10:45:38.674 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.674 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.674 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.674 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.674 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.674 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.674 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.675 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.675 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.675 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.675 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.675 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.675 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.675 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.675 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.676 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.676 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.676 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.676 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.676 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.676 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.676 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.677 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.677 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.677 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.677 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.677 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.677 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.677 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.677 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.678 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.678 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.678 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.678 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.679 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.679 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.679 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.679 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.679 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.679 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.679 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.679 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.680 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.680 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.680 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.680 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.680 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.680 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.681 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.681 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.681 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.681 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.681 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.681 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.682 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.682 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.682 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.682 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.682 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.682 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-04-20T10:45:38.683 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-04-20T10:45:38.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.683 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.683 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-04-20T10:45:38.683 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.683 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.683 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.684 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T10:45:38.684 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.684 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.684 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-04-20T10:45:38.684 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-04-20T10:45:38.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.685 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.685 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.685 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.685 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.685 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.685 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-04-20T10:45:38.685 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.686 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.686 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-04-20T10:45:38.686 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-04-20T10:45:38.686 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.686 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.686 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:38.687 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.687 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.687 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.687 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T10:45:38.687 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.687 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.687 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_no_wait": { 2024-04-20T10:45:38.688 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T10:45:38.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.688 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.688 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.688 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.688 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.688 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.688 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.689 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T10:45:38.689 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.689 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.689 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.689 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-04-20T10:45:38.690 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.690 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.690 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.690 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-04-20T10:45:38.690 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-04-20T10:45:38.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:38.690 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.691 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-04-20T10:45:38.691 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.691 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.691 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.691 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.691 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.691 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.692 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-04-20T10:45:38.692 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-04-20T10:45:38.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.692 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.692 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.692 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.692 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.693 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.693 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.693 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.693 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.693 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:38.693 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:38.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.694 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.694 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:38.694 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.694 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.694 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.694 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.695 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.695 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.695 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.695 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.695 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.695 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.696 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.696 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.696 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.696 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.696 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.696 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.696 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.697 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.697 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.697 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.697 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.697 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.697 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.698 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.698 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.698 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.698 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.698 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.698 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.699 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.699 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.699 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.699 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.699 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.699 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.699 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.700 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.700 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.700 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.700 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.700 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.700 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.700 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.700 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.701 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.701 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.701 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.701 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.701 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.701 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.701 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.701 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.702 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.702 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.702 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.702 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.702 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.702 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.702 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.703 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.703 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.703 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.703 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.703 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.704 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-04-20T10:45:38.704 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-04-20T10:45:38.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.704 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.704 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.704 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.704 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.704 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.705 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.705 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.705 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.705 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-04-20T10:45:38.705 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-04-20T10:45:38.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.706 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.706 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-04-20T10:45:38.706 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.706 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.706 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.706 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.707 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.707 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.707 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-04-20T10:45:38.707 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-04-20T10:45:38.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.707 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.708 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T10:45:38.708 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.708 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.708 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.708 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.708 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.708 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.709 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-20T10:45:38.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.709 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.709 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 25147, 2024-04-20T10:45:38.709 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "z", 2024-04-20T10:45:38.709 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_features": 4540701547738038271, 2024-04-20T10:45:38.709 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-04-20T10:45:38.710 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.710 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-04-20T10:45:38.710 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.710 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.710 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.710 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:38.710 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:38.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.711 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.711 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.711 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:38.711 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.711 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.711 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.711 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T10:45:38.711 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.712 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.712 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.712 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.712 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.712 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.712 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.713 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.713 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.713 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.713 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.713 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.713 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.713 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.713 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.714 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.714 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.714 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.714 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.714 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.714 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.715 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.715 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.715 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.715 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.715 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.715 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.715 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.715 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.715 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.716 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.716 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.716 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.716 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.716 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.717 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.717 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.717 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.717 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.717 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.717 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.717 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.717 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.718 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.718 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.718 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.718 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.718 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.718 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.719 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.719 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.719 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.719 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.719 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.719 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.720 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.720 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.720 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.720 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.720 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.720 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-04-20T10:45:38.720 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-04-20T10:45:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-04-20T10:45:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.722 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.722 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-04-20T10:45:38.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-04-20T10:45:38.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.722 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.723 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-04-20T10:45:38.723 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.723 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.723 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.723 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-04-20T10:45:38.723 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.723 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.723 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.724 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-04-20T10:45:38.724 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-04-20T10:45:38.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.724 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.724 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.724 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.724 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.725 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.725 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-04-20T10:45:38.725 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.725 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.725 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-04-20T10:45:38.725 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-04-20T10:45:38.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.726 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.726 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.726 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.726 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.726 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.726 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-04-20T10:45:38.726 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.727 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.727 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.727 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-04-20T10:45:38.727 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-04-20T10:45:38.727 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.727 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.728 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-04-20T10:45:38.728 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.728 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.728 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.728 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-04-20T10:45:38.728 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.728 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.729 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-04-20T10:45:38.729 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-04-20T10:45:38.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.729 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.729 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.729 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.729 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.730 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.730 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-04-20T10:45:38.730 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.730 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.730 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-04-20T10:45:38.730 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-04-20T10:45:38.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.731 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.731 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.731 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.731 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.731 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.731 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T10:45:38.732 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.732 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.732 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.732 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-04-20T10:45:38.732 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-04-20T10:45:38.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.733 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.733 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.733 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.733 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.733 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.733 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-04-20T10:45:38.733 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.734 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.734 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-04-20T10:45:38.734 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.735 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.735 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.735 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-04-20T10:45:38.735 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-04-20T10:45:38.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.735 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.735 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.736 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.736 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.736 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.736 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T10:45:38.736 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.736 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.736 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-04-20T10:45:38.737 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-04-20T10:45:38.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.737 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.737 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.737 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-04-20T10:45:38.737 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.737 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.737 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.738 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T10:45:38.738 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T10:45:38.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.738 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.738 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-04-20T10:45:38.738 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-04-20T10:45:38.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:38.739 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.739 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:38.739 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-04-20T10:45:38.739 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-04-20T10:45:38.739 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.739 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T10:45:38.739 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T10:45:38.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.740 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.740 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-04-20T10:45:38.740 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-04-20T10:45:38.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:38.740 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.741 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-04-20T10:45:38.741 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-20T10:45:38.741 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-04-20T10:45:38.741 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.741 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T10:45:38.741 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.741 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.742 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-04-20T10:45:38.742 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-04-20T10:45:38.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.742 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.742 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T10:45:38.742 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.743 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.743 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.743 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T10:45:38.743 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T10:45:38.743 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.743 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.743 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-04-20T10:45:38.744 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-04-20T10:45:38.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:38.744 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.744 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-04-20T10:45:38.744 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-04-20T10:45:38.744 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-04-20T10:45:38.744 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.745 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-04-20T10:45:38.745 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T10:45:38.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.745 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.745 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-04-20T10:45:38.745 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-04-20T10:45:38.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.746 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.746 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-04-20T10:45:38.746 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.746 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.746 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.747 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T10:45:38.747 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T10:45:38.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.747 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.747 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-04-20T10:45:38.747 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-04-20T10:45:38.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:38.748 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.748 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:38.748 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-04-20T10:45:38.748 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-04-20T10:45:38.748 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.748 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T10:45:38.749 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T10:45:38.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.749 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.749 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.749 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.750 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.750 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.750 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.750 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.750 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.750 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.750 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.751 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.751 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.751 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.751 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.751 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.751 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.752 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.752 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.752 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.752 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.752 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.753 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.753 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.753 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.753 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.753 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.753 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.754 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.754 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.754 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.754 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.755 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.755 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.755 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.755 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.755 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.755 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.755 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.755 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.756 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.756 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.756 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.756 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.756 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.756 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.757 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.757 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.757 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.757 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.757 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.757 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.758 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.758 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.758 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.758 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.758 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-04-20T10:45:38.758 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-04-20T10:45:38.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:38.759 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.759 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:38.759 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.759 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.759 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.759 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T10:45:38.760 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.760 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.760 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-04-20T10:45:38.760 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-04-20T10:45:38.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.760 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.761 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-04-20T10:45:38.761 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.761 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.761 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.761 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-04-20T10:45:38.761 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-04-20T10:45:38.762 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-04-20T10:45:38.762 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-04-20T10:45:38.762 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-04-20T10:45:38.762 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.762 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-04-20T10:45:38.762 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.763 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.763 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-04-20T10:45:38.763 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-04-20T10:45:38.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.763 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.764 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.764 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.764 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.764 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.764 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T10:45:38.764 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.764 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.765 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:38.765 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:38.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.765 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.765 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:38.766 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.766 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.766 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.766 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T10:45:38.766 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.766 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.767 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-04-20T10:45:38.767 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-04-20T10:45:38.767 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.767 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.767 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.767 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-20T10:45:38.768 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-20T10:45:38.768 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.768 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.768 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T10:45:38.768 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T10:45:38.768 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.768 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.769 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-04-20T10:45:38.769 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-04-20T10:45:38.769 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:38.769 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.769 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.769 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.769 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.770 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.770 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.770 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T10:45:38.770 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.770 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.771 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.771 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.771 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-04-20T10:45:38.771 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.771 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.772 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.772 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-04-20T10:45:38.772 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-04-20T10:45:38.772 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:38.772 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.772 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.772 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-04-20T10:45:38.773 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.773 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.773 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.773 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T10:45:38.773 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.774 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.774 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-04-20T10:45:38.774 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-04-20T10:45:38.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.774 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.775 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-04-20T10:45:38.775 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.775 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.775 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.775 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T10:45:38.775 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.775 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.776 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-04-20T10:45:38.776 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-04-20T10:45:38.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.776 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.776 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-04-20T10:45:38.776 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.776 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.776 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.777 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T10:45:38.777 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.777 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.777 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-04-20T10:45:38.777 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-04-20T10:45:38.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.778 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.778 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.778 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.778 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.778 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.778 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T10:45:38.778 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T10:45:38.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.779 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.779 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-04-20T10:45:38.779 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-04-20T10:45:38.779 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.779 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.779 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.779 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:38.780 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.780 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.780 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.780 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T10:45:38.780 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.780 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.781 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-04-20T10:45:38.781 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T10:45:38.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:38.781 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.781 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-04-20T10:45:38.781 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.781 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.781 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.782 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T10:45:38.782 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.782 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.782 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-04-20T10:45:38.782 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-04-20T10:45:38.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.783 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.783 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.783 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.783 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.783 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.783 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-04-20T10:45:38.783 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-04-20T10:45:38.784 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-04-20T10:45:38.784 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.784 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T10:45:38.784 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.784 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.784 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.784 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-04-20T10:45:38.785 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-04-20T10:45:38.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.785 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.785 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.785 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.785 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.785 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.785 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T10:45:38.786 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.786 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.786 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-04-20T10:45:38.786 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-04-20T10:45:38.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.787 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.787 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.787 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.787 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.787 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.787 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T10:45:38.787 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.788 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.788 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-04-20T10:45:38.788 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-04-20T10:45:38.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.788 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.788 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.789 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.789 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.789 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.789 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T10:45:38.789 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.789 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.790 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-04-20T10:45:38.790 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-04-20T10:45:38.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.790 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.790 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T10:45:38.790 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.790 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.791 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.791 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-20T10:45:38.791 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.791 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.791 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-04-20T10:45:38.791 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-04-20T10:45:38.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.792 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.792 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.792 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T10:45:38.792 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.792 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.792 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.792 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-04-20T10:45:38.792 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.793 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.793 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-04-20T10:45:38.793 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-04-20T10:45:38.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.793 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.793 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T10:45:38.794 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.794 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.794 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.794 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-04-20T10:45:38.794 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.794 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.795 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-04-20T10:45:38.795 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-04-20T10:45:38.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.795 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.795 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T10:45:38.795 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.795 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.796 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.796 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-20T10:45:38.796 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.796 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.796 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-04-20T10:45:38.796 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-04-20T10:45:38.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.797 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.797 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T10:45:38.797 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.797 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.797 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.797 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-04-20T10:45:38.798 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.798 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.798 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-04-20T10:45:38.798 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-04-20T10:45:38.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.799 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.799 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T10:45:38.799 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.799 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.799 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.799 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-04-20T10:45:38.800 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.800 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.800 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-04-20T10:45:38.800 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-04-20T10:45:38.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.801 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.801 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T10:45:38.801 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.801 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.801 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.801 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-04-20T10:45:38.801 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.802 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.802 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-04-20T10:45:38.802 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-04-20T10:45:38.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.803 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.803 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T10:45:38.803 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.803 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.803 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.803 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-04-20T10:45:38.803 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.803 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.804 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.804 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-04-20T10:45:38.804 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-04-20T10:45:38.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.804 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.805 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T10:45:38.805 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.805 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.805 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.805 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-04-20T10:45:38.805 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.806 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.806 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-04-20T10:45:38.806 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-04-20T10:45:38.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.806 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.807 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T10:45:38.807 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.807 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.807 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.807 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-04-20T10:45:38.807 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.808 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.808 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-04-20T10:45:38.808 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-04-20T10:45:38.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.808 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.808 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T10:45:38.809 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.809 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.809 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.809 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-20T10:45:38.809 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.809 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.810 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-04-20T10:45:38.810 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-04-20T10:45:38.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.810 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.810 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T10:45:38.810 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.810 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.811 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.811 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-04-20T10:45:38.811 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.811 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.811 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-04-20T10:45:38.811 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-04-20T10:45:38.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.812 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.812 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T10:45:38.812 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.813 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.813 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.813 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-20T10:45:38.813 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.813 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.814 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-04-20T10:45:38.814 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-04-20T10:45:38.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.814 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.814 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T10:45:38.815 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.815 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.815 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.815 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-04-20T10:45:38.815 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.815 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.815 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.815 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_samba": { 2024-04-20T10:45:38.816 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_samba", 2024-04-20T10:45:38.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.816 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.816 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T10:45:38.816 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.816 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.816 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.817 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Samba/SMB container image", 2024-04-20T10:45:38.817 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.817 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.817 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-04-20T10:45:38.817 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-04-20T10:45:38.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.818 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.818 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.818 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T10:45:38.818 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.818 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.818 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.818 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-04-20T10:45:38.818 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.819 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.819 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.819 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-04-20T10:45:38.819 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-04-20T10:45:38.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.819 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.819 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.820 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.820 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.820 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.820 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.820 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T10:45:38.820 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.820 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.821 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-04-20T10:45:38.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-04-20T10:45:38.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.821 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.821 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:38.821 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.821 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.822 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.822 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T10:45:38.822 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.822 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.822 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-04-20T10:45:38.822 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T10:45:38.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.823 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.823 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-04-20T10:45:38.823 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.823 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.823 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.823 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T10:45:38.824 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.824 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.824 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-04-20T10:45:38.824 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-04-20T10:45:38.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.824 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.825 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-04-20T10:45:38.825 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.825 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.825 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.825 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T10:45:38.825 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.825 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.826 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-04-20T10:45:38.826 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-04-20T10:45:38.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.826 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.826 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-04-20T10:45:38.826 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-04-20T10:45:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-04-20T10:45:38.827 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-04-20T10:45:38.828 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.828 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.828 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.828 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.828 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.828 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.828 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.828 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T10:45:38.829 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.829 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.829 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-04-20T10:45:38.829 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-04-20T10:45:38.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.829 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.830 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:38.830 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.830 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.830 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.830 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-04-20T10:45:38.830 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.830 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.831 INFO:teuthology.orchestra.run.smithi042.stdout: "grafana_dashboards_path": { 2024-04-20T10:45:38.831 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "grafana_dashboards_path", 2024-04-20T10:45:38.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.831 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.831 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T10:45:38.831 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.831 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.832 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.832 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T10:45:38.832 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.832 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.832 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-04-20T10:45:38.832 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-04-20T10:45:38.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.833 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.833 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:38.833 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.833 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.833 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.833 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-04-20T10:45:38.833 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.834 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.834 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.834 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-04-20T10:45:38.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-04-20T10:45:38.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.834 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.834 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.835 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.835 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.835 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.835 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T10:45:38.835 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.835 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.835 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-04-20T10:45:38.836 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-04-20T10:45:38.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.836 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.836 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.836 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.836 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.836 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.837 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T10:45:38.837 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.837 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.837 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.837 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.838 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.838 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.838 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.838 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.838 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.838 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.838 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.839 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.839 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.839 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.839 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.839 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.839 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.839 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.839 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.840 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-04-20T10:45:38.840 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-04-20T10:45:38.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.840 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.840 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.840 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.840 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.841 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.841 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T10:45:38.841 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.841 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.841 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.841 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.842 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.842 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.842 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.842 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.842 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.842 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T10:45:38.842 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.843 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.843 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.843 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.843 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.844 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.844 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.844 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.844 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.844 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.844 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.844 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.844 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.844 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.845 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.845 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.845 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.845 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.845 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.845 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.845 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.846 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.846 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.846 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.846 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.846 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.846 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.847 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.847 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.847 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T10:45:38.847 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T10:45:38.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.847 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.848 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.848 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.848 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.848 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.848 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T10:45:38.848 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.848 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.849 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T10:45:38.849 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T10:45:38.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.849 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.849 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-04-20T10:45:38.850 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.850 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.850 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.850 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T10:45:38.850 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.850 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-04-20T10:45:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-04-20T10:45:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.851 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-04-20T10:45:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-04-20T10:45:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-04-20T10:45:38.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.853 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.853 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.853 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.853 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.853 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.853 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T10:45:38.853 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.853 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.854 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.854 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-04-20T10:45:38.854 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-04-20T10:45:38.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.854 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.854 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.854 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.855 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.855 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.855 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-04-20T10:45:38.855 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.855 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.855 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.855 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-04-20T10:45:38.856 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-04-20T10:45:38.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.856 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.856 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-04-20T10:45:38.856 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.856 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.856 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.856 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-04-20T10:45:38.857 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-04-20T10:45:38.857 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.857 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T10:45:38.857 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.857 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.857 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-04-20T10:45:38.857 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-04-20T10:45:38.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.858 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.858 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-04-20T10:45:38.858 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.858 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.858 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.858 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T10:45:38.859 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.859 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.859 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-04-20T10:45:38.859 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-04-20T10:45:38.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.859 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.860 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T10:45:38.860 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.860 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.860 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.860 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T10:45:38.860 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.860 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.861 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-04-20T10:45:38.861 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-04-20T10:45:38.861 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.861 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.861 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.861 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.861 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.861 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.862 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.862 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T10:45:38.862 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.862 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.862 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.862 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-04-20T10:45:38.862 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-04-20T10:45:38.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.863 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.863 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.863 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.863 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.863 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.863 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T10:45:38.863 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.864 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.864 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-04-20T10:45:38.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-04-20T10:45:38.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.864 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.864 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.864 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.865 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.865 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.865 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T10:45:38.865 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.865 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.865 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.865 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-04-20T10:45:38.866 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-04-20T10:45:38.866 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.866 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.866 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.866 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.866 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.866 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.867 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T10:45:38.867 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.867 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.867 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.867 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-04-20T10:45:38.867 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-04-20T10:45:38.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.867 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.868 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.868 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.868 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.868 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.868 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T10:45:38.868 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.868 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.869 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-04-20T10:45:38.869 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-04-20T10:45:38.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.869 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.869 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-04-20T10:45:38.869 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.870 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.870 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.870 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-04-20T10:45:38.870 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.870 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.870 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-04-20T10:45:38.870 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-04-20T10:45:38.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.871 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.871 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.871 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.871 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.871 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.871 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.871 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T10:45:38.872 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.872 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.872 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-04-20T10:45:38.872 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-04-20T10:45:38.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.872 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.873 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.873 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.873 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.873 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.873 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T10:45:38.873 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.873 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.874 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-04-20T10:45:38.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-04-20T10:45:38.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.874 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.874 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.874 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.874 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.875 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.875 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T10:45:38.875 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.875 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.876 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-04-20T10:45:38.876 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-04-20T10:45:38.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.876 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.876 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.876 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.876 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.876 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.877 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T10:45:38.877 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.877 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.877 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.877 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-04-20T10:45:38.878 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-04-20T10:45:38.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.878 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.878 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.878 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.878 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.878 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.879 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T10:45:38.879 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.879 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.879 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-04-20T10:45:38.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-04-20T10:45:38.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.880 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.880 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.880 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.880 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.880 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.881 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T10:45:38.881 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.881 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.882 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-04-20T10:45:38.882 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.882 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.882 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.882 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.882 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.883 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.883 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.883 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.883 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.883 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.884 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.884 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.884 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.884 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.884 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.884 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.884 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.884 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.885 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.885 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.885 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.885 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.885 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.886 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.886 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.886 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.886 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.886 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.886 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.887 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.887 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.887 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.887 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.887 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.887 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.887 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.888 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.888 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.888 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.888 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.888 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.888 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.888 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.888 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.889 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.889 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.889 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.889 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.889 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.889 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.889 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.890 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.890 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.890 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.890 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.891 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.891 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.891 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.891 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.891 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.891 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.892 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-04-20T10:45:38.892 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-04-20T10:45:38.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.892 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.892 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.892 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-04-20T10:45:38.892 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.892 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.892 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.893 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T10:45:38.893 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.893 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.893 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.893 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-04-20T10:45:38.893 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-04-20T10:45:38.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:38.894 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.894 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.894 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-04-20T10:45:38.894 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.894 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.894 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.894 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T10:45:38.894 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.895 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.895 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.895 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.895 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:38.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:38.895 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-04-20T10:45:38.895 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:38.895 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:38.896 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:38.896 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T10:45:38.896 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T10:45:38.896 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.896 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.896 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.896 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.896 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.896 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.897 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.897 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.897 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.897 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.897 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T10:45:38.897 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T10:45:38.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.898 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.898 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.898 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.898 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.898 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.898 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.898 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.899 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.899 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T10:45:38.899 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T10:45:38.899 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.899 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.899 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.899 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.900 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.900 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.900 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.900 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.900 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.900 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.900 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.901 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-04-20T10:45:38.901 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T10:45:38.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.901 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.901 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.901 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.901 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.901 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.901 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.902 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.902 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.902 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.902 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T10:45:38.902 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T10:45:38.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.902 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.903 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.903 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.903 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.903 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.903 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.903 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.904 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.904 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T10:45:38.904 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T10:45:38.904 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.904 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.904 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.904 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.904 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.905 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.905 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.905 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.905 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.905 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.905 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.905 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T10:45:38.905 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T10:45:38.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.906 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.906 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.906 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.906 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.906 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.906 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.907 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.907 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.907 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T10:45:38.907 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T10:45:38.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.907 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.908 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.908 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.908 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.908 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.908 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.908 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.908 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.909 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T10:45:38.909 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T10:45:38.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.909 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.909 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.909 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.909 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.909 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.910 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.910 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.910 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.910 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.910 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.910 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T10:45:38.910 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T10:45:38.910 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.911 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.911 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.911 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.911 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.911 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.911 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.911 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.911 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.912 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.912 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T10:45:38.912 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T10:45:38.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.912 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.912 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.913 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.913 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.913 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.913 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.913 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.913 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.913 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.913 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T10:45:38.914 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T10:45:38.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.914 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.914 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.914 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.914 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.914 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.914 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.915 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.915 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.915 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.915 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T10:45:38.915 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T10:45:38.915 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.915 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.916 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.916 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.916 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.916 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.916 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.916 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.916 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.916 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.916 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.917 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T10:45:38.917 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T10:45:38.917 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.917 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.917 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.917 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.917 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.917 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.918 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.918 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.918 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.918 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.918 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.918 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T10:45:38.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T10:45:38.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.919 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.919 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.919 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-20T10:45:38.919 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.919 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.919 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.919 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.919 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.920 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.920 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T10:45:38.920 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T10:45:38.920 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.920 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.920 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.920 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.921 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.921 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.921 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.921 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.921 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.921 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.921 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-04-20T10:45:38.921 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-04-20T10:45:38.922 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.922 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.922 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.922 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.922 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.922 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.922 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.922 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.923 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.923 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.923 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.923 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T10:45:38.923 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T10:45:38.923 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.923 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.924 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.924 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-20T10:45:38.924 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.924 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.924 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.924 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.924 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.924 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.924 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.925 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T10:45:38.925 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T10:45:38.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.925 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.925 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.925 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.926 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.926 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.926 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.926 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.926 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.926 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T10:45:38.926 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T10:45:38.927 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.927 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.927 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.927 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.927 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.927 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.927 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.927 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.928 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.928 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.928 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T10:45:38.928 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T10:45:38.928 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.928 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.928 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.928 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.929 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.929 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.929 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.929 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.929 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.929 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.929 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.929 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T10:45:38.930 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T10:45:38.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.930 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.930 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.930 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.930 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.930 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T10:45:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T10:45:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.931 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "[]", 2024-04-20T10:45:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.932 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T10:45:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T10:45:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "{}", 2024-04-20T10:45:38.933 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T10:45:38.934 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T10:45:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.935 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T10:45:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T10:45:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.936 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.937 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T10:45:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T10:45:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.938 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T10:45:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T10:45:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.939 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.940 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T10:45:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T10:45:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.941 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T10:45:38.942 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T10:45:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.943 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T10:45:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T10:45:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.944 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T10:45:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T10:45:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.946 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T10:45:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T10:45:38.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.948 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T10:45:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T10:45:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.949 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.950 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T10:45:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T10:45:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T10:45:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.951 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T10:45:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T10:45:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.954 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T10:45:38.954 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T10:45:38.954 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.954 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.954 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-04-20T10:45:38.954 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.954 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.955 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.955 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.955 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.955 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.955 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.955 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T10:45:38.955 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T10:45:38.955 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.956 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.956 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.956 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-04-20T10:45:38.956 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.956 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.956 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.956 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.956 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.957 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.957 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T10:45:38.957 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T10:45:38.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.957 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.958 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.958 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.958 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.958 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.958 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.958 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.958 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.958 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.959 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T10:45:38.959 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T10:45:38.959 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.959 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.959 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.959 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-20T10:45:38.959 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.959 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.960 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.960 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.960 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.960 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.960 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.960 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T10:45:38.960 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T10:45:38.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.961 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.961 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.961 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.961 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.961 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.961 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.961 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.962 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.962 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T10:45:38.962 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T10:45:38.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.962 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.962 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.963 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.963 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.963 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.963 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.963 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.963 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.963 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.963 INFO:teuthology.orchestra.run.smithi042.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T10:45:38.964 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T10:45:38.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.964 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.964 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.964 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.964 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.964 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.964 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.965 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.965 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.965 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T10:45:38.965 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T10:45:38.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.966 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.966 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.966 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:38.966 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.966 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.966 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.966 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.966 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.967 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.967 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T10:45:38.967 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T10:45:38.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.967 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.967 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.967 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:38.968 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.968 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.968 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.968 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.968 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.968 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.968 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.968 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T10:45:38.969 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T10:45:38.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.969 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.969 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-20T10:45:38.969 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.969 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.970 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.970 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.970 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.970 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.970 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-04-20T10:45:38.970 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-04-20T10:45:38.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.971 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.971 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.971 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.971 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.971 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.971 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.971 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.972 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.972 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-04-20T10:45:38.972 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-04-20T10:45:38.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.972 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.972 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.972 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.973 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.973 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.973 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.973 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.973 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.973 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.973 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.973 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-04-20T10:45:38.974 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-04-20T10:45:38.974 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.974 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.974 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.974 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.974 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.974 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.974 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.974 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-04-20T10:45:38.975 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.975 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.975 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.975 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-04-20T10:45:38.975 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-04-20T10:45:38.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.975 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.976 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-04-20T10:45:38.976 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.976 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.976 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.976 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.976 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.976 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.977 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-04-20T10:45:38.977 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-04-20T10:45:38.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.977 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.977 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.977 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.977 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.978 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.978 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.978 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.978 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.978 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:38.978 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:38.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.979 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.979 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.979 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.979 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.979 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.979 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.979 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.979 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.979 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.980 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.980 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.980 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.980 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.980 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.980 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.980 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.980 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.981 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:38.981 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:38.981 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.981 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.981 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.981 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.981 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.981 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.981 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.982 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.982 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.982 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.982 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:38.982 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:38.982 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.982 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.983 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:38.983 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.983 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.983 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.983 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:38.983 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:38.983 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:38.983 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.984 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:38.984 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:38.984 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.984 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.984 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.984 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.984 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:38.985 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:38.985 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.985 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.985 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:38.985 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.985 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.985 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.985 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.985 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.986 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.986 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.986 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-04-20T10:45:38.986 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-04-20T10:45:38.986 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.986 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.987 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:38.987 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.987 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.987 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.987 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-04-20T10:45:38.987 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.987 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.988 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.988 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-04-20T10:45:38.988 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T10:45:38.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.988 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.988 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.988 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:38.989 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.989 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.989 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.989 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.989 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.989 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.989 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.990 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-20T10:45:38.990 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-20T10:45:38.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.990 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.990 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.990 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-04-20T10:45:38.990 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.990 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.991 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.991 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.991 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.991 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.991 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.991 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-20T10:45:38.991 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-20T10:45:38.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.992 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.992 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.992 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-04-20T10:45:38.992 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.992 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.992 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.992 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.992 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.992 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.993 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.993 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-04-20T10:45:38.993 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-04-20T10:45:38.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:38.993 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.993 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:38.993 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.994 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.994 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.994 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.994 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.994 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.994 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.995 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-04-20T10:45:38.995 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-04-20T10:45:38.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.995 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.995 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.995 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-04-20T10:45:38.995 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.995 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.995 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.996 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.996 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.996 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.996 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-04-20T10:45:38.996 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-04-20T10:45:38.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:38.996 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.997 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.997 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-04-20T10:45:38.997 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:38.997 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:38.997 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:38.997 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:38.997 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-04-20T10:45:38.997 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:38.997 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.998 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.998 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:38.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:38.998 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-04-20T10:45:38.998 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-04-20T10:45:38.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:38.998 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:38.999 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:38.999 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-04-20T10:45:38.999 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-04-20T10:45:38.999 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-04-20T10:45:38.999 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:38.999 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:38.999 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:38.999 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:38.999 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.000 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-04-20T10:45:39.000 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-04-20T10:45:39.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.000 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.000 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.000 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.000 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.001 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.001 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.001 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.001 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.001 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.001 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.001 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.002 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-04-20T10:45:39.002 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.002 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.002 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.002 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-04-20T10:45:39.002 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-04-20T10:45:39.002 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.002 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.002 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.003 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.003 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.003 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.003 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.003 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-04-20T10:45:39.003 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.003 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.003 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.004 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.004 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.004 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.004 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.004 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.004 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.005 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.005 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.005 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.005 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.005 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.005 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.005 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.005 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.005 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.006 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.006 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.006 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.006 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.006 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.006 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.009 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.009 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.009 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.009 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.009 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.009 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.009 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.009 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.009 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.010 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.010 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.010 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.010 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.010 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.010 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.011 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.011 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.011 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.011 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.011 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.011 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.011 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.011 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.012 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.012 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-04-20T10:45:39.012 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-04-20T10:45:39.012 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.012 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.012 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.012 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-04-20T10:45:39.012 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.013 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.013 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.013 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T10:45:39.013 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.013 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.013 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.013 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-04-20T10:45:39.013 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-04-20T10:45:39.014 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.014 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.014 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.014 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-04-20T10:45:39.014 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.014 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.014 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.014 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T10:45:39.014 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.015 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.015 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.015 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-04-20T10:45:39.015 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-04-20T10:45:39.015 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.015 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.015 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.015 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-04-20T10:45:39.016 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.016 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.016 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.016 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-04-20T10:45:39.016 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.016 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.016 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.017 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-04-20T10:45:39.017 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-04-20T10:45:39.017 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.017 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.017 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-04-20T10:45:39.017 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.017 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.017 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.018 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T10:45:39.018 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.018 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.018 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.018 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-04-20T10:45:39.018 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-04-20T10:45:39.018 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.018 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.019 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.019 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.019 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.019 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.019 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.019 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T10:45:39.019 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.019 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.019 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.020 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:39.020 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:39.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.020 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.020 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.020 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:39.020 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.021 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.021 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.021 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T10:45:39.021 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.021 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.021 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-04-20T10:45:39.021 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-04-20T10:45:39.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.022 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.022 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-04-20T10:45:39.022 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.022 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.022 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.022 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T10:45:39.023 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.023 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.023 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.023 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.023 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.023 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.023 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-04-20T10:45:39.023 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.024 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.024 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.024 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.024 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.024 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.024 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.024 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.024 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.025 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.025 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.025 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.025 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.025 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.025 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.025 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.025 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.025 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.026 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.026 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.026 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.026 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.026 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.026 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.027 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.027 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.027 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.027 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.027 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.027 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.027 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.027 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.027 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.028 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.028 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.028 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.028 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.028 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.028 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.028 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.028 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.029 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.029 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.029 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.029 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.029 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.029 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.029 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.029 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.029 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.030 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.030 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.030 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.030 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.030 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.030 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.030 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.030 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.031 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.031 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.031 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.031 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.031 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.031 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.031 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.032 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.032 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-04-20T10:45:39.032 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-04-20T10:45:39.032 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.032 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.032 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.032 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-04-20T10:45:39.033 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.033 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.033 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.033 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.033 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.033 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.033 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-04-20T10:45:39.034 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-04-20T10:45:39.034 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.034 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.034 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.034 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-04-20T10:45:39.034 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.034 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.034 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.035 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.035 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.035 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.035 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.035 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:39.035 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:39.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.036 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.036 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.036 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:39.036 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.036 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.036 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.036 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.036 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.036 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.037 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.037 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.037 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.037 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.037 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-04-20T10:45:39.037 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-04-20T10:45:39.037 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-04-20T10:45:39.037 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.038 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-04-20T10:45:39.038 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-04-20T10:45:39.038 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.038 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.038 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-04-20T10:45:39.038 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.038 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.039 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.039 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T10:45:39.039 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.039 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.039 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-04-20T10:45:39.039 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-04-20T10:45:39.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.040 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.040 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.040 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-04-20T10:45:39.040 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.040 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.040 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.040 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T10:45:39.040 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.041 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.041 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.041 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-04-20T10:45:39.041 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-04-20T10:45:39.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.041 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.041 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.042 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.042 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.042 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.042 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-04-20T10:45:39.042 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.042 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.042 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.043 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:39.043 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:39.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.043 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.043 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-04-20T10:45:39.043 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-04-20T10:45:39.043 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.043 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.044 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T10:45:39.044 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.044 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.044 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.044 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.044 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.045 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.045 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.045 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.045 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.045 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.045 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.045 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.045 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.046 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.046 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.046 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.046 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.046 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.046 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.046 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.047 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.047 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.047 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.047 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.047 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.047 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.047 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.047 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.047 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.048 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.048 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.048 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.048 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.048 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.048 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.048 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.049 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.049 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.049 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.049 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.049 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.049 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.049 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.049 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.049 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.050 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.050 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.050 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.050 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.050 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.050 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.050 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.051 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.051 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.051 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.051 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.051 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.051 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.051 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.051 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.051 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.052 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.052 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.052 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.052 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.052 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-04-20T10:45:39.052 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-04-20T10:45:39.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.053 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.053 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.053 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.053 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.053 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.053 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.053 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-04-20T10:45:39.053 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.053 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.054 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.054 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-04-20T10:45:39.054 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-04-20T10:45:39.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.054 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.054 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-04-20T10:45:39.055 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.055 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.055 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.055 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-04-20T10:45:39.055 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.055 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.055 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-04-20T10:45:39.056 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-04-20T10:45:39.056 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.056 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.056 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.056 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-04-20T10:45:39.056 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.056 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.056 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.056 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T10:45:39.057 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.057 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.057 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-04-20T10:45:39.057 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-04-20T10:45:39.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.057 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.058 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-20T10:45:39.058 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-20T10:45:39.058 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-04-20T10:45:39.058 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.058 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T10:45:39.058 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.058 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.059 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-04-20T10:45:39.059 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-04-20T10:45:39.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.059 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.059 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.059 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.059 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-04-20T10:45:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-04-20T10:45:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-04-20T10:45:39.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.061 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.061 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-04-20T10:45:39.061 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.061 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.061 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.061 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T10:45:39.061 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.062 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.062 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.062 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-04-20T10:45:39.062 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.062 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.063 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.063 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.063 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.063 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.063 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.063 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.063 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.064 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.064 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.064 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.064 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.064 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.064 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.064 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.064 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.064 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.065 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.065 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.065 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.065 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.065 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.065 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.065 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.065 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.066 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.066 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.066 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.066 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.066 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.066 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.066 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.066 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.070 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.070 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.070 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.070 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.070 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.070 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.070 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.071 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.071 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.071 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.071 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.071 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.071 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.071 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.071 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.071 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.072 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.072 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.072 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.072 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.072 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.072 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.072 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.073 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.073 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.073 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.073 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.073 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.073 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.073 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.073 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.074 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.074 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-04-20T10:45:39.074 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.074 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.075 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.075 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.075 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.075 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.075 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.075 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.075 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.076 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.076 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.076 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.076 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.076 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.076 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.076 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.076 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.076 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.077 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.077 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.077 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.077 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.077 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.078 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.078 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.078 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.078 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.078 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.078 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.078 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.078 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.079 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.079 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.079 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.079 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.079 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.079 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.080 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.080 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.080 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.080 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.080 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.080 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.080 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.080 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.080 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.081 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.081 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.081 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.081 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.081 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.082 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.082 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.082 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.082 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.082 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.082 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.082 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.082 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.083 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.083 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.083 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-04-20T10:45:39.083 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.083 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.083 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.084 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-04-20T10:45:39.084 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-04-20T10:45:39.084 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.084 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.084 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.084 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-04-20T10:45:39.084 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.084 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.085 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.085 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-20T10:45:39.085 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.085 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.085 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.085 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-04-20T10:45:39.085 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-04-20T10:45:39.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.086 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.086 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:39.086 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-04-20T10:45:39.086 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.086 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.086 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-20T10:45:39.086 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.087 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.087 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.087 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.087 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.087 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.088 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.088 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.088 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.088 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.088 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.088 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.088 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.088 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.088 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.089 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.089 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.089 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.089 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.089 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.089 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.089 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.090 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.090 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.090 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.090 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.090 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.090 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.091 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.091 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.091 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.091 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.091 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.091 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.091 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.092 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.092 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.092 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.092 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.092 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.092 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.092 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.092 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.092 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.093 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.093 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.093 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.093 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.093 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.093 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.094 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.094 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.094 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.094 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.094 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.094 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.094 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.095 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.095 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.095 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-04-20T10:45:39.095 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.096 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.096 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.096 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-04-20T10:45:39.096 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-04-20T10:45:39.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.096 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.096 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.096 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-04-20T10:45:39.097 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.097 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.097 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.097 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-04-20T10:45:39.097 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T10:45:39.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.097 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.097 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.098 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.098 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.098 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.098 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.098 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.098 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.099 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.099 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.099 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.099 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.099 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.099 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.099 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.099 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.099 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.100 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.100 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.100 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.100 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.101 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.101 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.101 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.101 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.101 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.101 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.101 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.102 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.102 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.102 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.102 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.103 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.103 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.103 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.103 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.103 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.103 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.103 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.103 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.103 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.104 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.104 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.104 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.104 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.104 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.104 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.105 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.105 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.105 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.105 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.105 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.105 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.105 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.106 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.106 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.106 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-04-20T10:45:39.106 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-04-20T10:45:39.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.106 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.107 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.107 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.107 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.107 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.107 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.107 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-04-20T10:45:39.107 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T10:45:39.107 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.107 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.108 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-04-20T10:45:39.108 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-04-20T10:45:39.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.108 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.108 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-04-20T10:45:39.108 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.109 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.109 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.109 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-04-20T10:45:39.109 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.109 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.110 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-04-20T10:45:39.110 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-04-20T10:45:39.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.110 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.110 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-04-20T10:45:39.111 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.111 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.111 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.111 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T10:45:39.111 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.111 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.112 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-04-20T10:45:39.112 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-04-20T10:45:39.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.112 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.112 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.112 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.113 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.113 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.113 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-04-20T10:45:39.113 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.113 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.113 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-04-20T10:45:39.114 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-04-20T10:45:39.114 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.114 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.114 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-04-20T10:45:39.114 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.115 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.115 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.115 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T10:45:39.115 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T10:45:39.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.115 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.116 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.116 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-04-20T10:45:39.116 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.116 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.116 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.116 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.117 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.117 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.117 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.117 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.117 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.117 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.118 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.118 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.118 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.118 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.118 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.118 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.119 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.119 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.119 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.119 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.119 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.119 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.119 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.120 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.120 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.120 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.120 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.120 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.121 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.121 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.121 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.121 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.122 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.122 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.122 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.122 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.122 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.122 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.123 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.123 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.123 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.123 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.123 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.123 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.123 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.123 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.124 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.124 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.124 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.124 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.124 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.124 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.124 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.125 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.125 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.125 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.125 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.125 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.125 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.125 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.126 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.126 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.126 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.126 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.126 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-04-20T10:45:39.126 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.127 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.127 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.127 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.127 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.127 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.127 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.127 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.127 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.127 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.128 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.128 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.128 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.128 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.128 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.128 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.128 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.128 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.128 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.129 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.129 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.129 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.129 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.129 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.130 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.130 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.130 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.130 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.130 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.130 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.130 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.131 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.131 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.131 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.131 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.131 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.131 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.131 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.132 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.132 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.132 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.132 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.132 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.132 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.132 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.132 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.132 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.133 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.133 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.133 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.133 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.133 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.133 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.134 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.134 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.134 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.134 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.134 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.135 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.135 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.135 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.135 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.135 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.135 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.136 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-04-20T10:45:39.136 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.136 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.136 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.136 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.136 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.136 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.137 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.137 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.137 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.137 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.137 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.137 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.137 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.137 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.137 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.138 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.138 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.138 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.138 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.138 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.138 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.138 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.139 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.139 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.139 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.139 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.139 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.139 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.139 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.140 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.140 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.140 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.140 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.140 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.141 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.141 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.141 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.141 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.141 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.141 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.141 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.142 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.142 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.142 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.142 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.142 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.142 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.142 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.142 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.142 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.143 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.143 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.143 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.143 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.143 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.144 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.144 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.144 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.144 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.144 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.144 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.145 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.145 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-04-20T10:45:39.145 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.145 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.145 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.145 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-04-20T10:45:39.145 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-04-20T10:45:39.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T10:45:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.146 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.147 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.148 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.149 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.151 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.152 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.154 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-04-20T10:45:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-04-20T10:45:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.155 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-04-20T10:45:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-04-20T10:45:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-04-20T10:45:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-04-20T10:45:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.156 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-04-20T10:45:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.157 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.158 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.159 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.160 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.161 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.162 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.163 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.164 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-04-20T10:45:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.166 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.167 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.168 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.171 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.172 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.173 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.174 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-04-20T10:45:39.175 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.176 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.177 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.178 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.182 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.183 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.184 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-04-20T10:45:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-04-20T10:45:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.186 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-04-20T10:45:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-04-20T10:45:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-04-20T10:45:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T10:45:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.187 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-04-20T10:45:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.188 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-04-20T10:45:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-04-20T10:45:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.189 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T10:45:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.190 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-04-20T10:45:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-04-20T10:45:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.191 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.192 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.193 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.194 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.195 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.196 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.197 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.199 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.201 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-04-20T10:45:39.202 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-04-20T10:45:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-04-20T10:45:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.203 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-04-20T10:45:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-20T10:45:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-04-20T10:45:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-04-20T10:45:39.206 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-04-20T10:45:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-04-20T10:45:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.207 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.208 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-04-20T10:45:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-04-20T10:45:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.209 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T10:45:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T10:45:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.210 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.211 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.212 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.213 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.215 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.216 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.217 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.218 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-04-20T10:45:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-04-20T10:45:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.219 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.220 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T10:45:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T10:45:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-04-20T10:45:39.221 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.222 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-04-20T10:45:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-04-20T10:45:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-04-20T10:45:39.223 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.224 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-20T10:45:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-20T10:45:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-04-20T10:45:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.225 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T10:45:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-20T10:45:39.226 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-20T10:45:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-04-20T10:45:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.227 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T10:45:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-04-20T10:45:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-04-20T10:45:39.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-04-20T10:45:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.229 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-04-20T10:45:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-04-20T10:45:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.230 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-04-20T10:45:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-04-20T10:45:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-04-20T10:45:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.231 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-04-20T10:45:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-04-20T10:45:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-04-20T10:45:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-04-20T10:45:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-04-20T10:45:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.233 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.234 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.234 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.234 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.234 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-04-20T10:45:39.234 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.234 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.234 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.234 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.235 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.235 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.235 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.235 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.235 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.235 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.235 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.235 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.235 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.236 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.236 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.236 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.236 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.236 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.236 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.236 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.236 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.236 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.237 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.237 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.237 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.237 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.237 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.237 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.237 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.238 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.238 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.238 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.238 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.238 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.238 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.238 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.239 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.239 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.239 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.239 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.239 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.239 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.239 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.240 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.240 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.240 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.240 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.240 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.240 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.240 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.240 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.240 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.241 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.241 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.241 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.241 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.241 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.241 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.242 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.242 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.242 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.242 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.242 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.242 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.242 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:38.001+0000 7f06bb600640 1 -- 172.21.15.42:0/3688765759 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f06980a3d20 msgr2=0x7f06980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:39.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:38.001+0000 7f06bb600640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f06980a3d20 0x7f06980a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f06b4109710 tx=0x7f06ac009500 comp rx=0 tx=0).stop 2024-04-20T10:45:39.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:38.002+0000 7f06bb600640 1 -- 172.21.15.42:0/3688765759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06b410e670 msgr2=0x7f06b41a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:45:39.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:38.002+0000 7f06bb600640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06b410e670 0x7f06b41a36a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f06a402c180 tx=0x7f06a4004270 comp rx=0 tx=0).stop 2024-04-20T10:45:39.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:38.002+0000 7f06bb600640 1 -- 172.21.15.42:0/3688765759 shutdown_connections 2024-04-20T10:45:39.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:38.002+0000 7f06bb600640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f06980a3d20 0x7f06980a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:39.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:38.002+0000 7f06bb600640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06b410e670 0x7f06b41a36a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:39.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:38.002+0000 7f06bb600640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06b4108f90 0x7f06b419f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:39.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:38.002+0000 7f06bb600640 1 --2- 172.21.15.42:0/3688765759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06b41085d0 0x7f06b419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:45:39.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:38.002+0000 7f06bb600640 1 -- 172.21.15.42:0/3688765759 >> 172.21.15.42:0/3688765759 conn(0x7f06b4076580 msgr2=0x7f06b4077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:45:39.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:38.002+0000 7f06bb600640 1 -- 172.21.15.42:0/3688765759 shutdown_connections 2024-04-20T10:45:39.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:45:38.002+0000 7f06bb600640 1 -- 172.21.15.42:0/3688765759 wait complete. 2024-04-20T10:45:39.245 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:37 smithi162 ceph-mon[21854]: pgmap v361: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 170 KiB/s wr, 9 op/s 2024-04-20T10:45:39.245 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:37 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/402704261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:45:39.245 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:37 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1762176653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:45:39.245 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:37 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3688765759' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-20T10:45:39.246 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:37 smithi046 ceph-mon[21574]: pgmap v361: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 170 KiB/s wr, 9 op/s 2024-04-20T10:45:39.246 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:37 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/402704261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:45:39.246 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:37 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1762176653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:45:39.246 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:37 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3688765759' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-20T10:45:39.249 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-04-20T10:45:39.249 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-04-20T10:45:39.249 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.249 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.250 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:39.250 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.250 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.250 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.250 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.250 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.250 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.251 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-04-20T10:45:39.251 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T10:45:39.251 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.251 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.251 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.251 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.251 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.252 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.252 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.252 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.252 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.252 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.252 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.252 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-04-20T10:45:39.253 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-04-20T10:45:39.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.253 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.253 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.253 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.253 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.253 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.253 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.253 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.254 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.254 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-04-20T10:45:39.254 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.255 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.255 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.255 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-04-20T10:45:39.255 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-04-20T10:45:39.255 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.255 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.255 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.255 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.255 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.256 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.256 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.256 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.256 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.256 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.256 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.256 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-04-20T10:45:39.256 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-04-20T10:45:39.257 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.257 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.257 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.257 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.257 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.257 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.257 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.257 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.258 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.258 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.258 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.258 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.258 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.258 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.258 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.259 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.259 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.259 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.259 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.259 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.259 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.259 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.260 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.260 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.260 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.260 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.260 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.260 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.260 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.260 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.261 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.261 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.261 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.261 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.261 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.261 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.261 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.262 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.262 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.262 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.262 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.262 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.262 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.263 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.263 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.263 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.263 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.263 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.263 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.263 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.263 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.264 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.264 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.264 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.264 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.264 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.264 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.264 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.265 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.265 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.265 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.265 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.265 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.265 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.265 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.266 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.266 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.266 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.266 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-20T10:45:39.266 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-20T10:45:39.266 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.267 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.267 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.267 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.267 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.267 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.267 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.267 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.267 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.268 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.268 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-20T10:45:39.268 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-20T10:45:39.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.268 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.268 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.268 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.269 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.269 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.269 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.269 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.269 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.270 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-04-20T10:45:39.270 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.270 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.270 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.270 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.270 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.271 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.271 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.274 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.274 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.274 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.274 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.274 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.274 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.275 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.275 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.275 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.275 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.275 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.276 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.276 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.276 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.276 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.276 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.277 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.277 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.277 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.277 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.277 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.277 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.277 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.277 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.278 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.278 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.278 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.278 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.278 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.278 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.278 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.279 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.279 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.279 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.279 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.279 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.279 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.279 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.279 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.280 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.280 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.280 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.280 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.280 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.280 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.280 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.281 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.281 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.281 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.281 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.281 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.281 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.281 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.282 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.282 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.282 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.282 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-04-20T10:45:39.282 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.282 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.282 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.282 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.283 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.283 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.283 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.283 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.283 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.283 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.283 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.283 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.284 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.284 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.284 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.284 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.284 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.284 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.284 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.284 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.284 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.285 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.285 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.285 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.285 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.285 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.285 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.286 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.286 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.286 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.286 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.286 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.286 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.286 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.286 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.287 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.287 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.287 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.287 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.287 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.287 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.288 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.288 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.288 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.288 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.288 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.288 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.288 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.288 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.289 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.289 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.289 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.289 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.289 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.289 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.289 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.290 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.290 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.290 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.290 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.290 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.290 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.290 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-04-20T10:45:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-04-20T10:45:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-04-20T10:45:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.292 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.292 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T10:45:39.292 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.292 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.292 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.292 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.293 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.293 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-04-20T10:45:39.293 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.293 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.293 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.293 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.293 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.294 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.294 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.294 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.294 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.294 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.294 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.294 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.295 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.295 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.295 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.295 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.295 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.295 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.295 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.295 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.296 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.296 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.296 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.296 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.296 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.297 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.297 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.297 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.297 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.297 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.297 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.298 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.298 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.298 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.298 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.298 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.299 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.299 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.299 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.299 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.299 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.299 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.299 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.299 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.300 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.300 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.300 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.300 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.300 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.300 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.301 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.301 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.301 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.301 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.301 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.301 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.302 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.302 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.302 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.302 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.302 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.302 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.302 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.302 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.303 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-04-20T10:45:39.303 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-04-20T10:45:39.303 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.303 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.303 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.304 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.304 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.304 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.304 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.304 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.304 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.304 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.304 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.305 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-04-20T10:45:39.305 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-04-20T10:45:39.305 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.305 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.305 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-04-20T10:45:39.306 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.306 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.306 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.306 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.306 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.306 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.307 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.307 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-04-20T10:45:39.307 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-04-20T10:45:39.307 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.307 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.307 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.307 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.308 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.308 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.308 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.308 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.308 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.308 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.308 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.309 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-04-20T10:45:39.309 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-04-20T10:45:39.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.309 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.309 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.309 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.310 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.310 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.310 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.310 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.310 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.310 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.311 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-04-20T10:45:39.311 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-04-20T10:45:39.311 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:39.311 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.311 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.311 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.311 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.311 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.312 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.312 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.312 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.312 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.312 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.313 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-04-20T10:45:39.313 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-04-20T10:45:39.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.313 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.314 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.314 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.314 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.314 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.314 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.314 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.315 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.315 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.315 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-04-20T10:45:39.315 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-04-20T10:45:39.315 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.315 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.315 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.315 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.316 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.316 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.316 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.316 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.316 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.316 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.316 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.317 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-04-20T10:45:39.317 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-04-20T10:45:39.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.317 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.317 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.317 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.317 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.317 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.318 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.318 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.318 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.318 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.318 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-04-20T10:45:39.318 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-04-20T10:45:39.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.319 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.319 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.319 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.319 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-20T10:45:39.319 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-04-20T10:45:39.319 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.319 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.319 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.320 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.320 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.320 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-04-20T10:45:39.320 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-04-20T10:45:39.320 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.320 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.320 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.321 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.321 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.321 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.321 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.321 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.321 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.321 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.321 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.322 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-04-20T10:45:39.322 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-04-20T10:45:39.322 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.322 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.322 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.322 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.322 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.323 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.323 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.323 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.323 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.323 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.323 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.323 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-04-20T10:45:39.323 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-04-20T10:45:39.324 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.324 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.324 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.324 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.324 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.324 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.324 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.324 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.324 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.325 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.325 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.325 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-04-20T10:45:39.325 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.326 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.326 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.326 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-04-20T10:45:39.326 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-04-20T10:45:39.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.326 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.326 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.327 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.327 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.327 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.327 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T10:45:39.327 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.327 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.327 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.327 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-04-20T10:45:39.328 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-04-20T10:45:39.328 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.328 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.328 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.328 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.328 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.328 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.328 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.328 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-04-20T10:45:39.329 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.329 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.329 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.329 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.329 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.329 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.330 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.330 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.330 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.330 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.330 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.330 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.330 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.330 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.331 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.331 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.331 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.331 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.331 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.331 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.331 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.331 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.332 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.332 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.332 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.332 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.332 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.332 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.332 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.332 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.332 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.333 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.333 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.333 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.333 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.333 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.333 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.333 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.334 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.334 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.334 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.334 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.334 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.334 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.334 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.335 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.335 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.335 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.335 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.335 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.335 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.335 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.335 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.335 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.336 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.336 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.336 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.336 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.336 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.336 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.336 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.337 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.337 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.337 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.337 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.337 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.337 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.337 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.338 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-04-20T10:45:39.338 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.338 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.338 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.338 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.338 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.339 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.339 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.339 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.339 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.339 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.339 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.339 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.340 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.340 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.340 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.340 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.340 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.340 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.340 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.340 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.340 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.341 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.341 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.341 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.341 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.341 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.341 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.341 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.341 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.342 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.342 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.342 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.342 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.342 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.342 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.343 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.343 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.343 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.343 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.343 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.343 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.343 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.343 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.344 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.344 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.344 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.344 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.344 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.344 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.344 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.344 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.345 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.345 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.345 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.345 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.345 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.345 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.345 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.345 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.346 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.346 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.346 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.346 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.346 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.346 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.346 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.346 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.346 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.347 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-04-20T10:45:39.347 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.347 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.347 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.347 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.348 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.348 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.348 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.348 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.348 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.348 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.349 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.349 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.349 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.349 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.349 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.349 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.350 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.350 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.350 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.350 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.350 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.350 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.350 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.350 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.351 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.351 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.351 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.351 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.351 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.351 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.351 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.351 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.352 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.352 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.352 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.352 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.352 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.352 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.352 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.353 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.353 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.353 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.353 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.353 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.353 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.353 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.353 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.353 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.354 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.354 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.354 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.354 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.354 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.354 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.355 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.355 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.355 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.355 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.355 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.355 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.355 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.355 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.355 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.356 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.356 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-04-20T10:45:39.356 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.356 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.357 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.357 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-04-20T10:45:39.357 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-04-20T10:45:39.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.357 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.357 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T10:45:39.357 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.357 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.358 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.358 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.358 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.358 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.358 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.358 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:39.358 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:39.358 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.359 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.359 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.359 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-04-20T10:45:39.359 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.359 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.359 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.359 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.359 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.359 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.360 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.360 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.360 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.360 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.360 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.360 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.360 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.361 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.361 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.361 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.361 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.361 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.361 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.361 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.361 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.361 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.362 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.362 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.362 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.362 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.362 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.362 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.362 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.363 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.363 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.363 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.363 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.363 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.363 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.363 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.363 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.364 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.364 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.364 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.364 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.364 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.364 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.364 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.365 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.365 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.365 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.365 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.365 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.365 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.365 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.365 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.365 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.366 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.366 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.366 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.366 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.366 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.366 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.366 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.367 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.367 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.367 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.367 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.367 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.367 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.367 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.367 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.367 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.368 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.368 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.368 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.368 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.368 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-04-20T10:45:39.369 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.369 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.369 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.369 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-04-20T10:45:39.369 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-04-20T10:45:39.369 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.369 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.369 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.370 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.370 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.370 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.370 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.370 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T10:45:39.370 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.370 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.370 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.371 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-04-20T10:45:39.371 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-04-20T10:45:39.371 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.371 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.371 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.371 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.372 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.372 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.372 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.372 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T10:45:39.372 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.372 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.373 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.373 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-04-20T10:45:39.373 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-04-20T10:45:39.373 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.373 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.373 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.373 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.374 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.374 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.374 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.374 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T10:45:39.374 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.374 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.375 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.375 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-04-20T10:45:39.375 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-04-20T10:45:39.375 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.375 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.375 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.375 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.376 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.376 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.376 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.376 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T10:45:39.376 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.376 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.376 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.377 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-04-20T10:45:39.377 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-04-20T10:45:39.377 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.377 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.377 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.377 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.377 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.378 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.378 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.378 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T10:45:39.378 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.378 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.378 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.379 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-04-20T10:45:39.379 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-04-20T10:45:39.379 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.379 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.379 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.379 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.380 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.380 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.380 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.380 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.380 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.380 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.381 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-04-20T10:45:39.381 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-04-20T10:45:39.381 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.381 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.381 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.381 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.381 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.382 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.382 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.382 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.382 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.382 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.382 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.383 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-04-20T10:45:39.383 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-04-20T10:45:39.383 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.383 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.383 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.383 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T10:45:39.383 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.384 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.384 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.384 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.384 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.384 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.384 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-04-20T10:45:39.385 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-04-20T10:45:39.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.385 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.385 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.385 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.385 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.385 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.385 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.385 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.386 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.386 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.386 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.386 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:39.386 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:39.386 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.386 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.386 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-04-20T10:45:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-04-20T10:45:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.387 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-04-20T10:45:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-04-20T10:45:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-04-20T10:45:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.389 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-04-20T10:45:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-04-20T10:45:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-04-20T10:45:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-04-20T10:45:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.392 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.394 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.395 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.397 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.398 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.399 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.400 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-04-20T10:45:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-04-20T10:45:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-04-20T10:45:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-04-20T10:45:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-04-20T10:45:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-04-20T10:45:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T10:45:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.406 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-04-20T10:45:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.407 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.408 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.409 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.410 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.412 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.413 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.414 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.415 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-04-20T10:45:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.416 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.417 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.418 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.419 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.421 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.422 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.424 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-04-20T10:45:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-04-20T10:45:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.425 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-04-20T10:45:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T10:45:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-04-20T10:45:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-04-20T10:45:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.427 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-04-20T10:45:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-04-20T10:45:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-04-20T10:45:39.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T10:45:39.429 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_no_wait": { 2024-04-20T10:45:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T10:45:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.430 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T10:45:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.431 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-04-20T10:45:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-04-20T10:45:39.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-04-20T10:45:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-04-20T10:45:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.433 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-04-20T10:45:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-04-20T10:45:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.434 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.436 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.438 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.439 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.440 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.443 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.444 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.445 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-04-20T10:45:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-04-20T10:45:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.446 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.447 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-04-20T10:45:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-04-20T10:45:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-04-20T10:45:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.448 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-04-20T10:45:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-04-20T10:45:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.449 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T10:45:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.450 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-20T10:45:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-04-20T10:45:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-04-20T10:45:39.451 INFO:teuthology.orchestra.run.smithi042.stdout: "dashboard", 2024-04-20T10:45:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-04-20T10:45:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-04-20T10:45:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "restful", 2024-04-20T10:45:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_schedule" 2024-04-20T10:45:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-04-20T10:45:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-04-20T10:45:39.452 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:39.453 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T10:45:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.454 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.455 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.456 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.457 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.458 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.459 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.460 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.461 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.462 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-04-20T10:45:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-04-20T10:45:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.463 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-04-20T10:45:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-04-20T10:45:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-04-20T10:45:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.464 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-04-20T10:45:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-04-20T10:45:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-04-20T10:45:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-04-20T10:45:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.466 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-04-20T10:45:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-04-20T10:45:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-04-20T10:45:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.468 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-04-20T10:45:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.469 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-04-20T10:45:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-04-20T10:45:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-04-20T10:45:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.470 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-04-20T10:45:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-04-20T10:45:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-04-20T10:45:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.471 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-04-20T10:45:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.472 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-04-20T10:45:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-04-20T10:45:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.473 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T10:45:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.474 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-04-20T10:45:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-04-20T10:45:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-04-20T10:45:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.476 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-04-20T10:45:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-04-20T10:45:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-04-20T10:45:39.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T10:45:39.480 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-04-20T10:45:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-04-20T10:45:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-04-20T10:45:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T10:45:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T10:45:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.482 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-04-20T10:45:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-04-20T10:45:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-04-20T10:45:39.483 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-04-20T10:45:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T10:45:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T10:45:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-04-20T10:45:39.484 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-04-20T10:45:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-04-20T10:45:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-20T10:45:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-04-20T10:45:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T10:45:39.485 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-04-20T10:45:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-04-20T10:45:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.486 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T10:45:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T10:45:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T10:45:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-04-20T10:45:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-04-20T10:45:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-04-20T10:45:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-04-20T10:45:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-04-20T10:45:39.488 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-04-20T10:45:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T10:45:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-04-20T10:45:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-04-20T10:45:39.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-04-20T10:45:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T10:45:39.490 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T10:45:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-04-20T10:45:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-04-20T10:45:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.491 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-04-20T10:45:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-04-20T10:45:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T10:45:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T10:45:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.493 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.494 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.495 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.496 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.497 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.499 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.500 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-04-20T10:45:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-04-20T10:45:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T10:45:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-04-20T10:45:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-04-20T10:45:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-04-20T10:45:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.503 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-04-20T10:45:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-04-20T10:45:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-04-20T10:45:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-04-20T10:45:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-04-20T10:45:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-04-20T10:45:39.504 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-04-20T10:45:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-04-20T10:45:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.505 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T10:45:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.507 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T10:45:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-04-20T10:45:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-04-20T10:45:39.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-20T10:45:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-20T10:45:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T10:45:39.509 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T10:45:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-04-20T10:45:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-04-20T10:45:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.510 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T10:45:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-04-20T10:45:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-04-20T10:45:39.512 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-04-20T10:45:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-04-20T10:45:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T10:45:39.513 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-04-20T10:45:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-04-20T10:45:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-04-20T10:45:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T10:45:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-04-20T10:45:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-04-20T10:45:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-04-20T10:45:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.516 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T10:45:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-04-20T10:45:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-04-20T10:45:39.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T10:45:39.518 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T10:45:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-04-20T10:45:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-04-20T10:45:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.519 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T10:45:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-04-20T10:45:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T10:45:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-04-20T10:45:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.521 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T10:45:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-04-20T10:45:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-04-20T10:45:39.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-04-20T10:45:39.523 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-04-20T10:45:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-04-20T10:45:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T10:45:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-04-20T10:45:39.524 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-04-20T10:45:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T10:45:39.525 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-04-20T10:45:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-04-20T10:45:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.526 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T10:45:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-04-20T10:45:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-04-20T10:45:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.528 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T10:45:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-04-20T10:45:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-04-20T10:45:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.529 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T10:45:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-20T10:45:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-04-20T10:45:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-04-20T10:45:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T10:45:39.531 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-04-20T10:45:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.532 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-04-20T10:45:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-04-20T10:45:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T10:45:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.533 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-04-20T10:45:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-04-20T10:45:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-04-20T10:45:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T10:45:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-20T10:45:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.535 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-04-20T10:45:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-04-20T10:45:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T10:45:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.536 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-04-20T10:45:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-04-20T10:45:39.537 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-04-20T10:45:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T10:45:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.538 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-04-20T10:45:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-04-20T10:45:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-04-20T10:45:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.539 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T10:45:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-04-20T10:45:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-04-20T10:45:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-04-20T10:45:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T10:45:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.541 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-04-20T10:45:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-04-20T10:45:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-04-20T10:45:39.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T10:45:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-04-20T10:45:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-04-20T10:45:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-04-20T10:45:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.544 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T10:45:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-04-20T10:45:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-04-20T10:45:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-04-20T10:45:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T10:45:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.546 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-20T10:45:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-04-20T10:45:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-04-20T10:45:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.547 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T10:45:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-04-20T10:45:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.548 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-04-20T10:45:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-04-20T10:45:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T10:45:39.549 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-20T10:45:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.550 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-04-20T10:45:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-04-20T10:45:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T10:45:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.551 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-04-20T10:45:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_samba": { 2024-04-20T10:45:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_samba", 2024-04-20T10:45:39.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T10:45:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Samba/SMB container image", 2024-04-20T10:45:39.553 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-04-20T10:45:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-04-20T10:45:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.554 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T10:45:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-04-20T10:45:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-04-20T10:45:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-04-20T10:45:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.556 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.557 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.557 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.557 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T10:45:39.557 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.557 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.557 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.557 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-04-20T10:45:39.558 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-04-20T10:45:39.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.558 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.558 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:39.558 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.558 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.558 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.558 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T10:45:39.559 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.559 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.559 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-04-20T10:45:39.559 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T10:45:39.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.559 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.560 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-04-20T10:45:39.560 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.560 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.560 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.560 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T10:45:39.560 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.561 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.561 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-04-20T10:45:39.561 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-04-20T10:45:39.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.561 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.561 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.561 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-04-20T10:45:39.561 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.562 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.562 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.562 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T10:45:39.562 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.562 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.562 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.562 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-04-20T10:45:39.563 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-04-20T10:45:39.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.563 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.563 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-04-20T10:45:39.563 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.563 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.563 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.563 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-04-20T10:45:39.564 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.564 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.564 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-04-20T10:45:39.564 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-04-20T10:45:39.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.564 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.565 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.565 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.565 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.565 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.565 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.565 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T10:45:39.565 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.566 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.566 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-04-20T10:45:39.566 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-04-20T10:45:39.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.566 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.566 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:39.567 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.567 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.567 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.567 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-04-20T10:45:39.567 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.567 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.568 INFO:teuthology.orchestra.run.smithi042.stdout: "grafana_dashboards_path": { 2024-04-20T10:45:39.568 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "grafana_dashboards_path", 2024-04-20T10:45:39.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.568 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.568 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T10:45:39.568 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.568 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.568 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.569 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T10:45:39.569 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.569 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.569 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-04-20T10:45:39.569 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-04-20T10:45:39.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.570 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.570 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:39.570 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.570 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.570 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.570 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-04-20T10:45:39.570 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.570 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.571 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.571 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-04-20T10:45:39.571 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-04-20T10:45:39.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.571 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.571 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.572 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.572 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.572 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.572 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T10:45:39.572 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.572 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.573 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-04-20T10:45:39.573 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-04-20T10:45:39.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.573 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.573 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.573 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.573 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.573 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.573 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.574 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T10:45:39.574 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.574 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.574 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.574 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.574 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.575 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.575 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.575 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.575 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.575 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.575 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.575 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.575 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.576 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.576 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.576 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.576 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.576 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.576 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.576 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.576 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-04-20T10:45:39.577 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-04-20T10:45:39.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.577 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.577 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.577 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.577 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.577 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.577 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T10:45:39.578 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.578 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.578 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.578 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.578 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.579 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.579 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.579 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.579 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.579 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T10:45:39.579 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.579 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.580 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.580 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.580 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.580 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.580 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.581 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.581 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.581 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.581 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.581 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.581 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.581 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.581 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.581 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.582 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.582 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.582 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.582 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.582 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.583 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.583 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.583 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.583 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.583 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.583 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.583 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.584 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.584 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.584 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T10:45:39.584 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T10:45:39.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.584 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.585 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.585 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.585 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.585 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.585 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T10:45:39.585 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.586 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.586 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T10:45:39.586 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T10:45:39.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.586 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.586 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.586 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-04-20T10:45:39.586 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.587 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.587 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.587 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T10:45:39.587 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.587 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.587 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.587 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-04-20T10:45:39.587 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-04-20T10:45:39.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.588 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.588 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:39.588 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.588 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.588 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.588 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-04-20T10:45:39.589 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.589 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.589 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-04-20T10:45:39.589 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-04-20T10:45:39.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.589 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.590 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:39.590 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.590 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.590 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.590 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T10:45:39.590 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.590 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.591 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-04-20T10:45:39.591 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-04-20T10:45:39.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.591 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.591 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.591 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.592 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.592 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.592 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-04-20T10:45:39.592 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.592 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.592 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.592 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-04-20T10:45:39.592 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-04-20T10:45:39.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.593 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.593 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-04-20T10:45:39.593 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.593 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.593 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.593 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-04-20T10:45:39.594 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-04-20T10:45:39.594 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.594 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T10:45:39.594 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.594 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.594 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.594 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-04-20T10:45:39.595 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-04-20T10:45:39.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.595 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.595 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-04-20T10:45:39.595 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.595 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.595 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.596 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T10:45:39.596 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.596 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.596 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.596 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-04-20T10:45:39.596 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-04-20T10:45:39.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.596 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.597 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T10:45:39.597 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.597 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.597 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.597 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T10:45:39.597 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.597 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.597 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.598 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-04-20T10:45:39.598 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-04-20T10:45:39.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.598 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.598 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.598 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.599 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.599 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.599 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T10:45:39.599 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.599 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.599 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-04-20T10:45:39.599 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-04-20T10:45:39.600 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.600 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.600 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.600 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.600 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.600 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.600 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T10:45:39.600 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.601 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.601 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.601 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-04-20T10:45:39.601 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-04-20T10:45:39.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.601 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.601 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.602 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.602 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.602 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.602 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T10:45:39.602 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.602 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.603 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-04-20T10:45:39.603 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-04-20T10:45:39.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.603 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.603 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.603 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.603 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.603 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.604 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T10:45:39.604 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.604 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.604 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-04-20T10:45:39.604 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-04-20T10:45:39.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.604 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.605 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.605 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.605 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.605 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.605 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T10:45:39.605 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.605 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.605 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.606 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-04-20T10:45:39.606 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-04-20T10:45:39.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.606 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.606 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-04-20T10:45:39.606 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.606 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.607 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.607 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-04-20T10:45:39.607 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.607 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.607 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-04-20T10:45:39.607 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-04-20T10:45:39.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.608 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.608 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.608 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.608 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.608 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.608 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T10:45:39.608 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.609 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.609 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-04-20T10:45:39.609 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-04-20T10:45:39.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.609 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.609 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.610 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.610 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.610 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.610 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T10:45:39.610 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.610 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.610 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.610 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-04-20T10:45:39.611 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-04-20T10:45:39.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.611 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.611 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.611 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.611 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.611 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.612 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T10:45:39.612 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.612 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.612 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.612 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-04-20T10:45:39.612 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-04-20T10:45:39.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.612 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.613 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.613 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.613 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.613 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.613 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T10:45:39.613 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.613 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.614 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-04-20T10:45:39.614 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-04-20T10:45:39.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.614 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.614 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.615 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.615 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.615 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.615 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T10:45:39.615 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.615 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.615 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-04-20T10:45:39.616 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-04-20T10:45:39.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.616 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.616 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.616 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.616 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.616 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.617 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T10:45:39.617 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.617 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.618 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-04-20T10:45:39.618 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.618 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.618 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.618 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.618 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.618 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.619 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.619 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.619 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.619 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.619 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.619 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.619 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.620 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.620 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.620 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.620 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.620 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.620 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.620 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.620 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.621 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.621 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.621 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.621 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.621 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.622 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.622 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.622 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.622 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.622 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.622 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.622 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.623 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.623 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.623 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.623 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.623 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.623 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.623 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.624 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.624 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.624 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.624 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.624 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.624 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.624 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.624 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.625 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.625 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.625 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.625 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.625 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.625 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.626 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.626 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.626 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.626 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.626 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.626 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.627 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-04-20T10:45:39.627 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-04-20T10:45:39.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.627 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.627 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-04-20T10:45:39.627 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.627 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.628 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.628 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T10:45:39.628 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.628 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.628 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-04-20T10:45:39.628 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-04-20T10:45:39.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.629 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.629 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-04-20T10:45:39.629 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.629 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.629 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.629 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T10:45:39.629 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.630 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.630 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-04-20T10:45:39.631 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.631 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.631 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.631 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T10:45:39.631 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T10:45:39.631 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.631 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.631 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:39.632 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.632 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.632 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.632 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.632 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.632 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.632 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T10:45:39.633 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T10:45:39.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.633 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.633 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.633 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.633 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.633 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.634 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.634 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.634 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.634 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T10:45:39.634 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T10:45:39.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.634 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.635 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.635 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.635 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.635 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.635 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.635 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.635 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.635 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.636 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-04-20T10:45:39.636 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T10:45:39.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.636 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.636 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.636 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.636 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.637 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.637 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.637 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.637 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.637 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T10:45:39.637 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T10:45:39.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.638 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.638 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.638 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.638 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.638 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.638 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.638 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.639 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.639 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.639 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T10:45:39.639 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T10:45:39.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.639 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.639 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.640 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.640 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.640 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.640 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.640 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.640 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.641 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T10:45:39.641 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T10:45:39.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.641 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.641 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.641 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.641 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.641 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.642 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.642 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.642 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.642 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T10:45:39.642 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T10:45:39.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.643 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.643 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.643 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.643 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.643 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.643 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.643 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.644 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.644 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T10:45:39.644 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T10:45:39.644 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.644 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.644 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.644 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.645 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.645 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.645 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.645 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.645 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.645 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.645 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.645 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T10:45:39.646 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T10:45:39.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.646 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.646 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.646 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.646 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.646 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.647 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.647 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.647 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.647 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T10:45:39.647 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T10:45:39.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.647 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.648 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.648 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.648 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.648 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.648 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.648 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.649 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.649 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T10:45:39.649 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T10:45:39.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.649 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.649 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.649 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.649 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.650 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.650 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.650 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.650 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.650 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.650 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.650 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T10:45:39.650 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T10:45:39.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.651 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.651 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.651 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.651 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.651 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.651 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.651 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.652 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.652 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.652 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T10:45:39.652 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T10:45:39.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.652 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.653 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.653 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.653 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.653 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.653 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.653 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.653 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.654 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T10:45:39.654 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T10:45:39.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.654 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.654 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-20T10:45:39.654 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.655 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.655 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.655 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.655 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.655 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.655 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T10:45:39.655 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T10:45:39.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.656 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.656 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.656 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.656 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.656 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.656 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.656 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.656 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.657 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.657 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-04-20T10:45:39.657 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-04-20T10:45:39.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.657 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.658 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.658 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.658 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.658 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.658 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.658 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.658 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.658 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.659 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T10:45:39.659 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T10:45:39.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.659 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.659 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.659 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-20T10:45:39.659 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.659 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.659 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.660 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.660 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.660 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.660 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.660 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T10:45:39.660 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T10:45:39.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.661 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.661 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.661 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.661 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.661 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.661 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.661 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.662 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.662 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.662 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T10:45:39.662 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T10:45:39.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.662 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.662 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.663 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.663 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.663 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.663 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.663 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.663 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.663 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.664 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T10:45:39.664 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T10:45:39.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.664 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.664 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.664 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.664 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.664 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.664 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.665 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.665 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.665 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.665 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T10:45:39.665 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T10:45:39.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.665 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.666 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.666 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.666 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.666 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.666 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.666 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.666 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.667 INFO:teuthology.orchestra.run.smithi042.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T10:45:39.667 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T10:45:39.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.667 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.667 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "[]", 2024-04-20T10:45:39.667 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.667 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.668 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.668 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.668 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.668 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.668 INFO:teuthology.orchestra.run.smithi042.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T10:45:39.668 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T10:45:39.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.669 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.669 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.669 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "{}", 2024-04-20T10:45:39.669 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.669 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.669 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.669 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.670 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.670 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.670 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T10:45:39.670 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T10:45:39.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.670 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.671 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.671 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.671 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.671 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.671 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.671 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.671 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.672 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T10:45:39.672 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T10:45:39.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.672 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.672 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.672 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.673 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.673 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.673 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.673 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.673 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.673 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T10:45:39.674 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T10:45:39.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.674 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.674 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.674 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.674 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.674 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.675 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.675 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.675 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.675 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.675 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T10:45:39.675 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T10:45:39.675 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.675 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.676 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.676 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.676 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.676 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.676 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.676 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.676 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.676 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.677 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T10:45:39.677 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T10:45:39.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.677 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.677 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.677 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.677 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.678 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.678 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.678 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.678 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.678 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.678 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T10:45:39.678 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T10:45:39.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.679 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.679 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.679 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.679 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.679 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.679 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.679 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.680 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.680 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T10:45:39.680 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T10:45:39.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.680 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.680 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.680 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.681 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.681 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.681 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.681 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.681 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.681 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.681 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.683 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T10:45:39.683 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T10:45:39.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.684 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.684 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.684 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.684 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.684 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.684 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.684 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T10:45:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T10:45:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.685 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.686 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T10:45:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T10:45:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.687 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T10:45:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T10:45:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T10:45:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.689 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T10:45:39.690 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T10:45:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.691 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T10:45:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T10:45:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-04-20T10:45:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.693 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T10:45:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T10:45:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-04-20T10:45:39.694 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T10:45:39.695 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T10:45:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.696 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T10:45:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T10:45:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.697 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-20T10:45:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.698 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T10:45:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T10:45:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.699 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T10:45:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T10:45:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.700 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.701 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T10:45:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T10:45:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.702 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T10:45:39.703 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T10:45:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.704 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T10:45:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T10:45:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.705 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T10:45:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T10:45:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-20T10:45:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-04-20T10:45:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-04-20T10:45:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.709 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-04-20T10:45:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-04-20T10:45:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.710 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.711 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-04-20T10:45:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-04-20T10:45:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.712 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-04-20T10:45:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-04-20T10:45:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-04-20T10:45:39.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-04-20T10:45:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.714 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-04-20T10:45:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-04-20T10:45:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.715 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.716 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.717 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.718 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.719 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.720 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.721 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.722 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.723 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.724 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-04-20T10:45:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-04-20T10:45:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.725 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-04-20T10:45:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-04-20T10:45:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T10:45:39.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.727 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-20T10:45:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-20T10:45:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.728 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-04-20T10:45:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.729 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-20T10:45:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-20T10:45:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-04-20T10:45:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.730 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-04-20T10:45:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-04-20T10:45:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.731 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.732 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-04-20T10:45:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-04-20T10:45:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-04-20T10:45:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-04-20T10:45:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-04-20T10:45:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-04-20T10:45:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-04-20T10:45:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-04-20T10:45:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-04-20T10:45:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.737 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-04-20T10:45:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-04-20T10:45:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-04-20T10:45:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-04-20T10:45:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-04-20T10:45:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.739 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-04-20T10:45:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-04-20T10:45:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-04-20T10:45:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.741 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-04-20T10:45:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.743 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.744 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.745 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.748 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.749 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.750 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-04-20T10:45:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-04-20T10:45:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-04-20T10:45:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T10:45:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-04-20T10:45:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-04-20T10:45:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-04-20T10:45:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.753 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T10:45:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-04-20T10:45:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-04-20T10:45:39.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-04-20T10:45:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-04-20T10:45:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.755 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-04-20T10:45:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-04-20T10:45:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.756 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-04-20T10:45:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T10:45:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.757 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-04-20T10:45:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-04-20T10:45:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.758 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T10:45:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.759 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T10:45:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.760 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-04-20T10:45:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-04-20T10:45:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-04-20T10:45:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.761 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T10:45:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-04-20T10:45:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.764 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.766 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.767 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.768 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.769 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.770 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-04-20T10:45:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-04-20T10:45:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.771 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-04-20T10:45:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.772 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-04-20T10:45:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-04-20T10:45:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-04-20T10:45:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.773 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:39.774 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-20T10:45:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.775 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-04-20T10:45:39.776 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-04-20T10:45:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-04-20T10:45:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-04-20T10:45:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-04-20T10:45:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.777 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-04-20T10:45:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T10:45:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-04-20T10:45:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-04-20T10:45:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-04-20T10:45:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T10:45:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-04-20T10:45:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-04-20T10:45:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.781 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-04-20T10:45:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.782 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-04-20T10:45:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-04-20T10:45:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.783 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T10:45:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.784 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.785 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.787 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.789 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.790 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.791 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.792 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-04-20T10:45:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-04-20T10:45:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.793 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-04-20T10:45:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-04-20T10:45:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-04-20T10:45:39.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-04-20T10:45:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-04-20T10:45:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-04-20T10:45:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-04-20T10:45:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-04-20T10:45:39.796 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T10:45:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-04-20T10:45:39.797 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-04-20T10:45:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-20T10:45:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-20T10:45:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-04-20T10:45:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T10:45:39.798 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-04-20T10:45:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-04-20T10:45:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-04-20T10:45:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-04-20T10:45:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-04-20T10:45:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-04-20T10:45:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.801 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T10:45:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.802 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-04-20T10:45:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.804 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.805 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.806 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.808 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.809 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.810 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.811 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-04-20T10:45:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.812 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.813 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.813 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.813 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.813 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.813 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.813 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.813 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.813 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.814 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.814 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.814 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.814 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.814 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.814 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.815 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.815 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.815 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.815 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.815 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.815 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.815 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.816 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.816 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.816 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.816 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.816 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.817 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.817 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.817 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.817 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.817 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.817 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.817 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.817 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.817 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.818 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.818 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.818 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.818 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.818 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.818 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.819 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.819 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.819 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.819 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.819 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.819 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.819 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.820 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.820 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.820 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.820 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-04-20T10:45:39.821 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.821 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.821 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.821 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-04-20T10:45:39.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-04-20T10:45:39.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.821 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.822 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-04-20T10:45:39.822 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.822 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.822 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.822 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-20T10:45:39.822 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.823 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.823 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-04-20T10:45:39.823 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-04-20T10:45:39.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.823 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.824 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:39.824 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-04-20T10:45:39.824 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.824 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.824 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-20T10:45:39.824 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.824 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.825 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.825 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.825 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.825 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.825 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.825 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.826 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.826 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.826 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.826 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.826 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.826 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.826 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.826 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.826 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.827 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.827 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.827 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.827 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.828 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.828 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.828 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.828 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.828 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.828 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.828 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.828 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.829 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.829 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.829 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.829 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.830 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.830 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.830 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.830 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.830 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.830 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.830 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.830 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.831 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.831 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.831 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.831 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.831 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.831 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.831 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.832 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.832 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.832 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.832 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.832 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.832 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.833 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.833 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.833 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.833 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.833 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-04-20T10:45:39.834 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.834 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.834 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.834 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-04-20T10:45:39.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-04-20T10:45:39.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.834 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.835 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-04-20T10:45:39.835 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.835 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.835 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.835 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-04-20T10:45:39.835 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T10:45:39.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.835 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.836 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.836 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.836 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.836 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.836 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.836 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.837 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.837 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.837 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.837 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.837 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.837 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.837 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.837 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.838 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.838 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.838 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.838 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.838 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.839 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.839 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.839 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.839 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.839 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.839 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.839 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.840 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.840 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.840 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.840 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.840 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.840 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.841 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.841 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.841 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.841 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.841 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.841 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.841 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.841 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.842 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.842 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.842 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.842 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.842 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.842 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.843 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.843 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.843 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.843 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.843 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.843 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.843 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.844 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.844 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.844 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-04-20T10:45:39.844 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-04-20T10:45:39.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.844 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.844 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.844 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.845 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.845 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.845 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.845 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-04-20T10:45:39.845 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T10:45:39.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.845 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.845 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-04-20T10:45:39.846 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-04-20T10:45:39.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.846 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.846 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-04-20T10:45:39.846 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.846 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.846 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.847 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-04-20T10:45:39.847 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.847 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.847 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-04-20T10:45:39.847 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-04-20T10:45:39.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.847 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.848 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-04-20T10:45:39.848 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.848 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.848 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.848 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T10:45:39.848 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.849 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.849 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-04-20T10:45:39.849 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-04-20T10:45:39.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.849 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.849 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.850 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.850 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.850 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.850 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-04-20T10:45:39.850 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.850 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.850 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-04-20T10:45:39.851 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-04-20T10:45:39.851 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.851 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.851 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-04-20T10:45:39.851 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.851 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.851 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.851 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T10:45:39.852 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T10:45:39.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.852 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.852 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.852 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.852 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.852 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-04-20T10:45:39.852 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.853 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.853 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.853 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.853 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.853 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.853 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.853 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.854 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.854 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.854 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.854 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.854 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.854 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.854 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.854 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.855 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.855 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.855 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.855 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.855 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.855 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.856 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.856 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.856 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.856 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.856 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.856 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.856 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.857 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.857 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.857 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.857 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.857 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.857 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.858 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.858 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.858 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.858 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.858 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.858 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.858 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.858 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.859 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.859 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.859 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.859 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.859 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.859 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.860 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.860 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.860 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.860 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.860 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.860 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.860 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.861 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.861 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.861 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.861 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.861 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.861 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.861 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-04-20T10:45:39.862 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.862 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.862 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.862 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.862 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.862 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.862 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.863 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.863 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.863 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.863 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.863 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.863 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.863 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.863 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.863 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.864 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.864 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.864 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.864 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.864 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.864 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.865 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.865 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.865 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.865 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.865 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.865 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.865 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.865 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.865 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.866 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.866 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.866 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.866 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.866 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.866 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.866 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.867 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.867 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.867 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.867 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.867 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.867 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.867 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.867 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.868 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.868 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.868 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.868 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.868 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.868 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.868 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.869 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.869 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.869 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.869 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.870 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.870 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.870 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.870 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.870 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.870 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.871 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-04-20T10:45:39.871 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.871 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.871 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.872 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.872 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.872 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.872 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.872 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.873 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.873 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.873 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.873 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.873 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.873 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.873 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.873 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.874 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.874 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.874 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.874 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.874 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.875 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.875 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.875 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.875 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.875 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.875 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.875 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.875 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.876 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.876 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.876 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.876 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.876 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.876 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.877 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.877 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.877 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.877 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.877 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.877 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.877 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.877 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.878 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.878 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.878 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.878 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.878 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.878 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.878 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.879 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.879 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.879 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.879 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.879 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.879 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.879 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.879 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.880 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.880 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-04-20T10:45:39.880 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.880 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.881 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.881 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-04-20T10:45:39.881 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-04-20T10:45:39.881 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.881 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.881 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.881 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.881 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.881 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.882 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.882 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T10:45:39.882 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.882 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.882 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.882 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.883 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.883 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.883 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.883 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.883 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.883 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.883 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.884 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.884 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.884 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.884 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.884 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.884 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.884 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.884 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.884 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.885 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.885 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.887 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.887 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.888 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.888 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.888 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.888 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.888 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.888 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.888 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.889 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.889 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.889 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.889 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.889 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.889 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.889 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.889 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.890 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.890 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.890 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.890 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.890 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.890 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.890 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.890 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.891 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.891 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.891 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.891 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.892 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.892 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.892 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.892 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.892 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.892 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.892 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.893 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-04-20T10:45:39.893 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-04-20T10:45:39.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.893 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.893 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.893 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.893 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.894 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.894 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-04-20T10:45:39.894 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-04-20T10:45:39.894 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-04-20T10:45:39.894 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.894 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-04-20T10:45:39.894 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.894 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.895 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.895 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.895 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-04-20T10:45:39.895 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.895 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.895 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.896 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.896 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.896 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.896 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.896 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.896 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.896 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.896 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.897 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.897 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.897 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.897 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.897 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.897 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.897 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.897 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.897 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.898 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.898 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.898 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.898 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.899 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.899 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.899 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.899 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.899 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.899 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.899 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.899 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.900 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.900 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.900 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.900 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.900 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.900 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.901 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.901 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.901 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.901 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.901 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.901 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.901 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.901 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.901 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.901 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.902 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.902 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.902 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.902 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.902 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.902 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.903 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.903 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.903 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.903 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.903 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.903 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.903 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.904 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-04-20T10:45:39.904 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.904 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.904 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.904 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.904 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.905 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.905 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.905 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.905 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.905 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.905 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.905 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.905 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.905 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.906 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.906 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.906 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.906 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.906 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.906 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.906 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.907 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.907 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.907 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.907 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.907 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.908 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.908 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.908 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.908 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.908 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.908 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.909 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.909 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.909 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.909 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.909 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.909 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.909 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.909 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.910 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.910 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.910 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.910 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.910 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.910 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.910 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.910 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.911 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.911 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.911 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.911 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.911 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.911 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.912 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.912 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.912 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.912 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.912 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.912 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.913 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.913 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.913 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.913 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.913 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-04-20T10:45:39.913 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.913 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.914 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.914 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.914 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.914 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.914 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.914 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.914 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.915 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.915 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.915 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.915 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.915 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.915 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.915 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.915 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.916 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.916 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.916 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.916 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.916 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.916 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.916 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.916 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.917 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.917 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.917 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.917 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.917 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.917 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.917 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.917 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.918 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.918 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.918 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.918 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.918 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.918 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.919 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.919 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.919 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.919 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.919 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.919 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.919 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.920 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.920 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.920 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.920 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.920 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.920 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.920 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.921 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.921 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.921 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.921 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.921 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.921 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.921 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.922 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.922 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.922 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.922 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:39.922 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:39.922 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.922 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.923 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.923 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:39.923 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.923 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.923 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.923 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.923 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.923 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.923 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.924 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-04-20T10:45:39.924 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-04-20T10:45:39.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:39.924 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.924 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.924 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-04-20T10:45:39.924 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-04-20T10:45:39.924 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.925 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.925 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-04-20T10:45:39.925 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T10:45:39.925 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.925 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.926 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-04-20T10:45:39.926 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.926 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.926 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.926 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-04-20T10:45:39.926 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-04-20T10:45:39.926 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.926 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.927 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.927 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.927 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.927 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.927 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T10:45:39.927 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.927 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.928 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.928 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-04-20T10:45:39.928 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-04-20T10:45:39.928 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.928 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.928 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.928 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.929 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.929 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.929 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.929 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.929 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.929 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.929 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.930 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.930 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.930 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.930 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.930 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.930 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.931 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.931 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.931 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.931 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.931 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.931 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.931 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.931 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.931 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.931 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.932 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.932 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.932 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.932 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.932 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.932 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.932 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.933 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.933 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.933 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.933 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.933 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.933 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.933 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.933 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.934 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.934 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.934 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.934 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.934 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.934 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.934 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.935 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.935 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.935 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.935 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.935 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.935 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.935 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.935 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.935 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.937 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.937 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.937 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.937 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.937 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.937 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.937 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-04-20T10:45:39.937 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-04-20T10:45:39.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.938 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.938 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.938 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-04-20T10:45:39.938 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.938 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.938 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.938 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-04-20T10:45:39.938 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.939 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.939 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-20T10:45:39.939 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-20T10:45:39.939 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:39.939 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.939 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.940 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-20T10:45:39.940 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.940 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.940 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.940 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-04-20T10:45:39.940 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.940 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.940 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.941 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.941 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-04-20T10:45:39.941 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.941 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.941 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.941 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-04-20T10:45:39.941 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-04-20T10:45:39.942 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.942 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.942 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.942 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.942 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.942 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.942 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.942 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.943 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.943 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.943 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.943 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-04-20T10:45:39.943 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-04-20T10:45:39.943 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.943 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.943 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.944 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.944 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.944 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.944 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.944 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T10:45:39.944 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T10:45:39.944 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.944 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.945 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.945 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.945 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.945 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.945 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.945 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.945 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.945 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.946 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.946 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.946 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.946 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.946 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.946 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.946 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.946 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.947 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.947 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.947 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.947 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.947 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.947 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.948 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.948 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.948 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.948 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.948 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.948 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.948 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.949 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.949 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.949 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.949 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.949 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.950 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.950 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.950 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.950 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.950 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.950 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.950 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.950 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.950 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.951 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.951 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.951 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.951 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.951 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.951 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.951 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.952 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.952 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.952 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.952 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.952 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.952 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.952 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.953 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.953 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-04-20T10:45:39.953 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-04-20T10:45:39.953 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.953 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.953 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.953 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.953 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.954 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.954 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.954 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.954 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.954 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.954 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.954 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.954 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T10:45:39.955 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T10:45:39.955 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.955 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.955 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.955 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-04-20T10:45:39.955 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.955 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.955 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.956 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.956 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.956 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.956 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.956 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-04-20T10:45:39.956 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-04-20T10:45:39.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:39.956 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.957 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-04-20T10:45:39.957 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.957 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.957 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.957 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.957 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.957 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.958 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-20T10:45:39.958 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-20T10:45:39.958 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.958 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.958 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.958 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-04-20T10:45:39.958 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.958 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.959 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.959 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T10:45:39.959 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.959 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.959 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.959 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-20T10:45:39.959 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-20T10:45:39.959 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.960 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.960 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.960 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-04-20T10:45:39.960 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.960 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.960 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.960 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T10:45:39.960 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.961 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.961 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-04-20T10:45:39.961 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-04-20T10:45:39.961 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.961 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-04-20T10:45:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.962 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-04-20T10:45:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-04-20T10:45:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-04-20T10:45:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.963 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-04-20T10:45:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-04-20T10:45:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.964 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-04-20T10:45:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-04-20T10:45:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-04-20T10:45:39.965 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-04-20T10:45:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-04-20T10:45:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-04-20T10:45:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.967 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.968 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.969 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.969 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.969 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.969 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.969 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.970 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.971 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.972 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.973 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.973 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.973 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.973 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.973 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.974 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.974 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.974 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.975 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.975 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.975 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.976 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.976 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.976 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.976 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.976 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.976 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.977 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.977 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.977 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.977 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.977 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.977 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.977 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.978 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.978 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.978 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.978 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.978 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.978 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.979 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.979 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.979 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.979 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.979 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.979 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-04-20T10:45:39.979 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-04-20T10:45:39.979 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:39.980 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.980 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.980 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-20T10:45:39.980 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.980 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.980 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.980 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.980 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.980 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.981 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.981 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-04-20T10:45:39.981 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T10:45:39.981 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.981 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.981 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.981 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.982 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.982 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.982 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.982 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.982 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.982 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.982 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-04-20T10:45:39.983 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-04-20T10:45:39.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.983 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.983 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.983 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.983 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.983 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.984 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.984 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.984 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.984 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.984 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:39.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:39.984 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-04-20T10:45:39.985 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:39.985 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:39.985 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:39.985 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-04-20T10:45:39.985 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-04-20T10:45:39.985 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.985 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.985 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.985 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:39.986 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.986 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.986 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.986 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.986 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.986 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.987 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-04-20T10:45:39.987 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-04-20T10:45:39.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.987 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.987 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.987 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.987 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.988 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.988 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.988 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.988 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.988 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.988 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:39.988 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:39.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.989 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.989 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.989 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.989 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.989 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.989 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.989 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.989 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.990 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.990 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.990 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.990 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.990 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.990 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.990 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.991 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.991 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:39.991 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:39.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.991 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.991 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.992 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.992 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.992 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.992 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.992 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.992 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.992 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.992 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:39.993 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:39.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.993 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.993 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:39.993 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.993 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.993 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:39.993 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:39.994 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:39.994 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:39.994 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:39.994 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:39.994 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:39.994 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:39.994 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.994 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.995 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.995 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.995 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:39.995 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:39.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:39.995 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.995 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:39.995 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:39.996 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.996 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.996 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.996 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.996 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.996 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.997 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-20T10:45:39.997 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-20T10:45:39.997 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.997 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.997 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.997 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.997 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.997 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.998 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.998 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.998 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:39.998 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:39.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:39.998 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-20T10:45:39.998 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-20T10:45:39.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:39.999 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:39.999 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:39.999 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:39.999 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:39.999 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:39.999 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:39.999 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:39.999 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:39.999 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.000 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.000 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.000 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.000 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-04-20T10:45:40.000 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:40.000 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:40.001 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:40.001 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:40.001 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:40.001 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.001 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.001 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.001 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.001 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.001 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.002 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.002 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.002 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.002 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.002 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.002 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.002 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.002 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.003 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.003 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.003 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.003 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.003 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:40.003 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:40.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.003 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.004 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.004 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.004 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.004 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.004 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.004 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.005 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.005 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:40.005 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:40.005 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.005 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.005 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:40.005 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.006 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.006 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.006 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.006 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.006 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.006 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.006 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.006 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.007 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.007 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.007 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.007 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.007 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.007 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:40.007 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:40.007 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.008 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.008 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.008 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.008 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.008 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.008 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.008 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.009 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.009 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.009 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.009 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.009 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-04-20T10:45:40.009 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:40.009 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:40.010 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:40.010 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:40.010 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:40.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.010 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.010 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.010 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.011 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.011 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.011 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.011 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.011 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.011 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.011 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.011 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.011 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.012 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.012 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.012 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.012 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.012 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:40.012 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:40.012 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.013 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.013 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.013 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.013 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.013 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.013 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.013 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.013 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.014 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.014 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:40.014 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:40.014 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.014 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.014 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.014 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:40.014 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.015 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.015 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.015 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.015 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.015 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.015 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.015 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.015 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.016 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.016 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.016 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.016 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.016 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.016 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:40.016 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:40.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.017 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.017 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.017 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.017 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.017 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.017 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.017 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.018 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.018 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.018 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-04-20T10:45:40.018 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-04-20T10:45:40.018 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.018 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.018 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.018 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-04-20T10:45:40.019 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.019 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.019 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.019 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T10:45:40.019 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.019 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.019 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.019 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.020 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.020 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.020 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-04-20T10:45:40.020 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:40.020 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:40.020 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:40.020 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:40.021 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:40.021 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.021 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.021 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.021 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.021 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.021 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.021 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.021 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.022 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.022 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.022 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.022 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.022 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.022 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.022 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.022 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.024 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.024 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.024 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.024 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.024 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.024 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.024 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.024 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.025 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:40.025 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:40.025 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.025 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.025 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.025 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:40.026 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.026 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.026 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.026 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.026 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.026 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.026 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.026 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.027 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.027 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.027 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.027 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.027 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.027 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.027 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:40.028 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:40.028 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.028 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.028 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.028 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.028 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.028 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.028 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.028 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.029 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.029 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.029 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.029 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-04-20T10:45:40.029 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-04-20T10:45:40.029 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.029 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.030 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.030 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.030 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.030 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.030 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.030 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.030 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.030 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.031 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.031 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-04-20T10:45:40.031 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-04-20T10:45:40.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.031 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.031 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-04-20T10:45:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.032 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-04-20T10:45:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-04-20T10:45:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.033 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-04-20T10:45:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-04-20T10:45:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:40.034 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-04-20T10:45:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-04-20T10:45:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-20T10:45:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.036 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-04-20T10:45:40.037 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-04-20T10:45:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.038 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-04-20T10:45:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-04-20T10:45:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.040 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-04-20T10:45:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-04-20T10:45:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.041 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-04-20T10:45:40.042 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-04-20T10:45:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-20T10:45:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-04-20T10:45:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.043 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-04-20T10:45:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-04-20T10:45:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.044 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.045 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-04-20T10:45:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-04-20T10:45:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.046 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-04-20T10:45:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-04-20T10:45:40.047 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.048 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-04-20T10:45:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:40.049 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-04-20T10:45:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-04-20T10:45:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.050 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T10:45:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-04-20T10:45:40.051 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-04-20T10:45:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-04-20T10:45:40.052 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.053 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.054 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.055 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.056 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:40.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.058 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.059 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.060 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.061 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-04-20T10:45:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:40.062 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.063 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.064 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.065 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.066 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.071 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-04-20T10:45:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.072 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.073 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.074 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.075 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.076 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.077 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.079 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-04-20T10:45:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-04-20T10:45:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-04-20T10:45:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.081 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T10:45:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.082 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-04-20T10:45:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.085 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.086 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.089 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.091 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.092 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.094 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-04-20T10:45:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-04-20T10:45:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-04-20T10:45:40.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T10:45:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-04-20T10:45:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-04-20T10:45:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.098 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T10:45:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.099 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-04-20T10:45:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-04-20T10:45:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.100 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T10:45:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-04-20T10:45:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-04-20T10:45:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.101 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T10:45:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-04-20T10:45:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-04-20T10:45:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.103 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T10:45:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-04-20T10:45:40.104 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-04-20T10:45:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.105 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-04-20T10:45:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-04-20T10:45:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.106 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-04-20T10:45:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-04-20T10:45:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T10:45:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.108 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-04-20T10:45:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-04-20T10:45:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.109 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.110 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.111 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-04-20T10:45:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-04-20T10:45:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.112 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-04-20T10:45:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-04-20T10:45:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-04-20T10:45:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.113 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-04-20T10:45:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-04-20T10:45:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.114 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.115 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-04-20T10:45:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-04-20T10:45:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.116 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:40.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.118 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.120 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.121 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.122 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.123 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.124 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-04-20T10:45:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-04-20T10:45:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.126 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-04-20T10:45:40.127 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-04-20T10:45:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.128 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-04-20T10:45:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-04-20T10:45:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T10:45:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.130 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-04-20T10:45:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:40.131 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.132 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.133 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.134 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:40.135 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.136 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.137 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.138 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-04-20T10:45:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:40.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.141 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.142 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.143 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:40.144 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.145 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.146 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.147 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.148 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-04-20T10:45:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-04-20T10:45:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-04-20T10:45:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.149 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T10:45:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-04-20T10:45:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-04-20T10:45:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.150 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-04-20T10:45:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.151 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-04-20T10:45:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-04-20T10:45:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-20T10:45:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.153 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.153 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.153 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T10:45:40.153 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.153 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.153 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_no_wait": { 2024-04-20T10:45:40.153 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T10:45:40.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.154 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.154 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-20T10:45:40.154 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.154 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.154 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.154 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T10:45:40.155 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.155 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.155 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.155 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.155 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.155 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-04-20T10:45:40.155 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-20T10:45:40.156 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-20T10:45:40.156 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-20T10:45:40.156 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-04-20T10:45:40.156 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-04-20T10:45:40.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-20T10:45:40.156 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.156 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-04-20T10:45:40.156 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.157 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.157 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.157 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.157 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.157 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.157 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-04-20T10:45:40.158 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-04-20T10:45:40.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.158 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.158 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.158 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.158 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.158 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.158 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.159 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.159 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.159 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-20T10:45:40.159 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-20T10:45:40.159 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-20T10:45:40.159 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.160 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.160 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-20T10:45:40.160 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.160 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.160 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.160 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.160 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.160 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.160 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.161 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-20T10:45:40.161 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-20T10:45:40.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.161 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.161 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.161 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.161 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.161 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.162 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.162 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.162 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.162 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.162 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.162 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.162 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.162 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.163 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.163 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.163 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.163 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-20T10:45:40.163 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-20T10:45:40.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.163 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.164 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.164 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.164 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.164 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.164 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.164 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.164 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.164 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.164 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.165 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-20T10:45:40.165 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-20T10:45:40.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.165 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.165 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.165 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-20T10:45:40.165 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.165 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.166 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-20T10:45:40.166 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-20T10:45:40.166 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-20T10:45:40.166 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-20T10:45:40.166 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-20T10:45:40.166 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-20T10:45:40.166 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-20T10:45:40.166 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.166 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.167 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.167 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.167 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.167 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-20T10:45:40.167 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-20T10:45:40.167 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-20T10:45:40.167 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.167 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-20T10:45:40.168 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-20T10:45:40.168 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.168 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.168 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.168 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.168 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.168 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.168 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.169 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-04-20T10:45:40.169 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-04-20T10:45:40.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.169 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.169 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.169 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-20T10:45:40.169 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.169 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.170 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.170 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.170 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.170 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.170 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-04-20T10:45:40.170 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-04-20T10:45:40.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-20T10:45:40.171 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.171 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.171 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-04-20T10:45:40.171 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.171 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.171 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.171 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.171 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.171 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.172 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.172 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-04-20T10:45:40.172 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-04-20T10:45:40.172 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-20T10:45:40.172 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-20T10:45:40.172 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-20T10:45:40.172 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T10:45:40.173 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-20T10:45:40.173 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-20T10:45:40.173 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-20T10:45:40.173 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-20T10:45:40.173 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-20T10:45:40.173 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-20T10:45:40.173 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-20T10:45:40.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.174 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.174 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-04-20T10:45:40.174 INFO:teuthology.orchestra.run.smithi042.stdout: "dashboard": "https://172.21.15.42:8443/" 2024-04-20T10:45:40.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.174 INFO:teuthology.orchestra.run.smithi042.stdout: "always_on_modules": { 2024-04-20T10:45:40.174 INFO:teuthology.orchestra.run.smithi042.stdout: "octopus": [ 2024-04-20T10:45:40.174 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-04-20T10:45:40.175 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-04-20T10:45:40.175 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-04-20T10:45:40.175 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-04-20T10:45:40.175 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-04-20T10:45:40.175 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-04-20T10:45:40.175 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-04-20T10:45:40.175 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-04-20T10:45:40.175 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-04-20T10:45:40.176 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-04-20T10:45:40.176 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.176 INFO:teuthology.orchestra.run.smithi042.stdout: "pacific": [ 2024-04-20T10:45:40.176 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-04-20T10:45:40.176 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-04-20T10:45:40.176 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-04-20T10:45:40.176 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-04-20T10:45:40.176 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-04-20T10:45:40.177 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-04-20T10:45:40.177 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-04-20T10:45:40.177 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-04-20T10:45:40.177 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-04-20T10:45:40.177 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-04-20T10:45:40.177 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.177 INFO:teuthology.orchestra.run.smithi042.stdout: "quincy": [ 2024-04-20T10:45:40.177 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-04-20T10:45:40.177 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-04-20T10:45:40.178 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-04-20T10:45:40.178 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-04-20T10:45:40.178 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-04-20T10:45:40.178 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-04-20T10:45:40.178 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-04-20T10:45:40.178 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-04-20T10:45:40.178 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-04-20T10:45:40.178 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-04-20T10:45:40.178 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.179 INFO:teuthology.orchestra.run.smithi042.stdout: "reef": [ 2024-04-20T10:45:40.179 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-04-20T10:45:40.179 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-04-20T10:45:40.179 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-04-20T10:45:40.179 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-04-20T10:45:40.179 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-04-20T10:45:40.179 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-04-20T10:45:40.179 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-04-20T10:45:40.180 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-04-20T10:45:40.180 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-04-20T10:45:40.180 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-04-20T10:45:40.180 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-20T10:45:40.180 INFO:teuthology.orchestra.run.smithi042.stdout: "squid": [ 2024-04-20T10:45:40.180 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-04-20T10:45:40.180 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-04-20T10:45:40.180 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-04-20T10:45:40.180 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-04-20T10:45:40.181 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-04-20T10:45:40.181 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-04-20T10:45:40.181 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-04-20T10:45:40.181 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-04-20T10:45:40.181 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-04-20T10:45:40.181 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-04-20T10:45:40.181 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-20T10:45:40.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.181 INFO:teuthology.orchestra.run.smithi042.stdout: "last_failure_osd_epoch": 88, 2024-04-20T10:45:40.182 INFO:teuthology.orchestra.run.smithi042.stdout: "active_clients": [ 2024-04-20T10:45:40.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.182 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-04-20T10:45:40.182 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-04-20T10:45:40.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-04-20T10:45:40.182 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-04-20T10:45:40.182 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1035655327 2024-04-20T10:45:40.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.183 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-20T10:45:40.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.183 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "libcephsqlite", 2024-04-20T10:45:40.183 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-04-20T10:45:40.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.183 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-04-20T10:45:40.183 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-04-20T10:45:40.184 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2125699279 2024-04-20T10:45:40.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.184 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-20T10:45:40.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.184 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-04-20T10:45:40.184 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-04-20T10:45:40.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-04-20T10:45:40.185 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-04-20T10:45:40.185 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2094513841 2024-04-20T10:45:40.185 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.185 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-20T10:45:40.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.185 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-04-20T10:45:40.185 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-04-20T10:45:40.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-04-20T10:45:40.186 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-04-20T10:45:40.186 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 385963668 2024-04-20T10:45:40.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.186 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-20T10:45:40.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.186 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-04-20T10:45:40.187 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-04-20T10:45:40.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.187 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-04-20T10:45:40.187 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-04-20T10:45:40.187 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 208200255 2024-04-20T10:45:40.187 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.187 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-20T10:45:40.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:40.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.188 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-04-20T10:45:40.188 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-04-20T10:45:40.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-20T10:45:40.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-04-20T10:45:40.188 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-04-20T10:45:40.188 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1357331212 2024-04-20T10:45:40.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.188 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-20T10:45:40.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:40.189 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-20T10:45:40.189 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:45:40.195 DEBUG:tasks.check_counter:Getting stats from a 2024-04-20T10:45:40.196 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-20T10:45:40.196 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:37 smithi042 ceph-mon[17078]: pgmap v361: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 170 KiB/s wr, 9 op/s 2024-04-20T10:45:40.196 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:37 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/402704261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:45:40.197 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:37 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1762176653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:45:40.197 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:37 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3688765759' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-20T10:45:40.262 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:39 smithi042 ceph-mon[17078]: pgmap v362: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 7 op/s 2024-04-20T10:45:40.262 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:39 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:45:40.263 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:39 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:45:40.263 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:39 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:45:40.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:39 smithi046 ceph-mon[21574]: pgmap v362: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 7 op/s 2024-04-20T10:45:40.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:39 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:45:40.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:39 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:45:40.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:39 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:45:40.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:39 smithi162 ceph-mon[21854]: pgmap v362: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 7 op/s 2024-04-20T10:45:40.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:39 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T10:45:40.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:39 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:45:40.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:39 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:45:40.631 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:45:41.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:40 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:45:41.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:40 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:45:41.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:40 smithi042 ceph-mon[17078]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:45:41.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:40 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:45:41.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:40 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:45:41.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:40 smithi046 ceph-mon[21574]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:45:41.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:40 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T10:45:41.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:40 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T10:45:41.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:40 smithi162 ceph-mon[21854]: from='mgr.15024 ' entity='mgr.x' 2024-04-20T10:45:42.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:41 smithi046 ceph-mon[21574]: pgmap v363: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 4 op/s 2024-04-20T10:45:42.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:41 smithi042 ceph-mon[17078]: pgmap v363: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 4 op/s 2024-04-20T10:45:42.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:41 smithi162 ceph-mon[21854]: pgmap v363: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 4 op/s 2024-04-20T10:45:43.067 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:45:43.068 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T10:45:43.068 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-04-20T10:45:43.068 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-04-20T10:45:43.068 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2992, 2024-04-20T10:45:43.068 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 448, 2024-04-20T10:45:43.068 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-04-20T10:45:43.069 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-04-20T10:45:43.069 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.004933138, 2024-04-20T10:45:43.069 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000088420, 2024-04-20T10:45:43.069 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.001355947, 2024-04-20T10:45:43.069 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:45:43.070 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:43.070 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-04-20T10:45:43.070 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.001611605, 2024-04-20T10:45:43.070 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000537201 2024-04-20T10:45:43.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:43.070 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:43.070 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:43.070 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:43.071 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:43.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:43.071 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-20T10:45:43.071 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 448 2024-04-20T10:45:43.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:43.071 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T10:45:43.071 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-04-20T10:45:43.071 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 175, 2024-04-20T10:45:43.072 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-04-20T10:45:43.072 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 186912, 2024-04-20T10:45:43.072 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-04-20T10:45:43.072 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-04-20T10:45:43.072 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.034609027, 2024-04-20T10:45:43.072 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.025027045, 2024-04-20T10:45:43.072 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.003745139, 2024-04-20T10:45:43.072 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:45:43.072 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:43.073 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 172, 2024-04-20T10:45:43.073 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.026880620, 2024-04-20T10:45:43.073 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000156282 2024-04-20T10:45:43.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:43.073 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:43.073 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:43.073 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:43.073 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:43.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:43.074 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T10:45:43.074 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 186912 2024-04-20T10:45:43.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:43.074 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T10:45:43.074 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 240, 2024-04-20T10:45:43.074 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 228, 2024-04-20T10:45:43.075 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1733120, 2024-04-20T10:45:43.075 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 140704, 2024-04-20T10:45:43.075 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-04-20T10:45:43.075 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-04-20T10:45:43.075 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.102670954, 2024-04-20T10:45:43.075 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.034028191, 2024-04-20T10:45:43.075 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.080115581, 2024-04-20T10:45:43.075 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.011404792, 2024-04-20T10:45:43.076 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:43.076 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 227, 2024-04-20T10:45:43.076 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.024271482, 2024-04-20T10:45:43.076 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000106922 2024-04-20T10:45:43.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:43.076 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:43.076 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:43.077 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:43.077 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:43.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:43.077 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1733120, 2024-04-20T10:45:43.077 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 140704 2024-04-20T10:45:43.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:43.077 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-04-20T10:45:43.078 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-04-20T10:45:43.078 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-04-20T10:45:43.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:43.078 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-04-20T10:45:43.078 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-04-20T10:45:43.078 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-04-20T10:45:43.078 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T10:45:43.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-04-20T10:45:43.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T10:45:43.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-04-20T10:45:43.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T10:45:43.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T10:45:43.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T10:45:43.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T10:45:43.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T10:45:43.079 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-04-20T10:45:43.080 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T10:45:43.080 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T10:45:43.080 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-04-20T10:45:43.080 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-04-20T10:45:43.080 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-04-20T10:45:43.080 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-04-20T10:45:43.080 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T10:45:43.080 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T10:45:43.080 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T10:45:43.081 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T10:45:43.081 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T10:45:43.081 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-04-20T10:45:43.081 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-04-20T10:45:43.081 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-04-20T10:45:43.081 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T10:45:43.081 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T10:45:43.081 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-04-20T10:45:43.082 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-04-20T10:45:43.082 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-04-20T10:45:43.082 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-04-20T10:45:43.082 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T10:45:43.082 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-04-20T10:45:43.082 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T10:45:43.082 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-04-20T10:45:43.082 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148977, 2024-04-20T10:45:43.082 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 30, 2024-04-20T10:45:43.083 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-04-20T10:45:43.083 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-04-20T10:45:43.083 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-04-20T10:45:43.083 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-04-20T10:45:43.083 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-04-20T10:45:43.083 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-04-20T10:45:43.083 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-04-20T10:45:43.084 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-04-20T10:45:43.084 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-04-20T10:45:43.084 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-04-20T10:45:43.084 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T10:45:43.084 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-04-20T10:45:43.084 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-04-20T10:45:43.084 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-04-20T10:45:43.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-04-20T10:45:43.085 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-04-20T10:45:43.085 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-04-20T10:45:43.085 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-04-20T10:45:43.085 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-04-20T10:45:43.085 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-04-20T10:45:43.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:43.085 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T10:45:43.085 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-04-20T10:45:43.086 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-04-20T10:45:43.086 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-04-20T10:45:43.086 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 242, 2024-04-20T10:45:43.086 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1699274, 2024-04-20T10:45:43.086 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-04-20T10:45:43.086 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 242, 2024-04-20T10:45:43.086 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-04-20T10:45:43.087 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-04-20T10:45:43.087 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 242, 2024-04-20T10:45:43.087 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1699274, 2024-04-20T10:45:43.087 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-04-20T10:45:43.087 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:43.087 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:43.087 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:43.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:43.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:43.088 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:45:43.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:43 smithi042 ceph-mon[17078]: pgmap v364: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 612 KiB/s wr, 45 op/s 2024-04-20T10:45:43.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:43 smithi046 ceph-mon[21574]: pgmap v364: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 612 KiB/s wr, 45 op/s 2024-04-20T10:45:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:43 smithi162 ceph-mon[21854]: pgmap v364: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 612 KiB/s wr, 45 op/s 2024-04-20T10:45:43.945 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-20T10:45:43.945 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-20T10:45:43.945 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-20T10:45:43.946 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-20T10:45:43.946 DEBUG:tasks.check_counter:Getting stats from d 2024-04-20T10:45:43.946 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-20T10:45:44.423 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:45:46.036 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:45 smithi042 ceph-mon[17078]: pgmap v365: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 641 KiB/s wr, 46 op/s 2024-04-20T10:45:46.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:45 smithi046 ceph-mon[21574]: pgmap v365: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 641 KiB/s wr, 46 op/s 2024-04-20T10:45:46.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:45 smithi162 ceph-mon[21854]: pgmap v365: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 641 KiB/s wr, 46 op/s 2024-04-20T10:45:46.631 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:45:46.631 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T10:45:46.631 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-04-20T10:45:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-04-20T10:45:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2992, 2024-04-20T10:45:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 448, 2024-04-20T10:45:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-04-20T10:45:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-04-20T10:45:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.004807100, 2024-04-20T10:45:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000056695, 2024-04-20T10:45:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.001504113, 2024-04-20T10:45:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:45:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-04-20T10:45:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.001504102, 2024-04-20T10:45:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000501367 2024-04-20T10:45:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-20T10:45:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 448 2024-04-20T10:45:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T10:45:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-04-20T10:45:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 181, 2024-04-20T10:45:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-04-20T10:45:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 187968, 2024-04-20T10:45:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-04-20T10:45:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-04-20T10:45:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.037669505, 2024-04-20T10:45:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.025911510, 2024-04-20T10:45:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.006107479, 2024-04-20T10:45:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:45:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 178, 2024-04-20T10:45:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.028178805, 2024-04-20T10:45:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000158307 2024-04-20T10:45:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T10:45:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 187968 2024-04-20T10:45:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T10:45:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 247, 2024-04-20T10:45:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 235, 2024-04-20T10:45:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1735024, 2024-04-20T10:45:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 145184, 2024-04-20T10:45:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-04-20T10:45:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-04-20T10:45:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.095778223, 2024-04-20T10:45:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.024122706, 2024-04-20T10:45:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.078833471, 2024-04-20T10:45:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.011284542, 2024-04-20T10:45:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 234, 2024-04-20T10:45:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.028648327, 2024-04-20T10:45:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000122428 2024-04-20T10:45:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1735024, 2024-04-20T10:45:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 145184 2024-04-20T10:45:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-04-20T10:45:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-04-20T10:45:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-04-20T10:45:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-04-20T10:45:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-04-20T10:45:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-04-20T10:45:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T10:45:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-04-20T10:45:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T10:45:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-04-20T10:45:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T10:45:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T10:45:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T10:45:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T10:45:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T10:45:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-04-20T10:45:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T10:45:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T10:45:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-04-20T10:45:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-04-20T10:45:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-04-20T10:45:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-04-20T10:45:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T10:45:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T10:45:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T10:45:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T10:45:46.643 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T10:45:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-04-20T10:45:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-04-20T10:45:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-04-20T10:45:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T10:45:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T10:45:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-04-20T10:45:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-04-20T10:45:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-04-20T10:45:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-04-20T10:45:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T10:45:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-04-20T10:45:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T10:45:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-04-20T10:45:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 150008, 2024-04-20T10:45:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 31, 2024-04-20T10:45:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-04-20T10:45:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-04-20T10:45:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-04-20T10:45:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-04-20T10:45:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-04-20T10:45:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-04-20T10:45:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-04-20T10:45:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-04-20T10:45:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-04-20T10:45:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-04-20T10:45:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T10:45:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-04-20T10:45:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-04-20T10:45:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-04-20T10:45:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-04-20T10:45:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-04-20T10:45:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-04-20T10:45:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-04-20T10:45:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-04-20T10:45:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-04-20T10:45:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T10:45:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-04-20T10:45:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-04-20T10:45:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-04-20T10:45:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 249, 2024-04-20T10:45:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1700184, 2024-04-20T10:45:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-04-20T10:45:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 249, 2024-04-20T10:45:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-04-20T10:45:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-04-20T10:45:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 249, 2024-04-20T10:45:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1700184, 2024-04-20T10:45:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-04-20T10:45:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:46.652 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:45:47.510 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-20T10:45:47.511 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-20T10:45:47.511 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-20T10:45:47.511 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-20T10:45:47.511 DEBUG:tasks.check_counter:Getting stats from g 2024-04-20T10:45:47.511 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-20T10:45:47.988 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:45:48.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:47 smithi046 ceph-mon[21574]: pgmap v366: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 651 KiB/s wr, 47 op/s 2024-04-20T10:45:48.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:47 smithi042 ceph-mon[17078]: pgmap v366: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 651 KiB/s wr, 47 op/s 2024-04-20T10:45:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:47 smithi162 ceph-mon[21854]: pgmap v366: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 651 KiB/s wr, 47 op/s 2024-04-20T10:45:50.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:49 smithi046 ceph-mon[21574]: pgmap v367: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 644 KiB/s wr, 47 op/s 2024-04-20T10:45:50.089 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:49 smithi042 ceph-mon[17078]: pgmap v367: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 644 KiB/s wr, 47 op/s 2024-04-20T10:45:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:49 smithi162 ceph-mon[21854]: pgmap v367: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 644 KiB/s wr, 47 op/s 2024-04-20T10:45:50.308 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:45:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T10:45:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 247, 2024-04-20T10:45:50.309 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 233, 2024-04-20T10:45:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1736928, 2024-04-20T10:45:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 143072, 2024-04-20T10:45:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-04-20T10:45:50.310 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-04-20T10:45:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.113501849, 2024-04-20T10:45:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.037884614, 2024-04-20T10:45:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.082907217, 2024-04-20T10:45:50.311 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.012016577, 2024-04-20T10:45:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 233, 2024-04-20T10:45:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.030053446, 2024-04-20T10:45:50.312 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000128984 2024-04-20T10:45:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1736928, 2024-04-20T10:45:50.314 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 143072 2024-04-20T10:45:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T10:45:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-04-20T10:45:50.315 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 177, 2024-04-20T10:45:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-04-20T10:45:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 187264, 2024-04-20T10:45:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-04-20T10:45:50.316 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-04-20T10:45:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.034838721, 2024-04-20T10:45:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.025963304, 2024-04-20T10:45:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.003535205, 2024-04-20T10:45:50.317 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:45:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 174, 2024-04-20T10:45:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.029495061, 2024-04-20T10:45:50.318 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000169511 2024-04-20T10:45:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:50.319 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:50.320 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T10:45:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 187264 2024-04-20T10:45:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T10:45:50.321 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 0, 2024-04-20T10:45:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 2, 2024-04-20T10:45:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 0, 2024-04-20T10:45:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 0, 2024-04-20T10:45:50.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-04-20T10:45:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-04-20T10:45:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.000996064, 2024-04-20T10:45:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T10:45:50.323 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.000823876, 2024-04-20T10:45:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:45:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:50.324 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:50.325 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:50.326 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:50.330 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T10:45:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T10:45:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-04-20T10:45:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-04-20T10:45:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-04-20T10:45:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-04-20T10:45:50.332 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-04-20T10:45:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-04-20T10:45:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T10:45:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-04-20T10:45:50.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T10:45:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-04-20T10:45:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T10:45:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T10:45:50.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T10:45:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T10:45:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T10:45:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-04-20T10:45:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T10:45:50.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T10:45:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-04-20T10:45:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-04-20T10:45:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-04-20T10:45:50.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-04-20T10:45:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T10:45:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T10:45:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T10:45:50.337 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T10:45:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T10:45:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-04-20T10:45:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-04-20T10:45:50.338 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-04-20T10:45:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T10:45:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T10:45:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-04-20T10:45:50.339 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-04-20T10:45:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-04-20T10:45:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-04-20T10:45:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T10:45:50.340 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-04-20T10:45:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T10:45:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-04-20T10:45:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 150008, 2024-04-20T10:45:50.341 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 31, 2024-04-20T10:45:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-04-20T10:45:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-04-20T10:45:50.342 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-04-20T10:45:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-04-20T10:45:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-04-20T10:45:50.343 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-04-20T10:45:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-04-20T10:45:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-04-20T10:45:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-04-20T10:45:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-04-20T10:45:50.344 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T10:45:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-04-20T10:45:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-04-20T10:45:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-04-20T10:45:50.345 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-04-20T10:45:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-04-20T10:45:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-04-20T10:45:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-04-20T10:45:50.346 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-04-20T10:45:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-04-20T10:45:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T10:45:50.347 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-04-20T10:45:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-04-20T10:45:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-04-20T10:45:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 245, 2024-04-20T10:45:50.348 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1699664, 2024-04-20T10:45:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-04-20T10:45:50.349 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 245, 2024-04-20T10:45:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-04-20T10:45:50.350 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-04-20T10:45:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 245, 2024-04-20T10:45:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1699664, 2024-04-20T10:45:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-04-20T10:45:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:50.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:50.352 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:45:51.189 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-20T10:45:51.189 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-20T10:45:51.189 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-20T10:45:51.189 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-20T10:45:51.189 DEBUG:tasks.check_counter:Getting stats from j 2024-04-20T10:45:51.190 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-20T10:45:51.697 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.a/config 2024-04-20T10:45:52.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:51 smithi046 ceph-mon[21574]: pgmap v368: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 640 KiB/s wr, 46 op/s 2024-04-20T10:45:52.061 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:51 smithi042 ceph-mon[17078]: pgmap v368: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 640 KiB/s wr, 46 op/s 2024-04-20T10:45:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:51 smithi162 ceph-mon[21854]: pgmap v368: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 640 KiB/s wr, 46 op/s 2024-04-20T10:45:54.005 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:53 smithi042 ceph-mon[17078]: pgmap v369: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 644 KiB/s wr, 47 op/s 2024-04-20T10:45:54.009 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-20T10:45:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T10:45:54.009 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-04-20T10:45:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-04-20T10:45:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2992, 2024-04-20T10:45:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 448, 2024-04-20T10:45:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-04-20T10:45:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-04-20T10:45:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.004517701, 2024-04-20T10:45:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000082500, 2024-04-20T10:45:54.010 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.001431015, 2024-04-20T10:45:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:45:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-04-20T10:45:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.001865920, 2024-04-20T10:45:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000621973 2024-04-20T10:45:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:54.011 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-20T10:45:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 448 2024-04-20T10:45:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T10:45:54.012 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-04-20T10:45:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 183, 2024-04-20T10:45:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-04-20T10:45:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 188320, 2024-04-20T10:45:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-04-20T10:45:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-04-20T10:45:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.036674655, 2024-04-20T10:45:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.026766096, 2024-04-20T10:45:54.013 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.004346494, 2024-04-20T10:45:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:45:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 180, 2024-04-20T10:45:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.029953362, 2024-04-20T10:45:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000166407 2024-04-20T10:45:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:54.014 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T10:45:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 188320 2024-04-20T10:45:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T10:45:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 250, 2024-04-20T10:45:54.015 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 238, 2024-04-20T10:45:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1735840, 2024-04-20T10:45:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 147104, 2024-04-20T10:45:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-04-20T10:45:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-04-20T10:45:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.098531868, 2024-04-20T10:45:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.025317606, 2024-04-20T10:45:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.086202832, 2024-04-20T10:45:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.011692621, 2024-04-20T10:45:54.016 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 237, 2024-04-20T10:45:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.036877982, 2024-04-20T10:45:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000155603 2024-04-20T10:45:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:54.017 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1735840, 2024-04-20T10:45:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 147104 2024-04-20T10:45:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-04-20T10:45:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-04-20T10:45:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-04-20T10:45:54.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-04-20T10:45:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-04-20T10:45:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-04-20T10:45:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T10:45:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-04-20T10:45:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T10:45:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-04-20T10:45:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T10:45:54.019 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T10:45:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T10:45:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T10:45:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T10:45:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-04-20T10:45:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T10:45:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T10:45:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-04-20T10:45:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-04-20T10:45:54.020 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-04-20T10:45:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-04-20T10:45:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T10:45:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T10:45:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T10:45:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T10:45:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T10:45:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-04-20T10:45:54.021 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-04-20T10:45:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-04-20T10:45:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T10:45:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T10:45:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-04-20T10:45:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-04-20T10:45:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-04-20T10:45:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-04-20T10:45:54.022 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T10:45:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-04-20T10:45:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T10:45:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-04-20T10:45:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 149723, 2024-04-20T10:45:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 31, 2024-04-20T10:45:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-04-20T10:45:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-04-20T10:45:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-04-20T10:45:54.023 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-04-20T10:45:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-04-20T10:45:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-04-20T10:45:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-04-20T10:45:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-04-20T10:45:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-04-20T10:45:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-04-20T10:45:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T10:45:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-04-20T10:45:54.024 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-04-20T10:45:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-04-20T10:45:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-04-20T10:45:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-04-20T10:45:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-04-20T10:45:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-04-20T10:45:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-04-20T10:45:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-04-20T10:45:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-20T10:45:54.025 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T10:45:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-04-20T10:45:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-04-20T10:45:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-04-20T10:45:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 252, 2024-04-20T10:45:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1700574, 2024-04-20T10:45:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-04-20T10:45:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 252, 2024-04-20T10:45:54.026 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-04-20T10:45:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-04-20T10:45:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 252, 2024-04-20T10:45:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1700574, 2024-04-20T10:45:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-04-20T10:45:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-20T10:45:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-20T10:45:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-20T10:45:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:54.027 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-20T10:45:54.028 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-20T10:45:54.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:53 smithi046 ceph-mon[21574]: pgmap v369: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 644 KiB/s wr, 47 op/s 2024-04-20T10:45:54.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:53 smithi162 ceph-mon[21854]: pgmap v369: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 644 KiB/s wr, 47 op/s 2024-04-20T10:45:54.820 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-20T10:45:54.820 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-20T10:45:54.820 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-20T10:45:54.820 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-20T10:45:54.820 DEBUG:tasks.check_counter:Getting stats from b 2024-04-20T10:45:54.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-20T10:45:55.156 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T10:45:55.932 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:55 smithi046 ceph-mon[21574]: pgmap v370: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 61 KiB/s wr, 6 op/s 2024-04-20T10:45:56.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:55 smithi042 ceph-mon[17078]: pgmap v370: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 61 KiB/s wr, 6 op/s 2024-04-20T10:45:56.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:55 smithi162 ceph-mon[21854]: pgmap v370: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 61 KiB/s wr, 6 op/s 2024-04-20T10:45:57.818 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-20T10:45:57.818 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T10:45:57.818 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 375828, 2024-04-20T10:45:57.818 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 375204, 2024-04-20T10:45:57.818 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 179110026, 2024-04-20T10:45:57.819 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 477038602, 2024-04-20T10:45:57.819 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 22, 2024-04-20T10:45:57.819 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 5, 2024-04-20T10:45:57.819 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 19.598541269, 2024-04-20T10:45:57.819 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 6.831324096, 2024-04-20T10:45:57.819 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 4.407216455, 2024-04-20T10:45:57.819 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 6.883002302, 2024-04-20T10:45:57.819 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:57.819 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 375203, 2024-04-20T10:45:57.820 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 61.669364363, 2024-04-20T10:45:57.820 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000164362 2024-04-20T10:45:57.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.820 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:57.820 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 947, 2024-04-20T10:45:57.820 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000137178, 2024-04-20T10:45:57.820 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000144 2024-04-20T10:45:57.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.821 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 348320, 2024-04-20T10:45:57.821 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 219216 2024-04-20T10:45:57.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.821 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T10:45:57.821 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 361624, 2024-04-20T10:45:57.821 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 361524, 2024-04-20T10:45:57.821 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 173499143, 2024-04-20T10:45:57.821 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 432017907, 2024-04-20T10:45:57.821 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 145, 2024-04-20T10:45:57.822 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 5, 2024-04-20T10:45:57.822 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 18.688032968, 2024-04-20T10:45:57.822 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 6.533973261, 2024-04-20T10:45:57.822 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 4.303804225, 2024-04-20T10:45:57.822 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 6.503799655, 2024-04-20T10:45:57.822 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:57.822 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 361523, 2024-04-20T10:45:57.822 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 67.563285857, 2024-04-20T10:45:57.822 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000186885 2024-04-20T10:45:57.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.823 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:57.823 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 242, 2024-04-20T10:45:57.823 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000041965, 2024-04-20T10:45:57.823 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000173 2024-04-20T10:45:57.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.823 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1816224, 2024-04-20T10:45:57.823 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 178224 2024-04-20T10:45:57.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.824 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T10:45:57.824 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 429424, 2024-04-20T10:45:57.824 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 618511, 2024-04-20T10:45:57.824 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 188366510, 2024-04-20T10:45:57.824 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 427792817, 2024-04-20T10:45:57.824 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 15, 2024-04-20T10:45:57.824 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 5, 2024-04-20T10:45:57.824 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 25.307430027, 2024-04-20T10:45:57.824 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 10.262008154, 2024-04-20T10:45:57.825 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 30.653679965, 2024-04-20T10:45:57.825 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 3.567823556, 2024-04-20T10:45:57.825 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:45:57.825 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 618508, 2024-04-20T10:45:57.825 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 45.723531977, 2024-04-20T10:45:57.825 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000073925 2024-04-20T10:45:57.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.825 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:45:57.825 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 580149, 2024-04-20T10:45:57.826 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.096007758, 2024-04-20T10:45:57.826 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000165 2024-04-20T10:45:57.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.826 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 101014064, 2024-04-20T10:45:57.826 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 262288400 2024-04-20T10:45:57.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.826 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-04-20T10:45:57.826 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 1, 2024-04-20T10:45:57.827 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-04-20T10:45:57.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.827 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-MDSRank": { 2024-04-20T10:45:57.827 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-04-20T10:45:57.827 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-04-20T10:45:57.827 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 582434, 2024-04-20T10:45:57.827 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 304.263181692, 2024-04-20T10:45:57.827 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000522399 2024-04-20T10:45:57.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:45:57.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.828 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-PurgeQueue": { 2024-04-20T10:45:57.828 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-04-20T10:45:57.828 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-04-20T10:45:57.828 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1820, 2024-04-20T10:45:57.828 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.225993006, 2024-04-20T10:45:57.828 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000124171 2024-04-20T10:45:57.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:45:57.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.829 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-04-20T10:45:57.829 INFO:teuthology.orchestra.run.smithi046.stdout: "request": 100941, 2024-04-20T10:45:57.829 INFO:teuthology.orchestra.run.smithi046.stdout: "reply": 100941, 2024-04-20T10:45:57.829 INFO:teuthology.orchestra.run.smithi046.stdout: "reply_latency": { 2024-04-20T10:45:57.829 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 100941, 2024-04-20T10:45:57.829 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 327.163476482, 2024-04-20T10:45:57.829 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003241135 2024-04-20T10:45:57.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.829 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_reply": 0, 2024-04-20T10:45:57.830 INFO:teuthology.orchestra.run.smithi046.stdout: "forward": 0, 2024-04-20T10:45:57.830 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_complete": 0, 2024-04-20T10:45:57.830 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_keys": 0, 2024-04-20T10:45:57.830 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_commit": 6322, 2024-04-20T10:45:57.830 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_split": 235, 2024-04-20T10:45:57.830 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_merge": 0, 2024-04-20T10:45:57.830 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes": 121070, 2024-04-20T10:45:57.830 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_top": 14076, 2024-04-20T10:45:57.830 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_bottom": 106994, 2024-04-20T10:45:57.831 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pin_tail": 0, 2024-04-20T10:45:57.831 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pinned": 100961, 2024-04-20T10:45:57.831 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_expired": 0, 2024-04-20T10:45:57.831 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_with_caps": 90305, 2024-04-20T10:45:57.831 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": 90306, 2024-04-20T10:45:57.831 INFO:teuthology.orchestra.run.smithi046.stdout: "subtrees": 2, 2024-04-20T10:45:57.831 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse": 101350, 2024-04-20T10:45:57.831 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_hit": 100437, 2024-04-20T10:45:57.832 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_forward": 0, 2024-04-20T10:45:57.832 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_discover": 0, 2024-04-20T10:45:57.832 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_dir_fetch": 0, 2024-04-20T10:45:57.832 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_remote_ino": 0, 2024-04-20T10:45:57.832 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_lock": 0, 2024-04-20T10:45:57.832 INFO:teuthology.orchestra.run.smithi046.stdout: "load_cent": 0, 2024-04-20T10:45:57.832 INFO:teuthology.orchestra.run.smithi046.stdout: "q": 0, 2024-04-20T10:45:57.832 INFO:teuthology.orchestra.run.smithi046.stdout: "exported": 0, 2024-04-20T10:45:57.832 INFO:teuthology.orchestra.run.smithi046.stdout: "exported_inodes": 0, 2024-04-20T10:45:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "imported": 0, 2024-04-20T10:45:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "imported_inodes": 0, 2024-04-20T10:45:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_dir_fetch": 0, 2024-04-20T10:45:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_backtrace_fetch": 835456, 2024-04-20T10:45:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_peer_discover": 0, 2024-04-20T10:45:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rfiles": 85013, 2024-04-20T10:45:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rbytes": 1558153240, 2024-04-20T10:45:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rsnaps": 6, 2024-04-20T10:45:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_fetch": 835456, 2024-04-20T10:45:57.834 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_set_tag": 0, 2024-04-20T10:45:57.834 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T10:45:57.834 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_inotable_repaired": 0, 2024-04-20T10:45:57.834 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_inodes": 48978, 2024-04-20T10:45:57.834 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_base_inodes": 18, 2024-04-20T10:45:57.834 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dirfrag_rstats": 48960, 2024-04-20T10:45:57.834 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_file_inodes": 786478, 2024-04-20T10:45:57.834 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_inode_file_caps": 0, 2024-04-20T10:45:57.834 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_revoke": 6179, 2024-04-20T10:45:57.835 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_grant": 97402, 2024-04-20T10:45:57.835 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_trunc": 73, 2024-04-20T10:45:57.835 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T10:45:57.835 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T10:45:57.835 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps": 144078, 2024-04-20T10:45:57.835 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps_dirty": 138064, 2024-04-20T10:45:57.835 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_cap_release": 755, 2024-04-20T10:45:57.835 INFO:teuthology.orchestra.run.smithi046.stdout: "process_request_cap_release": 13800 2024-04-20T10:45:57.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.836 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache": { 2024-04-20T10:45:57.836 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update": 0, 2024-04-20T10:45:57.836 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update_receipt": 0, 2024-04-20T10:45:57.836 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_try_discover": 0, 2024-04-20T10:45:57.836 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_send_discover": 0, 2024-04-20T10:45:57.836 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_handle_discover": 0, 2024-04-20T10:45:57.836 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays": 1, 2024-04-20T10:45:57.836 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_delayed": 0, 2024-04-20T10:45:57.836 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_enqueuing": 0, 2024-04-20T10:45:57.837 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_created": 1225, 2024-04-20T10:45:57.837 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_enqueued": 1224, 2024-04-20T10:45:57.837 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_reintegrated": 0, 2024-04-20T10:45:57.837 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_migrated": 0, 2024-04-20T10:45:57.837 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_processing": 0, 2024-04-20T10:45:57.837 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_enqueued": 0, 2024-04-20T10:45:57.837 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_prioritized": 0, 2024-04-20T10:45:57.837 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_started": 0, 2024-04-20T10:45:57.837 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_completed": 0, 2024-04-20T10:45:57.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_path": 0, 2024-04-20T10:45:57.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_quiesce_inode": 0, 2024-04-20T10:45:57.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_enqueue_scrub": 18, 2024-04-20T10:45:57.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_exportdir": 0, 2024-04-20T10:45:57.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_flush": 0, 2024-04-20T10:45:57.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragmentdir": 238, 2024-04-20T10:45:57.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragstats": 0, 2024-04-20T10:45:57.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_inodestats": 0 2024-04-20T10:45:57.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.839 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log": { 2024-04-20T10:45:57.839 INFO:teuthology.orchestra.run.smithi046.stdout: "evlrg": 0, 2024-04-20T10:45:57.839 INFO:teuthology.orchestra.run.smithi046.stdout: "evadd": 243585, 2024-04-20T10:45:57.839 INFO:teuthology.orchestra.run.smithi046.stdout: "evex": 147481, 2024-04-20T10:45:57.839 INFO:teuthology.orchestra.run.smithi046.stdout: "evtrm": 147481, 2024-04-20T10:45:57.839 INFO:teuthology.orchestra.run.smithi046.stdout: "ev": 96104, 2024-04-20T10:45:57.839 INFO:teuthology.orchestra.run.smithi046.stdout: "evexg": 0, 2024-04-20T10:45:57.839 INFO:teuthology.orchestra.run.smithi046.stdout: "evexd": 12290, 2024-04-20T10:45:57.840 INFO:teuthology.orchestra.run.smithi046.stdout: "segadd": 308, 2024-04-20T10:45:57.840 INFO:teuthology.orchestra.run.smithi046.stdout: "segex": 180, 2024-04-20T10:45:57.840 INFO:teuthology.orchestra.run.smithi046.stdout: "segtrm": 180, 2024-04-20T10:45:57.840 INFO:teuthology.orchestra.run.smithi046.stdout: "seg": 128, 2024-04-20T10:45:57.840 INFO:teuthology.orchestra.run.smithi046.stdout: "segmjr": 8, 2024-04-20T10:45:57.840 INFO:teuthology.orchestra.run.smithi046.stdout: "segexg": 0, 2024-04-20T10:45:57.840 INFO:teuthology.orchestra.run.smithi046.stdout: "segexd": 16, 2024-04-20T10:45:57.840 INFO:teuthology.orchestra.run.smithi046.stdout: "expos": 363925891, 2024-04-20T10:45:57.840 INFO:teuthology.orchestra.run.smithi046.stdout: "wrpos": 688415461, 2024-04-20T10:45:57.841 INFO:teuthology.orchestra.run.smithi046.stdout: "rdpos": 0, 2024-04-20T10:45:57.841 INFO:teuthology.orchestra.run.smithi046.stdout: "jlat": { 2024-04-20T10:45:57.841 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 12068, 2024-04-20T10:45:57.841 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 1109.871173202, 2024-04-20T10:45:57.841 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.091968111 2024-04-20T10:45:57.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.841 INFO:teuthology.orchestra.run.smithi046.stdout: "replayed": 0 2024-04-20T10:45:57.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.841 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mem": { 2024-04-20T10:45:57.842 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 119567, 2024-04-20T10:45:57.842 INFO:teuthology.orchestra.run.smithi046.stdout: "ino+": 120809, 2024-04-20T10:45:57.842 INFO:teuthology.orchestra.run.smithi046.stdout: "ino-": 1242, 2024-04-20T10:45:57.842 INFO:teuthology.orchestra.run.smithi046.stdout: "dir": 6957, 2024-04-20T10:45:57.842 INFO:teuthology.orchestra.run.smithi046.stdout: "dir+": 7300, 2024-04-20T10:45:57.842 INFO:teuthology.orchestra.run.smithi046.stdout: "dir-": 343, 2024-04-20T10:45:57.842 INFO:teuthology.orchestra.run.smithi046.stdout: "dn": 121070, 2024-04-20T10:45:57.842 INFO:teuthology.orchestra.run.smithi046.stdout: "dn+": 122698, 2024-04-20T10:45:57.842 INFO:teuthology.orchestra.run.smithi046.stdout: "dn-": 1628, 2024-04-20T10:45:57.843 INFO:teuthology.orchestra.run.smithi046.stdout: "cap": 90306, 2024-04-20T10:45:57.843 INFO:teuthology.orchestra.run.smithi046.stdout: "cap+": 91546, 2024-04-20T10:45:57.843 INFO:teuthology.orchestra.run.smithi046.stdout: "cap-": 1240, 2024-04-20T10:45:57.843 INFO:teuthology.orchestra.run.smithi046.stdout: "rss": 610220, 2024-04-20T10:45:57.843 INFO:teuthology.orchestra.run.smithi046.stdout: "heap": 231800 2024-04-20T10:45:57.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.843 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server": { 2024-04-20T10:45:57.843 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_client_request": 107380, 2024-04-20T10:45:57.843 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_server_request": 0, 2024-04-20T10:45:57.844 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_request": 100941, 2024-04-20T10:45:57.844 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_session": 33, 2024-04-20T10:45:57.844 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_peer_request": 0, 2024-04-20T10:45:57.844 INFO:teuthology.orchestra.run.smithi046.stdout: "req_create_latency": { 2024-04-20T10:45:57.844 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 86067, 2024-04-20T10:45:57.844 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 259.207773530, 2024-04-20T10:45:57.844 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003011697 2024-04-20T10:45:57.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.845 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getattr_latency": { 2024-04-20T10:45:57.845 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 238, 2024-04-20T10:45:57.845 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.106871995, 2024-04-20T10:45:57.845 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000449041 2024-04-20T10:45:57.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.845 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getfilelock_latency": { 2024-04-20T10:45:57.845 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.845 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.845 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.846 INFO:teuthology.orchestra.run.smithi046.stdout: "req_link_latency": { 2024-04-20T10:45:57.846 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.846 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.846 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.846 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookup_latency": { 2024-04-20T10:45:57.846 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 4, 2024-04-20T10:45:57.846 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000854274, 2024-04-20T10:45:57.847 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000213568 2024-04-20T10:45:57.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.847 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookuphash_latency": { 2024-04-20T10:45:57.847 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.847 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.847 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.847 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupino_latency": { 2024-04-20T10:45:57.847 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2, 2024-04-20T10:45:57.848 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000474134, 2024-04-20T10:45:57.848 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000237067 2024-04-20T10:45:57.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.848 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupname_latency": { 2024-04-20T10:45:57.848 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.848 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.848 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.848 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupparent_latency": { 2024-04-20T10:45:57.849 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.849 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.849 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.849 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupsnap_latency": { 2024-04-20T10:45:57.850 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.850 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.850 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.850 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lssnap_latency": { 2024-04-20T10:45:57.850 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 11, 2024-04-20T10:45:57.851 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.007371493, 2024-04-20T10:45:57.851 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000670135 2024-04-20T10:45:57.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.851 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mkdir_latency": { 2024-04-20T10:45:57.851 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 5408, 2024-04-20T10:45:57.851 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 15.806581910, 2024-04-20T10:45:57.851 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002922814 2024-04-20T10:45:57.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.851 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mknod_latency": { 2024-04-20T10:45:57.852 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.852 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.852 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.852 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mksnap_latency": { 2024-04-20T10:45:57.852 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 11, 2024-04-20T10:45:57.852 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 2.608868478, 2024-04-20T10:45:57.852 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.237169861 2024-04-20T10:45:57.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.853 INFO:teuthology.orchestra.run.smithi046.stdout: "req_open_latency": { 2024-04-20T10:45:57.853 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 708, 2024-04-20T10:45:57.853 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 1.742543122, 2024-04-20T10:45:57.853 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002461219 2024-04-20T10:45:57.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.853 INFO:teuthology.orchestra.run.smithi046.stdout: "req_readdir_latency": { 2024-04-20T10:45:57.853 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 7027, 2024-04-20T10:45:57.853 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 37.300546767, 2024-04-20T10:45:57.853 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.005308175 2024-04-20T10:45:57.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.854 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rename_latency": { 2024-04-20T10:45:57.854 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 30, 2024-04-20T10:45:57.854 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.089083641, 2024-04-20T10:45:57.854 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002969454 2024-04-20T10:45:57.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.854 INFO:teuthology.orchestra.run.smithi046.stdout: "req_renamesnap_latency": { 2024-04-20T10:45:57.854 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.855 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.855 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.855 INFO:teuthology.orchestra.run.smithi046.stdout: "req_snapdiff_latency": { 2024-04-20T10:45:57.855 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.855 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.855 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.855 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmdir_latency": { 2024-04-20T10:45:57.856 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 109, 2024-04-20T10:45:57.856 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 1.390633735, 2024-04-20T10:45:57.856 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.012758107 2024-04-20T10:45:57.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.856 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmsnap_latency": { 2024-04-20T10:45:57.856 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 5, 2024-04-20T10:45:57.856 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.582066457, 2024-04-20T10:45:57.856 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.116413291 2024-04-20T10:45:57.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.857 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmxattr_latency": { 2024-04-20T10:45:57.857 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.857 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.857 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.857 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setattr_latency": { 2024-04-20T10:45:57.857 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 136, 2024-04-20T10:45:57.857 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 4.495862670, 2024-04-20T10:45:57.858 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.033057813 2024-04-20T10:45:57.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.858 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setdirlayout_latency": { 2024-04-20T10:45:57.858 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.858 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.858 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.858 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setfilelock_latency": { 2024-04-20T10:45:57.858 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.859 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.859 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.859 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setlayout_latency": { 2024-04-20T10:45:57.859 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.859 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.859 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.859 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setxattr_latency": { 2024-04-20T10:45:57.860 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 7, 2024-04-20T10:45:57.860 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.020053784, 2024-04-20T10:45:57.860 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002864826 2024-04-20T10:45:57.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.860 INFO:teuthology.orchestra.run.smithi046.stdout: "req_symlink_latency": { 2024-04-20T10:45:57.860 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 62, 2024-04-20T10:45:57.860 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.057502733, 2024-04-20T10:45:57.860 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000927463 2024-04-20T10:45:57.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.861 INFO:teuthology.orchestra.run.smithi046.stdout: "req_unlink_latency": { 2024-04-20T10:45:57.861 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1116, 2024-04-20T10:45:57.861 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 3.746387759, 2024-04-20T10:45:57.861 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003356978 2024-04-20T10:45:57.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.861 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_revoke_eviction": 0, 2024-04-20T10:45:57.861 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_acquisition_throttle": 0, 2024-04-20T10:45:57.861 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getvxattr_latency": { 2024-04-20T10:45:57.862 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.862 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.862 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:45:57.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.862 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessions": { 2024-04-20T10:45:57.862 INFO:teuthology.orchestra.run.smithi046.stdout: "session_count": 2, 2024-04-20T10:45:57.862 INFO:teuthology.orchestra.run.smithi046.stdout: "session_add": 5, 2024-04-20T10:45:57.862 INFO:teuthology.orchestra.run.smithi046.stdout: "session_remove": 3, 2024-04-20T10:45:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_open": 2, 2024-04-20T10:45:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_stale": 0, 2024-04-20T10:45:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: "total_load": 5116, 2024-04-20T10:45:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: "average_load": 2558, 2024-04-20T10:45:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: "avg_session_uptime": 685, 2024-04-20T10:45:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: "mdthresh_evicted": 0 2024-04-20T10:45:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-04-20T10:45:57.863 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-04-20T10:45:57.864 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-04-20T10:45:57.864 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T10:45:57.864 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-04-20T10:45:57.864 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T10:45:57.864 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-04-20T10:45:57.864 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T10:45:57.864 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T10:45:57.864 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T10:45:57.864 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T10:45:57.865 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T10:45:57.865 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-04-20T10:45:57.865 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T10:45:57.865 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T10:45:57.865 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-04-20T10:45:57.865 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-04-20T10:45:57.865 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-04-20T10:45:57.865 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-04-20T10:45:57.865 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T10:45:57.866 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T10:45:57.866 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T10:45:57.866 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T10:45:57.866 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T10:45:57.866 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-04-20T10:45:57.866 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-04-20T10:45:57.866 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-04-20T10:45:57.866 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T10:45:57.867 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T10:45:57.867 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-04-20T10:45:57.867 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-04-20T10:45:57.867 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-04-20T10:45:57.867 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-04-20T10:45:57.867 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T10:45:57.867 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-04-20T10:45:57.867 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T10:45:57.867 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-04-20T10:45:57.868 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 3646486, 2024-04-20T10:45:57.868 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 90421, 2024-04-20T10:45:57.868 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 792, 2024-04-20T10:45:57.868 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 9, 2024-04-20T10:45:57.868 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-04-20T10:45:57.868 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-04-20T10:45:57.868 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-04-20T10:45:57.868 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-04-20T10:45:57.868 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-04-20T10:45:57.869 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-04-20T10:45:57.869 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 8224, 2024-04-20T10:45:57.869 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 171, 2024-04-20T10:45:57.869 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T10:45:57.869 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-04-20T10:45:57.869 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-04-20T10:45:57.869 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-04-20T10:45:57.869 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 420870219, 2024-04-20T10:45:57.869 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 4959598, 2024-04-20T10:45:57.870 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-04-20T10:45:57.870 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-04-20T10:45:57.870 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-04-20T10:45:57.870 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-04-20T10:45:57.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.870 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter": { 2024-04-20T10:45:57.870 INFO:teuthology.orchestra.run.smithi046.stdout: "op_active": 0, 2024-04-20T10:45:57.870 INFO:teuthology.orchestra.run.smithi046.stdout: "op_laggy": 0, 2024-04-20T10:45:57.871 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send": 919319, 2024-04-20T10:45:57.871 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send_bytes": 810916069, 2024-04-20T10:45:57.871 INFO:teuthology.orchestra.run.smithi046.stdout: "op_resend": 32, 2024-04-20T10:45:57.871 INFO:teuthology.orchestra.run.smithi046.stdout: "op_reply": 919319, 2024-04-20T10:45:57.871 INFO:teuthology.orchestra.run.smithi046.stdout: "op_latency": { 2024-04-20T10:45:57.871 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 919319, 2024-04-20T10:45:57.871 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 11044.704624331, 2024-04-20T10:45:57.871 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.012014006 2024-04-20T10:45:57.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.872 INFO:teuthology.orchestra.run.smithi046.stdout: "op_inflight": 0, 2024-04-20T10:45:57.872 INFO:teuthology.orchestra.run.smithi046.stdout: "oplen_avg": { 2024-04-20T10:45:57.872 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 919319, 2024-04-20T10:45:57.872 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 1051798 2024-04-20T10:45:57.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.872 INFO:teuthology.orchestra.run.smithi046.stdout: "op": 919319, 2024-04-20T10:45:57.872 INFO:teuthology.orchestra.run.smithi046.stdout: "op_r": 835572, 2024-04-20T10:45:57.872 INFO:teuthology.orchestra.run.smithi046.stdout: "op_w": 83747, 2024-04-20T10:45:57.872 INFO:teuthology.orchestra.run.smithi046.stdout: "op_rmw": 0, 2024-04-20T10:45:57.873 INFO:teuthology.orchestra.run.smithi046.stdout: "op_pg": 0, 2024-04-20T10:45:57.873 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_stat": 736, 2024-04-20T10:45:57.873 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_create": 62337, 2024-04-20T10:45:57.873 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_read": 98, 2024-04-20T10:45:57.873 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_write": 12148, 2024-04-20T10:45:57.873 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writefull": 109, 2024-04-20T10:45:57.873 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writesame": 0, 2024-04-20T10:45:57.873 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_append": 0, 2024-04-20T10:45:57.873 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_zero": 0, 2024-04-20T10:45:57.874 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_truncate": 190, 2024-04-20T10:45:57.874 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_delete": 2029, 2024-04-20T10:45:57.874 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_mapext": 0, 2024-04-20T10:45:57.874 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_sparse_read": 0, 2024-04-20T10:45:57.874 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_clonerange": 0, 2024-04-20T10:45:57.874 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_getxattr": 835474, 2024-04-20T10:45:57.874 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_setxattr": 124680, 2024-04-20T10:45:57.874 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_cmpxattr": 0, 2024-04-20T10:45:57.875 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_rmxattr": 0, 2024-04-20T10:45:57.875 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_resetxattrs": 0, 2024-04-20T10:45:57.875 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_call": 0, 2024-04-20T10:45:57.875 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_watch": 0, 2024-04-20T10:45:57.875 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_notify": 0, 2024-04-20T10:45:57.875 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T10:45:57.875 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls": 0, 2024-04-20T10:45:57.875 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls_filter": 0, 2024-04-20T10:45:57.875 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_other": 73, 2024-04-20T10:45:57.876 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_active": 0, 2024-04-20T10:45:57.876 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_send": 0, 2024-04-20T10:45:57.876 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_resend": 0, 2024-04-20T10:45:57.876 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_ping": 0, 2024-04-20T10:45:57.876 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_active": 0, 2024-04-20T10:45:57.876 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_send": 0, 2024-04-20T10:45:57.876 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_resend": 0, 2024-04-20T10:45:57.876 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_active": 0, 2024-04-20T10:45:57.876 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_send": 0, 2024-04-20T10:45:57.877 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_resend": 0, 2024-04-20T10:45:57.877 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_active": 0, 2024-04-20T10:45:57.877 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_send": 0, 2024-04-20T10:45:57.877 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_resend": 0, 2024-04-20T10:45:57.877 INFO:teuthology.orchestra.run.smithi046.stdout: "command_active": 0, 2024-04-20T10:45:57.877 INFO:teuthology.orchestra.run.smithi046.stdout: "command_send": 0, 2024-04-20T10:45:57.877 INFO:teuthology.orchestra.run.smithi046.stdout: "command_resend": 0, 2024-04-20T10:45:57.877 INFO:teuthology.orchestra.run.smithi046.stdout: "map_epoch": 98, 2024-04-20T10:45:57.877 INFO:teuthology.orchestra.run.smithi046.stdout: "map_full": 0, 2024-04-20T10:45:57.878 INFO:teuthology.orchestra.run.smithi046.stdout: "map_inc": 14, 2024-04-20T10:45:57.878 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_sessions": 12, 2024-04-20T10:45:57.878 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_open": 12, 2024-04-20T10:45:57.878 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_close": 0, 2024-04-20T10:45:57.878 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_laggy": 0, 2024-04-20T10:45:57.878 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_wr": 13567, 2024-04-20T10:45:57.878 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_rd": 0, 2024-04-20T10:45:57.878 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_del": 357 2024-04-20T10:45:57.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.879 INFO:teuthology.orchestra.run.smithi046.stdout: "oft": { 2024-04-20T10:45:57.879 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_objs": 1, 2024-04-20T10:45:57.879 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_kv_pairs": 0, 2024-04-20T10:45:57.879 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_updates": 0, 2024-04-20T10:45:57.879 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_removes": 0 2024-04-20T10:45:57.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.879 INFO:teuthology.orchestra.run.smithi046.stdout: "osd-slow-ops": { 2024-04-20T10:45:57.879 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_ops_count": 0 2024-04-20T10:45:57.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.880 INFO:teuthology.orchestra.run.smithi046.stdout: "purge_queue": { 2024-04-20T10:45:57.880 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops": 0, 2024-04-20T10:45:57.880 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops_high_water": 32, 2024-04-20T10:45:57.880 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing": 0, 2024-04-20T10:45:57.880 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_high_water": 32, 2024-04-20T10:45:57.880 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed_ops": 1224, 2024-04-20T10:45:57.880 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed": 1224, 2024-04-20T10:45:57.880 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_item_in_journal": 0 2024-04-20T10:45:57.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.881 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T10:45:57.881 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-20T10:45:57.881 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-20T10:45:57.881 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-20T10:45:57.881 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 1166872, 2024-04-20T10:45:57.881 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 423802108, 2024-04-20T10:45:57.881 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-20T10:45:57.881 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 1166872, 2024-04-20T10:45:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-20T10:45:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-20T10:45:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 1166872, 2024-04-20T10:45:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 423802108, 2024-04-20T10:45:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-20T10:45:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.882 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:45:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_bytes": { 2024-04-20T10:45:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-20T10:45:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-20T10:45:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-20T10:45:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-04-20T10:45:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-04-20T10:45:57.883 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-20T10:45:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-04-20T10:45:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 919319, 2024-04-20T10:45:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 811039749, 2024-04-20T10:45:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 919319, 2024-04-20T10:45:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 811039749, 2024-04-20T10:45:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-20T10:45:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.884 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:45:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_ops": { 2024-04-20T10:45:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-20T10:45:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1024, 2024-04-20T10:45:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-20T10:45:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-04-20T10:45:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-04-20T10:45:57.885 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-20T10:45:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-04-20T10:45:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 919319, 2024-04-20T10:45:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 919319, 2024-04-20T10:45:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 919319, 2024-04-20T10:45:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 919319, 2024-04-20T10:45:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-20T10:45:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.886 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:45:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle": { 2024-04-20T10:45:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-20T10:45:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-04-20T10:45:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-20T10:45:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 1224, 2024-04-20T10:45:57.887 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 123680, 2024-04-20T10:45:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-20T10:45:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 1224, 2024-04-20T10:45:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-20T10:45:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-20T10:45:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 80, 2024-04-20T10:45:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 123680, 2024-04-20T10:45:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-20T10:45:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:45:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:45:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle-0x5577e7c54500": { 2024-04-20T10:45:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-20T10:45:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-04-20T10:45:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-20T10:45:57.889 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 243585, 2024-04-20T10:45:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 684221157, 2024-04-20T10:45:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-20T10:45:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 243585, 2024-04-20T10:45:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-20T10:45:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-20T10:45:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 12068, 2024-04-20T10:45:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 684221157, 2024-04-20T10:45:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-20T10:45:57.890 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:45:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:45:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:45:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:45:57.891 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:45:57.891 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-20T10:45:58.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:57 smithi046 ceph-mon[21574]: pgmap v371: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 3 op/s 2024-04-20T10:45:58.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:57 smithi042 ceph-mon[17078]: pgmap v371: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 3 op/s 2024-04-20T10:45:58.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:57 smithi162 ceph-mon[21854]: pgmap v371: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 3 op/s 2024-04-20T10:45:58.650 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-20T10:45:58.650 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-20T10:45:58.650 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-20T10:45:58.650 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=235 2024-04-20T10:45:58.650 DEBUG:tasks.check_counter:Getting stats from e 2024-04-20T10:45:58.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-20T10:45:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T10:46:00.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:45:59 smithi046 ceph-mon[21574]: pgmap v372: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-04-20T10:46:00.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:45:59 smithi042 ceph-mon[17078]: pgmap v372: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-04-20T10:46:00.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:45:59 smithi162 ceph-mon[21854]: pgmap v372: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-04-20T10:46:01.195 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-20T10:46:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T10:46:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 250, 2024-04-20T10:46:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 238, 2024-04-20T10:46:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1735840, 2024-04-20T10:46:01.195 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 147104, 2024-04-20T10:46:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-04-20T10:46:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-04-20T10:46:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.045795157, 2024-04-20T10:46:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.012475080, 2024-04-20T10:46:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.038116850, 2024-04-20T10:46:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.005216270, 2024-04-20T10:46:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:01.196 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 237, 2024-04-20T10:46:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.009935604, 2024-04-20T10:46:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000041922 2024-04-20T10:46:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:01.197 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1735840, 2024-04-20T10:46:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 147104 2024-04-20T10:46:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T10:46:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 7, 2024-04-20T10:46:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 186, 2024-04-20T10:46:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 3472, 2024-04-20T10:46:01.198 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 267264, 2024-04-20T10:46:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-04-20T10:46:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-04-20T10:46:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.011378853, 2024-04-20T10:46:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.007374708, 2024-04-20T10:46:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.001871524, 2024-04-20T10:46:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:46:01.199 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 184, 2024-04-20T10:46:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.017615407, 2024-04-20T10:46:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000095735 2024-04-20T10:46:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:01.200 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-04-20T10:46:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 267264 2024-04-20T10:46:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T10:46:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-04-20T10:46:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-04-20T10:46:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-04-20T10:46:01.201 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-04-20T10:46:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-04-20T10:46:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-04-20T10:46:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.000275607, 2024-04-20T10:46:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T10:46:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000200414, 2024-04-20T10:46:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:46:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:01.202 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:01.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T10:46:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T10:46:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-04-20T10:46:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-04-20T10:46:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-04-20T10:46:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-04-20T10:46:01.204 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-04-20T10:46:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-04-20T10:46:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T10:46:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-04-20T10:46:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T10:46:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-04-20T10:46:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T10:46:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T10:46:01.205 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T10:46:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T10:46:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T10:46:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-04-20T10:46:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T10:46:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T10:46:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-04-20T10:46:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-04-20T10:46:01.206 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-04-20T10:46:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-04-20T10:46:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T10:46:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T10:46:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T10:46:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T10:46:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T10:46:01.207 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-04-20T10:46:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-04-20T10:46:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-04-20T10:46:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T10:46:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T10:46:01.208 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-04-20T10:46:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-04-20T10:46:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-04-20T10:46:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-04-20T10:46:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T10:46:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-04-20T10:46:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T10:46:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-04-20T10:46:01.209 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 148977, 2024-04-20T10:46:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 30, 2024-04-20T10:46:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-04-20T10:46:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-04-20T10:46:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-04-20T10:46:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-04-20T10:46:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-04-20T10:46:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-04-20T10:46:01.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-04-20T10:46:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-04-20T10:46:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-04-20T10:46:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-04-20T10:46:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T10:46:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-04-20T10:46:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-04-20T10:46:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-04-20T10:46:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-04-20T10:46:01.211 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-04-20T10:46:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-04-20T10:46:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-04-20T10:46:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-04-20T10:46:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-04-20T10:46:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:01.212 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T10:46:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-20T10:46:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-20T10:46:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-20T10:46:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 253, 2024-04-20T10:46:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1700601, 2024-04-20T10:46:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-20T10:46:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 253, 2024-04-20T10:46:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-20T10:46:01.213 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-20T10:46:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 253, 2024-04-20T10:46:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1700601, 2024-04-20T10:46:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-20T10:46:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:46:01.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:46:01.215 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-20T10:46:01.995 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:01 smithi046 ceph-mon[21574]: pgmap v373: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-04-20T10:46:02.026 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-20T10:46:02.026 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-20T10:46:02.026 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-20T10:46:02.026 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-04-20T10:46:02.027 DEBUG:tasks.check_counter:Getting stats from h 2024-04-20T10:46:02.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-20T10:46:02.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:01 smithi042 ceph-mon[17078]: pgmap v373: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-04-20T10:46:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:01 smithi162 ceph-mon[21854]: pgmap v373: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-04-20T10:46:02.280 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T10:46:03.883 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:03 smithi046 ceph-mon[21574]: pgmap v374: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-04-20T10:46:04.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:03 smithi042 ceph-mon[17078]: pgmap v374: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-04-20T10:46:04.132 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-20T10:46:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T10:46:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-04-20T10:46:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-04-20T10:46:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-04-20T10:46:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-04-20T10:46:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-04-20T10:46:04.133 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-04-20T10:46:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.001190823, 2024-04-20T10:46:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T10:46:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000120343, 2024-04-20T10:46:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:46:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:04.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T10:46:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T10:46:04.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T10:46:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 6, 2024-04-20T10:46:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 183, 2024-04-20T10:46:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 3312, 2024-04-20T10:46:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 188416, 2024-04-20T10:46:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 5, 2024-04-20T10:46:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-04-20T10:46:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.009930630, 2024-04-20T10:46:04.136 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.006369717, 2024-04-20T10:46:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.001636689, 2024-04-20T10:46:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:46:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 181, 2024-04-20T10:46:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.025376183, 2024-04-20T10:46:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000140199 2024-04-20T10:46:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:04.137 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-20T10:46:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 188416 2024-04-20T10:46:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T10:46:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 249, 2024-04-20T10:46:04.138 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 237, 2024-04-20T10:46:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1735568, 2024-04-20T10:46:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 146464, 2024-04-20T10:46:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-04-20T10:46:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-04-20T10:46:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.043776105, 2024-04-20T10:46:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.011751038, 2024-04-20T10:46:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.037083809, 2024-04-20T10:46:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.005550314, 2024-04-20T10:46:04.139 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 236, 2024-04-20T10:46:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.009429601, 2024-04-20T10:46:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000039955 2024-04-20T10:46:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:04.140 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1735568, 2024-04-20T10:46:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 146464 2024-04-20T10:46:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-04-20T10:46:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-04-20T10:46:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-04-20T10:46:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:04.141 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-04-20T10:46:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-04-20T10:46:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-04-20T10:46:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T10:46:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-04-20T10:46:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T10:46:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-04-20T10:46:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T10:46:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T10:46:04.142 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T10:46:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T10:46:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T10:46:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-04-20T10:46:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T10:46:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T10:46:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-04-20T10:46:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-04-20T10:46:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-04-20T10:46:04.143 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-04-20T10:46:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T10:46:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T10:46:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T10:46:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T10:46:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T10:46:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-04-20T10:46:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-04-20T10:46:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-04-20T10:46:04.144 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T10:46:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T10:46:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-04-20T10:46:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-04-20T10:46:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-04-20T10:46:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-04-20T10:46:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T10:46:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-04-20T10:46:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T10:46:04.145 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-04-20T10:46:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 148955, 2024-04-20T10:46:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 30, 2024-04-20T10:46:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-04-20T10:46:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-04-20T10:46:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-04-20T10:46:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-04-20T10:46:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-04-20T10:46:04.146 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-04-20T10:46:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-04-20T10:46:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-04-20T10:46:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-04-20T10:46:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-04-20T10:46:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T10:46:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-04-20T10:46:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-04-20T10:46:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-04-20T10:46:04.147 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-04-20T10:46:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-04-20T10:46:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-04-20T10:46:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-04-20T10:46:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-04-20T10:46:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-04-20T10:46:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T10:46:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-20T10:46:04.148 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-20T10:46:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-20T10:46:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 251, 2024-04-20T10:46:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1700444, 2024-04-20T10:46:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-20T10:46:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 251, 2024-04-20T10:46:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-20T10:46:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-20T10:46:04.149 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 251, 2024-04-20T10:46:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1700444, 2024-04-20T10:46:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-20T10:46:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:46:04.150 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:46:04.150 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-20T10:46:04.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:03 smithi162 ceph-mon[21854]: pgmap v374: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-04-20T10:46:04.851 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-04-20T10:46:04.851 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-04-20T10:46:04.851 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-04-20T10:46:04.851 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.h 2024-04-20T10:46:04.851 DEBUG:tasks.check_counter:Getting stats from k 2024-04-20T10:46:04.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-20T10:46:05.108 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.b/config 2024-04-20T10:46:05.966 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:05 smithi046 ceph-mon[21574]: pgmap v375: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 91 KiB/s wr, 19 op/s 2024-04-20T10:46:06.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:05 smithi042 ceph-mon[17078]: pgmap v375: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 91 KiB/s wr, 19 op/s 2024-04-20T10:46:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:05 smithi162 ceph-mon[21854]: pgmap v375: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 91 KiB/s wr, 19 op/s 2024-04-20T10:46:07.097 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-20T10:46:07.097 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T10:46:07.097 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-04-20T10:46:07.097 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-04-20T10:46:07.097 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-04-20T10:46:07.097 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-04-20T10:46:07.098 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-04-20T10:46:07.098 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-04-20T10:46:07.098 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.001081594, 2024-04-20T10:46:07.098 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T10:46:07.098 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000180073, 2024-04-20T10:46:07.098 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:46:07.098 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:07.099 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:07.099 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:07.099 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:07.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:07.099 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:07.099 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:07.099 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:07.099 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:07.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:07.100 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T10:46:07.100 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T10:46:07.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:07.100 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T10:46:07.100 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 6, 2024-04-20T10:46:07.100 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 188, 2024-04-20T10:46:07.100 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 3312, 2024-04-20T10:46:07.100 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 189296, 2024-04-20T10:46:07.101 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 5, 2024-04-20T10:46:07.101 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-04-20T10:46:07.101 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.011401144, 2024-04-20T10:46:07.101 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.007409894, 2024-04-20T10:46:07.101 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.001769822, 2024-04-20T10:46:07.101 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:46:07.101 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:07.101 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 186, 2024-04-20T10:46:07.102 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.017057436, 2024-04-20T10:46:07.102 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000091706 2024-04-20T10:46:07.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:07.102 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:07.102 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:07.102 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:07.102 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:07.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:07.103 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-20T10:46:07.103 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 189296 2024-04-20T10:46:07.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:07.103 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T10:46:07.103 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 255, 2024-04-20T10:46:07.103 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 243, 2024-04-20T10:46:07.104 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1737824, 2024-04-20T10:46:07.104 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 150000, 2024-04-20T10:46:07.104 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-04-20T10:46:07.104 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-04-20T10:46:07.104 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.046203825, 2024-04-20T10:46:07.104 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.011799176, 2024-04-20T10:46:07.104 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.038935319, 2024-04-20T10:46:07.104 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.005940231, 2024-04-20T10:46:07.104 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:07.105 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 242, 2024-04-20T10:46:07.105 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.010441665, 2024-04-20T10:46:07.105 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000043147 2024-04-20T10:46:07.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:07.105 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:07.105 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:07.105 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:07.105 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:07.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:07.106 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1737824, 2024-04-20T10:46:07.106 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 150000 2024-04-20T10:46:07.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:07.106 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-04-20T10:46:07.106 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-04-20T10:46:07.106 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-04-20T10:46:07.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:07.106 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-04-20T10:46:07.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-04-20T10:46:07.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-04-20T10:46:07.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T10:46:07.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-04-20T10:46:07.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T10:46:07.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-04-20T10:46:07.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T10:46:07.107 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T10:46:07.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T10:46:07.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T10:46:07.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T10:46:07.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-04-20T10:46:07.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T10:46:07.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T10:46:07.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-04-20T10:46:07.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-04-20T10:46:07.108 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-04-20T10:46:07.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-04-20T10:46:07.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T10:46:07.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T10:46:07.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T10:46:07.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T10:46:07.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T10:46:07.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-04-20T10:46:07.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-04-20T10:46:07.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-04-20T10:46:07.109 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T10:46:07.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T10:46:07.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-04-20T10:46:07.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-04-20T10:46:07.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-04-20T10:46:07.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-04-20T10:46:07.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T10:46:07.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-04-20T10:46:07.110 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T10:46:07.111 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-04-20T10:46:07.111 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 148670, 2024-04-20T10:46:07.111 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 30, 2024-04-20T10:46:07.111 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-04-20T10:46:07.111 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-04-20T10:46:07.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-04-20T10:46:07.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-04-20T10:46:07.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-04-20T10:46:07.111 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-04-20T10:46:07.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-04-20T10:46:07.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-04-20T10:46:07.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-04-20T10:46:07.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-04-20T10:46:07.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T10:46:07.112 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-04-20T10:46:07.112 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-04-20T10:46:07.112 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-04-20T10:46:07.112 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-04-20T10:46:07.113 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-04-20T10:46:07.113 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-04-20T10:46:07.113 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-04-20T10:46:07.113 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-04-20T10:46:07.113 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-04-20T10:46:07.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-20T10:46:07.113 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T10:46:07.113 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-04-20T10:46:07.113 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-04-20T10:46:07.113 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-04-20T10:46:07.114 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 257, 2024-04-20T10:46:07.114 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1701861, 2024-04-20T10:46:07.114 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-04-20T10:46:07.114 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 257, 2024-04-20T10:46:07.114 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-04-20T10:46:07.114 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-04-20T10:46:07.114 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 257, 2024-04-20T10:46:07.114 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1701861, 2024-04-20T10:46:07.115 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-04-20T10:46:07.115 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-04-20T10:46:07.115 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-04-20T10:46:07.115 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-04-20T10:46:07.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:46:07.115 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-20T10:46:07.115 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-20T10:46:07.748 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-20T10:46:07.748 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-20T10:46:07.749 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-20T10:46:07.749 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-20T10:46:07.749 DEBUG:tasks.check_counter:Getting stats from c 2024-04-20T10:46:07.749 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-20T10:46:08.001 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:46:08.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:07 smithi046 ceph-mon[21574]: pgmap v376: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 91 KiB/s wr, 19 op/s 2024-04-20T10:46:08.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:07 smithi042 ceph-mon[17078]: pgmap v376: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 91 KiB/s wr, 19 op/s 2024-04-20T10:46:08.123 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:07 smithi162 ceph-mon[21854]: pgmap v376: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 91 KiB/s wr, 19 op/s 2024-04-20T10:46:09.804 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:09 smithi162 ceph-mon[21854]: pgmap v377: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 86 KiB/s wr, 19 op/s 2024-04-20T10:46:09.804 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-20T10:46:09.804 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T10:46:09.804 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 0, 2024-04-20T10:46:09.804 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 2, 2024-04-20T10:46:09.805 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 0, 2024-04-20T10:46:09.805 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 0, 2024-04-20T10:46:09.805 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 4, 2024-04-20T10:46:09.805 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 0, 2024-04-20T10:46:09.805 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.001121075, 2024-04-20T10:46:09.805 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T10:46:09.805 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.000239540, 2024-04-20T10:46:09.805 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:46:09.806 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:09.806 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:09.806 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:09.806 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:09.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:09.806 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:09.806 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:09.806 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:09.807 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:09.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:09.807 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T10:46:09.807 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T10:46:09.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:09.807 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T10:46:09.807 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 255, 2024-04-20T10:46:09.807 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 241, 2024-04-20T10:46:09.807 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 1739104, 2024-04-20T10:46:09.808 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 148192, 2024-04-20T10:46:09.808 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 2, 2024-04-20T10:46:09.808 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 1, 2024-04-20T10:46:09.808 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.045700025, 2024-04-20T10:46:09.808 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.013690129, 2024-04-20T10:46:09.808 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.035466038, 2024-04-20T10:46:09.808 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.005516430, 2024-04-20T10:46:09.808 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:09.809 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 241, 2024-04-20T10:46:09.809 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.012163158, 2024-04-20T10:46:09.809 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000050469 2024-04-20T10:46:09.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:09.809 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:09.809 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:09.809 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:09.809 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:09.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:09.810 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 1739104, 2024-04-20T10:46:09.810 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 148192 2024-04-20T10:46:09.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:09.810 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T10:46:09.810 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 3, 2024-04-20T10:46:09.810 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 185, 2024-04-20T10:46:09.810 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 480, 2024-04-20T10:46:09.810 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 266816, 2024-04-20T10:46:09.811 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 6, 2024-04-20T10:46:09.811 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 3, 2024-04-20T10:46:09.811 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.011261781, 2024-04-20T10:46:09.811 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.007291545, 2024-04-20T10:46:09.811 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.001746247, 2024-04-20T10:46:09.811 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:46:09.811 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:09.811 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 181, 2024-04-20T10:46:09.812 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.027039471, 2024-04-20T10:46:09.812 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000149389 2024-04-20T10:46:09.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:09.812 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:09.812 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:09.812 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:09.812 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:09.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:09.813 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-20T10:46:09.813 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 266816 2024-04-20T10:46:09.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:09.813 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2024-04-20T10:46:09.813 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 0, 2024-04-20T10:46:09.813 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2024-04-20T10:46:09.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:09.813 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2024-04-20T10:46:09.813 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2024-04-20T10:46:09.814 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2024-04-20T10:46:09.814 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T10:46:09.814 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2024-04-20T10:46:09.814 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T10:46:09.814 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2024-04-20T10:46:09.814 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T10:46:09.814 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T10:46:09.814 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T10:46:09.815 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T10:46:09.815 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T10:46:09.815 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2024-04-20T10:46:09.815 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T10:46:09.815 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T10:46:09.815 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2024-04-20T10:46:09.815 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2024-04-20T10:46:09.815 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2024-04-20T10:46:09.815 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2024-04-20T10:46:09.816 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T10:46:09.816 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T10:46:09.816 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T10:46:09.816 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T10:46:09.816 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T10:46:09.816 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2024-04-20T10:46:09.816 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2024-04-20T10:46:09.816 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2024-04-20T10:46:09.817 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T10:46:09.817 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T10:46:09.817 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2024-04-20T10:46:09.817 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2024-04-20T10:46:09.817 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2024-04-20T10:46:09.817 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2024-04-20T10:46:09.817 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T10:46:09.817 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2024-04-20T10:46:09.817 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T10:46:09.818 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2024-04-20T10:46:09.818 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 148995, 2024-04-20T10:46:09.818 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 31, 2024-04-20T10:46:09.818 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2024-04-20T10:46:09.818 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2024-04-20T10:46:09.818 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2024-04-20T10:46:09.818 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2024-04-20T10:46:09.818 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2024-04-20T10:46:09.819 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2024-04-20T10:46:09.819 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2024-04-20T10:46:09.819 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2024-04-20T10:46:09.819 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 0, 2024-04-20T10:46:09.819 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 0, 2024-04-20T10:46:09.819 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T10:46:09.819 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2024-04-20T10:46:09.819 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2024-04-20T10:46:09.819 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2024-04-20T10:46:09.820 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 928, 2024-04-20T10:46:09.820 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 2, 2024-04-20T10:46:09.820 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2024-04-20T10:46:09.820 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2024-04-20T10:46:09.820 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2024-04-20T10:46:09.820 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2024-04-20T10:46:09.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:09.820 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T10:46:09.821 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-04-20T10:46:09.821 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-04-20T10:46:09.821 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-04-20T10:46:09.821 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 254, 2024-04-20T10:46:09.821 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 1700731, 2024-04-20T10:46:09.821 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-04-20T10:46:09.821 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 254, 2024-04-20T10:46:09.821 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-04-20T10:46:09.821 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-04-20T10:46:09.822 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 254, 2024-04-20T10:46:09.822 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 1700731, 2024-04-20T10:46:09.822 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-04-20T10:46:09.822 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:09.822 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:09.822 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:09.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:09.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:09.823 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-20T10:46:09.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:09 smithi042 ceph-mon[17078]: pgmap v377: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 86 KiB/s wr, 19 op/s 2024-04-20T10:46:10.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:09 smithi046 ceph-mon[21574]: pgmap v377: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 86 KiB/s wr, 19 op/s 2024-04-20T10:46:10.593 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-04-20T10:46:10.593 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-04-20T10:46:10.593 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-04-20T10:46:10.593 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.c 2024-04-20T10:46:10.593 DEBUG:tasks.check_counter:Getting stats from f 2024-04-20T10:46:10.593 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-20T10:46:10.848 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:46:12.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:11 smithi046 ceph-mon[21574]: pgmap v378: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 86 KiB/s wr, 19 op/s 2024-04-20T10:46:12.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:11 smithi046 ceph-mon[21574]: osdmap e99: 12 total, 12 up, 12 in 2024-04-20T10:46:12.084 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:11 smithi162 ceph-mon[21854]: pgmap v378: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 86 KiB/s wr, 19 op/s 2024-04-20T10:46:12.084 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:11 smithi162 ceph-mon[21854]: osdmap e99: 12 total, 12 up, 12 in 2024-04-20T10:46:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:11 smithi042 ceph-mon[17078]: pgmap v378: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 86 KiB/s wr, 19 op/s 2024-04-20T10:46:12.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:11 smithi042 ceph-mon[17078]: osdmap e99: 12 total, 12 up, 12 in 2024-04-20T10:46:12.682 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-20T10:46:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T10:46:12.682 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 253, 2024-04-20T10:46:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 241, 2024-04-20T10:46:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 1737280, 2024-04-20T10:46:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 148720, 2024-04-20T10:46:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 4, 2024-04-20T10:46:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 0, 2024-04-20T10:46:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.048620859, 2024-04-20T10:46:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.013826848, 2024-04-20T10:46:12.683 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.038219816, 2024-04-20T10:46:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.005431410, 2024-04-20T10:46:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 240, 2024-04-20T10:46:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.010999660, 2024-04-20T10:46:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000045831 2024-04-20T10:46:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:12.684 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 1737280, 2024-04-20T10:46:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 148720 2024-04-20T10:46:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T10:46:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 2, 2024-04-20T10:46:12.685 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 185, 2024-04-20T10:46:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 320, 2024-04-20T10:46:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 188672, 2024-04-20T10:46:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 5, 2024-04-20T10:46:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 3, 2024-04-20T10:46:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.011422712, 2024-04-20T10:46:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.007849736, 2024-04-20T10:46:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.001280876, 2024-04-20T10:46:12.686 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:46:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 182, 2024-04-20T10:46:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.018399487, 2024-04-20T10:46:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000101096 2024-04-20T10:46:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:12.687 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T10:46:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 188672 2024-04-20T10:46:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T10:46:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 4, 2024-04-20T10:46:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 4, 2024-04-20T10:46:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 2992, 2024-04-20T10:46:12.688 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 448, 2024-04-20T10:46:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 2, 2024-04-20T10:46:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 1, 2024-04-20T10:46:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.000529251, 2024-04-20T10:46:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.000035004, 2024-04-20T10:46:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.000489130, 2024-04-20T10:46:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:46:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:12.689 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 3, 2024-04-20T10:46:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000693431, 2024-04-20T10:46:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000231143 2024-04-20T10:46:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:12.690 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-20T10:46:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 448 2024-04-20T10:46:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2024-04-20T10:46:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 0, 2024-04-20T10:46:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2024-04-20T10:46:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2024-04-20T10:46:12.691 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2024-04-20T10:46:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2024-04-20T10:46:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T10:46:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2024-04-20T10:46:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T10:46:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2024-04-20T10:46:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T10:46:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T10:46:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T10:46:12.692 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T10:46:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T10:46:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2024-04-20T10:46:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T10:46:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T10:46:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2024-04-20T10:46:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2024-04-20T10:46:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2024-04-20T10:46:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2024-04-20T10:46:12.693 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T10:46:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T10:46:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T10:46:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T10:46:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T10:46:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2024-04-20T10:46:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2024-04-20T10:46:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2024-04-20T10:46:12.694 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T10:46:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T10:46:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2024-04-20T10:46:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2024-04-20T10:46:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2024-04-20T10:46:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2024-04-20T10:46:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T10:46:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2024-04-20T10:46:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T10:46:12.695 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2024-04-20T10:46:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 148955, 2024-04-20T10:46:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 30, 2024-04-20T10:46:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2024-04-20T10:46:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2024-04-20T10:46:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2024-04-20T10:46:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2024-04-20T10:46:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2024-04-20T10:46:12.696 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2024-04-20T10:46:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2024-04-20T10:46:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2024-04-20T10:46:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 0, 2024-04-20T10:46:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 0, 2024-04-20T10:46:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T10:46:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2024-04-20T10:46:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2024-04-20T10:46:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2024-04-20T10:46:12.697 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 928, 2024-04-20T10:46:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 2, 2024-04-20T10:46:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2024-04-20T10:46:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2024-04-20T10:46:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2024-04-20T10:46:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2024-04-20T10:46:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T10:46:12.698 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-04-20T10:46:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-04-20T10:46:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-04-20T10:46:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 255, 2024-04-20T10:46:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 1701601, 2024-04-20T10:46:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-04-20T10:46:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 255, 2024-04-20T10:46:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-04-20T10:46:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-04-20T10:46:12.699 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 255, 2024-04-20T10:46:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 1701601, 2024-04-20T10:46:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-04-20T10:46:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:12.700 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:12.700 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-20T10:46:13.381 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-20T10:46:13.382 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-20T10:46:13.382 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-20T10:46:13.382 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-04-20T10:46:13.382 DEBUG:tasks.check_counter:Getting stats from i 2024-04-20T10:46:13.382 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-20T10:46:13.625 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:46:14.013 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:13 smithi162 ceph-mon[21854]: pgmap v380: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:14.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:13 smithi046 ceph-mon[21574]: pgmap v380: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:14.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:13 smithi042 ceph-mon[17078]: pgmap v380: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:15.400 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-20T10:46:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T10:46:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 2140, 2024-04-20T10:46:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 2142, 2024-04-20T10:46:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 260723491, 2024-04-20T10:46:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 680530, 2024-04-20T10:46:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 8, 2024-04-20T10:46:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 4, 2024-04-20T10:46:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.341263144, 2024-04-20T10:46:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.057469403, 2024-04-20T10:46:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.199767425, 2024-04-20T10:46:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.066131324, 2024-04-20T10:46:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 2140, 2024-04-20T10:46:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.078291102, 2024-04-20T10:46:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000036584 2024-04-20T10:46:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T10:46:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T10:46:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T10:46:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 1202, 2024-04-20T10:46:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 1187, 2024-04-20T10:46:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 271913678, 2024-04-20T10:46:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 446182, 2024-04-20T10:46:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 6, 2024-04-20T10:46:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 5, 2024-04-20T10:46:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.262242211, 2024-04-20T10:46:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.035151482, 2024-04-20T10:46:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.200863582, 2024-04-20T10:46:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.029117472, 2024-04-20T10:46:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1187, 2024-04-20T10:46:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.041552489, 2024-04-20T10:46:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000035006 2024-04-20T10:46:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 1761168, 2024-04-20T10:46:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 151648 2024-04-20T10:46:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T10:46:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 842, 2024-04-20T10:46:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 1022, 2024-04-20T10:46:15.407 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 155468015, 2024-04-20T10:46:15.407 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 754867, 2024-04-20T10:46:15.407 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 8, 2024-04-20T10:46:15.407 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 6, 2024-04-20T10:46:15.407 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.146371537, 2024-04-20T10:46:15.407 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.030041702, 2024-04-20T10:46:15.407 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.086872911, 2024-04-20T10:46:15.407 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.021468438, 2024-04-20T10:46:15.408 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:15.408 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1019, 2024-04-20T10:46:15.408 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.045284678, 2024-04-20T10:46:15.408 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000044440 2024-04-20T10:46:15.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.408 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:15.408 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.408 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.409 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.409 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T10:46:15.409 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 488064 2024-04-20T10:46:15.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.409 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2024-04-20T10:46:15.409 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 1, 2024-04-20T10:46:15.409 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2024-04-20T10:46:15.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.410 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-MDSRank": { 2024-04-20T10:46:15.410 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2024-04-20T10:46:15.410 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2024-04-20T10:46:15.410 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 5991, 2024-04-20T10:46:15.410 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.726466520, 2024-04-20T10:46:15.410 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000121259 2024-04-20T10:46:15.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:15.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.411 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-PurgeQueue": { 2024-04-20T10:46:15.411 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2024-04-20T10:46:15.411 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2024-04-20T10:46:15.411 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.411 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.411 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.411 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:15.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.412 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-04-20T10:46:15.412 INFO:teuthology.orchestra.run.smithi162.stdout: "request": 0, 2024-04-20T10:46:15.412 INFO:teuthology.orchestra.run.smithi162.stdout: "reply": 0, 2024-04-20T10:46:15.412 INFO:teuthology.orchestra.run.smithi162.stdout: "reply_latency": { 2024-04-20T10:46:15.412 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.412 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.412 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.413 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_reply": 0, 2024-04-20T10:46:15.413 INFO:teuthology.orchestra.run.smithi162.stdout: "forward": 0, 2024-04-20T10:46:15.413 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_complete": 0, 2024-04-20T10:46:15.413 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_keys": 0, 2024-04-20T10:46:15.413 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_commit": 0, 2024-04-20T10:46:15.413 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_split": 0, 2024-04-20T10:46:15.413 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_merge": 0, 2024-04-20T10:46:15.413 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes": 121948, 2024-04-20T10:46:15.413 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_top": 49140, 2024-04-20T10:46:15.414 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_bottom": 72808, 2024-04-20T10:46:15.414 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pin_tail": 0, 2024-04-20T10:46:15.414 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pinned": 51748, 2024-04-20T10:46:15.414 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_expired": 0, 2024-04-20T10:46:15.414 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_with_caps": 0, 2024-04-20T10:46:15.414 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": 0, 2024-04-20T10:46:15.414 INFO:teuthology.orchestra.run.smithi162.stdout: "subtrees": 2, 2024-04-20T10:46:15.414 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse": 0, 2024-04-20T10:46:15.415 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_hit": 0, 2024-04-20T10:46:15.415 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_forward": 0, 2024-04-20T10:46:15.415 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_discover": 0, 2024-04-20T10:46:15.415 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_dir_fetch": 0, 2024-04-20T10:46:15.415 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_remote_ino": 0, 2024-04-20T10:46:15.415 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_lock": 0, 2024-04-20T10:46:15.415 INFO:teuthology.orchestra.run.smithi162.stdout: "load_cent": 0, 2024-04-20T10:46:15.415 INFO:teuthology.orchestra.run.smithi162.stdout: "q": 0, 2024-04-20T10:46:15.416 INFO:teuthology.orchestra.run.smithi162.stdout: "exported": 0, 2024-04-20T10:46:15.416 INFO:teuthology.orchestra.run.smithi162.stdout: "exported_inodes": 0, 2024-04-20T10:46:15.416 INFO:teuthology.orchestra.run.smithi162.stdout: "imported": 0, 2024-04-20T10:46:15.416 INFO:teuthology.orchestra.run.smithi162.stdout: "imported_inodes": 0, 2024-04-20T10:46:15.416 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_dir_fetch": 0, 2024-04-20T10:46:15.416 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_backtrace_fetch": 0, 2024-04-20T10:46:15.416 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_peer_discover": 0, 2024-04-20T10:46:15.416 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rfiles": 85013, 2024-04-20T10:46:15.417 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rbytes": 1558153240, 2024-04-20T10:46:15.417 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rsnaps": 6, 2024-04-20T10:46:15.417 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_fetch": 0, 2024-04-20T10:46:15.417 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_set_tag": 0, 2024-04-20T10:46:15.417 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T10:46:15.417 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_inotable_repaired": 0, 2024-04-20T10:46:15.417 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_inodes": 0, 2024-04-20T10:46:15.417 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T10:46:15.417 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dirfrag_rstats": 0, 2024-04-20T10:46:15.418 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_file_inodes": 0, 2024-04-20T10:46:15.418 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_inode_file_caps": 0, 2024-04-20T10:46:15.418 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_revoke": 0, 2024-04-20T10:46:15.418 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_grant": 0, 2024-04-20T10:46:15.418 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T10:46:15.418 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T10:46:15.418 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T10:46:15.418 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps": 0, 2024-04-20T10:46:15.419 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps_dirty": 0, 2024-04-20T10:46:15.419 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_cap_release": 0, 2024-04-20T10:46:15.419 INFO:teuthology.orchestra.run.smithi162.stdout: "process_request_cap_release": 0 2024-04-20T10:46:15.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.419 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache": { 2024-04-20T10:46:15.419 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update": 0, 2024-04-20T10:46:15.419 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update_receipt": 0, 2024-04-20T10:46:15.419 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_try_discover": 0, 2024-04-20T10:46:15.420 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_send_discover": 0, 2024-04-20T10:46:15.420 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_handle_discover": 0, 2024-04-20T10:46:15.420 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays": 0, 2024-04-20T10:46:15.420 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_delayed": 0, 2024-04-20T10:46:15.420 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_enqueuing": 0, 2024-04-20T10:46:15.420 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_created": 0, 2024-04-20T10:46:15.420 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_enqueued": 0, 2024-04-20T10:46:15.420 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_reintegrated": 0, 2024-04-20T10:46:15.420 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_migrated": 0, 2024-04-20T10:46:15.421 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_processing": 0, 2024-04-20T10:46:15.421 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_enqueued": 0, 2024-04-20T10:46:15.421 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_prioritized": 0, 2024-04-20T10:46:15.421 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_started": 0, 2024-04-20T10:46:15.421 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_completed": 0, 2024-04-20T10:46:15.421 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_quiesce_path": 0, 2024-04-20T10:46:15.421 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_quiesce_inode": 0, 2024-04-20T10:46:15.421 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T10:46:15.421 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_exportdir": 0, 2024-04-20T10:46:15.422 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_flush": 0, 2024-04-20T10:46:15.422 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragmentdir": 0, 2024-04-20T10:46:15.422 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragstats": 0, 2024-04-20T10:46:15.422 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_inodestats": 0 2024-04-20T10:46:15.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.422 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log": { 2024-04-20T10:46:15.422 INFO:teuthology.orchestra.run.smithi162.stdout: "evlrg": 0, 2024-04-20T10:46:15.422 INFO:teuthology.orchestra.run.smithi162.stdout: "evadd": 0, 2024-04-20T10:46:15.423 INFO:teuthology.orchestra.run.smithi162.stdout: "evex": 0, 2024-04-20T10:46:15.423 INFO:teuthology.orchestra.run.smithi162.stdout: "evtrm": 0, 2024-04-20T10:46:15.423 INFO:teuthology.orchestra.run.smithi162.stdout: "ev": 243646, 2024-04-20T10:46:15.423 INFO:teuthology.orchestra.run.smithi162.stdout: "evexg": 0, 2024-04-20T10:46:15.423 INFO:teuthology.orchestra.run.smithi162.stdout: "evexd": 0, 2024-04-20T10:46:15.423 INFO:teuthology.orchestra.run.smithi162.stdout: "segadd": 0, 2024-04-20T10:46:15.423 INFO:teuthology.orchestra.run.smithi162.stdout: "segex": 0, 2024-04-20T10:46:15.423 INFO:teuthology.orchestra.run.smithi162.stdout: "segtrm": 0, 2024-04-20T10:46:15.423 INFO:teuthology.orchestra.run.smithi162.stdout: "seg": 144, 2024-04-20T10:46:15.424 INFO:teuthology.orchestra.run.smithi162.stdout: "segmjr": 21, 2024-04-20T10:46:15.424 INFO:teuthology.orchestra.run.smithi162.stdout: "segexg": 0, 2024-04-20T10:46:15.424 INFO:teuthology.orchestra.run.smithi162.stdout: "segexd": 0, 2024-04-20T10:46:15.424 INFO:teuthology.orchestra.run.smithi162.stdout: "expos": 363925891, 2024-04-20T10:46:15.424 INFO:teuthology.orchestra.run.smithi162.stdout: "wrpos": 689273386, 2024-04-20T10:46:15.424 INFO:teuthology.orchestra.run.smithi162.stdout: "rdpos": 689260153, 2024-04-20T10:46:15.424 INFO:teuthology.orchestra.run.smithi162.stdout: "jlat": { 2024-04-20T10:46:15.424 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.425 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.425 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.425 INFO:teuthology.orchestra.run.smithi162.stdout: "replayed": 243646 2024-04-20T10:46:15.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.425 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_mem": { 2024-04-20T10:46:15.425 INFO:teuthology.orchestra.run.smithi162.stdout: "ino": 119601, 2024-04-20T10:46:15.425 INFO:teuthology.orchestra.run.smithi162.stdout: "ino+": 120825, 2024-04-20T10:46:15.426 INFO:teuthology.orchestra.run.smithi162.stdout: "ino-": 1224, 2024-04-20T10:46:15.426 INFO:teuthology.orchestra.run.smithi162.stdout: "dir": 6957, 2024-04-20T10:46:15.426 INFO:teuthology.orchestra.run.smithi162.stdout: "dir+": 7300, 2024-04-20T10:46:15.426 INFO:teuthology.orchestra.run.smithi162.stdout: "dir-": 343, 2024-04-20T10:46:15.426 INFO:teuthology.orchestra.run.smithi162.stdout: "dn": 121948, 2024-04-20T10:46:15.426 INFO:teuthology.orchestra.run.smithi162.stdout: "dn+": 122025, 2024-04-20T10:46:15.426 INFO:teuthology.orchestra.run.smithi162.stdout: "dn-": 77, 2024-04-20T10:46:15.426 INFO:teuthology.orchestra.run.smithi162.stdout: "cap": 0, 2024-04-20T10:46:15.426 INFO:teuthology.orchestra.run.smithi162.stdout: "cap+": 0, 2024-04-20T10:46:15.427 INFO:teuthology.orchestra.run.smithi162.stdout: "cap-": 0, 2024-04-20T10:46:15.427 INFO:teuthology.orchestra.run.smithi162.stdout: "rss": 494472, 2024-04-20T10:46:15.427 INFO:teuthology.orchestra.run.smithi162.stdout: "heap": 215416 2024-04-20T10:46:15.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.427 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_server": { 2024-04-20T10:46:15.427 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_client_request": 0, 2024-04-20T10:46:15.427 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_server_request": 0, 2024-04-20T10:46:15.427 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_request": 0, 2024-04-20T10:46:15.428 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_session": 0, 2024-04-20T10:46:15.428 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_peer_request": 0, 2024-04-20T10:46:15.428 INFO:teuthology.orchestra.run.smithi162.stdout: "req_create_latency": { 2024-04-20T10:46:15.428 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.428 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.428 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.428 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getattr_latency": { 2024-04-20T10:46:15.428 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.429 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.429 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.429 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getfilelock_latency": { 2024-04-20T10:46:15.429 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.429 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.429 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.430 INFO:teuthology.orchestra.run.smithi162.stdout: "req_link_latency": { 2024-04-20T10:46:15.430 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.430 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.430 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.430 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookup_latency": { 2024-04-20T10:46:15.430 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.430 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.430 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.431 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookuphash_latency": { 2024-04-20T10:46:15.431 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.431 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.431 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.431 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupino_latency": { 2024-04-20T10:46:15.431 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.432 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.432 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.432 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupname_latency": { 2024-04-20T10:46:15.432 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.432 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.432 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.432 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupparent_latency": { 2024-04-20T10:46:15.433 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.433 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.433 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.433 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupsnap_latency": { 2024-04-20T10:46:15.433 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.433 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.434 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.434 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lssnap_latency": { 2024-04-20T10:46:15.434 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.434 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.435 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.435 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mkdir_latency": { 2024-04-20T10:46:15.435 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.435 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.436 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.436 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mknod_latency": { 2024-04-20T10:46:15.436 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.436 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.436 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.436 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mksnap_latency": { 2024-04-20T10:46:15.437 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.437 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.437 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.437 INFO:teuthology.orchestra.run.smithi162.stdout: "req_open_latency": { 2024-04-20T10:46:15.437 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.437 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.437 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.438 INFO:teuthology.orchestra.run.smithi162.stdout: "req_readdir_latency": { 2024-04-20T10:46:15.438 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.438 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.438 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.438 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rename_latency": { 2024-04-20T10:46:15.438 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.439 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.439 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.439 INFO:teuthology.orchestra.run.smithi162.stdout: "req_renamesnap_latency": { 2024-04-20T10:46:15.439 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.439 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.439 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.439 INFO:teuthology.orchestra.run.smithi162.stdout: "req_snapdiff_latency": { 2024-04-20T10:46:15.440 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.440 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.440 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.440 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmdir_latency": { 2024-04-20T10:46:15.440 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.440 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.440 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.441 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmsnap_latency": { 2024-04-20T10:46:15.441 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.441 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.441 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.441 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmxattr_latency": { 2024-04-20T10:46:15.441 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.441 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.442 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.442 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setattr_latency": { 2024-04-20T10:46:15.442 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.442 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.442 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.442 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setdirlayout_latency": { 2024-04-20T10:46:15.442 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.443 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.443 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.443 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setfilelock_latency": { 2024-04-20T10:46:15.443 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.443 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.443 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.444 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setlayout_latency": { 2024-04-20T10:46:15.444 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.444 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.444 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.444 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setxattr_latency": { 2024-04-20T10:46:15.444 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.444 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.444 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.445 INFO:teuthology.orchestra.run.smithi162.stdout: "req_symlink_latency": { 2024-04-20T10:46:15.445 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.445 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.445 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.445 INFO:teuthology.orchestra.run.smithi162.stdout: "req_unlink_latency": { 2024-04-20T10:46:15.445 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.446 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.446 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.446 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_revoke_eviction": 0, 2024-04-20T10:46:15.446 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_acquisition_throttle": 0, 2024-04-20T10:46:15.446 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getvxattr_latency": { 2024-04-20T10:46:15.446 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.446 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.447 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:15.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.447 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_sessions": { 2024-04-20T10:46:15.447 INFO:teuthology.orchestra.run.smithi162.stdout: "session_count": 2, 2024-04-20T10:46:15.447 INFO:teuthology.orchestra.run.smithi162.stdout: "session_add": 5, 2024-04-20T10:46:15.447 INFO:teuthology.orchestra.run.smithi162.stdout: "session_remove": 3, 2024-04-20T10:46:15.447 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_open": 4, 2024-04-20T10:46:15.447 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_stale": 0, 2024-04-20T10:46:15.448 INFO:teuthology.orchestra.run.smithi162.stdout: "total_load": 0, 2024-04-20T10:46:15.448 INFO:teuthology.orchestra.run.smithi162.stdout: "average_load": 0, 2024-04-20T10:46:15.448 INFO:teuthology.orchestra.run.smithi162.stdout: "avg_session_uptime": 5085, 2024-04-20T10:46:15.448 INFO:teuthology.orchestra.run.smithi162.stdout: "mdthresh_evicted": 0 2024-04-20T10:46:15.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.448 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2024-04-20T10:46:15.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2024-04-20T10:46:15.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2024-04-20T10:46:15.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T10:46:15.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2024-04-20T10:46:15.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T10:46:15.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2024-04-20T10:46:15.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T10:46:15.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T10:46:15.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T10:46:15.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T10:46:15.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T10:46:15.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2024-04-20T10:46:15.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T10:46:15.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T10:46:15.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2024-04-20T10:46:15.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2024-04-20T10:46:15.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2024-04-20T10:46:15.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2024-04-20T10:46:15.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T10:46:15.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T10:46:15.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T10:46:15.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T10:46:15.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T10:46:15.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2024-04-20T10:46:15.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2024-04-20T10:46:15.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2024-04-20T10:46:15.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T10:46:15.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T10:46:15.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2024-04-20T10:46:15.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2024-04-20T10:46:15.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2024-04-20T10:46:15.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2024-04-20T10:46:15.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T10:46:15.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2024-04-20T10:46:15.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T10:46:15.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2024-04-20T10:46:15.452 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 5339410, 2024-04-20T10:46:15.453 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 137916, 2024-04-20T10:46:15.453 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2024-04-20T10:46:15.453 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2024-04-20T10:46:15.453 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2024-04-20T10:46:15.453 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2024-04-20T10:46:15.453 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2024-04-20T10:46:15.453 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2024-04-20T10:46:15.453 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2024-04-20T10:46:15.453 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2024-04-20T10:46:15.454 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 8344, 2024-04-20T10:46:15.454 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 174, 2024-04-20T10:46:15.454 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T10:46:15.454 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2024-04-20T10:46:15.454 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2024-04-20T10:46:15.454 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2024-04-20T10:46:15.454 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 386188359, 2024-04-20T10:46:15.454 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 4330068, 2024-04-20T10:46:15.455 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2024-04-20T10:46:15.455 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2024-04-20T10:46:15.455 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2024-04-20T10:46:15.455 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2024-04-20T10:46:15.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.455 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter": { 2024-04-20T10:46:15.455 INFO:teuthology.orchestra.run.smithi162.stdout: "op_active": 0, 2024-04-20T10:46:15.455 INFO:teuthology.orchestra.run.smithi162.stdout: "op_laggy": 0, 2024-04-20T10:46:15.455 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send": 3905, 2024-04-20T10:46:15.456 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send_bytes": 26, 2024-04-20T10:46:15.456 INFO:teuthology.orchestra.run.smithi162.stdout: "op_resend": 4, 2024-04-20T10:46:15.456 INFO:teuthology.orchestra.run.smithi162.stdout: "op_reply": 3905, 2024-04-20T10:46:15.456 INFO:teuthology.orchestra.run.smithi162.stdout: "op_latency": { 2024-04-20T10:46:15.456 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 3905, 2024-04-20T10:46:15.456 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 7.453850592, 2024-04-20T10:46:15.456 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.001908796 2024-04-20T10:46:15.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.457 INFO:teuthology.orchestra.run.smithi162.stdout: "op_inflight": 0, 2024-04-20T10:46:15.457 INFO:teuthology.orchestra.run.smithi162.stdout: "oplen_avg": { 2024-04-20T10:46:15.457 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 3905, 2024-04-20T10:46:15.457 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 3906 2024-04-20T10:46:15.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.457 INFO:teuthology.orchestra.run.smithi162.stdout: "op": 3905, 2024-04-20T10:46:15.457 INFO:teuthology.orchestra.run.smithi162.stdout: "op_r": 3905, 2024-04-20T10:46:15.457 INFO:teuthology.orchestra.run.smithi162.stdout: "op_w": 0, 2024-04-20T10:46:15.457 INFO:teuthology.orchestra.run.smithi162.stdout: "op_rmw": 0, 2024-04-20T10:46:15.458 INFO:teuthology.orchestra.run.smithi162.stdout: "op_pg": 0, 2024-04-20T10:46:15.458 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_stat": 2426, 2024-04-20T10:46:15.458 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_create": 0, 2024-04-20T10:46:15.458 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_read": 1476, 2024-04-20T10:46:15.458 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_write": 0, 2024-04-20T10:46:15.458 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writefull": 0, 2024-04-20T10:46:15.458 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writesame": 0, 2024-04-20T10:46:15.458 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_append": 0, 2024-04-20T10:46:15.459 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_zero": 0, 2024-04-20T10:46:15.459 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_truncate": 0, 2024-04-20T10:46:15.459 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_delete": 0, 2024-04-20T10:46:15.459 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_mapext": 0, 2024-04-20T10:46:15.459 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_sparse_read": 0, 2024-04-20T10:46:15.459 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_clonerange": 0, 2024-04-20T10:46:15.459 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_getxattr": 2, 2024-04-20T10:46:15.459 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_setxattr": 0, 2024-04-20T10:46:15.459 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_cmpxattr": 0, 2024-04-20T10:46:15.460 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_rmxattr": 0, 2024-04-20T10:46:15.460 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_resetxattrs": 0, 2024-04-20T10:46:15.460 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_call": 0, 2024-04-20T10:46:15.460 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_watch": 0, 2024-04-20T10:46:15.460 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_notify": 0, 2024-04-20T10:46:15.460 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T10:46:15.460 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls": 0, 2024-04-20T10:46:15.460 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls_filter": 0, 2024-04-20T10:46:15.461 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_other": 0, 2024-04-20T10:46:15.461 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_active": 0, 2024-04-20T10:46:15.461 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_send": 0, 2024-04-20T10:46:15.461 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_resend": 0, 2024-04-20T10:46:15.461 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_ping": 0, 2024-04-20T10:46:15.461 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_active": 0, 2024-04-20T10:46:15.461 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_send": 0, 2024-04-20T10:46:15.461 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_resend": 0, 2024-04-20T10:46:15.461 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_active": 0, 2024-04-20T10:46:15.462 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_send": 0, 2024-04-20T10:46:15.462 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_resend": 0, 2024-04-20T10:46:15.462 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_active": 0, 2024-04-20T10:46:15.462 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_send": 0, 2024-04-20T10:46:15.462 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_resend": 0, 2024-04-20T10:46:15.462 INFO:teuthology.orchestra.run.smithi162.stdout: "command_active": 0, 2024-04-20T10:46:15.462 INFO:teuthology.orchestra.run.smithi162.stdout: "command_send": 0, 2024-04-20T10:46:15.462 INFO:teuthology.orchestra.run.smithi162.stdout: "command_resend": 0, 2024-04-20T10:46:15.463 INFO:teuthology.orchestra.run.smithi162.stdout: "map_epoch": 99, 2024-04-20T10:46:15.463 INFO:teuthology.orchestra.run.smithi162.stdout: "map_full": 0, 2024-04-20T10:46:15.463 INFO:teuthology.orchestra.run.smithi162.stdout: "map_inc": 14, 2024-04-20T10:46:15.463 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_sessions": 11, 2024-04-20T10:46:15.463 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_open": 11, 2024-04-20T10:46:15.463 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_close": 0, 2024-04-20T10:46:15.463 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_laggy": 0, 2024-04-20T10:46:15.463 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_wr": 0, 2024-04-20T10:46:15.463 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_rd": 2, 2024-04-20T10:46:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_del": 0 2024-04-20T10:46:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "oft": { 2024-04-20T10:46:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_objs": 0, 2024-04-20T10:46:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_kv_pairs": 0, 2024-04-20T10:46:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_updates": 0, 2024-04-20T10:46:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_removes": 0 2024-04-20T10:46:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "osd-slow-ops": { 2024-04-20T10:46:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_ops_count": 0 2024-04-20T10:46:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "purge_queue": { 2024-04-20T10:46:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops": 0, 2024-04-20T10:46:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops_high_water": 0, 2024-04-20T10:46:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing": 0, 2024-04-20T10:46:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_high_water": 0, 2024-04-20T10:46:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executed_ops": 0, 2024-04-20T10:46:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executed": 0, 2024-04-20T10:46:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_item_in_journal": 0 2024-04-20T10:46:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T10:46:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-04-20T10:46:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-04-20T10:46:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-04-20T10:46:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 4180, 2024-04-20T10:46:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 687712061, 2024-04-20T10:46:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-04-20T10:46:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 4180, 2024-04-20T10:46:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-04-20T10:46:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-04-20T10:46:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 4180, 2024-04-20T10:46:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 687712061, 2024-04-20T10:46:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-04-20T10:46:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_bytes": { 2024-04-20T10:46:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-04-20T10:46:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-04-20T10:46:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-04-20T10:46:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-04-20T10:46:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-04-20T10:46:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-04-20T10:46:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-04-20T10:46:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 3905, 2024-04-20T10:46:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 685273400, 2024-04-20T10:46:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 3905, 2024-04-20T10:46:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 685273400, 2024-04-20T10:46:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-04-20T10:46:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_ops": { 2024-04-20T10:46:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-04-20T10:46:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 1024, 2024-04-20T10:46:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-04-20T10:46:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-04-20T10:46:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-04-20T10:46:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-04-20T10:46:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-04-20T10:46:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 3905, 2024-04-20T10:46:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 3905, 2024-04-20T10:46:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 3905, 2024-04-20T10:46:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 3905, 2024-04-20T10:46:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-04-20T10:46:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle": { 2024-04-20T10:46:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-04-20T10:46:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2024-04-20T10:46:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-04-20T10:46:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-04-20T10:46:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-04-20T10:46:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-04-20T10:46:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-04-20T10:46:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-04-20T10:46:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-04-20T10:46:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 0, 2024-04-20T10:46:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 0, 2024-04-20T10:46:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-04-20T10:46:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle-0x56386ce966e0": { 2024-04-20T10:46:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-04-20T10:46:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2024-04-20T10:46:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-04-20T10:46:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-04-20T10:46:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-04-20T10:46:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-04-20T10:46:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-04-20T10:46:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-04-20T10:46:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-04-20T10:46:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 0, 2024-04-20T10:46:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 0, 2024-04-20T10:46:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-04-20T10:46:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:15.477 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-20T10:46:16.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:15 smithi046 ceph-mon[21574]: pgmap v381: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:16.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:15 smithi042 ceph-mon[17078]: pgmap v381: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:15 smithi162 ceph-mon[21854]: pgmap v381: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:16.264 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-20T10:46:16.264 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-20T10:46:16.264 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T10:46:16.264 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-20T10:46:16.264 DEBUG:tasks.check_counter:Getting stats from l 2024-04-20T10:46:16.264 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 83e8f766-feff-11ee-bc92-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-20T10:46:16.502 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/mon.c/config 2024-04-20T10:46:18.041 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:17 smithi162 ceph-mon[21854]: pgmap v382: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:18.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:17 smithi046 ceph-mon[21574]: pgmap v382: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:18.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:17 smithi042 ceph-mon[17078]: pgmap v382: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:18.312 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-20T10:46:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T10:46:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 0, 2024-04-20T10:46:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 2, 2024-04-20T10:46:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 0, 2024-04-20T10:46:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 0, 2024-04-20T10:46:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 4, 2024-04-20T10:46:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 0, 2024-04-20T10:46:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.001029270, 2024-04-20T10:46:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T10:46:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.000230273, 2024-04-20T10:46:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:46:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T10:46:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T10:46:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T10:46:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 263, 2024-04-20T10:46:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 249, 2024-04-20T10:46:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 1741904, 2024-04-20T10:46:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 153008, 2024-04-20T10:46:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 2, 2024-04-20T10:46:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 1, 2024-04-20T10:46:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.050118001, 2024-04-20T10:46:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.014538200, 2024-04-20T10:46:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.039097941, 2024-04-20T10:46:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.006138485, 2024-04-20T10:46:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 249, 2024-04-20T10:46:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.012875277, 2024-04-20T10:46:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000051707 2024-04-20T10:46:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 1741904, 2024-04-20T10:46:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 153008 2024-04-20T10:46:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T10:46:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 2, 2024-04-20T10:46:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 189, 2024-04-20T10:46:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 320, 2024-04-20T10:46:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 189376, 2024-04-20T10:46:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 5, 2024-04-20T10:46:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 3, 2024-04-20T10:46:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.011506825, 2024-04-20T10:46:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.008020509, 2024-04-20T10:46:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.001206418, 2024-04-20T10:46:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T10:46:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T10:46:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 186, 2024-04-20T10:46:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.019828865, 2024-04-20T10:46:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000106606 2024-04-20T10:46:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-04-20T10:46:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T10:46:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 189376 2024-04-20T10:46:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2024-04-20T10:46:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 0, 2024-04-20T10:46:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2024-04-20T10:46:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2024-04-20T10:46:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2024-04-20T10:46:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2024-04-20T10:46:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T10:46:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2024-04-20T10:46:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T10:46:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2024-04-20T10:46:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T10:46:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T10:46:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T10:46:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T10:46:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T10:46:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2024-04-20T10:46:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T10:46:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T10:46:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2024-04-20T10:46:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2024-04-20T10:46:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2024-04-20T10:46:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2024-04-20T10:46:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T10:46:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T10:46:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T10:46:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T10:46:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T10:46:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2024-04-20T10:46:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2024-04-20T10:46:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2024-04-20T10:46:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T10:46:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T10:46:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2024-04-20T10:46:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2024-04-20T10:46:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2024-04-20T10:46:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2024-04-20T10:46:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T10:46:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2024-04-20T10:46:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T10:46:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2024-04-20T10:46:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 148732, 2024-04-20T10:46:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 31, 2024-04-20T10:46:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2024-04-20T10:46:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2024-04-20T10:46:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2024-04-20T10:46:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2024-04-20T10:46:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2024-04-20T10:46:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2024-04-20T10:46:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2024-04-20T10:46:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2024-04-20T10:46:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 0, 2024-04-20T10:46:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 0, 2024-04-20T10:46:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T10:46:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2024-04-20T10:46:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2024-04-20T10:46:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2024-04-20T10:46:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 928, 2024-04-20T10:46:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 2, 2024-04-20T10:46:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2024-04-20T10:46:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2024-04-20T10:46:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2024-04-20T10:46:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2024-04-20T10:46:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-20T10:46:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T10:46:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-04-20T10:46:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-04-20T10:46:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-04-20T10:46:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 261, 2024-04-20T10:46:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 1702381, 2024-04-20T10:46:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-04-20T10:46:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 261, 2024-04-20T10:46:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-04-20T10:46:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-04-20T10:46:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 261, 2024-04-20T10:46:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 1702381, 2024-04-20T10:46:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-04-20T10:46:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-04-20T10:46:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-04-20T10:46:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-04-20T10:46:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-20T10:46:18.332 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-20T10:46:19.062 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-20T10:46:19.062 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-20T10:46:19.062 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-20T10:46:19.062 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-20T10:46:19.063 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-20T10:46:19.075 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-20T10:46:19.075 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:46:19.076 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:46:19.133 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-20T10:46:19.134 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:46:19.180 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:46:19.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.447+0000 7f99ee800640 1 -- 172.21.15.42:0/3753492502 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e8106130 msgr2=0x7f99e81065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:19.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.447+0000 7f99ee800640 1 --2- 172.21.15.42:0/3753492502 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e8106130 0x7f99e81065c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009ca0 tx=0x7f99e002c1d0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.447+0000 7f99ee800640 1 -- 172.21.15.42:0/3753492502 shutdown_connections 2024-04-20T10:46:19.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.447+0000 7f99ee800640 1 --2- 172.21.15.42:0/3753492502 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e8106130 0x7f99e81065c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.447+0000 7f99ee800640 1 --2- 172.21.15.42:0/3753492502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99e8101650 0x7f99e8105b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.447+0000 7f99ee800640 1 --2- 172.21.15.42:0/3753492502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e8100d30 0x7f99e8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.447+0000 7f99ee800640 1 -- 172.21.15.42:0/3753492502 >> 172.21.15.42:0/3753492502 conn(0x7f99e80fc780 msgr2=0x7f99e80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:19.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.447+0000 7f99ee800640 1 -- 172.21.15.42:0/3753492502 shutdown_connections 2024-04-20T10:46:19.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.447+0000 7f99ee800640 1 -- 172.21.15.42:0/3753492502 wait complete. 2024-04-20T10:46:19.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.448+0000 7f99ee800640 1 Processor -- start 2024-04-20T10:46:19.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.448+0000 7f99ee800640 1 -- start start 2024-04-20T10:46:19.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.448+0000 7f99ee800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e8100d30 0x7f99e81a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:19.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ee800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99e8101650 0x7f99e81a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:19.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ee800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e8106130 0x7f99e819ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:19.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ee800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8118750 con 0x7f99e8101650 2024-04-20T10:46:19.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ee800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99e81185d0 con 0x7f99e8106130 2024-04-20T10:46:19.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ee800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f99e81188d0 con 0x7f99e8100d30 2024-04-20T10:46:19.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ed400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e8100d30 0x7f99e81a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:19.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99eca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99e8101650 0x7f99e81a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:19.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ed400640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e8100d30 0x7f99e81a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:44682/0 (socket says 172.21.15.42:44682) 2024-04-20T10:46:19.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ed400640 1 -- 172.21.15.42:0/1591774213 learned_addr learned my addr 172.21.15.42:0/1591774213 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:46:19.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ede00640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e8106130 0x7f99e819ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:19.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ed400640 1 -- 172.21.15.42:0/1591774213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e8106130 msgr2=0x7f99e819ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:19.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ed400640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e8106130 0x7f99e819ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ed400640 1 -- 172.21.15.42:0/1591774213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99e8101650 msgr2=0x7f99e81a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:19.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ed400640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99e8101650 0x7f99e81a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99ed400640 1 -- 172.21.15.42:0/1591774213 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e819f210 con 0x7f99e8100d30 2024-04-20T10:46:19.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.449+0000 7f99eca00640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99e8101650 0x7f99e81a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:46:19.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.450+0000 7f99ed400640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e8100d30 0x7f99e81a48e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00e9f0 tx=0x7f99dc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:19.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.450+0000 7f99e6000640 1 -- 172.21.15.42:0/1591774213 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc00cd80 con 0x7f99e8100d30 2024-04-20T10:46:19.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.450+0000 7f99e6000640 1 -- 172.21.15.42:0/1591774213 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99dc005500 con 0x7f99e8100d30 2024-04-20T10:46:19.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.450+0000 7f99e6000640 1 -- 172.21.15.42:0/1591774213 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc010690 con 0x7f99e8100d30 2024-04-20T10:46:19.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.450+0000 7f99ee800640 1 -- 172.21.15.42:0/1591774213 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e819f500 con 0x7f99e8100d30 2024-04-20T10:46:19.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.450+0000 7f99ee800640 1 -- 172.21.15.42:0/1591774213 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e81ab7a0 con 0x7f99e8100d30 2024-04-20T10:46:19.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.451+0000 7f99ee800640 1 -- 172.21.15.42:0/1591774213 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e80690f0 con 0x7f99e8100d30 2024-04-20T10:46:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.453+0000 7f99e6000640 1 -- 172.21.15.42:0/1591774213 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f99dc0046c0 con 0x7f99e8100d30 2024-04-20T10:46:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.455+0000 7f99e6000640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f99d00a3d20 0x7f99d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.455+0000 7f99e6000640 1 -- 172.21.15.42:0/1591774213 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f99dc0d0c10 con 0x7f99e8100d30 2024-04-20T10:46:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.455+0000 7f99eca00640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f99d00a3d20 0x7f99d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.456+0000 7f99eca00640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f99d00a3d20 0x7f99d00a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f99e80698a0 tx=0x7f99d800b360 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:19.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.457+0000 7f99e6000640 1 -- 172.21.15.42:0/1591774213 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f99dc09d220 con 0x7f99e8100d30 2024-04-20T10:46:19.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.669+0000 7f99ee800640 1 -- 172.21.15.42:0/1591774213 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99e819ffc0 con 0x7f99e8100d30 2024-04-20T10:46:19.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.671+0000 7f99e6000640 1 -- 172.21.15.42:0/1591774213 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23191 (secure 0 0 0) 0x7f99dc09cb20 con 0x7f99e8100d30 2024-04-20T10:46:19.673 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:46:19.673 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":100,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:46:19.066036+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/230562472":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/1321607472":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/3804122415":"2024-04-21T10:34:00.299359+0000","172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/3610568749":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/869471821":"2024-04-21T10:34:00.299359+0000","172.21.15.42:6800/3610568749":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:46:19.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.674+0000 7f99ee800640 1 -- 172.21.15.42:0/1591774213 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f99d00a3d20 msgr2=0x7f99d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:19.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.674+0000 7f99ee800640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f99d00a3d20 0x7f99d00a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f99e80698a0 tx=0x7f99d800b360 comp rx=0 tx=0).stop 2024-04-20T10:46:19.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.674+0000 7f99ee800640 1 -- 172.21.15.42:0/1591774213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e8100d30 msgr2=0x7f99e81a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:19.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.674+0000 7f99ee800640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e8100d30 0x7f99e81a48e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00e9f0 tx=0x7f99dc00edf0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.674+0000 7f99ee800640 1 -- 172.21.15.42:0/1591774213 shutdown_connections 2024-04-20T10:46:19.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.674+0000 7f99ee800640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f99d00a3d20 0x7f99d00a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.674+0000 7f99ee800640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99e8106130 0x7f99e819ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.674+0000 7f99ee800640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99e8101650 0x7f99e81a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.674+0000 7f99ee800640 1 --2- 172.21.15.42:0/1591774213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e8100d30 0x7f99e81a48e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.674+0000 7f99ee800640 1 -- 172.21.15.42:0/1591774213 >> 172.21.15.42:0/1591774213 conn(0x7f99e80fc780 msgr2=0x7f99e8103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:19.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.675+0000 7f99ee800640 1 -- 172.21.15.42:0/1591774213 shutdown_connections 2024-04-20T10:46:19.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.675+0000 7f99ee800640 1 -- 172.21.15.42:0/1591774213 wait complete. 2024-04-20T10:46:19.698 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:26:16.537072+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:31:33.523692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:31:34.019519+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:46:19.698 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:46:19.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.911+0000 7fc449c00640 1 -- 172.21.15.42:0/262116847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc44410b060 msgr2=0x7fc44410d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:19.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.911+0000 7fc449c00640 1 --2- 172.21.15.42:0/262116847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc44410b060 0x7fc44410d450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009be0 tx=0x7fc43c02bef0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.911+0000 7fc449c00640 1 -- 172.21.15.42:0/262116847 shutdown_connections 2024-04-20T10:46:19.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.911+0000 7fc449c00640 1 --2- 172.21.15.42:0/262116847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc44410b060 0x7fc44410d450 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.911+0000 7fc449c00640 1 --2- 172.21.15.42:0/262116847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444100cf0 0x7fc444105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.911+0000 7fc449c00640 1 --2- 172.21.15.42:0/262116847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4440684f0 0x7fc444104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.911+0000 7fc449c00640 1 -- 172.21.15.42:0/262116847 >> 172.21.15.42:0/262116847 conn(0x7fc444076fb0 msgr2=0x7fc4440fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:19.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.911+0000 7fc449c00640 1 -- 172.21.15.42:0/262116847 shutdown_connections 2024-04-20T10:46:19.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.911+0000 7fc449c00640 1 -- 172.21.15.42:0/262116847 wait complete. 2024-04-20T10:46:19.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.912+0000 7fc449c00640 1 Processor -- start 2024-04-20T10:46:19.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.912+0000 7fc449c00640 1 -- start start 2024-04-20T10:46:19.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc449c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4440684f0 0x7fc4441a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:19.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc449c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc444100cf0 0x7fc4441a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:19.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc449c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc44410b060 0x7fc444109d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:19.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc449c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc444112360 con 0x7fc444100cf0 2024-04-20T10:46:19.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc449c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441121e0 con 0x7fc4440684f0 2024-04-20T10:46:19.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc449c00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441124e0 con 0x7fc44410b060 2024-04-20T10:46:19.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc43a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc444100cf0 0x7fc4441a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:19.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc43a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc444100cf0 0x7fc4441a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41928/0 (socket says 172.21.15.42:41928) 2024-04-20T10:46:19.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc43a000640 1 -- 172.21.15.42:0/3881173055 learned_addr learned my addr 172.21.15.42:0/3881173055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:46:19.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc443400640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc44410b060 0x7fc444109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:19.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc442a00640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4440684f0 0x7fc4441a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:19.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc43a000640 1 -- 172.21.15.42:0/3881173055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc44410b060 msgr2=0x7fc444109d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:19.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.913+0000 7fc43a000640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc44410b060 0x7fc444109d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.914+0000 7fc43a000640 1 -- 172.21.15.42:0/3881173055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4440684f0 msgr2=0x7fc4441a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:19.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.914+0000 7fc43a000640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4440684f0 0x7fc4441a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:19.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.914+0000 7fc43a000640 1 -- 172.21.15.42:0/3881173055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44410a600 con 0x7fc444100cf0 2024-04-20T10:46:19.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.914+0000 7fc43a000640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc444100cf0 0x7fc4441a0d60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00e9f0 tx=0x7fc42c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:19.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.914+0000 7fc43be00640 1 -- 172.21.15.42:0/3881173055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00ccc0 con 0x7fc444100cf0 2024-04-20T10:46:19.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.914+0000 7fc43be00640 1 -- 172.21.15.42:0/3881173055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc42c00ce60 con 0x7fc444100cf0 2024-04-20T10:46:19.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.914+0000 7fc449c00640 1 -- 172.21.15.42:0/3881173055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44410a8f0 con 0x7fc444100cf0 2024-04-20T10:46:19.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.914+0000 7fc43be00640 1 -- 172.21.15.42:0/3881173055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c010690 con 0x7fc444100cf0 2024-04-20T10:46:19.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.914+0000 7fc449c00640 1 -- 172.21.15.42:0/3881173055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4441a7770 con 0x7fc444100cf0 2024-04-20T10:46:19.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.916+0000 7fc449c00640 1 -- 172.21.15.42:0/3881173055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4441018a0 con 0x7fc444100cf0 2024-04-20T10:46:19.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.918+0000 7fc43be00640 1 -- 172.21.15.42:0/3881173055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc42c0046c0 con 0x7fc444100cf0 2024-04-20T10:46:19.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.919+0000 7fc43be00640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc4280a3d20 0x7fc4280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:19.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.919+0000 7fc43be00640 1 -- 172.21.15.42:0/3881173055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fc42c0d0bf0 con 0x7fc444100cf0 2024-04-20T10:46:19.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.919+0000 7fc442a00640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc4280a3d20 0x7fc4280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:19.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.920+0000 7fc442a00640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc4280a3d20 0x7fc4280a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc434005fd0 tx=0x7fc434009260 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:19.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:19.924+0000 7fc43be00640 1 -- 172.21.15.42:0/3881173055 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc42c09d200 con 0x7fc444100cf0 2024-04-20T10:46:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:19 smithi046 ceph-mon[21574]: pgmap v383: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:20.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:19 smithi046 ceph-mon[21574]: osdmap e100: 12 total, 12 up, 12 in 2024-04-20T10:46:20.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:19 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1591774213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:46:20.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:19 smithi042 ceph-mon[17078]: pgmap v383: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:20.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:19 smithi042 ceph-mon[17078]: osdmap e100: 12 total, 12 up, 12 in 2024-04-20T10:46:20.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:19 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1591774213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:46:20.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.131+0000 7fc449c00640 1 -- 172.21.15.42:0/3881173055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc4441a7940 con 0x7fc444100cf0 2024-04-20T10:46:20.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.132+0000 7fc43be00640 1 -- 172.21.15.42:0/3881173055 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fc42c09cb00 con 0x7fc444100cf0 2024-04-20T10:46:20.133 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-20T10:46:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.135+0000 7fc449c00640 1 -- 172.21.15.42:0/3881173055 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc4280a3d20 msgr2=0x7fc4280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.135+0000 7fc449c00640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc4280a3d20 0x7fc4280a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc434005fd0 tx=0x7fc434009260 comp rx=0 tx=0).stop 2024-04-20T10:46:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.135+0000 7fc449c00640 1 -- 172.21.15.42:0/3881173055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc444100cf0 msgr2=0x7fc4441a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.135+0000 7fc449c00640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc444100cf0 0x7fc4441a0d60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00e9f0 tx=0x7fc42c00edf0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.135+0000 7fc449c00640 1 -- 172.21.15.42:0/3881173055 shutdown_connections 2024-04-20T10:46:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.135+0000 7fc449c00640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc4280a3d20 0x7fc4280a61e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.135+0000 7fc449c00640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc44410b060 0x7fc444109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.135+0000 7fc449c00640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc444100cf0 0x7fc4441a0d60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.135+0000 7fc449c00640 1 --2- 172.21.15.42:0/3881173055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4440684f0 0x7fc4441a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.135+0000 7fc449c00640 1 -- 172.21.15.42:0/3881173055 >> 172.21.15.42:0/3881173055 conn(0x7fc444076fb0 msgr2=0x7fc44410c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.135+0000 7fc449c00640 1 -- 172.21.15.42:0/3881173055 shutdown_connections 2024-04-20T10:46:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.135+0000 7fc449c00640 1 -- 172.21.15.42:0/3881173055 wait complete. 2024-04-20T10:46:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:19 smithi162 ceph-mon[21854]: pgmap v383: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:19 smithi162 ceph-mon[21854]: osdmap e100: 12 total, 12 up, 12 in 2024-04-20T10:46:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:19 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1591774213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:46:20.155 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:46:20.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.363+0000 7f4b61800640 1 -- 172.21.15.42:0/3097822825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c108ea0 msgr2=0x7f4b5c1152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:20.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.363+0000 7f4b61800640 1 --2- 172.21.15.42:0/3097822825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c108ea0 0x7f4b5c1152c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009900 tx=0x7f4b5402bcd0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.363+0000 7f4b61800640 1 -- 172.21.15.42:0/3097822825 shutdown_connections 2024-04-20T10:46:20.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.363+0000 7f4b61800640 1 --2- 172.21.15.42:0/3097822825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c108ea0 0x7f4b5c1152c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.363+0000 7f4b61800640 1 --2- 172.21.15.42:0/3097822825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c108500 0x7f4b5c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.363+0000 7f4b61800640 1 --2- 172.21.15.42:0/3097822825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c10d3d0 0x7f4b5c10d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.363+0000 7f4b61800640 1 -- 172.21.15.42:0/3097822825 >> 172.21.15.42:0/3097822825 conn(0x7f4b5c077d80 msgr2=0x7f4b5c106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:20.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.363+0000 7f4b61800640 1 -- 172.21.15.42:0/3097822825 shutdown_connections 2024-04-20T10:46:20.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.363+0000 7f4b61800640 1 -- 172.21.15.42:0/3097822825 wait complete. 2024-04-20T10:46:20.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.364+0000 7f4b61800640 1 Processor -- start 2024-04-20T10:46:20.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.364+0000 7f4b61800640 1 -- start start 2024-04-20T10:46:20.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b61800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c108500 0x7f4b5c1a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:20.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b61800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c108ea0 0x7f4b5c1a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:20.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b61800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c10d3d0 0x7f4b5c19ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:20.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b61800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c118840 con 0x7f4b5c108ea0 2024-04-20T10:46:20.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b61800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c1186c0 con 0x7f4b5c108500 2024-04-20T10:46:20.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b61800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c1189c0 con 0x7f4b5c10d3d0 2024-04-20T10:46:20.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b5b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c108ea0 0x7f4b5c1a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:20.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b5b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c108ea0 0x7f4b5c1a4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41942/0 (socket says 172.21.15.42:41942) 2024-04-20T10:46:20.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b5b400640 1 -- 172.21.15.42:0/1303892231 learned_addr learned my addr 172.21.15.42:0/1303892231 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:46:20.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b60e00640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c10d3d0 0x7f4b5c19ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:20.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b5be00640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c108500 0x7f4b5c1a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:20.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b5b400640 1 -- 172.21.15.42:0/1303892231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c10d3d0 msgr2=0x7f4b5c19ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:20.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b5b400640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c10d3d0 0x7f4b5c19ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b5b400640 1 -- 172.21.15.42:0/1303892231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c108500 msgr2=0x7f4b5c1a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:20.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b5b400640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c108500 0x7f4b5c1a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.365+0000 7f4b5b400640 1 -- 172.21.15.42:0/1303892231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c19f200 con 0x7f4b5c108ea0 2024-04-20T10:46:20.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.366+0000 7f4b5b400640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c108ea0 0x7f4b5c1a4e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800ebc0 tx=0x7f4b4800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:20.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.366+0000 7f4b58c00640 1 -- 172.21.15.42:0/1303892231 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48010630 con 0x7f4b5c108ea0 2024-04-20T10:46:20.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.366+0000 7f4b58c00640 1 -- 172.21.15.42:0/1303892231 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b48010cb0 con 0x7f4b5c108ea0 2024-04-20T10:46:20.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.366+0000 7f4b58c00640 1 -- 172.21.15.42:0/1303892231 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48018bd0 con 0x7f4b5c108ea0 2024-04-20T10:46:20.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.366+0000 7f4b61800640 1 -- 172.21.15.42:0/1303892231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c19f4f0 con 0x7f4b5c108ea0 2024-04-20T10:46:20.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.366+0000 7f4b61800640 1 -- 172.21.15.42:0/1303892231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c1ab790 con 0x7f4b5c108ea0 2024-04-20T10:46:20.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.367+0000 7f4b61800640 1 -- 172.21.15.42:0/1303892231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5c0690f0 con 0x7f4b5c108ea0 2024-04-20T10:46:20.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.370+0000 7f4b58c00640 1 -- 172.21.15.42:0/1303892231 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4b480107d0 con 0x7f4b5c108ea0 2024-04-20T10:46:20.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.372+0000 7f4b58c00640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4b400a3d20 0x7f4b400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.372+0000 7f4b58c00640 1 -- 172.21.15.42:0/1303892231 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f4b480d1060 con 0x7f4b5c108ea0 2024-04-20T10:46:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.372+0000 7f4b5be00640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4b400a3d20 0x7f4b400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:20.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.373+0000 7f4b5be00640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4b400a3d20 0x7f4b400a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c19fbc0 tx=0x7f4b4c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:20.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.374+0000 7f4b58c00640 1 -- 172.21.15.42:0/1303892231 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4b4809d020 con 0x7f4b5c108ea0 2024-04-20T10:46:20.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.583+0000 7f4b61800640 1 -- 172.21.15.42:0/1303892231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4b5c1aba20 con 0x7f4b5c108ea0 2024-04-20T10:46:20.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.584+0000 7f4b58c00640 1 -- 172.21.15.42:0/1303892231 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f4b480a1320 con 0x7f4b5c108ea0 2024-04-20T10:46:20.585 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-20T10:46:20.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.587+0000 7f4b61800640 1 -- 172.21.15.42:0/1303892231 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4b400a3d20 msgr2=0x7f4b400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:20.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.587+0000 7f4b61800640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4b400a3d20 0x7f4b400a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c19fbc0 tx=0x7f4b4c005f90 comp rx=0 tx=0).stop 2024-04-20T10:46:20.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.587+0000 7f4b61800640 1 -- 172.21.15.42:0/1303892231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c108ea0 msgr2=0x7f4b5c1a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:20.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.587+0000 7f4b61800640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c108ea0 0x7f4b5c1a4e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800ebc0 tx=0x7f4b4800c6a0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.587+0000 7f4b61800640 1 -- 172.21.15.42:0/1303892231 shutdown_connections 2024-04-20T10:46:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.587+0000 7f4b61800640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f4b400a3d20 0x7f4b400a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.587+0000 7f4b61800640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b5c10d3d0 0x7f4b5c19ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.587+0000 7f4b61800640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b5c108ea0 0x7f4b5c1a4e10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.587+0000 7f4b61800640 1 --2- 172.21.15.42:0/1303892231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b5c108500 0x7f4b5c1a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.587+0000 7f4b61800640 1 -- 172.21.15.42:0/1303892231 >> 172.21.15.42:0/1303892231 conn(0x7f4b5c077d80 msgr2=0x7f4b5c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.587+0000 7f4b61800640 1 -- 172.21.15.42:0/1303892231 shutdown_connections 2024-04-20T10:46:20.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.587+0000 7f4b61800640 1 -- 172.21.15.42:0/1303892231 wait complete. 2024-04-20T10:46:20.609 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:46:20.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.830+0000 7f8466600640 1 -- 172.21.15.42:0/2084501536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8460109250 msgr2=0x7f8460110e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:20.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.830+0000 7f8466600640 1 --2- 172.21.15.42:0/2084501536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8460109250 0x7f8460110e20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8458009ca0 tx=0x7f845802c100 comp rx=0 tx=0).stop 2024-04-20T10:46:20.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.830+0000 7f8466600640 1 -- 172.21.15.42:0/2084501536 shutdown_connections 2024-04-20T10:46:20.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.831+0000 7f8466600640 1 --2- 172.21.15.42:0/2084501536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8460109250 0x7f8460110e20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.831+0000 7f8466600640 1 --2- 172.21.15.42:0/2084501536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8460104b30 0x7f8460108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.831+0000 7f8466600640 1 --2- 172.21.15.42:0/2084501536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8460104180 0x7f8460104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.831+0000 7f8466600640 1 -- 172.21.15.42:0/2084501536 >> 172.21.15.42:0/2084501536 conn(0x7f84600ff9c0 msgr2=0x7f8460101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.831+0000 7f8466600640 1 -- 172.21.15.42:0/2084501536 shutdown_connections 2024-04-20T10:46:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.831+0000 7f8466600640 1 -- 172.21.15.42:0/2084501536 wait complete. 2024-04-20T10:46:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.832+0000 7f8466600640 1 Processor -- start 2024-04-20T10:46:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.832+0000 7f8466600640 1 -- start start 2024-04-20T10:46:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.832+0000 7f8466600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8460104180 0x7f84601a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8466600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8460109250 0x7f84601a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8466600640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f846019a690 0x7f846019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8466600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8460114310 con 0x7f8460109250 2024-04-20T10:46:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8466600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8460114190 con 0x7f8460104180 2024-04-20T10:46:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8466600640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8460114490 con 0x7f846019a690 2024-04-20T10:46:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8465200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8460104180 0x7f84601a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8457e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8460109250 0x7f84601a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8465200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8460104180 0x7f84601a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:34890/0 (socket says 172.21.15.42:34890) 2024-04-20T10:46:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8465200640 1 -- 172.21.15.42:0/2607283541 learned_addr learned my addr 172.21.15.42:0/2607283541 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:46:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8465c00640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f846019a690 0x7f846019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8465200640 1 -- 172.21.15.42:0/2607283541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f846019a690 msgr2=0x7f846019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8465200640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f846019a690 0x7f846019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8465200640 1 -- 172.21.15.42:0/2607283541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8460109250 msgr2=0x7f84601a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8465200640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8460109250 0x7f84601a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8465200640 1 -- 172.21.15.42:0/2607283541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846019b290 con 0x7f8460104180 2024-04-20T10:46:20.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.833+0000 7f8457e00640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8460109250 0x7f84601a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:46:20.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.834+0000 7f8465200640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8460104180 0x7f84601a0580 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f845000e9f0 tx=0x7f845000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:20.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.834+0000 7f845e000640 1 -- 172.21.15.42:0/2607283541 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845000cd80 con 0x7f8460104180 2024-04-20T10:46:20.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.834+0000 7f845e000640 1 -- 172.21.15.42:0/2607283541 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8450005500 con 0x7f8460104180 2024-04-20T10:46:20.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.834+0000 7f8466600640 1 -- 172.21.15.42:0/2607283541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84601a7320 con 0x7f8460104180 2024-04-20T10:46:20.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.834+0000 7f845e000640 1 -- 172.21.15.42:0/2607283541 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450010690 con 0x7f8460104180 2024-04-20T10:46:20.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.834+0000 7f8466600640 1 -- 172.21.15.42:0/2607283541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84601a7720 con 0x7f8460104180 2024-04-20T10:46:20.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.836+0000 7f8466600640 1 -- 172.21.15.42:0/2607283541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8460105650 con 0x7f8460104180 2024-04-20T10:46:20.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.837+0000 7f845e000640 1 -- 172.21.15.42:0/2607283541 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f84500046c0 con 0x7f8460104180 2024-04-20T10:46:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.838+0000 7f845e000640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f84440a3da0 0x7f84440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.839+0000 7f845e000640 1 -- 172.21.15.42:0/2607283541 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f84500d0bc0 con 0x7f8460104180 2024-04-20T10:46:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.839+0000 7f8457e00640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f84440a3da0 0x7f84440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.839+0000 7f8457e00640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f84440a3da0 0x7f84440a6260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f846019be90 tx=0x7f8448005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:20.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:20.843+0000 7f845e000640 1 -- 172.21.15.42:0/2607283541 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f845009d230 con 0x7f8460104180 2024-04-20T10:46:21.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.050+0000 7f8466600640 1 -- 172.21.15.42:0/2607283541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f846010c4e0 con 0x7f8460104180 2024-04-20T10:46:21.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.050+0000 7f845e000640 1 -- 172.21.15.42:0/2607283541 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f845009cb30 con 0x7f8460104180 2024-04-20T10:46:21.051 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-20T10:46:21.052 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:20 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3881173055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:46:21.052 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:20 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1303892231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:46:21.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:20 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3881173055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:46:21.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:20 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1303892231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:46:21.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.053+0000 7f8466600640 1 -- 172.21.15.42:0/2607283541 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f84440a3da0 msgr2=0x7f84440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:21.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.053+0000 7f8466600640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f84440a3da0 0x7f84440a6260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f846019be90 tx=0x7f8448005f90 comp rx=0 tx=0).stop 2024-04-20T10:46:21.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.053+0000 7f8466600640 1 -- 172.21.15.42:0/2607283541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8460104180 msgr2=0x7f84601a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:21.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.053+0000 7f8466600640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8460104180 0x7f84601a0580 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f845000e9f0 tx=0x7f845000edf0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.054+0000 7f8466600640 1 -- 172.21.15.42:0/2607283541 shutdown_connections 2024-04-20T10:46:21.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.054+0000 7f8466600640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f84440a3da0 0x7f84440a6260 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.054+0000 7f8466600640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f846019a690 0x7f846019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.054+0000 7f8466600640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8460109250 0x7f84601a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.054+0000 7f8466600640 1 --2- 172.21.15.42:0/2607283541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8460104180 0x7f84601a0580 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.054+0000 7f8466600640 1 -- 172.21.15.42:0/2607283541 >> 172.21.15.42:0/2607283541 conn(0x7f84600ff9c0 msgr2=0x7f8460101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:21.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.054+0000 7f8466600640 1 -- 172.21.15.42:0/2607283541 shutdown_connections 2024-04-20T10:46:21.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.054+0000 7f8466600640 1 -- 172.21.15.42:0/2607283541 wait complete. 2024-04-20T10:46:21.072 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:20 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3881173055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T10:46:21.072 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:20 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1303892231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T10:46:21.076 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:46:21.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.291+0000 7f2255200640 1 -- 172.21.15.42:0/261392514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2250102fc0 msgr2=0x7f22501033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:21.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.291+0000 7f2255200640 1 --2- 172.21.15.42:0/261392514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2250102fc0 0x7f22501033a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2238009900 tx=0x7f223802bcd0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.291+0000 7f2255200640 1 -- 172.21.15.42:0/261392514 shutdown_connections 2024-04-20T10:46:21.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.291+0000 7f2255200640 1 --2- 172.21.15.42:0/261392514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f225010c3e0 0x7f225010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.291+0000 7f2255200640 1 --2- 172.21.15.42:0/261392514 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22501038e0 0x7f225010bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.291+0000 7f2255200640 1 --2- 172.21.15.42:0/261392514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2250102fc0 0x7f22501033a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.291+0000 7f2255200640 1 -- 172.21.15.42:0/261392514 >> 172.21.15.42:0/261392514 conn(0x7f22500fc7e0 msgr2=0x7f22500fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:21.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.291+0000 7f2255200640 1 -- 172.21.15.42:0/261392514 shutdown_connections 2024-04-20T10:46:21.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.291+0000 7f2255200640 1 -- 172.21.15.42:0/261392514 wait complete. 2024-04-20T10:46:21.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.292+0000 7f2255200640 1 Processor -- start 2024-04-20T10:46:21.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.292+0000 7f2255200640 1 -- start start 2024-04-20T10:46:21.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f2255200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2250102fc0 0x7f22501a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:21.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f2255200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22501038e0 0x7f22501a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:21.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f2255200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225010c3e0 0x7f225019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f2255200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22501121a0 con 0x7f2250102fc0 2024-04-20T10:46:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f2255200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2250112020 con 0x7f22501038e0 2024-04-20T10:46:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f2255200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2250112320 con 0x7f225010c3e0 2024-04-20T10:46:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f224f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2250102fc0 0x7f22501a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f224f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2250102fc0 0x7f22501a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41982/0 (socket says 172.21.15.42:41982) 2024-04-20T10:46:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f224f400640 1 -- 172.21.15.42:0/535000646 learned_addr learned my addr 172.21.15.42:0/535000646 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:46:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f224ea00640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22501038e0 0x7f22501a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f224fe00640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225010c3e0 0x7f225019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f224f400640 1 -- 172.21.15.42:0/535000646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225010c3e0 msgr2=0x7f225019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.293+0000 7f224f400640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225010c3e0 0x7f225019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.294+0000 7f224f400640 1 -- 172.21.15.42:0/535000646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22501038e0 msgr2=0x7f22501a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.294+0000 7f224f400640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22501038e0 0x7f22501a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.294+0000 7f224f400640 1 -- 172.21.15.42:0/535000646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225019afb0 con 0x7f2250102fc0 2024-04-20T10:46:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.294+0000 7f224ea00640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22501038e0 0x7f22501a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:46:21.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.294+0000 7f224f400640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2250102fc0 0x7f22501a0620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2238005940 tx=0x7f223802cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:21.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.294+0000 7f2247e00640 1 -- 172.21.15.42:0/535000646 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238041070 con 0x7f2250102fc0 2024-04-20T10:46:21.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.294+0000 7f2247e00640 1 -- 172.21.15.42:0/535000646 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2238004450 con 0x7f2250102fc0 2024-04-20T10:46:21.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.294+0000 7f2255200640 1 -- 172.21.15.42:0/535000646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225019b240 con 0x7f2250102fc0 2024-04-20T10:46:21.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.294+0000 7f2247e00640 1 -- 172.21.15.42:0/535000646 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238003d80 con 0x7f2250102fc0 2024-04-20T10:46:21.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.294+0000 7f2255200640 1 -- 172.21.15.42:0/535000646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22501a7470 con 0x7f2250102fc0 2024-04-20T10:46:21.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.296+0000 7f2255200640 1 -- 172.21.15.42:0/535000646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22500699d0 con 0x7f2250102fc0 2024-04-20T10:46:21.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.298+0000 7f2247e00640 1 -- 172.21.15.42:0/535000646 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2238044070 con 0x7f2250102fc0 2024-04-20T10:46:21.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.299+0000 7f2247e00640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f22340a3d20 0x7f22340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:21.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.299+0000 7f2247e00640 1 -- 172.21.15.42:0/535000646 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f22380f22c0 con 0x7f2250102fc0 2024-04-20T10:46:21.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.300+0000 7f224ea00640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f22340a3d20 0x7f22340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:21.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.300+0000 7f224ea00640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f22340a3d20 0x7f22340a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f22500fed60 tx=0x7f22400095d0 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:21.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.304+0000 7f2247e00640 1 -- 172.21.15.42:0/535000646 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f22380be9e0 con 0x7f2250102fc0 2024-04-20T10:46:21.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.528+0000 7f2255200640 1 -- 172.21.15.42:0/535000646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f22501a7640 con 0x7f2250102fc0 2024-04-20T10:46:21.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.529+0000 7f2247e00640 1 -- 172.21.15.42:0/535000646 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+944 (secure 0 0 0) 0x7f22380be2e0 con 0x7f2250102fc0 2024-04-20T10:46:21.530 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/230562472 2024-04-21T10:34:00.299359+0000 2024-04-20T10:46:21.530 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1321607472 2024-04-21T10:34:00.299359+0000 2024-04-20T10:46:21.531 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3804122415 2024-04-21T10:34:00.299359+0000 2024-04-20T10:46:21.531 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2743527860 2024-04-21T10:23:30.366298+0000 2024-04-20T10:46:21.531 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2743527860 2024-04-21T10:23:30.366298+0000 2024-04-20T10:46:21.531 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3840708146 2024-04-21T10:23:30.366298+0000 2024-04-20T10:46:21.531 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1852528586 2024-04-21T10:23:30.366298+0000 2024-04-20T10:46:21.531 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2022912579 2024-04-21T10:22:55.593824+0000 2024-04-20T10:46:21.531 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/3610568749 2024-04-21T10:34:00.299359+0000 2024-04-20T10:46:21.531 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2862444953 2024-04-21T10:22:55.593824+0000 2024-04-20T10:46:21.531 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2556893432 2024-04-21T10:22:55.593824+0000 2024-04-20T10:46:21.532 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/869471821 2024-04-21T10:34:00.299359+0000 2024-04-20T10:46:21.532 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/3610568749 2024-04-21T10:34:00.299359+0000 2024-04-20T10:46:21.532 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2734400038 2024-04-21T10:23:30.366298+0000 2024-04-20T10:46:21.532 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3551554049 2024-04-21T10:22:55.593824+0000 2024-04-20T10:46:21.532 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2556893432 2024-04-21T10:22:55.593824+0000 2024-04-20T10:46:21.533 INFO:teuthology.orchestra.run.smithi042.stderr:listed 16 entries 2024-04-20T10:46:21.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.532+0000 7f2255200640 1 -- 172.21.15.42:0/535000646 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f22340a3d20 msgr2=0x7f22340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:21.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.532+0000 7f2255200640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f22340a3d20 0x7f22340a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f22500fed60 tx=0x7f22400095d0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.532+0000 7f2255200640 1 -- 172.21.15.42:0/535000646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2250102fc0 msgr2=0x7f22501a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:21.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.532+0000 7f2255200640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2250102fc0 0x7f22501a0620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2238005940 tx=0x7f223802cc40 comp rx=0 tx=0).stop 2024-04-20T10:46:21.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.532+0000 7f2255200640 1 -- 172.21.15.42:0/535000646 shutdown_connections 2024-04-20T10:46:21.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.532+0000 7f2255200640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f22340a3d20 0x7f22340a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.532+0000 7f2255200640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225010c3e0 0x7f225019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.532+0000 7f2255200640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22501038e0 0x7f22501a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.532+0000 7f2255200640 1 --2- 172.21.15.42:0/535000646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2250102fc0 0x7f22501a0620 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:21.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.532+0000 7f2255200640 1 -- 172.21.15.42:0/535000646 >> 172.21.15.42:0/535000646 conn(0x7f22500fc7e0 msgr2=0x7f2250101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:21.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.532+0000 7f2255200640 1 -- 172.21.15.42:0/535000646 shutdown_connections 2024-04-20T10:46:21.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:21.533+0000 7f2255200640 1 -- 172.21.15.42:0/535000646 wait complete. 2024-04-20T10:46:21.554 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-20T10:46:21.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:46:21.554 DEBUG:teuthology.orchestra.run.smithi042:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-20T10:46:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:21 smithi046 ceph-mon[21574]: pgmap v385: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:22.053 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:21 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2607283541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:46:22.054 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:21 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/535000646' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T10:46:22.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:21 smithi042 ceph-mon[17078]: pgmap v385: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:22.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:21 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2607283541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:46:22.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:21 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/535000646' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T10:46:22.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:21 smithi162 ceph-mon[21854]: pgmap v385: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 4 op/s 2024-04-20T10:46:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:21 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2607283541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T10:46:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:21 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/535000646' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T10:46:24.103 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:23 smithi042 ceph-mon[17078]: pgmap v386: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-04-20T10:46:24.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:23 smithi162 ceph-mon[21854]: pgmap v386: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-04-20T10:46:24.303 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:23 smithi046 ceph-mon[21574]: pgmap v386: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-04-20T10:46:24.875 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-04-20T10:46:24.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:46:24.875 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T10:46:24.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:46:24.927 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-04-20T10:46:24.969 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T10:46:24.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:46:24.970 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:46:25.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:46:25.106 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-04-20T10:46:25.130 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:25 smithi042 ceph-mon[17078]: pgmap v387: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-04-20T10:46:25.224 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-20T10:46:25.237 INFO:tasks.cephadm:Teardown begin 2024-04-20T10:46:25.237 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T10:46:25.238 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:46:25.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:25 smithi162 ceph-mon[21854]: pgmap v387: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-04-20T10:46:25.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.460+0000 7f617b200640 1 -- 172.21.15.42:0/504889057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f617c10e470 msgr2=0x7f617c10e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:25.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.460+0000 7f617b200640 1 --2- 172.21.15.42:0/504889057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f617c10e470 0x7f617c10e850 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f616c009900 tx=0x7f616c02bcd0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.460+0000 7f617b200640 1 -- 172.21.15.42:0/504889057 shutdown_connections 2024-04-20T10:46:25.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.460+0000 7f617b200640 1 --2- 172.21.15.42:0/504889057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f617c108da0 0x7f617c115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.460+0000 7f617b200640 1 --2- 172.21.15.42:0/504889057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f617c108400 0x7f617c108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.460+0000 7f617b200640 1 --2- 172.21.15.42:0/504889057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f617c10e470 0x7f617c10e850 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.460+0000 7f617b200640 1 -- 172.21.15.42:0/504889057 >> 172.21.15.42:0/504889057 conn(0x7f617c077ae0 msgr2=0x7f617c079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:25.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.460+0000 7f617b200640 1 -- 172.21.15.42:0/504889057 shutdown_connections 2024-04-20T10:46:25.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.460+0000 7f617b200640 1 -- 172.21.15.42:0/504889057 wait complete. 2024-04-20T10:46:25.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.461+0000 7f617b200640 1 Processor -- start 2024-04-20T10:46:25.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.461+0000 7f617b200640 1 -- start start 2024-04-20T10:46:25.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f617b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f617c108400 0x7f617c1a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:25.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f617b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f617c108da0 0x7f617c1a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:25.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f617b200640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f617c10e470 0x7f617c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:25.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f617b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f617c118750 con 0x7f617c108da0 2024-04-20T10:46:25.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f617b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f617c1185d0 con 0x7f617c108400 2024-04-20T10:46:25.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f617b200640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f617c1188d0 con 0x7f617c10e470 2024-04-20T10:46:25.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f6179400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f617c108da0 0x7f617c1a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:25.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f6179e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f617c108400 0x7f617c1a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:25.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f6179400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f617c108da0 0x7f617c1a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42000/0 (socket says 172.21.15.42:42000) 2024-04-20T10:46:25.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f6179400640 1 -- 172.21.15.42:0/555842184 learned_addr learned my addr 172.21.15.42:0/555842184 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:46:25.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f617a800640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f617c10e470 0x7f617c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:25.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f6179400640 1 -- 172.21.15.42:0/555842184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f617c10e470 msgr2=0x7f617c19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:25.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f6179400640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f617c10e470 0x7f617c19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f6179400640 1 -- 172.21.15.42:0/555842184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f617c108400 msgr2=0x7f617c1a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:25.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.462+0000 7f6179400640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f617c108400 0x7f617c1a4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.463+0000 7f6179400640 1 -- 172.21.15.42:0/555842184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c19f170 con 0x7f617c108da0 2024-04-20T10:46:25.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.463+0000 7f6179400640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f617c108da0 0x7f617c1a4d80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f616800e9f0 tx=0x7f616800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:25.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.463+0000 7f6172a00640 1 -- 172.21.15.42:0/555842184 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616800cd80 con 0x7f617c108da0 2024-04-20T10:46:25.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.463+0000 7f617b200640 1 -- 172.21.15.42:0/555842184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617c19f460 con 0x7f617c108da0 2024-04-20T10:46:25.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.463+0000 7f6172a00640 1 -- 172.21.15.42:0/555842184 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6168005500 con 0x7f617c108da0 2024-04-20T10:46:25.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.463+0000 7f6172a00640 1 -- 172.21.15.42:0/555842184 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168018630 con 0x7f617c108da0 2024-04-20T10:46:25.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.463+0000 7f617b200640 1 -- 172.21.15.42:0/555842184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617c1ab700 con 0x7f617c108da0 2024-04-20T10:46:25.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.465+0000 7f617b200640 1 -- 172.21.15.42:0/555842184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f617c0690f0 con 0x7f617c108da0 2024-04-20T10:46:25.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.468+0000 7f6172a00640 1 -- 172.21.15.42:0/555842184 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f61680046c0 con 0x7f617c108da0 2024-04-20T10:46:25.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.469+0000 7f6172a00640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f61600a3d20 0x7f61600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:25.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.469+0000 7f6172a00640 1 -- 172.21.15.42:0/555842184 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f61680d0dd0 con 0x7f617c108da0 2024-04-20T10:46:25.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.469+0000 7f6179e00640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f61600a3d20 0x7f61600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:25.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.470+0000 7f6179e00640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f61600a3d20 0x7f61600a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f616c02efd0 tx=0x7f616c02cf90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:25.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.474+0000 7f6172a00640 1 -- 172.21.15.42:0/555842184 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f616809d3e0 con 0x7f617c108da0 2024-04-20T10:46:25.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:25 smithi046 ceph-mon[21574]: pgmap v387: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-04-20T10:46:25.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.733+0000 7f617b200640 1 -- 172.21.15.42:0/555842184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f617c19ff20 con 0x7f617c108da0 2024-04-20T10:46:25.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.735+0000 7f6172a00640 1 -- 172.21.15.42:0/555842184 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f616809cce0 con 0x7f617c108da0 2024-04-20T10:46:25.736 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:46:25.736 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:46:25.736 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:46:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.738+0000 7f617b200640 1 -- 172.21.15.42:0/555842184 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f61600a3d20 msgr2=0x7f61600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.738+0000 7f617b200640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f61600a3d20 0x7f61600a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f616c02efd0 tx=0x7f616c02cf90 comp rx=0 tx=0).stop 2024-04-20T10:46:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.738+0000 7f617b200640 1 -- 172.21.15.42:0/555842184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f617c108da0 msgr2=0x7f617c1a4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.738+0000 7f617b200640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f617c108da0 0x7f617c1a4d80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f616800e9f0 tx=0x7f616800edf0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.738+0000 7f617b200640 1 -- 172.21.15.42:0/555842184 shutdown_connections 2024-04-20T10:46:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.738+0000 7f617b200640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f61600a3d20 0x7f61600a61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.738+0000 7f617b200640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f617c10e470 0x7f617c19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.738+0000 7f617b200640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f617c108da0 0x7f617c1a4d80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.738+0000 7f617b200640 1 --2- 172.21.15.42:0/555842184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f617c108400 0x7f617c1a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.738+0000 7f617b200640 1 -- 172.21.15.42:0/555842184 >> 172.21.15.42:0/555842184 conn(0x7f617c077ae0 msgr2=0x7f617c079190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:25.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.738+0000 7f617b200640 1 -- 172.21.15.42:0/555842184 shutdown_connections 2024-04-20T10:46:25.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.738+0000 7f617b200640 1 -- 172.21.15.42:0/555842184 wait complete. 2024-04-20T10:46:25.761 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:46:25.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.974+0000 7f2640800640 1 -- 172.21.15.42:0/278951644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26381043a0 msgr2=0x7f2638104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:25.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.974+0000 7f2640800640 1 --2- 172.21.15.42:0/278951644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26381043a0 0x7f2638104780 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f262c009900 tx=0x7f262c02bc30 comp rx=0 tx=0).stop 2024-04-20T10:46:25.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.975+0000 7f2640800640 1 -- 172.21.15.42:0/278951644 shutdown_connections 2024-04-20T10:46:25.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.975+0000 7f2640800640 1 --2- 172.21.15.42:0/278951644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2638109470 0x7f2638111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.975+0000 7f2640800640 1 --2- 172.21.15.42:0/278951644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2638104d50 0x7f2638108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.975+0000 7f2640800640 1 --2- 172.21.15.42:0/278951644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26381043a0 0x7f2638104780 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.975+0000 7f2640800640 1 -- 172.21.15.42:0/278951644 >> 172.21.15.42:0/278951644 conn(0x7f26380ffc60 msgr2=0x7f2638102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:25.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.975+0000 7f2640800640 1 -- 172.21.15.42:0/278951644 shutdown_connections 2024-04-20T10:46:25.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.975+0000 7f2640800640 1 -- 172.21.15.42:0/278951644 wait complete. 2024-04-20T10:46:25.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.976+0000 7f2640800640 1 Processor -- start 2024-04-20T10:46:25.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.976+0000 7f2640800640 1 -- start start 2024-04-20T10:46:25.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f2640800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26381043a0 0x7f26381a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:25.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f2640800640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2638109470 0x7f26381a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:25.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f2640800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f263819a8f0 0x7f263819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:25.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f2640800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2638114570 con 0x7f26381043a0 2024-04-20T10:46:25.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f2640800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26381143f0 con 0x7f263819a8f0 2024-04-20T10:46:25.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f2640800640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f26381146f0 con 0x7f2638109470 2024-04-20T10:46:25.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f263de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26381043a0 0x7f26381a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:25.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f263de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26381043a0 0x7f26381a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42028/0 (socket says 172.21.15.42:42028) 2024-04-20T10:46:25.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f263de00640 1 -- 172.21.15.42:0/2604244127 learned_addr learned my addr 172.21.15.42:0/2604244127 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:46:25.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f263e800640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f263819a8f0 0x7f263819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:25.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f263d400640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2638109470 0x7f26381a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:25.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f263de00640 1 -- 172.21.15.42:0/2604244127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2638109470 msgr2=0x7f26381a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:25.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f263de00640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2638109470 0x7f26381a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.977+0000 7f263de00640 1 -- 172.21.15.42:0/2604244127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f263819a8f0 msgr2=0x7f263819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:25.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.978+0000 7f263de00640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f263819a8f0 0x7f263819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:25.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.978+0000 7f263de00640 1 -- 172.21.15.42:0/2604244127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263819b410 con 0x7f26381043a0 2024-04-20T10:46:25.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.978+0000 7f263de00640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26381043a0 0x7f26381a06f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f262c02c240 tx=0x7f262c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:25.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.978+0000 7f2632a00640 1 -- 172.21.15.42:0/2604244127 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c043020 con 0x7f26381043a0 2024-04-20T10:46:25.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.978+0000 7f2632a00640 1 -- 172.21.15.42:0/2604244127 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262c0054e0 con 0x7f26381043a0 2024-04-20T10:46:25.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.978+0000 7f2640800640 1 -- 172.21.15.42:0/2604244127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263819b6a0 con 0x7f26381043a0 2024-04-20T10:46:25.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.978+0000 7f2632a00640 1 -- 172.21.15.42:0/2604244127 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c003b30 con 0x7f26381043a0 2024-04-20T10:46:25.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.979+0000 7f2640800640 1 -- 172.21.15.42:0/2604244127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26381a78a0 con 0x7f26381043a0 2024-04-20T10:46:25.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.980+0000 7f2640800640 1 -- 172.21.15.42:0/2604244127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26380691d0 con 0x7f26381043a0 2024-04-20T10:46:25.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.982+0000 7f2632a00640 1 -- 172.21.15.42:0/2604244127 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f262c046020 con 0x7f26381043a0 2024-04-20T10:46:25.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.984+0000 7f2632a00640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f26200a3d20 0x7f26200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:25.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.984+0000 7f2632a00640 1 -- 172.21.15.42:0/2604244127 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f262c0f2770 con 0x7f26381043a0 2024-04-20T10:46:25.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.985+0000 7f263d400640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f26200a3d20 0x7f26200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:25.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.985+0000 7f263d400640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f26200a3d20 0x7f26200a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26381033a0 tx=0x7f2628005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:25.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:25.989+0000 7f2632a00640 1 -- 172.21.15.42:0/2604244127 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f262c04d020 con 0x7f26381043a0 2024-04-20T10:46:26.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.244+0000 7f2640800640 1 -- 172.21.15.42:0/2604244127 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2638104d50 con 0x7f26381043a0 2024-04-20T10:46:26.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.245+0000 7f2632a00640 1 -- 172.21.15.42:0/2604244127 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11828 (secure 0 0 0) 0x7f262c0bee90 con 0x7f26381043a0 2024-04-20T10:46:26.246 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:46:26.247 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/2523412756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":2523412756},{"type":"v1","addr":"172.21.15.42:6839","nonce":2523412756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/3615312070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":3615312070},{"type":"v1","addr":"172.21.15.42:6841","nonce":3615312070}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1854750771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1854750771},{"type":"v1","addr":"172.21.15.46:6833","nonce":1854750771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24463,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/4094679184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":4094679184},{"type":"v1","addr":"172.21.15.162:6833","nonce":4094679184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2947757102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2947757102},{"type":"v1","addr":"172.21.15.42:6835","nonce":2947757102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4194525549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4194525549},{"type":"v1","addr":"172.21.15.46:6835","nonce":4194525549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/2898732601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":2898732601},{"type":"v1","addr":"172.21.15.42:6837","nonce":2898732601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3787981189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3787981189},{"type":"v1","addr":"172.21.15.46:6837","nonce":3787981189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2486205910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2486205910},{"type":"v1","addr":"172.21.15.162:6835","nonce":2486205910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/2056054864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":2056054864},{"type":"v1","addr":"172.21.15.162:6837","nonce":2056054864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T10:31:35.067527+0000","modified":"2024-04-20T10:31:39.263696+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24584},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24568":{"gid":24568,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/3710674499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":3710674499},{"type":"v1","addr":"172.21.15.162:6839","nonce":3710674499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24584":{"gid":24584,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6839/2026912868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":2026912868},{"type":"v1","addr":"172.21.15.46:6839","nonce":2026912868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24584,"qdb_cluster":[24584]},"id":1}]} 2024-04-20T10:46:26.247 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-04-20T10:46:26.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.248+0000 7f2640800640 1 -- 172.21.15.42:0/2604244127 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f26200a3d20 msgr2=0x7f26200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:26.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.248+0000 7f2640800640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f26200a3d20 0x7f26200a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26381033a0 tx=0x7f2628005f90 comp rx=0 tx=0).stop 2024-04-20T10:46:26.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.248+0000 7f2640800640 1 -- 172.21.15.42:0/2604244127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26381043a0 msgr2=0x7f26381a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.248+0000 7f2640800640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26381043a0 0x7f26381a06f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f262c02c240 tx=0x7f262c0042f0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.248+0000 7f2640800640 1 -- 172.21.15.42:0/2604244127 shutdown_connections 2024-04-20T10:46:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.248+0000 7f2640800640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f26200a3d20 0x7f26200a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.248+0000 7f2640800640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f263819a8f0 0x7f263819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.248+0000 7f2640800640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2638109470 0x7f26381a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.248+0000 7f2640800640 1 --2- 172.21.15.42:0/2604244127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26381043a0 0x7f26381a06f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.248+0000 7f2640800640 1 -- 172.21.15.42:0/2604244127 >> 172.21.15.42:0/2604244127 conn(0x7f26380ffc60 msgr2=0x7f2638101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.249+0000 7f2640800640 1 -- 172.21.15.42:0/2604244127 shutdown_connections 2024-04-20T10:46:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.249+0000 7f2640800640 1 -- 172.21.15.42:0/2604244127 wait complete. 2024-04-20T10:46:26.269 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:46:26.347 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:26 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/555842184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:46:26.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:26 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/555842184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:46:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.487+0000 7f0eabe00640 1 -- 172.21.15.42:0/2721171161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac1047f0 msgr2=0x7f0eac104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.487+0000 7f0eabe00640 1 --2- 172.21.15.42:0/2721171161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac1047f0 0x7f0eac104ca0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009ca0 tx=0x7f0ea402c100 comp rx=0 tx=0).stop 2024-04-20T10:46:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.487+0000 7f0eabe00640 1 -- 172.21.15.42:0/2721171161 shutdown_connections 2024-04-20T10:46:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.487+0000 7f0eabe00640 1 --2- 172.21.15.42:0/2721171161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac1047f0 0x7f0eac104ca0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.487+0000 7f0eabe00640 1 --2- 172.21.15.42:0/2721171161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac107110 0x7f0eac069630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.487+0000 7f0eabe00640 1 --2- 172.21.15.42:0/2721171161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac106ca0 0x7f0eac0690f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.487+0000 7f0eabe00640 1 -- 172.21.15.42:0/2721171161 >> 172.21.15.42:0/2721171161 conn(0x7f0eac0767a0 msgr2=0x7f0eac076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.487+0000 7f0eabe00640 1 -- 172.21.15.42:0/2721171161 shutdown_connections 2024-04-20T10:46:26.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.487+0000 7f0eabe00640 1 -- 172.21.15.42:0/2721171161 wait complete. 2024-04-20T10:46:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.488+0000 7f0eabe00640 1 Processor -- start 2024-04-20T10:46:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.489+0000 7f0eabe00640 1 -- start start 2024-04-20T10:46:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.489+0000 7f0eabe00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac1047f0 0x7f0eac1a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.489+0000 7f0eabe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac106ca0 0x7f0eac1a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.489+0000 7f0eabe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac107110 0x7f0eac19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.489+0000 7f0eabe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac118680 con 0x7f0eac107110 2024-04-20T10:46:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.489+0000 7f0eabe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac118500 con 0x7f0eac106ca0 2024-04-20T10:46:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.489+0000 7f0eabe00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac118800 con 0x7f0eac1047f0 2024-04-20T10:46:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.489+0000 7f0eab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac107110 0x7f0eac19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.489+0000 7f0eab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac107110 0x7f0eac19e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42050/0 (socket says 172.21.15.42:42050) 2024-04-20T10:46:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.489+0000 7f0eaaa00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac1047f0 0x7f0eac1a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.489+0000 7f0ea2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac106ca0 0x7f0eac1a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.489+0000 7f0eab400640 1 -- 172.21.15.42:0/1119177868 learned_addr learned my addr 172.21.15.42:0/1119177868 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:46:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.490+0000 7f0eab400640 1 -- 172.21.15.42:0/1119177868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac1047f0 msgr2=0x7f0eac1a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.490+0000 7f0eab400640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac1047f0 0x7f0eac1a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.490+0000 7f0eab400640 1 -- 172.21.15.42:0/1119177868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac106ca0 msgr2=0x7f0eac1a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.490+0000 7f0eab400640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac106ca0 0x7f0eac1a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.490+0000 7f0eab400640 1 -- 172.21.15.42:0/1119177868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac19f120 con 0x7f0eac107110 2024-04-20T10:46:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.490+0000 7f0ea2000640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac106ca0 0x7f0eac1a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:46:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.490+0000 7f0eaaa00640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac1047f0 0x7f0eac1a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:46:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.490+0000 7f0eab400640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac107110 0x7f0eac19e9a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4037040 tx=0x7f0ea4032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.490+0000 7f0ea3e00640 1 -- 172.21.15.42:0/1119177868 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4039040 con 0x7f0eac107110 2024-04-20T10:46:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.491+0000 7f0ea3e00640 1 -- 172.21.15.42:0/1119177868 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea4044070 con 0x7f0eac107110 2024-04-20T10:46:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.491+0000 7f0eabe00640 1 -- 172.21.15.42:0/1119177868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac19f410 con 0x7f0eac107110 2024-04-20T10:46:26.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.491+0000 7f0ea3e00640 1 -- 172.21.15.42:0/1119177868 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40042a0 con 0x7f0eac107110 2024-04-20T10:46:26.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.491+0000 7f0eabe00640 1 -- 172.21.15.42:0/1119177868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac1ab660 con 0x7f0eac107110 2024-04-20T10:46:26.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.492+0000 7f0eabe00640 1 -- 172.21.15.42:0/1119177868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eac068ca0 con 0x7f0eac107110 2024-04-20T10:46:26.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.494+0000 7f0ea3e00640 1 -- 172.21.15.42:0/1119177868 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0ea4003970 con 0x7f0eac107110 2024-04-20T10:46:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.495+0000 7f0ea3e00640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0e900a3d20 0x7f0e900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.495+0000 7f0ea3e00640 1 -- 172.21.15.42:0/1119177868 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f0ea40f2bb0 con 0x7f0eac107110 2024-04-20T10:46:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.496+0000 7f0eaaa00640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0e900a3d20 0x7f0e900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:26.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.496+0000 7f0eaaa00640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0e900a3d20 0x7f0e900a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0eac101920 tx=0x7f0e9c005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:26.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.500+0000 7f0ea3e00640 1 -- 172.21.15.42:0/1119177868 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0ea40bf360 con 0x7f0eac107110 2024-04-20T10:46:26.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:26 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/555842184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:46:26.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.707+0000 7f0eabe00640 1 -- 172.21.15.42:0/1119177868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0eac1ab920 con 0x7f0eac107110 2024-04-20T10:46:26.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.710+0000 7f0ea3e00640 1 -- 172.21.15.42:0/1119177868 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23191 (secure 0 0 0) 0x7f0ea40bec60 con 0x7f0eac107110 2024-04-20T10:46:26.711 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-20T10:46:26.712 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":100,"fsid":"83e8f766-feff-11ee-bc92-c7b262605968","created":"2024-04-20T10:22:21.221805+0000","modified":"2024-04-20T10:46:19.066036+0000","last_up_change":"2024-04-20T10:30:36.944268+0000","last_in_change":"2024-04-20T10:30:17.510763+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:26:16.537072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:31:33.523692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:31:34.019519+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"47367eda-0111-48e6-891e-e8c78dab4d43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6803","nonce":1817026493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6805","nonce":1817026493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6809","nonce":1817026493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":1817026493},{"type":"v1","addr":"172.21.15.42:6807","nonce":1817026493}]},"public_addr":"172.21.15.42:6803/1817026493","cluster_addr":"172.21.15.42:6805/1817026493","heartbeat_back_addr":"172.21.15.42:6809/1817026493","heartbeat_front_addr":"172.21.15.42:6807/1817026493","state":["exists","up"]},{"osd":1,"uuid":"a4764a30-900f-4c2c-a5e4-99d346d82f17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6801","nonce":1972645025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6803","nonce":1972645025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6807","nonce":1972645025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1972645025},{"type":"v1","addr":"172.21.15.46:6805","nonce":1972645025}]},"public_addr":"172.21.15.46:6801/1972645025","cluster_addr":"172.21.15.46:6803/1972645025","heartbeat_back_addr":"172.21.15.46:6807/1972645025","heartbeat_front_addr":"172.21.15.46:6805/1972645025","state":["exists","up"]},{"osd":2,"uuid":"f69aba92-96e7-45c8-a47a-114aafa47178","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6801","nonce":1877464810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6803","nonce":1877464810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6807","nonce":1877464810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1877464810},{"type":"v1","addr":"172.21.15.162:6805","nonce":1877464810}]},"public_addr":"172.21.15.162:6801/1877464810","cluster_addr":"172.21.15.162:6803/1877464810","heartbeat_back_addr":"172.21.15.162:6807/1877464810","heartbeat_front_addr":"172.21.15.162:6805/1877464810","state":["exists","up"]},{"osd":3,"uuid":"620d380c-c93c-4b80-b7c2-f6db56a31e5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6811","nonce":2352652467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6813","nonce":2352652467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6817","nonce":2352652467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2352652467},{"type":"v1","addr":"172.21.15.42:6815","nonce":2352652467}]},"public_addr":"172.21.15.42:6811/2352652467","cluster_addr":"172.21.15.42:6813/2352652467","heartbeat_back_addr":"172.21.15.42:6817/2352652467","heartbeat_front_addr":"172.21.15.42:6815/2352652467","state":["exists","up"]},{"osd":4,"uuid":"d67ed7c7-d9b2-4e22-ad08-633fa00b77f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6809","nonce":4097597758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6811","nonce":4097597758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6815","nonce":4097597758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":4097597758},{"type":"v1","addr":"172.21.15.46:6813","nonce":4097597758}]},"public_addr":"172.21.15.46:6809/4097597758","cluster_addr":"172.21.15.46:6811/4097597758","heartbeat_back_addr":"172.21.15.46:6815/4097597758","heartbeat_front_addr":"172.21.15.46:6813/4097597758","state":["exists","up"]},{"osd":5,"uuid":"fe973de7-c11c-402d-89ce-138f33c0a706","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6809","nonce":2005776096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6811","nonce":2005776096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6815","nonce":2005776096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2005776096},{"type":"v1","addr":"172.21.15.162:6813","nonce":2005776096}]},"public_addr":"172.21.15.162:6809/2005776096","cluster_addr":"172.21.15.162:6811/2005776096","heartbeat_back_addr":"172.21.15.162:6815/2005776096","heartbeat_front_addr":"172.21.15.162:6813/2005776096","state":["exists","up"]},{"osd":6,"uuid":"67420dab-1d28-4626-9f42-6774ce3b19af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6819","nonce":2702679714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6821","nonce":2702679714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6825","nonce":2702679714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2702679714},{"type":"v1","addr":"172.21.15.42:6823","nonce":2702679714}]},"public_addr":"172.21.15.42:6819/2702679714","cluster_addr":"172.21.15.42:6821/2702679714","heartbeat_back_addr":"172.21.15.42:6825/2702679714","heartbeat_front_addr":"172.21.15.42:6823/2702679714","state":["exists","up"]},{"osd":7,"uuid":"13021f92-8ad1-4ff6-b0c7-52d1a2d174a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6817","nonce":2431095643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6819","nonce":2431095643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6823","nonce":2431095643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2431095643},{"type":"v1","addr":"172.21.15.46:6821","nonce":2431095643}]},"public_addr":"172.21.15.46:6817/2431095643","cluster_addr":"172.21.15.46:6819/2431095643","heartbeat_back_addr":"172.21.15.46:6823/2431095643","heartbeat_front_addr":"172.21.15.46:6821/2431095643","state":["exists","up"]},{"osd":8,"uuid":"8185b6db-3e13-40ed-950c-b330c983331e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6817","nonce":3373880377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6819","nonce":3373880377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6823","nonce":3373880377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3373880377},{"type":"v1","addr":"172.21.15.162:6821","nonce":3373880377}]},"public_addr":"172.21.15.162:6817/3373880377","cluster_addr":"172.21.15.162:6819/3373880377","heartbeat_back_addr":"172.21.15.162:6823/3373880377","heartbeat_front_addr":"172.21.15.162:6821/3373880377","state":["exists","up"]},{"osd":9,"uuid":"1cd34bd8-f661-4903-88cb-7c44fd178f1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6827","nonce":661029076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6829","nonce":661029076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6833","nonce":661029076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":661029076},{"type":"v1","addr":"172.21.15.42:6831","nonce":661029076}]},"public_addr":"172.21.15.42:6827/661029076","cluster_addr":"172.21.15.42:6829/661029076","heartbeat_back_addr":"172.21.15.42:6833/661029076","heartbeat_front_addr":"172.21.15.42:6831/661029076","state":["exists","up"]},{"osd":10,"uuid":"6d4fba56-28b6-449d-9b0c-fc927cbe6aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6825","nonce":1339678617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6827","nonce":1339678617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6831","nonce":1339678617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1339678617},{"type":"v1","addr":"172.21.15.46:6829","nonce":1339678617}]},"public_addr":"172.21.15.46:6825/1339678617","cluster_addr":"172.21.15.46:6827/1339678617","heartbeat_back_addr":"172.21.15.46:6831/1339678617","heartbeat_front_addr":"172.21.15.46:6829/1339678617","state":["exists","up"]},{"osd":11,"uuid":"6b31e91e-bfbb-4672-b2c3-55fc49d9aae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6825","nonce":1519508339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6827","nonce":1519508339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6831","nonce":1519508339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1519508339},{"type":"v1","addr":"172.21.15.162:6829","nonce":1519508339}]},"public_addr":"172.21.15.162:6825/1519508339","cluster_addr":"172.21.15.162:6827/1519508339","heartbeat_back_addr":"172.21.15.162:6831/1519508339","heartbeat_front_addr":"172.21.15.162:6829/1519508339","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:24.490783+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:25:49.333086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:12.878049+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:26:47.243208+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:13.564464+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:38.534685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:14.260612+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:28:41.465377+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:06.894379+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:29:41.582746+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:09.834410+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:30:35.635447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/230562472":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/1321607472":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/3804122415":"2024-04-21T10:34:00.299359+0000","172.21.15.42:6801/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:6800/2743527860":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3840708146":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/1852528586":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/2022912579":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/3610568749":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/2862444953":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6800/2556893432":"2024-04-21T10:22:55.593824+0000","172.21.15.42:0/869471821":"2024-04-21T10:34:00.299359+0000","172.21.15.42:6800/3610568749":"2024-04-21T10:34:00.299359+0000","172.21.15.42:0/2734400038":"2024-04-21T10:23:30.366298+0000","172.21.15.42:0/3551554049":"2024-04-21T10:22:55.593824+0000","172.21.15.42:6801/2556893432":"2024-04-21T10:22:55.593824+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:46:26.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.712+0000 7f0eabe00640 1 -- 172.21.15.42:0/1119177868 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0e900a3d20 msgr2=0x7f0e900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:26.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.712+0000 7f0eabe00640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0e900a3d20 0x7f0e900a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0eac101920 tx=0x7f0e9c005f90 comp rx=0 tx=0).stop 2024-04-20T10:46:26.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.713+0000 7f0eabe00640 1 -- 172.21.15.42:0/1119177868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac107110 msgr2=0x7f0eac19e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:26.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.713+0000 7f0eabe00640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac107110 0x7f0eac19e9a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4037040 tx=0x7f0ea4032f30 comp rx=0 tx=0).stop 2024-04-20T10:46:26.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.713+0000 7f0eabe00640 1 -- 172.21.15.42:0/1119177868 shutdown_connections 2024-04-20T10:46:26.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.713+0000 7f0eabe00640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f0e900a3d20 0x7f0e900a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.713+0000 7f0eabe00640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac107110 0x7f0eac19e9a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.713+0000 7f0eabe00640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eac106ca0 0x7f0eac1a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.713+0000 7f0eabe00640 1 --2- 172.21.15.42:0/1119177868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0eac1047f0 0x7f0eac1a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.713+0000 7f0eabe00640 1 -- 172.21.15.42:0/1119177868 >> 172.21.15.42:0/1119177868 conn(0x7f0eac0767a0 msgr2=0x7f0eac101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:26.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.713+0000 7f0eabe00640 1 -- 172.21.15.42:0/1119177868 shutdown_connections 2024-04-20T10:46:26.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.713+0000 7f0eabe00640 1 -- 172.21.15.42:0/1119177868 wait complete. 2024-04-20T10:46:26.735 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T10:46:26.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.946+0000 7fbadb000640 1 -- 172.21.15.42:0/1330110584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad410d3d0 msgr2=0x7fbad410d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:26.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.946+0000 7fbadb000640 1 --2- 172.21.15.42:0/1330110584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad410d3d0 0x7fbad410d7b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009900 tx=0x7fbac802bcd0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.947+0000 7fbadb000640 1 -- 172.21.15.42:0/1330110584 shutdown_connections 2024-04-20T10:46:26.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.947+0000 7fbadb000640 1 --2- 172.21.15.42:0/1330110584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad4108ea0 0x7fbad41152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.947+0000 7fbadb000640 1 --2- 172.21.15.42:0/1330110584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad4108500 0x7fbad4108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.947+0000 7fbadb000640 1 --2- 172.21.15.42:0/1330110584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad410d3d0 0x7fbad410d7b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.947+0000 7fbadb000640 1 -- 172.21.15.42:0/1330110584 >> 172.21.15.42:0/1330110584 conn(0x7fbad4077d80 msgr2=0x7fbad4106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:26.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.947+0000 7fbadb000640 1 -- 172.21.15.42:0/1330110584 shutdown_connections 2024-04-20T10:46:26.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.947+0000 7fbadb000640 1 -- 172.21.15.42:0/1330110584 wait complete. 2024-04-20T10:46:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.948+0000 7fbadb000640 1 Processor -- start 2024-04-20T10:46:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.948+0000 7fbadb000640 1 -- start start 2024-04-20T10:46:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.948+0000 7fbadb000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad4108500 0x7fbad41a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.948+0000 7fbadb000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad4108ea0 0x7fbad41a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.948+0000 7fbadb000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad410d3d0 0x7fbad419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.948+0000 7fbadb000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4118840 con 0x7fbad4108ea0 2024-04-20T10:46:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.948+0000 7fbadb000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41186c0 con 0x7fbad410d3d0 2024-04-20T10:46:26.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.948+0000 7fbadb000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41189c0 con 0x7fbad4108500 2024-04-20T10:46:26.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.949+0000 7fbad9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad4108ea0 0x7fbad41a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:26.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.949+0000 7fbad9c00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad4108500 0x7fbad41a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:26.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.949+0000 7fbad9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad4108ea0 0x7fbad41a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42066/0 (socket says 172.21.15.42:42066) 2024-04-20T10:46:26.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.949+0000 7fbad9200640 1 -- 172.21.15.42:0/4214854733 learned_addr learned my addr 172.21.15.42:0/4214854733 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:46:26.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.949+0000 7fbada600640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad410d3d0 0x7fbad419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:26.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.949+0000 7fbad9c00640 1 -- 172.21.15.42:0/4214854733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad410d3d0 msgr2=0x7fbad419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:26.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.949+0000 7fbad9c00640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad410d3d0 0x7fbad419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.949+0000 7fbad9c00640 1 -- 172.21.15.42:0/4214854733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad4108ea0 msgr2=0x7fbad41a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.949+0000 7fbad9c00640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad4108ea0 0x7fbad41a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.949+0000 7fbad9c00640 1 -- 172.21.15.42:0/4214854733 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad419f2c0 con 0x7fbad4108500 2024-04-20T10:46:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.949+0000 7fbad9c00640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad4108500 0x7fbad41a4940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbac802c210 tx=0x7fbac8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.950+0000 7fbad2000640 1 -- 172.21.15.42:0/4214854733 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8042020 con 0x7fbad4108500 2024-04-20T10:46:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.950+0000 7fbadb000640 1 -- 172.21.15.42:0/4214854733 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad419f550 con 0x7fbad4108500 2024-04-20T10:46:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.950+0000 7fbad2000640 1 -- 172.21.15.42:0/4214854733 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac802ce00 con 0x7fbad4108500 2024-04-20T10:46:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.950+0000 7fbad2000640 1 -- 172.21.15.42:0/4214854733 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8003a50 con 0x7fbad4108500 2024-04-20T10:46:26.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.950+0000 7fbadb000640 1 -- 172.21.15.42:0/4214854733 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41ab800 con 0x7fbad4108500 2024-04-20T10:46:26.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.951+0000 7fbadb000640 1 -- 172.21.15.42:0/4214854733 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad40690f0 con 0x7fbad4108500 2024-04-20T10:46:26.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.953+0000 7fbad2000640 1 -- 172.21.15.42:0/4214854733 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbac8045020 con 0x7fbad4108500 2024-04-20T10:46:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.955+0000 7fbad2000640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbabc0a3d20 0x7fbabc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.955+0000 7fbad2000640 1 -- 172.21.15.42:0/4214854733 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fbac80f22c0 con 0x7fbad4108500 2024-04-20T10:46:26.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.955+0000 7fbad9200640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbabc0a3d20 0x7fbabc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:26.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.956+0000 7fbad9200640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbabc0a3d20 0x7fbabc0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbad4107f80 tx=0x7fbac4005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:26.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:26.959+0000 7fbad2000640 1 -- 172.21.15.42:0/4214854733 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbac804c020 con 0x7fbad4108500 2024-04-20T10:46:27.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.206+0000 7fbadb000640 1 -- 172.21.15.42:0/4214854733 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbad4110910 con 0x7fbad4108500 2024-04-20T10:46:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.219+0000 7fbad2000640 1 -- 172.21.15.42:0/4214854733 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) ==== 161+0+0 (secure 0 0 0) 0x7fbac80be9e0 con 0x7fbad4108500 2024-04-20T10:46:27.220 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T10:46:27.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.222+0000 7fbadb000640 1 -- 172.21.15.42:0/4214854733 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbabc0a3d20 msgr2=0x7fbabc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:27.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.222+0000 7fbadb000640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbabc0a3d20 0x7fbabc0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbad4107f80 tx=0x7fbac4005f90 comp rx=0 tx=0).stop 2024-04-20T10:46:27.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.222+0000 7fbadb000640 1 -- 172.21.15.42:0/4214854733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad4108500 msgr2=0x7fbad41a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:27.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.222+0000 7fbadb000640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad4108500 0x7fbad41a4940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbac802c210 tx=0x7fbac8005e80 comp rx=0 tx=0).stop 2024-04-20T10:46:27.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.222+0000 7fbadb000640 1 -- 172.21.15.42:0/4214854733 shutdown_connections 2024-04-20T10:46:27.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.222+0000 7fbadb000640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fbabc0a3d20 0x7fbabc0a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:27.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.222+0000 7fbadb000640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbad410d3d0 0x7fbad419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:27.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.222+0000 7fbadb000640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbad4108ea0 0x7fbad41a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:27.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.222+0000 7fbadb000640 1 --2- 172.21.15.42:0/4214854733 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbad4108500 0x7fbad41a4940 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:27.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.222+0000 7fbadb000640 1 -- 172.21.15.42:0/4214854733 >> 172.21.15.42:0/4214854733 conn(0x7fbad4077d80 msgr2=0x7fbad4078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:27.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.222+0000 7fbadb000640 1 -- 172.21.15.42:0/4214854733 shutdown_connections 2024-04-20T10:46:27.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.222+0000 7fbadb000640 1 -- 172.21.15.42:0/4214854733 wait complete. 2024-04-20T10:46:27.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T10:46:27.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:27 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/2604244127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:46:27.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:27 smithi042 ceph-mon[17078]: pgmap v388: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 297 KiB/s wr, 9 op/s 2024-04-20T10:46:27.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:27 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1119177868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:46:27.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:27 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a[17073]: 2024-04-20T10:46:27.209+0000 7f56ce600700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T10:46:27.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:27 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/2604244127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:46:27.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:27 smithi162 ceph-mon[21854]: pgmap v388: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 297 KiB/s wr, 9 op/s 2024-04-20T10:46:27.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:27 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1119177868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:46:27.400 INFO:journalctl@ceph.mds.i.smithi162.stdout:Apr 20 10:46:27 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-i[52560]: ignoring --setuser ceph since I am not root 2024-04-20T10:46:27.400 INFO:journalctl@ceph.mds.i.smithi162.stdout:Apr 20 10:46:27 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-i[52560]: ignoring --setgroup ceph since I am not root 2024-04-20T10:46:27.401 INFO:journalctl@ceph.mds.i.smithi162.stdout:Apr 20 10:46:27 smithi162 ceph-mds[52564]: -- 172.21.15.162:0/2766539074 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c3a03fa40 con 0x562c3a012c00 2024-04-20T10:46:27.401 INFO:journalctl@ceph.mds.i.smithi162.stdout:Apr 20 10:46:27 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-i[52560]: starting mds.i at 2024-04-20T10:46:27.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.441+0000 7f96eb000640 1 -- 172.21.15.42:0/284347923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96e4074b50 msgr2=0x7f96e410e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:27.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.441+0000 7f96eb000640 1 --2- 172.21.15.42:0/284347923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96e4074b50 0x7f96e410e300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009900 tx=0x7f96cc02bcd0 comp rx=0 tx=0).stop 2024-04-20T10:46:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.441+0000 7f96eb000640 1 -- 172.21.15.42:0/284347923 shutdown_connections 2024-04-20T10:46:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.441+0000 7f96eb000640 1 --2- 172.21.15.42:0/284347923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96e410e840 0x7f96e4110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.441+0000 7f96eb000640 1 --2- 172.21.15.42:0/284347923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96e4074b50 0x7f96e410e300 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.441+0000 7f96eb000640 1 --2- 172.21.15.42:0/284347923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96e4074230 0x7f96e4074610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.441+0000 7f96eb000640 1 -- 172.21.15.42:0/284347923 >> 172.21.15.42:0/284347923 conn(0x7f96e40793b0 msgr2=0x7f96e40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.441+0000 7f96eb000640 1 -- 172.21.15.42:0/284347923 shutdown_connections 2024-04-20T10:46:27.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.441+0000 7f96eb000640 1 -- 172.21.15.42:0/284347923 wait complete. 2024-04-20T10:46:27.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.442+0000 7f96eb000640 1 Processor -- start 2024-04-20T10:46:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.442+0000 7f96eb000640 1 -- start start 2024-04-20T10:46:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.443+0000 7f96eb000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96e4074230 0x7f96e41a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.443+0000 7f96eb000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96e4074b50 0x7f96e41a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.443+0000 7f96eb000640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96e410e840 0x7f96e419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.443+0000 7f96eb000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96e4114530 con 0x7f96e4074230 2024-04-20T10:46:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.443+0000 7f96eb000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41143b0 con 0x7f96e4074b50 2024-04-20T10:46:27.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.443+0000 7f96eb000640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41146b0 con 0x7f96e410e840 2024-04-20T10:46:27.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.443+0000 7f96e3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96e4074230 0x7f96e41a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:27.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.443+0000 7f96e3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96e4074230 0x7f96e41a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42094/0 (socket says 172.21.15.42:42094) 2024-04-20T10:46:27.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.443+0000 7f96e3e00640 1 -- 172.21.15.42:0/719543908 learned_addr learned my addr 172.21.15.42:0/719543908 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:46:27.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.443+0000 7f96dbe00640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96e4074b50 0x7f96e41a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:27.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.443+0000 7f96e9000640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96e410e840 0x7f96e419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:27.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.444+0000 7f96e3e00640 1 -- 172.21.15.42:0/719543908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96e410e840 msgr2=0x7f96e419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:27.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.444+0000 7f96e3e00640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96e410e840 0x7f96e419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:27.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.444+0000 7f96e3e00640 1 -- 172.21.15.42:0/719543908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96e4074b50 msgr2=0x7f96e41a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:27.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.444+0000 7f96e3e00640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96e4074b50 0x7f96e41a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:27.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.444+0000 7f96e3e00640 1 -- 172.21.15.42:0/719543908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e419b030 con 0x7f96e4074230 2024-04-20T10:46:27.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.444+0000 7f96e9000640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96e410e840 0x7f96e419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:46:27.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.444+0000 7f96e3e00640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96e4074230 0x7f96e41a0700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f96d400e9f0 tx=0x7f96d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:27.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.444+0000 7f96e1600640 1 -- 172.21.15.42:0/719543908 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d400cd80 con 0x7f96e4074230 2024-04-20T10:46:27.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.444+0000 7f96e1600640 1 -- 172.21.15.42:0/719543908 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d4010910 con 0x7f96e4074230 2024-04-20T10:46:27.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.444+0000 7f96eb000640 1 -- 172.21.15.42:0/719543908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e419b320 con 0x7f96e4074230 2024-04-20T10:46:27.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.444+0000 7f96e1600640 1 -- 172.21.15.42:0/719543908 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4018990 con 0x7f96e4074230 2024-04-20T10:46:27.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.444+0000 7f96eb000640 1 -- 172.21.15.42:0/719543908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e41a75c0 con 0x7f96e4074230 2024-04-20T10:46:27.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.445+0000 7f96eb000640 1 -- 172.21.15.42:0/719543908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e40691d0 con 0x7f96e4074230 2024-04-20T10:46:27.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.448+0000 7f96e1600640 1 -- 172.21.15.42:0/719543908 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f96d4010ac0 con 0x7f96e4074230 2024-04-20T10:46:27.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.449+0000 7f96e1600640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f96c80a3d20 0x7f96c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:27.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.449+0000 7f96e1600640 1 -- 172.21.15.42:0/719543908 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8613+0+0 (secure 0 0 0) 0x7f96d40d1050 con 0x7f96e4074230 2024-04-20T10:46:27.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.449+0000 7f96dbe00640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f96c80a3d20 0x7f96c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:27.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.450+0000 7f96dbe00640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f96c80a3d20 0x7f96c80a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f96cc02efd0 tx=0x7f96cc02cf90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:27.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.456+0000 7f96e1600640 1 -- 172.21.15.42:0/719543908 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f96d409d690 con 0x7f96e4074230 2024-04-20T10:46:27.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:27 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/2604244127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T10:46:27.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:27 smithi046 ceph-mon[21574]: pgmap v388: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 297 KiB/s wr, 9 op/s 2024-04-20T10:46:27.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:27 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1119177868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T10:46:27.554 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:46:27 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b[52186]: ignoring --setuser ceph since I am not root 2024-04-20T10:46:27.554 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:46:27 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b[52186]: ignoring --setgroup ceph since I am not root 2024-04-20T10:46:27.554 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:46:27 smithi046 ceph-mds[52190]: -- 172.21.15.46:0/1413089124 <== mon.2 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae255b3a40 con 0x55ae25586c00 2024-04-20T10:46:27.554 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:46:27 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b[52186]: starting mds.b at 2024-04-20T10:46:27.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:27.693+0000 7f96eb000640 1 -- 172.21.15.42:0/719543908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f96e402d5d0 con 0x7f96e4074230 2024-04-20T10:46:28.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.216+0000 7f96e1600640 1 -- 172.21.15.42:0/719543908 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f96d409cf90 con 0x7f96e4074230 2024-04-20T10:46:28.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.220+0000 7f96eb000640 1 -- 172.21.15.42:0/719543908 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f96c80a3d20 msgr2=0x7f96c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:28.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.220+0000 7f96eb000640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f96c80a3d20 0x7f96c80a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f96cc02efd0 tx=0x7f96cc02cf90 comp rx=0 tx=0).stop 2024-04-20T10:46:28.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.220+0000 7f96eb000640 1 -- 172.21.15.42:0/719543908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96e4074230 msgr2=0x7f96e41a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:28.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.220+0000 7f96eb000640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96e4074230 0x7f96e41a0700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f96d400e9f0 tx=0x7f96d400eeb0 comp rx=0 tx=0).stop 2024-04-20T10:46:28.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.220+0000 7f96eb000640 1 -- 172.21.15.42:0/719543908 shutdown_connections 2024-04-20T10:46:28.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.220+0000 7f96eb000640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7f96c80a3d20 0x7f96c80a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:28.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.220+0000 7f96eb000640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96e410e840 0x7f96e419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:28.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.220+0000 7f96eb000640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96e4074b50 0x7f96e41a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:28.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.220+0000 7f96eb000640 1 --2- 172.21.15.42:0/719543908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96e4074230 0x7f96e41a0700 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:28.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.220+0000 7f96eb000640 1 -- 172.21.15.42:0/719543908 >> 172.21.15.42:0/719543908 conn(0x7f96e40793b0 msgr2=0x7f96e40feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:28.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.220+0000 7f96eb000640 1 -- 172.21.15.42:0/719543908 shutdown_connections 2024-04-20T10:46:28.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.220+0000 7f96eb000640 1 -- 172.21.15.42:0/719543908 wait complete. 2024-04-20T10:46:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:28 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/4214854733' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T10:46:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:28 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T10:46:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:28 smithi042 ceph-mon[17078]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T10:46:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:28 smithi042 ceph-mon[17078]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T10:46:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:28 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T10:46:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:28 smithi042 ceph-mon[17078]: osdmap e101: 12 total, 12 up, 12 in 2024-04-20T10:46:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:28 smithi042 ceph-mon[17078]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-20T10:46:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:28 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/719543908' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T10:46:28.245 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T10:46:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:28 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/4214854733' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T10:46:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:28 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T10:46:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:28 smithi162 ceph-mon[21854]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T10:46:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:28 smithi162 ceph-mon[21854]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T10:46:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:28 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T10:46:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:28 smithi162 ceph-mon[21854]: osdmap e101: 12 total, 12 up, 12 in 2024-04-20T10:46:28.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:28 smithi162 ceph-mon[21854]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-20T10:46:28.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:28 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/719543908' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T10:46:28.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.449+0000 7fc6eac00640 1 -- 172.21.15.42:0/4153953802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6e41031c0 msgr2=0x7fc6e41035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:28.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.449+0000 7fc6eac00640 1 --2- 172.21.15.42:0/4153953802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6e41031c0 0x7fc6e41035a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4009900 tx=0x7fc6d402bcd0 comp rx=0 tx=0).stop 2024-04-20T10:46:28.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.449+0000 7fc6eac00640 1 -- 172.21.15.42:0/4153953802 shutdown_connections 2024-04-20T10:46:28.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.449+0000 7fc6eac00640 1 --2- 172.21.15.42:0/4153953802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e410c590 0x7fc6e410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:28.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.449+0000 7fc6eac00640 1 --2- 172.21.15.42:0/4153953802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e4103ae0 0x7fc6e410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:28.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.449+0000 7fc6eac00640 1 --2- 172.21.15.42:0/4153953802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6e41031c0 0x7fc6e41035a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:28.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.449+0000 7fc6eac00640 1 -- 172.21.15.42:0/4153953802 >> 172.21.15.42:0/4153953802 conn(0x7fc6e40771e0 msgr2=0x7fc6e40775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:28.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.449+0000 7fc6eac00640 1 -- 172.21.15.42:0/4153953802 shutdown_connections 2024-04-20T10:46:28.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.449+0000 7fc6eac00640 1 -- 172.21.15.42:0/4153953802 wait complete. 2024-04-20T10:46:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.450+0000 7fc6eac00640 1 Processor -- start 2024-04-20T10:46:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.450+0000 7fc6eac00640 1 -- start start 2024-04-20T10:46:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.451+0000 7fc6eac00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6e41031c0 0x7fc6e41a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.451+0000 7fc6eac00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e4103ae0 0x7fc6e41a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:28.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.451+0000 7fc6eac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e410c590 0x7fc6e419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:28.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.451+0000 7fc6eac00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e4112330 con 0x7fc6e4103ae0 2024-04-20T10:46:28.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.451+0000 7fc6eac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e41121b0 con 0x7fc6e410c590 2024-04-20T10:46:28.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.451+0000 7fc6eac00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e41124b0 con 0x7fc6e41031c0 2024-04-20T10:46:28.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.451+0000 7fc6e3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6e41031c0 0x7fc6e41a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:28.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.452+0000 7fc6e3e00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6e41031c0 0x7fc6e41a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.42:44850/0 (socket says 172.21.15.42:44850) 2024-04-20T10:46:28.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.452+0000 7fc6e3e00640 1 -- 172.21.15.42:0/1308705370 learned_addr learned my addr 172.21.15.42:0/1308705370 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:46:28.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.452+0000 7fc6e8c00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e410c590 0x7fc6e419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:28.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.452+0000 7fc6dbe00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e4103ae0 0x7fc6e41a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:28.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.452+0000 7fc6e3e00640 1 -- 172.21.15.42:0/1308705370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e410c590 msgr2=0x7fc6e419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:28.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.452+0000 7fc6e3e00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e410c590 0x7fc6e419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:28.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.452+0000 7fc6e3e00640 1 -- 172.21.15.42:0/1308705370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e4103ae0 msgr2=0x7fc6e41a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:28.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.452+0000 7fc6e3e00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e4103ae0 0x7fc6e41a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:28.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.452+0000 7fc6e3e00640 1 -- 172.21.15.42:0/1308705370 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e419b050 con 0x7fc6e41031c0 2024-04-20T10:46:28.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.452+0000 7fc6e8c00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e410c590 0x7fc6e419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:46:28.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.453+0000 7fc6e3e00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6e41031c0 0x7fc6e41a06c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4005940 tx=0x7fc6d402cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:28.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.453+0000 7fc6e1600640 1 -- 172.21.15.42:0/1308705370 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4041070 con 0x7fc6e41031c0 2024-04-20T10:46:28.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.453+0000 7fc6eac00640 1 -- 172.21.15.42:0/1308705370 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e419b2e0 con 0x7fc6e41031c0 2024-04-20T10:46:28.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.453+0000 7fc6e1600640 1 -- 172.21.15.42:0/1308705370 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d4004450 con 0x7fc6e41031c0 2024-04-20T10:46:28.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.453+0000 7fc6e1600640 1 -- 172.21.15.42:0/1308705370 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4003d80 con 0x7fc6e41031c0 2024-04-20T10:46:28.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.453+0000 7fc6eac00640 1 -- 172.21.15.42:0/1308705370 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e41a7510 con 0x7fc6e41031c0 2024-04-20T10:46:28.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.454+0000 7fc6eac00640 1 -- 172.21.15.42:0/1308705370 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e406ade0 con 0x7fc6e41031c0 2024-04-20T10:46:28.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.457+0000 7fc6e1600640 1 -- 172.21.15.42:0/1308705370 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fc6d4044070 con 0x7fc6e41031c0 2024-04-20T10:46:28.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.458+0000 7fc6e1600640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc6c80a3d20 0x7fc6c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:28.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.458+0000 7fc6e1600640 1 -- 172.21.15.42:0/1308705370 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8613+0+0 (secure 0 0 0) 0x7fc6d40f2450 con 0x7fc6e41031c0 2024-04-20T10:46:28.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.458+0000 7fc6dbe00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc6c80a3d20 0x7fc6c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:28.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.459+0000 7fc6dbe00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc6c80a3d20 0x7fc6c80a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4102e30 tx=0x7fc6cc005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:28.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.460+0000 7fc6e1600640 1 -- 172.21.15.42:0/1308705370 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc6d40be4f0 con 0x7fc6e41031c0 2024-04-20T10:46:28.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:28 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/4214854733' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T10:46:28.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:28 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T10:46:28.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:28 smithi046 ceph-mon[21574]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T10:46:28.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:28 smithi046 ceph-mon[21574]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T10:46:28.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:28 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T10:46:28.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:28 smithi046 ceph-mon[21574]: osdmap e101: 12 total, 12 up, 12 in 2024-04-20T10:46:28.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:28 smithi046 ceph-mon[21574]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-20T10:46:28.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:28 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/719543908' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T10:46:28.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:28.668+0000 7fc6eac00640 1 -- 172.21.15.42:0/1308705370 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc6e419bd60 con 0x7fc6e41031c0 2024-04-20T10:46:29.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.225+0000 7fc6e1600640 1 -- 172.21.15.42:0/1308705370 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7fc6d40c27f0 con 0x7fc6e41031c0 2024-04-20T10:46:29.227 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-04-20T10:46:29.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.232+0000 7fc6eac00640 1 -- 172.21.15.42:0/1308705370 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc6c80a3d20 msgr2=0x7fc6c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:29.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.232+0000 7fc6eac00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc6c80a3d20 0x7fc6c80a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4102e30 tx=0x7fc6cc005f90 comp rx=0 tx=0).stop 2024-04-20T10:46:29.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.232+0000 7fc6eac00640 1 -- 172.21.15.42:0/1308705370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6e41031c0 msgr2=0x7fc6e41a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:29.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.232+0000 7fc6eac00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6e41031c0 0x7fc6e41a06c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4005940 tx=0x7fc6d402cbd0 comp rx=0 tx=0).stop 2024-04-20T10:46:29.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.233+0000 7fc6eac00640 1 -- 172.21.15.42:0/1308705370 shutdown_connections 2024-04-20T10:46:29.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.233+0000 7fc6eac00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fc6c80a3d20 0x7fc6c80a61e0 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4102e30 tx=0x7fc6cc005f90 comp rx=0 tx=0).stop 2024-04-20T10:46:29.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.233+0000 7fc6eac00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e410c590 0x7fc6e419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:29.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.233+0000 7fc6eac00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6e4103ae0 0x7fc6e41a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:29.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.233+0000 7fc6eac00640 1 --2- 172.21.15.42:0/1308705370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6e41031c0 0x7fc6e41a06c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:29.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.233+0000 7fc6eac00640 1 -- 172.21.15.42:0/1308705370 >> 172.21.15.42:0/1308705370 conn(0x7fc6e40771e0 msgr2=0x7fc6e40fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:29.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.236+0000 7fc6eac00640 1 -- 172.21.15.42:0/1308705370 shutdown_connections 2024-04-20T10:46:29.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.236+0000 7fc6eac00640 1 -- 172.21.15.42:0/1308705370 wait complete. 2024-04-20T10:46:29.265 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:29 smithi042 ceph-mon[17078]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T10:46:29.266 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:29 smithi042 ceph-mon[17078]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T10:46:29.266 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:29 smithi042 ceph-mon[17078]: Cluster is now healthy 2024-04-20T10:46:29.266 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:29 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/719543908' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-20T10:46:29.266 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:29 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T10:46:29.266 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:29 smithi042 ceph-mon[17078]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T10:46:29.266 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:29 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.162:6838/2283334168,v1:172.21.15.162:6839/2283334168] up:boot 2024-04-20T10:46:29.266 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:29 smithi042 ceph-mon[17078]: mds.? [v2:172.21.15.46:6838/1161960475,v1:172.21.15.46:6839/1161960475] up:boot 2024-04-20T10:46:29.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:29 smithi042 ceph-mon[17078]: fsmap 12 up:standby 2024-04-20T10:46:29.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:29 smithi042 ceph-mon[17078]: pgmap v390: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-04-20T10:46:29.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:29 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/1308705370' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T10:46:29.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:29 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T10:46:29.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T10:46:29.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:29 smithi162 ceph-mon[21854]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T10:46:29.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:29 smithi162 ceph-mon[21854]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T10:46:29.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:29 smithi162 ceph-mon[21854]: Cluster is now healthy 2024-04-20T10:46:29.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:29 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/719543908' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-20T10:46:29.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:29 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T10:46:29.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:29 smithi162 ceph-mon[21854]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T10:46:29.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:29 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.162:6838/2283334168,v1:172.21.15.162:6839/2283334168] up:boot 2024-04-20T10:46:29.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:29 smithi162 ceph-mon[21854]: mds.? [v2:172.21.15.46:6838/1161960475,v1:172.21.15.46:6839/1161960475] up:boot 2024-04-20T10:46:29.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:29 smithi162 ceph-mon[21854]: fsmap 12 up:standby 2024-04-20T10:46:29.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:29 smithi162 ceph-mon[21854]: pgmap v390: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-04-20T10:46:29.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:29 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/1308705370' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T10:46:29.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:29 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T10:46:29.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.475+0000 7fa7cec00640 1 -- 172.21.15.42:0/1966532918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7c8116ff0 msgr2=0x7fa7c81193e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:29.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.475+0000 7fa7cec00640 1 --2- 172.21.15.42:0/1966532918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7c8116ff0 0x7fa7c81193e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009ca0 tx=0x7fa7c002c190 comp rx=0 tx=0).stop 2024-04-20T10:46:29.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.475+0000 7fa7cec00640 1 -- 172.21.15.42:0/1966532918 shutdown_connections 2024-04-20T10:46:29.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.475+0000 7fa7cec00640 1 --2- 172.21.15.42:0/1966532918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7c8116ff0 0x7fa7c81193e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:29.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.475+0000 7fa7cec00640 1 --2- 172.21.15.42:0/1966532918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c81094a0 0x7fa7c8109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:29.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.475+0000 7fa7cec00640 1 --2- 172.21.15.42:0/1966532918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c810c690 0x7fa7c8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:29.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.475+0000 7fa7cec00640 1 -- 172.21.15.42:0/1966532918 >> 172.21.15.42:0/1966532918 conn(0x7fa7c806d4d0 msgr2=0x7fa7c806d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:29.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.476+0000 7fa7cec00640 1 -- 172.21.15.42:0/1966532918 shutdown_connections 2024-04-20T10:46:29.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.476+0000 7fa7cec00640 1 -- 172.21.15.42:0/1966532918 wait complete. 2024-04-20T10:46:29.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.476+0000 7fa7cec00640 1 Processor -- start 2024-04-20T10:46:29.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.477+0000 7fa7cec00640 1 -- start start 2024-04-20T10:46:29.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.477+0000 7fa7cec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c81094a0 0x7fa7c81a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:29.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.477+0000 7fa7cec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7c810c690 0x7fa7c81a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:29.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.477+0000 7fa7cec00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c8116ff0 0x7fa7c81a7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.477+0000 7fa7cec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c811cd10 con 0x7fa7c810c690 2024-04-20T10:46:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.477+0000 7fa7cec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c811cb90 con 0x7fa7c81094a0 2024-04-20T10:46:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.477+0000 7fa7cec00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c811ce90 con 0x7fa7c8116ff0 2024-04-20T10:46:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.477+0000 7fa7c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c81094a0 0x7fa7c81a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.477+0000 7fa7c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c81094a0 0x7fa7c81a32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.42:49684/0 (socket says 172.21.15.42:49684) 2024-04-20T10:46:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.477+0000 7fa7c7e00640 1 -- 172.21.15.42:0/3438804678 learned_addr learned my addr 172.21.15.42:0/3438804678 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-20T10:46:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.477+0000 7fa7bfe00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7c810c690 0x7fa7c81a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:29.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.477+0000 7fa7ccc00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c8116ff0 0x7fa7c81a7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.478+0000 7fa7c7e00640 1 -- 172.21.15.42:0/3438804678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c8116ff0 msgr2=0x7fa7c81a7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.478+0000 7fa7c7e00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c8116ff0 0x7fa7c81a7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.478+0000 7fa7c7e00640 1 -- 172.21.15.42:0/3438804678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7c810c690 msgr2=0x7fa7c81a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.478+0000 7fa7c7e00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7c810c690 0x7fa7c81a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.478+0000 7fa7c7e00640 1 -- 172.21.15.42:0/3438804678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c81a82e0 con 0x7fa7c81094a0 2024-04-20T10:46:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.478+0000 7fa7c7e00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c81094a0 0x7fa7c81a32f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800e9f0 tx=0x7fa7b800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.478+0000 7fa7c5600640 1 -- 172.21.15.42:0/3438804678 <== mon.2 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b800cd80 con 0x7fa7c81094a0 2024-04-20T10:46:29.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.478+0000 7fa7cec00640 1 -- 172.21.15.42:0/3438804678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c81a85d0 con 0x7fa7c81094a0 2024-04-20T10:46:29.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.478+0000 7fa7c5600640 1 -- 172.21.15.42:0/3438804678 <== mon.2 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b8005500 con 0x7fa7c81094a0 2024-04-20T10:46:29.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.478+0000 7fa7c5600640 1 -- 172.21.15.42:0/3438804678 <== mon.2 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8010690 con 0x7fa7c81094a0 2024-04-20T10:46:29.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.478+0000 7fa7cec00640 1 -- 172.21.15.42:0/3438804678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c81afe60 con 0x7fa7c81094a0 2024-04-20T10:46:29.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.479+0000 7fa7cec00640 1 -- 172.21.15.42:0/3438804678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c8068d80 con 0x7fa7c81094a0 2024-04-20T10:46:29.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.481+0000 7fa7c5600640 1 -- 172.21.15.42:0/3438804678 <== mon.2 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa7b80046c0 con 0x7fa7c81094a0 2024-04-20T10:46:29.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.483+0000 7fa7c5600640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa7a80a3da0 0x7fa7a80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:46:29.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.483+0000 7fa7c5600640 1 -- 172.21.15.42:0/3438804678 <== mon.2 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8088+0+0 (secure 0 0 0) 0x7fa7b80d0990 con 0x7fa7c81094a0 2024-04-20T10:46:29.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.483+0000 7fa7bfe00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa7a80a3da0 0x7fa7a80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:46:29.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.484+0000 7fa7bfe00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa7a80a3da0 0x7fa7a80a6260 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8118f00 tx=0x7fa7b0005f90 comp rx=0 tx=0).ready entity=mgr.15024 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:46:29.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.487+0000 7fa7c5600640 1 -- 172.21.15.42:0/3438804678 <== mon.2 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa7b809d190 con 0x7fa7c81094a0 2024-04-20T10:46:29.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:29 smithi046 ceph-mon[21574]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T10:46:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:29 smithi046 ceph-mon[21574]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T10:46:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:29 smithi046 ceph-mon[21574]: Cluster is now healthy 2024-04-20T10:46:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:29 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/719543908' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-20T10:46:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:29 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T10:46:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:29 smithi046 ceph-mon[21574]: from='mgr.15024 172.21.15.42:0/1531666984' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T10:46:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:29 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.162:6838/2283334168,v1:172.21.15.162:6839/2283334168] up:boot 2024-04-20T10:46:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:29 smithi046 ceph-mon[21574]: mds.? [v2:172.21.15.46:6838/1161960475,v1:172.21.15.46:6839/1161960475] up:boot 2024-04-20T10:46:29.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:29 smithi046 ceph-mon[21574]: fsmap 12 up:standby 2024-04-20T10:46:29.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:29 smithi046 ceph-mon[21574]: pgmap v390: 97 pgs: 97 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-04-20T10:46:29.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:29 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/1308705370' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T10:46:29.555 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:29 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T10:46:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:29.686+0000 7fa7cec00640 1 -- 172.21.15.42:0/3438804678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa7c81b0090 con 0x7fa7c81094a0 2024-04-20T10:46:30.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.240+0000 7fa7c5600640 1 -- 172.21.15.42:0/3438804678 <== mon.2 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7fa7b80bd0d0 con 0x7fa7c81094a0 2024-04-20T10:46:30.241 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-04-20T10:46:30.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.248+0000 7fa7cec00640 1 -- 172.21.15.42:0/3438804678 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa7a80a3da0 msgr2=0x7fa7a80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:30.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.248+0000 7fa7cec00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa7a80a3da0 0x7fa7a80a6260 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8118f00 tx=0x7fa7b0005f90 comp rx=0 tx=0).stop 2024-04-20T10:46:30.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.248+0000 7fa7cec00640 1 -- 172.21.15.42:0/3438804678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c81094a0 msgr2=0x7fa7c81a32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:46:30.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.248+0000 7fa7cec00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c81094a0 0x7fa7c81a32f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800e9f0 tx=0x7fa7b800edf0 comp rx=0 tx=0).stop 2024-04-20T10:46:30.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.248+0000 7fa7cec00640 1 -- 172.21.15.42:0/3438804678 shutdown_connections 2024-04-20T10:46:30.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.248+0000 7fa7cec00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.42:6800/170978916,v1:172.21.15.42:6801/170978916] conn(0x7fa7a80a3da0 0x7fa7a80a6260 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:30.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.248+0000 7fa7cec00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7c8116ff0 0x7fa7c81a7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:30.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.248+0000 7fa7cec00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7c810c690 0x7fa7c81a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:30.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.248+0000 7fa7cec00640 1 --2- 172.21.15.42:0/3438804678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7c81094a0 0x7fa7c81a32f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:46:30.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.248+0000 7fa7cec00640 1 -- 172.21.15.42:0/3438804678 >> 172.21.15.42:0/3438804678 conn(0x7fa7c806d4d0 msgr2=0x7fa7c81173d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:46:30.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.249+0000 7fa7cec00640 1 -- 172.21.15.42:0/3438804678 shutdown_connections 2024-04-20T10:46:30.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-20T10:46:30.250+0000 7fa7cec00640 1 -- 172.21.15.42:0/3438804678 wait complete. 2024-04-20T10:46:30.281 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T10:46:30.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T10:46:30.412 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T10:46:30.445 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-20T10:46:30.446 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T10:46:30.495 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T10:46:30.512 DEBUG:teuthology.orchestra.run.smithi162:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T10:46:30.530 INFO:tasks.cephadm:Stopping all daemons... 2024-04-20T10:46:30.530 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-20T10:46:30.531 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.a 2024-04-20T10:46:30.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:30 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T10:46:30.553 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:30 smithi046 ceph-mon[21574]: osdmap e102: 12 total, 12 up, 12 in 2024-04-20T10:46:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:30 smithi046 ceph-mon[21574]: from='client.? 172.21.15.42:0/3438804678' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T10:46:30.554 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:30 smithi046 ceph-mon[21574]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T10:46:30.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:30 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T10:46:30.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:30 smithi042 ceph-mon[17078]: osdmap e102: 12 total, 12 up, 12 in 2024-04-20T10:46:30.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:30 smithi042 ceph-mon[17078]: from='client.? 172.21.15.42:0/3438804678' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T10:46:30.595 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:30 smithi042 ceph-mon[17078]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T10:46:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:30 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T10:46:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:30 smithi162 ceph-mon[21854]: osdmap e102: 12 total, 12 up, 12 in 2024-04-20T10:46:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:30 smithi162 ceph-mon[21854]: from='client.? 172.21.15.42:0/3438804678' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T10:46:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:30 smithi162 ceph-mon[21854]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T10:46:30.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:30 smithi042 systemd[1]: Stopping Ceph mon.a for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:46:31.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:31 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a[17073]: 2024-04-20T10:46:31.063+0000 7f56d5400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:46:31.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:31 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a[17073]: 2024-04-20T10:46:31.063+0000 7f56d5400700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-20T10:46:31.354 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:31 smithi042 podman[92100]: 2024-04-20 10:46:31.260767924 +0000 UTC m=+0.354300244 container died f39180ca248a36dae9b7cdcfbb967e6ac1f0ebbf0a7d992b6adf38d8f337c158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True) 2024-04-20T10:46:31.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:31 smithi042 podman[92100]: 2024-04-20 10:46:31.738215263 +0000 UTC m=+0.831747602 container cleanup f39180ca248a36dae9b7cdcfbb967e6ac1f0ebbf0a7d992b6adf38d8f337c158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-20T10:46:31.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:31 smithi042 bash[92100]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a 2024-04-20T10:46:32.353 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 20 10:46:31 smithi042 podman[92113]: 2024-04-20 10:46:31.963256667 +0000 UTC m=+0.684493806 container remove f39180ca248a36dae9b7cdcfbb967e6ac1f0ebbf0a7d992b6adf38d8f337c158 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-a, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:46:32.492 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.a.service' 2024-04-20T10:46:32.614 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:46:32.615 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-20T10:46:32.615 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-20T10:46:32.615 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.b 2024-04-20T10:46:33.004 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:32 smithi046 systemd[1]: Stopping Ceph mon.b for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:46:33.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:33 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-b[21569]: 2024-04-20T10:46:33.003+0000 7fa659854700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:46:33.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:33 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-b[21569]: 2024-04-20T10:46:33.003+0000 7fa659854700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-20T10:46:33.304 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:33 smithi046 podman[57654]: 2024-04-20 10:46:33.168389852 +0000 UTC m=+0.281377400 container died b88513ffbf0a20b5613bda69cf39900ff6c38bb50feb28732dbd3147f2805a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-20T10:46:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:33 smithi046 podman[57654]: 2024-04-20 10:46:33.555055578 +0000 UTC m=+0.668043130 container cleanup b88513ffbf0a20b5613bda69cf39900ff6c38bb50feb28732dbd3147f2805a5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-b, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-20T10:46:33.803 INFO:journalctl@ceph.mon.b.smithi046.stdout:Apr 20 10:46:33 smithi046 bash[57654]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-b 2024-04-20T10:46:33.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.b.service' 2024-04-20T10:46:33.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:46:33.978 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-20T10:46:33.978 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-20T10:46:33.978 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.c 2024-04-20T10:46:34.303 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:34 smithi162 systemd[1]: Stopping Ceph mon.c for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:46:34.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:34 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-c[21850]: 2024-04-20T10:46:34.302+0000 7fe94965c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:46:34.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:34 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-c[21850]: 2024-04-20T10:46:34.302+0000 7fe94965c700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-20T10:46:34.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:34 smithi162 podman[58491]: 2024-04-20 10:46:34.548281632 +0000 UTC m=+0.369544063 container died 78b2674133b17490d60e3824aa6c59b87cd815900137617b5dcf7450606d2be0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:46:35.498 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:35 smithi162 podman[58491]: 2024-04-20 10:46:35.270779476 +0000 UTC m=+1.092041881 container cleanup 78b2674133b17490d60e3824aa6c59b87cd815900137617b5dcf7450606d2be0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:46:35.498 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:35 smithi162 bash[58491]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-c 2024-04-20T10:46:35.751 INFO:journalctl@ceph.mon.c.smithi162.stdout:Apr 20 10:46:35 smithi162 podman[58505]: 2024-04-20 10:46:35.498255525 +0000 UTC m=+0.945269432 container remove 78b2674133b17490d60e3824aa6c59b87cd815900137617b5dcf7450606d2be0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mon-c, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-04-20T10:46:35.756 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mon.c.service' 2024-04-20T10:46:35.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:46:35.803 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-20T10:46:35.804 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-20T10:46:35.804 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.x 2024-04-20T10:46:36.353 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:46:35 smithi042 systemd[1]: Stopping Ceph mgr.x for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:46:36.851 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:46:36 smithi042 podman[92219]: 2024-04-20 10:46:36.546245711 +0000 UTC m=+0.268350870 container died 5bd72d6b593f6a53bcb340da07515b7a463d86bd77fe122b46b8eea14806f53c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-20T10:46:37.103 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:46:37 smithi042 podman[92219]: 2024-04-20 10:46:37.060036046 +0000 UTC m=+0.782141196 container cleanup 5bd72d6b593f6a53bcb340da07515b7a463d86bd77fe122b46b8eea14806f53c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:46:37.104 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:46:37 smithi042 bash[92219]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x 2024-04-20T10:46:37.606 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 20 10:46:37 smithi042 podman[92232]: 2024-04-20 10:46:37.303878059 +0000 UTC m=+0.738301853 container remove 5bd72d6b593f6a53bcb340da07515b7a463d86bd77fe122b46b8eea14806f53c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-04-20T10:46:37.821 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.x.service' 2024-04-20T10:46:37.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:46:37.942 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-20T10:46:37.942 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-20T10:46:37.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.y 2024-04-20T10:46:38.272 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:46:37 smithi046 systemd[1]: Stopping Ceph mgr.y for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:46:38.553 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:46:38 smithi046 podman[57790]: 2024-04-20 10:46:38.272823902 +0000 UTC m=+0.141835613 container died 2c1fd9edaa3bb6c02292f7a8f6d8364871cf6e6c5ce2255824f0056b89360abd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:46:38.912 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:46:38 smithi046 podman[57790]: 2024-04-20 10:46:38.661177075 +0000 UTC m=+0.530188770 container cleanup 2c1fd9edaa3bb6c02292f7a8f6d8364871cf6e6c5ce2255824f0056b89360abd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:46:38.912 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Apr 20 10:46:38 smithi046 bash[57790]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-y 2024-04-20T10:46:39.052 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.y.service' 2024-04-20T10:46:39.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:46:39.089 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-20T10:46:39.090 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-20T10:46:39.090 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.z 2024-04-20T10:46:39.399 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:46:39 smithi162 systemd[1]: Stopping Ceph mgr.z for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:46:39.727 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:46:39 smithi162 podman[58612]: 2024-04-20 10:46:39.422708854 +0000 UTC m=+0.146544164 container died 5fd52e305ebf258ae5439cca6095810c726c5fd3d3af7cb414f24c1045fb1723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-20T10:46:40.150 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:46:39 smithi162 podman[58612]: 2024-04-20 10:46:39.827020432 +0000 UTC m=+0.550855745 container cleanup 5fd52e305ebf258ae5439cca6095810c726c5fd3d3af7cb414f24c1045fb1723 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T10:46:40.150 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Apr 20 10:46:39 smithi162 bash[58612]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mgr-z 2024-04-20T10:46:41.011 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mgr.z.service' 2024-04-20T10:46:41.044 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:46:41.044 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-20T10:46:41.044 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-20T10:46:41.044 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.0 2024-04-20T10:46:41.604 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:41 smithi042 systemd[1]: Stopping Ceph osd.0 for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:46:42.104 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:41 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0[32186]: 2024-04-20T10:46:41.648+0000 7f4959600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:46:42.104 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:41 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0[32186]: 2024-04-20T10:46:41.648+0000 7f4959600700 -1 osd.0 103 *** Got signal Terminated *** 2024-04-20T10:46:42.104 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:41 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0[32186]: 2024-04-20T10:46:41.649+0000 7f4959600700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T10:46:47.604 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:47 smithi042 podman[92342]: 2024-04-20 10:46:47.329617559 +0000 UTC m=+5.836465344 container died 4a56c3de6357a51434a21281b967cc07c81d6b53c9da4597f11cbe75baf996af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-20T10:46:48.052 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:47 smithi042 podman[92342]: 2024-04-20 10:46:47.825089794 +0000 UTC m=+6.331937556 container cleanup 4a56c3de6357a51434a21281b967cc07c81d6b53c9da4597f11cbe75baf996af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-20T10:46:48.052 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:47 smithi042 bash[92342]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0 2024-04-20T10:46:48.353 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:48 smithi042 podman[92363]: 2024-04-20 10:46:48.051597749 +0000 UTC m=+0.702497016 container remove 4a56c3de6357a51434a21281b967cc07c81d6b53c9da4597f11cbe75baf996af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-20T10:46:49.103 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:48 smithi042 podman[92448]: 2024-04-20 10:46:48.829087702 +0000 UTC m=+0.035745247 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:46:49.104 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:48 smithi042 podman[92448]: 2024-04-20 10:46:48.952651927 +0000 UTC m=+0.159309500 container create d643afcf21288836658be48b9ada6efeb2fc5af5400698e3081c0cb6868a4976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:46:49.494 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:49 smithi042 podman[92448]: 2024-04-20 10:46:49.240580852 +0000 UTC m=+0.447238429 container init d643afcf21288836658be48b9ada6efeb2fc5af5400698e3081c0cb6868a4976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-20T10:46:49.494 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:49 smithi042 podman[92448]: 2024-04-20 10:46:49.252195666 +0000 UTC m=+0.458853236 container start d643afcf21288836658be48b9ada6efeb2fc5af5400698e3081c0cb6868a4976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-20T10:46:49.494 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:49 smithi042 podman[92448]: 2024-04-20 10:46:49.347880483 +0000 UTC m=+0.554538039 container attach d643afcf21288836658be48b9ada6efeb2fc5af5400698e3081c0cb6868a4976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-20T10:46:50.056 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 20 10:46:49 smithi042 podman[92448]: 2024-04-20 10:46:49.748690077 +0000 UTC m=+0.955347644 container died d643afcf21288836658be48b9ada6efeb2fc5af5400698e3081c0cb6868a4976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-20T10:46:50.477 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.0.service' 2024-04-20T10:46:50.597 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:46:50.597 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-20T10:46:50.597 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-20T10:46:50.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.1 2024-04-20T10:46:50.915 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:50 smithi046 systemd[1]: Stopping Ceph osd.1 for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:46:51.303 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:50 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1[26197]: 2024-04-20T10:46:50.915+0000 7fa025290700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:46:51.303 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:50 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1[26197]: 2024-04-20T10:46:50.915+0000 7fa025290700 -1 osd.1 103 *** Got signal Terminated *** 2024-04-20T10:46:51.303 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:50 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1[26197]: 2024-04-20T10:46:50.915+0000 7fa025290700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T10:46:56.750 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:56 smithi046 podman[57915]: 2024-04-20 10:46:56.470020751 +0000 UTC m=+5.675781198 container died d87c1be6ced8cae31c312fafc0345d34bce7a67de367670814a3e790ac738542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T10:46:57.053 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:56 smithi046 podman[57915]: 2024-04-20 10:46:56.849384181 +0000 UTC m=+6.055144629 container cleanup d87c1be6ced8cae31c312fafc0345d34bce7a67de367670814a3e790ac738542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:46:57.054 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:56 smithi046 bash[57915]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1 2024-04-20T10:46:57.480 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:57 smithi046 podman[57934]: 2024-04-20 10:46:57.117605429 +0000 UTC m=+0.647410500 container remove d87c1be6ced8cae31c312fafc0345d34bce7a67de367670814a3e790ac738542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:46:57.803 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:57 smithi046 podman[58024]: 2024-04-20 10:46:57.38424073 +0000 UTC m=+0.018516955 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:46:57.804 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:57 smithi046 podman[58024]: 2024-04-20 10:46:57.660228386 +0000 UTC m=+0.294504594 container create 89314e8ca49741b70711af42742b975b0895f46a438420c7e48eb21b4521bc03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-20T10:46:58.303 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:57 smithi046 podman[58024]: 2024-04-20 10:46:57.996304503 +0000 UTC m=+0.630580724 container init 89314e8ca49741b70711af42742b975b0895f46a438420c7e48eb21b4521bc03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-20T10:46:58.304 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:57 smithi046 podman[58024]: 2024-04-20 10:46:57.999679197 +0000 UTC m=+0.633955405 container start 89314e8ca49741b70711af42742b975b0895f46a438420c7e48eb21b4521bc03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-20T10:46:58.304 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:58 smithi046 podman[58024]: 2024-04-20 10:46:58.087816914 +0000 UTC m=+0.722093133 container attach 89314e8ca49741b70711af42742b975b0895f46a438420c7e48eb21b4521bc03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-20T10:46:59.054 INFO:journalctl@ceph.osd.1.smithi046.stdout:Apr 20 10:46:58 smithi046 podman[58024]: 2024-04-20 10:46:58.759172692 +0000 UTC m=+1.393448902 container died 89314e8ca49741b70711af42742b975b0895f46a438420c7e48eb21b4521bc03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True) 2024-04-20T10:46:59.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.1.service' 2024-04-20T10:46:59.551 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:46:59.551 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-20T10:46:59.551 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-20T10:46:59.552 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.2 2024-04-20T10:46:59.889 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:46:59 smithi162 systemd[1]: Stopping Ceph osd.2 for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:47:00.150 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:46:59 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2[27006]: 2024-04-20T10:46:59.888+0000 7fb9edcb4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:47:00.150 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:46:59 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2[27006]: 2024-04-20T10:46:59.888+0000 7fb9edcb4700 -1 osd.2 103 *** Got signal Terminated *** 2024-04-20T10:47:00.150 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:46:59 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2[27006]: 2024-04-20T10:46:59.888+0000 7fb9edcb4700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T10:47:05.650 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:47:05 smithi162 podman[58735]: 2024-04-20 10:47:05.392769401 +0000 UTC m=+5.640760861 container died 668860aca425d1e3cd53617c0340ad9bf7127f36772f48399f38e9c6628ad9fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-20T10:47:06.040 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:47:05 smithi162 podman[58735]: 2024-04-20 10:47:05.822227537 +0000 UTC m=+6.070218986 container cleanup 668860aca425d1e3cd53617c0340ad9bf7127f36772f48399f38e9c6628ad9fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-20T10:47:06.040 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:47:05 smithi162 bash[58735]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2 2024-04-20T10:47:06.399 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:47:06 smithi162 podman[58764]: 2024-04-20 10:47:06.040916602 +0000 UTC m=+0.649062567 container remove 668860aca425d1e3cd53617c0340ad9bf7127f36772f48399f38e9c6628ad9fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-20T10:47:06.720 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:47:06 smithi162 podman[58852]: 2024-04-20 10:47:06.341232213 +0000 UTC m=+0.019759181 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:47:06.720 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:47:06 smithi162 podman[58852]: 2024-04-20 10:47:06.450804232 +0000 UTC m=+0.129331176 container create d2415c219d5e1d5e60ab6a2d0b65aa4c6c784d76378d69f8a9fdcc9ab34d51a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-04-20T10:47:07.018 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:47:06 smithi162 podman[58852]: 2024-04-20 10:47:06.720406207 +0000 UTC m=+0.398933152 container init d2415c219d5e1d5e60ab6a2d0b65aa4c6c784d76378d69f8a9fdcc9ab34d51a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, ceph=True) 2024-04-20T10:47:07.018 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:47:06 smithi162 podman[58852]: 2024-04-20 10:47:06.723587362 +0000 UTC m=+0.402114311 container start d2415c219d5e1d5e60ab6a2d0b65aa4c6c784d76378d69f8a9fdcc9ab34d51a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:47:07.019 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:47:06 smithi162 podman[58852]: 2024-04-20 10:47:06.82052488 +0000 UTC m=+0.499051830 container attach d2415c219d5e1d5e60ab6a2d0b65aa4c6c784d76378d69f8a9fdcc9ab34d51a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:47:07.358 INFO:journalctl@ceph.osd.2.smithi162.stdout:Apr 20 10:47:07 smithi162 podman[58852]: 2024-04-20 10:47:07.018843812 +0000 UTC m=+0.697370769 container died d2415c219d5e1d5e60ab6a2d0b65aa4c6c784d76378d69f8a9fdcc9ab34d51a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-20T10:47:07.646 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.2.service' 2024-04-20T10:47:07.686 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:47:07.687 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-20T10:47:07.687 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-20T10:47:07.687 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.3 2024-04-20T10:47:08.103 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:07 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:07.845+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:08.103 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:07 smithi042 systemd[1]: Stopping Ceph osd.3 for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:47:08.604 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:08 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3[38659]: 2024-04-20T10:47:08.245+0000 7fac48200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:47:08.604 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:08 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3[38659]: 2024-04-20T10:47:08.245+0000 7fac48200700 -1 osd.3 103 *** Got signal Terminated *** 2024-04-20T10:47:08.604 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:08 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3[38659]: 2024-04-20T10:47:08.245+0000 7fac48200700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T10:47:08.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:08 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:08.453+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:09.104 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:08 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:08.816+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:09.474 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:09 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:09.187+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:09.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:09 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:09.475+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:10.056 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:09 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:09.864+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:10.120 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:09 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:09.847+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:10.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:10 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:10.055+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:10.520 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:10 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:10.163+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:10.899 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:10 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:10.521+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:11.053 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:10 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3[38659]: 2024-04-20T10:47:10.719+0000 7fac43a00700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.810087+0000 front 2024-04-20T10:46:44.810640+0000 (oldest deadline 2024-04-20T10:47:10.710087+0000) 2024-04-20T10:47:11.054 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:10 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:10.838+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:11.303 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:10 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:10.874+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:11.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:11 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:11.053+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:11.399 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:11 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:11.141+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:11.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:11 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:11.494+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:12.073 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:11 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3[38659]: 2024-04-20T10:47:11.679+0000 7fac43a00700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.810087+0000 front 2024-04-20T10:46:44.810640+0000 (oldest deadline 2024-04-20T10:47:10.710087+0000) 2024-04-20T10:47:12.074 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:11 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:11.850+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:12.303 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:11 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:11.837+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:12.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:12 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:12.073+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:12.511 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:12 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:12.169+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:12.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:12 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:12.510+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:13.064 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:12 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:12.872+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:13.064 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:12 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3[38659]: 2024-04-20T10:47:12.712+0000 7fac43a00700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.810087+0000 front 2024-04-20T10:46:44.810640+0000 (oldest deadline 2024-04-20T10:47:10.710087+0000) 2024-04-20T10:47:13.303 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:12 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:12.858+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:13.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:13 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:13.064+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:13.559 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:13 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:13.213+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:13.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:13 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:13.558+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:14.244 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:14 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:14.034+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:14.245 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:13 smithi042 podman[92554]: 2024-04-20 10:47:13.915675951 +0000 UTC m=+5.822059224 container died 835509bbc7e7d0a137b85f22e600f8c457155c02f9ceed43649255d97c5907c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:47:14.245 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:13 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:13.886+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:14.303 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:13 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:13.870+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:14.531 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:14 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:14.182+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:14.608 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:14 smithi042 podman[92554]: 2024-04-20 10:47:14.388808653 +0000 UTC m=+6.295191920 container cleanup 835509bbc7e7d0a137b85f22e600f8c457155c02f9ceed43649255d97c5907c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:47:14.608 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:14 smithi042 bash[92554]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3 2024-04-20T10:47:14.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:14 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:14.530+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:15.062 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:14 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:14.844+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:15.063 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:14 smithi042 podman[92585]: 2024-04-20 10:47:14.640181821 +0000 UTC m=+0.713434983 container remove 835509bbc7e7d0a137b85f22e600f8c457155c02f9ceed43649255d97c5907c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-20T10:47:15.303 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:14 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:14.886+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:15.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:15 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:15.061+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:15.484 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:15 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:15.231+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:15.805 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:15 smithi042 podman[92674]: 2024-04-20 10:47:15.388970766 +0000 UTC m=+0.039509305 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:47:15.805 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:15 smithi042 podman[92674]: 2024-04-20 10:47:15.516468091 +0000 UTC m=+0.167006715 container create 82fffa54517fe0b8a9ed3f0a90bc1b86b13582e6cf9ad0b2cae85c8015c6a9a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-20T10:47:15.805 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:15 smithi042 podman[92674]: 2024-04-20 10:47:15.803867873 +0000 UTC m=+0.454406451 container init 82fffa54517fe0b8a9ed3f0a90bc1b86b13582e6cf9ad0b2cae85c8015c6a9a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:47:15.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:15 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:15.483+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:16.075 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:15 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:15.892+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:16.075 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:15 smithi042 podman[92674]: 2024-04-20 10:47:15.81647773 +0000 UTC m=+0.467016308 container start 82fffa54517fe0b8a9ed3f0a90bc1b86b13582e6cf9ad0b2cae85c8015c6a9a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-04-20T10:47:16.075 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:15 smithi042 podman[92674]: 2024-04-20 10:47:15.91945375 +0000 UTC m=+0.569992292 container attach 82fffa54517fe0b8a9ed3f0a90bc1b86b13582e6cf9ad0b2cae85c8015c6a9a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-20T10:47:16.303 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:15 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:15.863+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:16.353 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 20 10:47:16 smithi042 podman[92674]: 2024-04-20 10:47:16.278223099 +0000 UTC m=+0.928761676 container died 82fffa54517fe0b8a9ed3f0a90bc1b86b13582e6cf9ad0b2cae85c8015c6a9a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T10:47:16.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:16 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:16.074+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:16.526 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:16 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:16.252+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:16.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:16 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:16.526+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:17.007 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:16 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:16.865+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:17.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.3.service' 2024-04-20T10:47:17.160 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:47:17.161 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-20T10:47:17.161 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-20T10:47:17.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.4 2024-04-20T10:47:17.189 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:16 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:16.831+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:17.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:17 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:17.093+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:17.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:17 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:17.093+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:17.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:17 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:17.240+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:17.470 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:17 smithi046 systemd[1]: Stopping Ceph osd.4 for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:47:17.803 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:17 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4[33146]: 2024-04-20T10:47:17.470+0000 7f151d16b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:47:17.803 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:17 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4[33146]: 2024-04-20T10:47:17.470+0000 7f151d16b700 -1 osd.4 103 *** Got signal Terminated *** 2024-04-20T10:47:17.804 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:17 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4[33146]: 2024-04-20T10:47:17.470+0000 7f151d16b700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T10:47:17.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:17 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:17.524+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:17.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:17 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:17.524+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:18.303 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:17 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:17.835+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:18.304 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:17 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:17.835+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:18.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:17 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:17.896+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:18.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:18 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:18.044+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:18.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:18 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:18.044+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:18.546 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:18 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:18.225+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:18.546 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:18 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:18.225+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:18.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:18 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:18.545+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:18.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:18 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:18.545+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:19.303 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:18 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:18.864+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:19.304 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:18 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:18.864+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:19.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:19 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:19.065+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:19.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:19 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:19.065+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:19.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:18 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:18.906+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:19.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:18 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:18.906+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:19.579 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:19 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:19.268+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:19.579 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:19 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:19.268+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:19.899 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:19 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:19.579+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:19.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:19 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:19.579+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:20.303 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:19 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:19.805+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:20.304 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:19 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:19.867+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:20.304 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:19 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:19.867+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:20.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:20 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:20.079+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:20.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:20 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:20.079+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:20.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:19 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:19.901+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:20.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:19 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:19.901+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:20.557 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:20 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:20.273+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:20.558 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:20 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:20.273+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:20.899 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:20 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:20.557+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:20.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:20 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:20.557+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:21.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:20 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:20.795+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:21.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:20 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:20.862+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:21.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:20 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:20.862+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:21.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:21 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:21.072+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:21.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:21 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:21.072+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:21.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:20 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:20.889+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:21.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:20 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:20.889+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:21.601 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:21 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:21.309+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:21.601 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:21 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:21.309+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:21.899 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:21 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:21.601+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:21.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:21 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:21.601+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:22.303 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:21 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4[33146]: 2024-04-20T10:47:21.923+0000 7f151975f700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.046038+0000 front 2024-04-20T10:46:53.046122+0000 (oldest deadline 2024-04-20T10:47:21.246762+0000) 2024-04-20T10:47:22.304 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:21 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:21.833+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:22.304 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:21 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:21.829+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:22.304 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:21 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:21.829+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:22.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:22 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:22.064+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:22.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:22 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:22.064+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:22.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:21 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:21.895+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:22.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:21 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:21.895+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:22.574 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:22 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:22.221+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:22.574 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:22 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:22.298+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:22.574 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:22 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:22.298+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:22.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:22 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:22.573+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:22.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:22 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:22.573+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:23.213 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:22 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:22.821+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:23.214 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:22 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:22.872+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:23.214 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:22 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:22.872+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:23.214 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:22 smithi046 podman[58152]: 2024-04-20 10:47:22.912529919 +0000 UTC m=+5.552061115 container died 656aab974eaeeb7de5c55f506eabf11468f25eed7798eece295029639c37e0d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-20T10:47:23.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:23 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:23.073+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:23.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:23 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:23.073+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:23.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:22 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:22.922+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:23.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:22 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:22.922+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:23.553 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:23 smithi046 podman[58152]: 2024-04-20 10:47:23.304079438 +0000 UTC m=+5.943610642 container cleanup 656aab974eaeeb7de5c55f506eabf11468f25eed7798eece295029639c37e0d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-20T10:47:23.554 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:23 smithi046 bash[58152]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4 2024-04-20T10:47:23.610 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:23 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:23.190+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:23.610 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:23 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:23.265+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:23.611 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:23 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:23.265+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:23.899 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:23 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:23.610+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:23.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:23 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:23.610+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:24.040 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:23 smithi046 podman[58171]: 2024-04-20 10:47:23.714176743 +0000 UTC m=+0.792091613 container remove 656aab974eaeeb7de5c55f506eabf11468f25eed7798eece295029639c37e0d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-20T10:47:24.041 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:23 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:23.861+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:24.041 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:23 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:23.837+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:24.041 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:23 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:23.837+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:24.303 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:24 smithi046 podman[58260]: 2024-04-20 10:47:23.943672399 +0000 UTC m=+0.018085977 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:47:24.304 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:24 smithi046 podman[58260]: 2024-04-20 10:47:24.065043155 +0000 UTC m=+0.139456742 container create 9bc000becb398794724c64011183e885fe24211d6f83fa40fd265edfabba066d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:47:24.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:24 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:24.116+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:24.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:24 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:24.116+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:24.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:23 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:23.948+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:24.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:23 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:23.948+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:24.587 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:24 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:24.153+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:24.587 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:24 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:24.242+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:24.587 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:24 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:24.242+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:24.649 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:24 smithi046 podman[58260]: 2024-04-20 10:47:24.342419744 +0000 UTC m=+0.416833303 container init 9bc000becb398794724c64011183e885fe24211d6f83fa40fd265edfabba066d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-20T10:47:24.649 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:24 smithi046 podman[58260]: 2024-04-20 10:47:24.34573508 +0000 UTC m=+0.420148634 container start 9bc000becb398794724c64011183e885fe24211d6f83fa40fd265edfabba066d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:47:24.649 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:24 smithi046 podman[58260]: 2024-04-20 10:47:24.434339723 +0000 UTC m=+0.508753290 container attach 9bc000becb398794724c64011183e885fe24211d6f83fa40fd265edfabba066d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-20T10:47:24.899 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:24 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:24.586+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:24.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:24 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:24.586+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:24.978 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:24 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:24.838+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:24.979 INFO:journalctl@ceph.osd.4.smithi046.stdout:Apr 20 10:47:24 smithi046 podman[58260]: 2024-04-20 10:47:24.649226507 +0000 UTC m=+0.723640074 container died 9bc000becb398794724c64011183e885fe24211d6f83fa40fd265edfabba066d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:47:24.979 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:24 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:24.818+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:24.979 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:24 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:24.818+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:25.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.4.service' 2024-04-20T10:47:25.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:25 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:25.119+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:25.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:25 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:25.119+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:25.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:24 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:24.914+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:25.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:24 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:24.914+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:25.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:47:25.369 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-20T10:47:25.369 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-20T10:47:25.369 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.5 2024-04-20T10:47:25.628 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:25 smithi162 systemd[1]: Stopping Ceph osd.5 for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:47:25.629 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:25 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:25.179+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:25.629 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:25 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:25.267+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:25.629 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:25 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:25.267+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:25.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:25 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:25.627+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:25.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:25 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:25.627+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:25.901 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:25 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:25.710+0000 7f7da34e4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:47:25.901 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:25 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:25.710+0000 7f7da34e4700 -1 osd.5 103 *** Got signal Terminated *** 2024-04-20T10:47:25.901 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:25 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:25.710+0000 7f7da34e4700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T10:47:26.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:25 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:25.808+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:26.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:25 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:25.808+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:26.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:25 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:25.777+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:26.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:25 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:25.777+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:26.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:25 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:25.905+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:26.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:25 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:25.905+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:26.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:26 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:26.107+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:26.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:26 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:26.107+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:26.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:26 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:26.304+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:26.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:26 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:26.304+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:26.651 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:26 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:26.227+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:27.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:26 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:26.810+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:27.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:26 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:26.810+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:27.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:26 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:26.741+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:27.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:26 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:26.741+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:27.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:26 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:26.677+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:27.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:26 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:26.677+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:27.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:27 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:27.066+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:27.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:27 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:27.066+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:27.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:26 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:26.914+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:27.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:26 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:26.914+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:27.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:26 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:26.914+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:27.640 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:27 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:27.225+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:27.641 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:27 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:27.305+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:27.641 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:27 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:27.305+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:27.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:27 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:27.640+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:27.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:27 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:27.640+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:27.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:27 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:27.640+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.653648+0000 front 2024-04-20T10:47:03.653668+0000 (oldest deadline 2024-04-20T10:47:27.153806+0000) 2024-04-20T10:47:28.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:27 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:27.769+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:28.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:27 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:27.769+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:28.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:27 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:27.770+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:28.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:27 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:27.770+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:28.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:28 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:28.084+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:28.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:28 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:28.084+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:28.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:27 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:27.918+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:28.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:27 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:27.918+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:28.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:27 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:27.918+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:28.649 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:28 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:28.233+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:28.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:28 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:28.233+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:28.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:28 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:28.304+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:28.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:28 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:28.304+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:29.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:28 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:28.739+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:29.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:28 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:28.739+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:29.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:28 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:28.759+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:29.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:28 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:28.759+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:29.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:28 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:28.659+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:29.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:28 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:28.659+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:29.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:28 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:28.659+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.653648+0000 front 2024-04-20T10:47:03.653668+0000 (oldest deadline 2024-04-20T10:47:27.153806+0000) 2024-04-20T10:47:29.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:29 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:29.091+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:29.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:29 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:29.091+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:29.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:28 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:28.961+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:29.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:28 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:28.961+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:29.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:28 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:28.961+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:29.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:29 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:29.267+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:29.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:29 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:29.267+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:29.651 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:29 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:29.236+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:29.651 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:29 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:29.236+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:30.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:29 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:29.698+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:30.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:29 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:29.698+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:30.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:29 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:29.795+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:30.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:29 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:29.795+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:30.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:29 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:29.795+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:30.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:29 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:29.680+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:30.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:29 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:29.680+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:30.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:29 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:29.680+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.653648+0000 front 2024-04-20T10:47:03.653668+0000 (oldest deadline 2024-04-20T10:47:27.153806+0000) 2024-04-20T10:47:30.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:30 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:30.043+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:30.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:30 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:30.043+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:30.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:30 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:30.043+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.191281+0000 front 2024-04-20T10:47:04.191192+0000 (oldest deadline 2024-04-20T10:47:29.491550+0000) 2024-04-20T10:47:30.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:29 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:29.958+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:30.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:29 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:29.958+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:30.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:29 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:29.958+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:30.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:30 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:30.267+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:30.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:30 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:30.267+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:30.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:30 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:30.239+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:30.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:30 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:30.239+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:30.651 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:30 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:30.239+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:31.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:30 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:30.693+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:31.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:30 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:30.693+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:31.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:30 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:30.777+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:31.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:30 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:30.777+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:31.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:30 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:30.777+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:31.070 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:30 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:30.690+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.349284+0000 front 2024-04-20T10:46:46.349454+0000 (oldest deadline 2024-04-20T10:47:07.449152+0000) 2024-04-20T10:47:31.070 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:30 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:30.690+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.552019+0000 front 2024-04-20T10:46:55.551936+0000 (oldest deadline 2024-04-20T10:47:17.251840+0000) 2024-04-20T10:47:31.070 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:30 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5[33697]: 2024-04-20T10:47:30.690+0000 7f7d9bad0700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.653648+0000 front 2024-04-20T10:47:03.653668+0000 (oldest deadline 2024-04-20T10:47:27.153806+0000) 2024-04-20T10:47:31.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:31 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:31.085+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:31.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:31 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:31.085+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:31.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:31 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:31.085+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.191281+0000 front 2024-04-20T10:47:04.191192+0000 (oldest deadline 2024-04-20T10:47:29.491550+0000) 2024-04-20T10:47:31.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:30 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:30.924+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:31.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:30 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:30.924+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:31.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:30 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:30.924+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:31.361 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:31 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:31.254+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:31.361 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:31 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:31.255+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:31.361 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:31 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:31.226+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:31.362 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:31 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:31.226+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:31.362 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:31 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:31.226+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:31.362 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:31 smithi162 podman[58959]: 2024-04-20 10:47:31.070513571 +0000 UTC m=+5.504842198 container died f39a63c56810f991e24ba23a59ceb4668b2eb71fb11a4a2f8853a79ea5c90407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-20T10:47:31.650 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:31 smithi162 podman[58959]: 2024-04-20 10:47:31.493941359 +0000 UTC m=+5.928269986 container cleanup f39a63c56810f991e24ba23a59ceb4668b2eb71fb11a4a2f8853a79ea5c90407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:47:31.650 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:31 smithi162 bash[58959]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5 2024-04-20T10:47:32.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:31 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:31.727+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:32.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:31 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:31.727+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:32.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:31 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:31.774+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:32.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:31 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:31.774+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:32.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:31 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:31.774+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:32.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:31 smithi162 podman[58976]: 2024-04-20 10:47:31.733438983 +0000 UTC m=+0.663207923 container remove f39a63c56810f991e24ba23a59ceb4668b2eb71fb11a4a2f8853a79ea5c90407 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-20T10:47:32.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:32 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:32.037+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:32.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:32 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:32.037+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:32.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:32 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:32.037+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.191281+0000 front 2024-04-20T10:47:04.191192+0000 (oldest deadline 2024-04-20T10:47:29.491550+0000) 2024-04-20T10:47:32.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:31 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:31.943+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:32.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:31 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:31.943+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:32.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:31 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:31.943+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:32.457 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:32 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:32.238+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:32.458 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:32 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:32.238+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:32.458 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:32 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:32.200+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:32.458 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:32 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:32.200+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:32.458 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:32 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:32.200+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:32.458 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:32 smithi162 podman[59067]: 2024-04-20 10:47:32.069291757 +0000 UTC m=+0.017784736 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:47:32.459 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:32 smithi162 podman[59067]: 2024-04-20 10:47:32.18848365 +0000 UTC m=+0.136976685 container create ee7735e94fda3f3c060fe88e536ef967bc6d4ac41302a7147f78946c450f3ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:47:32.459 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:32 smithi162 podman[59067]: 2024-04-20 10:47:32.458086333 +0000 UTC m=+0.406579318 container init ee7735e94fda3f3c060fe88e536ef967bc6d4ac41302a7147f78946c450f3ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-20T10:47:32.757 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:32 smithi162 podman[59067]: 2024-04-20 10:47:32.462057216 +0000 UTC m=+0.410550193 container start ee7735e94fda3f3c060fe88e536ef967bc6d4ac41302a7147f78946c450f3ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-20T10:47:32.757 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:32 smithi162 podman[59067]: 2024-04-20 10:47:32.574632943 +0000 UTC m=+0.523125933 container attach ee7735e94fda3f3c060fe88e536ef967bc6d4ac41302a7147f78946c450f3ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-20T10:47:33.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:32 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:32.760+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:33.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:32 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:32.760+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:33.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:32 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:32.795+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:33.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:32 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:32.795+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:33.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:32 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:32.795+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:33.068 INFO:journalctl@ceph.osd.5.smithi162.stdout:Apr 20 10:47:32 smithi162 podman[59067]: 2024-04-20 10:47:32.757294048 +0000 UTC m=+0.705787028 container died ee7735e94fda3f3c060fe88e536ef967bc6d4ac41302a7147f78946c450f3ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:47:33.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:33 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:33.029+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:33.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:33 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:33.029+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:33.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:33 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:33.029+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.191281+0000 front 2024-04-20T10:47:04.191192+0000 (oldest deadline 2024-04-20T10:47:29.491550+0000) 2024-04-20T10:47:33.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:32 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:32.986+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:33.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:32 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:32.986+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:33.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:32 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:32.986+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:33.370 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:33 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:33.261+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:33.370 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:33 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:33.261+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:33.370 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:33 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:33.153+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:33.370 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:33 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:33.153+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:33.370 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:33 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:33.153+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:33.394 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.5.service' 2024-04-20T10:47:33.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:47:33.432 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-20T10:47:33.432 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-20T10:47:33.432 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.6 2024-04-20T10:47:33.853 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:33 smithi042 systemd[1]: Stopping Ceph osd.6 for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:47:34.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:33 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:33.741+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:34.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:33 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:33.741+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:34.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:33 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:33.833+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:34.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:33 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:33.833+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:34.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:33 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:33.833+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:34.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:33 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:33.983+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:34.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:33 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:33.983+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:34.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:33 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:33.983+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:34.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:34.017+0000 7f89d1000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:47:34.355 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:34.017+0000 7f89d1000700 -1 osd.6 103 *** Got signal Terminated *** 2024-04-20T10:47:34.355 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:34.017+0000 7f89d1000700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T10:47:34.355 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:34.040+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:34.355 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:34.040+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:34.355 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:34.040+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.191281+0000 front 2024-04-20T10:47:04.191192+0000 (oldest deadline 2024-04-20T10:47:29.491550+0000) 2024-04-20T10:47:34.400 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:34 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:34.292+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:34.400 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:34 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:34.292+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:34.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:34 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:34.124+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:34.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:34 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:34.124+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:34.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:34 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:34.124+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:35.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:34 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:34.691+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:35.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:34 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:34.691+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:35.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:34 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:34.691+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:35.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:34 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:34.788+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:35.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:34 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:34.788+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:35.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:34 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:34.788+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:35.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:34.992+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:35.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:34.992+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:35.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:34.992+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.191281+0000 front 2024-04-20T10:47:04.191192+0000 (oldest deadline 2024-04-20T10:47:29.491550+0000) 2024-04-20T10:47:35.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:34.988+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:35.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:34.988+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:35.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:34 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:34.988+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:35.400 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:35 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:35.339+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:35.400 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:35 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:35.339+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:35.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:35 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:35.114+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:35.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:35 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:35.114+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:35.401 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:35 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:35.114+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:36.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:35 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:35.689+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:36.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:35 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:35.689+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:36.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:35 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:35.689+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:36.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:35 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:35.816+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:36.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:35 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:35.816+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:36.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:35 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:35.816+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:36.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:36 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:36.005+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:36.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:36 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:36.005+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:36.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:36 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:36.005+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.191281+0000 front 2024-04-20T10:47:04.191192+0000 (oldest deadline 2024-04-20T10:47:29.491550+0000) 2024-04-20T10:47:36.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:35 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:35.993+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:36.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:35 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:35.993+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:36.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:35 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:35.993+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:36.399 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:36 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:36.363+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:36.400 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:36 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:36.363+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:36.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:36 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:36.114+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:36.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:36 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:36.114+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:36.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:36 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:36.114+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:37.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:36 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:36.710+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:37.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:36 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:36.710+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:37.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:36 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:36.710+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:37.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:36 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:36.769+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:37.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:36 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:36.769+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:37.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:36 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:36.769+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:37.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:36 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:36.769+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:37.289 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:37.010+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:37.289 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:37.010+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:37.289 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:37.010+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.191281+0000 front 2024-04-20T10:47:04.191192+0000 (oldest deadline 2024-04-20T10:47:29.491550+0000) 2024-04-20T10:47:37.289 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:37.010+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.896005+0000 front 2024-04-20T10:47:12.896507+0000 (oldest deadline 2024-04-20T10:47:36.996049+0000) 2024-04-20T10:47:37.290 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:37.033+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:37.290 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:37.033+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:37.290 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:37.033+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:37.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:37 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:37.089+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:37.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:37 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:37.089+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:37.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:37 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:37.089+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:37.899 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:37 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:37.409+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:37.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:37 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:37.409+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:38.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:37 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:37.687+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:38.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:37 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:37.687+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:38.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:37 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:37.687+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:38.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:37 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:37.743+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:38.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:37 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:37.743+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:38.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:37 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:37.743+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:38.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:37 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:37.743+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:38.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:38 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:38.039+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:38.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:38 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:38.039+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:38.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:38 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:38.039+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:38.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:38 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:38.039+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:38.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:37.976+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:38.355 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:37.976+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:38.355 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:37.976+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.191281+0000 front 2024-04-20T10:47:04.191192+0000 (oldest deadline 2024-04-20T10:47:29.491550+0000) 2024-04-20T10:47:38.355 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:37.976+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.896005+0000 front 2024-04-20T10:47:12.896507+0000 (oldest deadline 2024-04-20T10:47:36.996049+0000) 2024-04-20T10:47:38.391 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:38 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:38.041+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:38.391 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:38 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:38.041+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:38.391 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:38 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:38.041+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:38.649 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:38 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:38.391+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:38.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:38 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:38.391+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:39.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:38 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:38.668+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:39.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:38 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:38.668+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:39.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:38 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:38.668+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:39.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:38 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:38.734+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:39.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:38 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:38.734+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:39.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:38 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:38.734+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:39.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:38 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:38.734+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:39.353 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:39.003+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:46.380857+0000 front 2024-04-20T10:46:46.380969+0000 (oldest deadline 2024-04-20T10:47:09.281012+0000) 2024-04-20T10:47:39.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:39.003+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.183579+0000 front 2024-04-20T10:46:52.183658+0000 (oldest deadline 2024-04-20T10:47:16.883838+0000) 2024-04-20T10:47:39.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:39.003+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.191281+0000 front 2024-04-20T10:47:04.191192+0000 (oldest deadline 2024-04-20T10:47:29.491550+0000) 2024-04-20T10:47:39.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6[45213]: 2024-04-20T10:47:39.003+0000 7f89cc800700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.896005+0000 front 2024-04-20T10:47:12.896507+0000 (oldest deadline 2024-04-20T10:47:36.996049+0000) 2024-04-20T10:47:39.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:39.010+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:39.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:39.010+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:39.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:39.010+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:39.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:39.010+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:39.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:39 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:39.045+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:39.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:39 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:39.045+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:39.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:39 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:39.045+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:39.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:39 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:39.406+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:39.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:39 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:39.406+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:39.968 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:39 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:39.713+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:39.969 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:39 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:39.713+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:39.969 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:39 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:39.713+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:39.969 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:39 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:39.761+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:39.969 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:39 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:39.761+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:39.969 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:39 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:39.761+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:39.969 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:39 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:39.761+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:39.974 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:39 smithi042 podman[92778]: 2024-04-20 10:47:39.663098724 +0000 UTC m=+5.780249213 container died 28c45975bacf3d701142a4ff264309c1b2781b2a8d876e39957f0b09e277e912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:47:40.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:40 smithi042 podman[92778]: 2024-04-20 10:47:40.110840788 +0000 UTC m=+6.227991274 container cleanup 28c45975bacf3d701142a4ff264309c1b2781b2a8d876e39957f0b09e277e912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True) 2024-04-20T10:47:40.354 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:40 smithi042 bash[92778]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6 2024-04-20T10:47:40.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:39.986+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:40.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:39.986+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:40.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:39.986+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:40.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:39 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:39.986+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:40.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:40 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:40.051+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:40.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:40 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:40.051+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:40.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:40 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:40.051+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:40.853 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:40 smithi042 podman[92808]: 2024-04-20 10:47:40.504634868 +0000 UTC m=+0.824486026 container remove 28c45975bacf3d701142a4ff264309c1b2781b2a8d876e39957f0b09e277e912 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-20T10:47:40.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:40 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:40.404+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:40.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:40 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:40.404+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:40.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:40 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:40.404+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:41.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:40 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:40.702+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:41.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:40 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:40.702+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:41.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:40 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:40.702+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:41.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:40 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:40.729+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:41.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:40 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:40.729+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:41.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:40 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:40.729+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:41.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:40 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:40.729+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:41.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:40 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:40.955+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:41.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:40 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:40.955+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:41.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:40 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:40.955+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:41.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:40 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:40.955+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:41.355 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:41 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:41.051+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:41.355 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:41 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:41.051+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:41.355 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:41 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:41.051+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:41.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:41 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:41.355+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:41.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:41 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:41.355+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:41.651 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:41 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:41.355+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:41.792 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:41 smithi042 podman[92895]: 2024-04-20 10:47:41.354424736 +0000 UTC m=+0.040481290 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:47:41.792 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:41 smithi042 podman[92895]: 2024-04-20 10:47:41.504664568 +0000 UTC m=+0.190721159 container create 8f0cb368cd79e1140c7f751cb3cdc8f583e00241b8a4159abe1dcbe12c3b0ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:47:41.792 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:41 smithi042 podman[92895]: 2024-04-20 10:47:41.791641041 +0000 UTC m=+0.477697639 container init 8f0cb368cd79e1140c7f751cb3cdc8f583e00241b8a4159abe1dcbe12c3b0ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:47:42.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:41 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:41.687+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:42.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:41 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:41.687+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:42.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:41 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:41.687+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:42.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:41 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:41.703+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:42.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:41 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:41.703+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:42.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:41 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:41.703+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:42.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:41 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:41.703+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:42.103 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:41 smithi042 podman[92895]: 2024-04-20 10:47:41.805012075 +0000 UTC m=+0.491068633 container start 8f0cb368cd79e1140c7f751cb3cdc8f583e00241b8a4159abe1dcbe12c3b0ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-20T10:47:42.103 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:41 smithi042 podman[92895]: 2024-04-20 10:47:41.882485117 +0000 UTC m=+0.568541693 container attach 8f0cb368cd79e1140c7f751cb3cdc8f583e00241b8a4159abe1dcbe12c3b0ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-20T10:47:42.103 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:41 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:41.940+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:42.104 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:41 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:41.940+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:42.104 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:41 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:41.940+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:42.104 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:41 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:41.940+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:42.352 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:42 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:42.093+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:42.353 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:42 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:42.093+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:42.353 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:42 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:42.093+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:42.603 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 20 10:47:42 smithi042 podman[92895]: 2024-04-20 10:47:42.273674156 +0000 UTC m=+0.959730784 container died 8f0cb368cd79e1140c7f751cb3cdc8f583e00241b8a4159abe1dcbe12c3b0ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-20T10:47:42.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:42 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:42.352+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:42.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:42 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:42.352+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:42.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:42 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:42.352+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:43.045 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:42 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:42.912+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:43.046 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:42 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:42.912+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:43.046 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:42 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:42.912+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:43.046 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:42 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:42.912+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:43.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:42 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:42.699+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:43.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:42 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:42.699+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:43.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:42 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:42.699+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:43.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:42 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:42.691+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:43.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:42 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:42.691+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:43.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:42 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:42.691+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:43.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:42 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:42.691+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:43.098 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.6.service' 2024-04-20T10:47:43.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:47:43.221 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-20T10:47:43.221 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-20T10:47:43.221 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.7 2024-04-20T10:47:43.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:43 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:43.060+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:43.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:43 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:43.060+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:43.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:43 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:43.060+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:43.553 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:43 smithi046 systemd[1]: Stopping Ceph osd.7 for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:47:43.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:43 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:43.402+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:43.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:43 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:43.402+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:43.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:43 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:43.402+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:43.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:43 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:43.402+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:44.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:43 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:43.690+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:44.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:43 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:43.690+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:44.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:43 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:43.690+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:44.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:43 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:43.690+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:44.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:43 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:43.566+0000 7fca93c43700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:47:44.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:43 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:43.566+0000 7fca93c43700 -1 osd.7 103 *** Got signal Terminated *** 2024-04-20T10:47:44.055 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:43 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:43.566+0000 7fca93c43700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T10:47:44.055 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:43 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:43.656+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:44.055 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:43 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:43.656+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:44.055 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:43 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:43.656+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:44.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:43 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:43.945+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:44.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:43 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:43.945+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:44.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:43 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:43.945+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:44.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:43 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:43.945+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:44.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:43 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:43.945+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:44.362 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:44 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:44.097+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:44.363 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:44 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:44.097+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:44.363 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:44 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:44.097+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:44.363 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:44 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:44.097+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:44.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:44 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:44.362+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:44.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:44 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:44.362+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:44.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:44 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:44.362+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:44.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:44 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:44.362+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:45.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:44 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:44.614+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:45.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:44 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:44.614+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:45.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:44 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:44.614+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:45.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:44 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:44.707+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:45.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:44 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:44.707+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:45.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:44 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:44.707+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:45.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:44 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:44.707+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:45.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:44 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:44.707+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:45.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:44 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:44.917+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:45.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:44 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:44.917+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:45.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:44 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:44.917+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:45.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:44 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:44.917+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:45.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:44 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:44.917+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:45.399 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:45 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:45.122+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:45.399 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:45 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:45.122+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:45.399 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:45 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:45.122+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:45.399 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:45 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:45.122+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:45.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:45 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:45.399+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:45.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:45 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:45.399+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:45.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:45 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:45.399+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:45.651 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:45 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:45.399+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:46.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:45 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:45.578+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:46.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:45 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:45.578+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:46.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:45 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:45.578+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:46.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:45 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:45.578+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:21.449557+0000 front 2024-04-20T10:47:21.449609+0000 (oldest deadline 2024-04-20T10:47:45.549748+0000) 2024-04-20T10:47:46.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:45 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:45.678+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:46.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:45 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:45.678+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:46.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:45 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:45.678+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:46.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:45 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:45.678+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:46.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:45 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:45.678+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:46.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:45 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:45.926+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:46.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:45 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:45.926+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:46.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:45 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:45.926+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:46.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:45 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:45.926+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:46.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:45 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:45.926+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:46.401 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:46 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:46.099+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:46.401 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:46 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:46.099+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:46.401 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:46 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:46.099+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:46.402 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:46 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:46.099+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:46.899 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:46 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:46.432+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:46.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:46 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:46.432+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:46.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:46 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:46.432+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:46.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:46 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:46.432+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:47.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:46 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:46.591+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:47.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:46 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:46.591+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:47.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:46 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:46.591+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:47.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:46 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:46.591+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:21.449557+0000 front 2024-04-20T10:47:21.449609+0000 (oldest deadline 2024-04-20T10:47:45.549748+0000) 2024-04-20T10:47:47.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:46 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:46.643+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:47.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:46 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:46.643+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:47.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:46 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:46.643+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:47.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:46 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:46.643+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:47.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:46 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:46.643+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:47.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:46 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:46.931+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:47.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:46 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:46.931+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:47.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:46 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:46.931+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:47.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:46 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:46.931+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:47.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:46 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:46.931+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:47.399 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:47 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:47.068+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:47.399 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:47 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:47.068+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:47.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:47 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:47.068+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:47.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:47 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:47.068+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:47.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:47 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:47.406+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:47.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:47 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:47.406+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:47.901 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:47 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:47.406+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:47.901 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:47 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:47.406+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:48.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:47 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:47.596+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:53.544456+0000 front 2024-04-20T10:46:53.544582+0000 (oldest deadline 2024-04-20T10:47:19.444578+0000) 2024-04-20T10:47:48.053 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:47 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:47.596+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.745625+0000 front 2024-04-20T10:47:04.745503+0000 (oldest deadline 2024-04-20T10:47:25.746202+0000) 2024-04-20T10:47:48.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:47 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:47.596+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.447376+0000 front 2024-04-20T10:47:10.447254+0000 (oldest deadline 2024-04-20T10:47:33.947317+0000) 2024-04-20T10:47:48.054 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:47 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7[39868]: 2024-04-20T10:47:47.596+0000 7fca8c22f700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:21.449557+0000 front 2024-04-20T10:47:21.449609+0000 (oldest deadline 2024-04-20T10:47:45.549748+0000) 2024-04-20T10:47:48.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:47 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:47.629+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:48.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:47 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:47.629+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:48.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:47 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:47.629+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:48.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:47 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:47.629+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:48.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:47 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:47.629+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:48.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:47 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:47.974+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:48.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:47 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:47.974+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:48.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:47 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:47.974+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:48.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:47 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:47.974+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:48.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:47 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:47.974+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:48.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:48 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:48.044+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:48.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:48 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:48.044+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:48.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:48 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:48.044+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:48.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:48 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:48.044+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:48.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:48 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:48.411+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:48.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:48 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:48.411+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:48.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:48 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:48.411+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:48.901 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:48 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:48.411+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:49.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:48 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:48.652+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:49.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:48 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:48.652+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:49.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:48 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:48.652+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:49.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:48 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:48.652+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:49.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:48 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:48.652+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:49.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:49 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:49.002+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:49.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:49 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:49.002+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:49.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:49 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:49.002+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:49.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:49 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:49.002+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:49.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:49 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:49.002+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:49.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:49 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:49.004+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:49.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:49 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:49.004+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:49.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:49 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:49.004+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:49.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:49 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:49.004+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:49.433 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:49 smithi046 podman[58377]: 2024-04-20 10:47:49.146167358 +0000 UTC m=+5.712459910 container died 3d5ff6106cdd24426cf251a5ae466013080bf0b2203d557c783a42bc31c65bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-20T10:47:49.803 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:49 smithi046 podman[58377]: 2024-04-20 10:47:49.549489749 +0000 UTC m=+6.115782297 container cleanup 3d5ff6106cdd24426cf251a5ae466013080bf0b2203d557c783a42bc31c65bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-20T10:47:49.803 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:49 smithi046 bash[58377]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7 2024-04-20T10:47:49.804 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:49 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:49.640+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:49.804 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:49 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:49.640+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:49.804 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:49 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:49.640+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:49.804 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:49 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:49.640+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:49.804 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:49 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:49.640+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:49.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:49 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:49.409+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:49.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:49 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:49.409+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:49.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:49 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:49.409+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:49.901 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:49 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:49.409+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:50.139 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:49 smithi046 podman[58394]: 2024-04-20 10:47:49.80891107 +0000 UTC m=+0.659146060 container remove 3d5ff6106cdd24426cf251a5ae466013080bf0b2203d557c783a42bc31c65bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-20T10:47:50.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:50.012+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:50.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:50.012+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:50.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:50.012+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:50.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:50.012+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:50.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:50.012+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:50.366 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:50 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:50.028+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:50.366 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:50 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:50.028+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:50.366 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:50 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:50.028+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:50.367 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:50 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:50.028+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:50.403 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:50 smithi046 podman[58489]: 2024-04-20 10:47:50.042805782 +0000 UTC m=+0.018156014 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:47:50.403 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:50 smithi046 podman[58489]: 2024-04-20 10:47:50.167977649 +0000 UTC m=+0.143327868 container create 84198a8d6bd6e12eb1849426659a5b0a97a46737fb854b6ec9bfee0a341c34f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-20T10:47:50.404 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:50 smithi046 podman[58489]: 2024-04-20 10:47:50.404043991 +0000 UTC m=+0.379394213 container init 84198a8d6bd6e12eb1849426659a5b0a97a46737fb854b6ec9bfee0a341c34f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-20T10:47:50.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:50 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:50.366+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:50.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:50 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:50.366+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:50.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:50 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:50.366+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:50.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:50 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:50.366+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:50.684 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:50 smithi046 podman[58489]: 2024-04-20 10:47:50.407514014 +0000 UTC m=+0.382864243 container start 84198a8d6bd6e12eb1849426659a5b0a97a46737fb854b6ec9bfee0a341c34f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-20T10:47:50.684 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:50 smithi046 podman[58489]: 2024-04-20 10:47:50.487269909 +0000 UTC m=+0.462620147 container attach 84198a8d6bd6e12eb1849426659a5b0a97a46737fb854b6ec9bfee0a341c34f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=) 2024-04-20T10:47:50.684 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:50 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:50.647+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:50.684 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:50 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:50.647+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:50.684 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:50 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:50.647+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:50.685 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:50 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:50.647+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:50.685 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:50 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:50.647+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:50.997 INFO:journalctl@ceph.osd.7.smithi046.stdout:Apr 20 10:47:50 smithi046 podman[58489]: 2024-04-20 10:47:50.711401171 +0000 UTC m=+0.686751383 container died 84198a8d6bd6e12eb1849426659a5b0a97a46737fb854b6ec9bfee0a341c34f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-7-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-20T10:47:51.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.7.service' 2024-04-20T10:47:51.338 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:51.016+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:51.339 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:51.016+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:51.339 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:51.016+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:51.339 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:51.016+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:51.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:50.962+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:51.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:50.962+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:51.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:50.962+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:51.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:50.962+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:51.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:50 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:50.962+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:51.357 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:47:51.358 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-20T10:47:51.358 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-20T10:47:51.358 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.8 2024-04-20T10:47:51.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:51.339+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:51.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:51.339+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:51.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:51.339+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:51.651 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:51.339+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:51.651 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:51 smithi162 systemd[1]: Stopping Ceph osd.8 for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:47:51.803 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:51 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:51.683+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:51.803 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:51 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:51.683+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:51.804 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:51 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:51.683+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:51.804 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:51 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:51.683+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:51.804 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:51 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:51.683+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:51.988 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:51.687+0000 7f5d6283d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:47:51.988 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:51.687+0000 7f5d6283d700 -1 osd.8 103 *** Got signal Terminated *** 2024-04-20T10:47:51.988 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:51.687+0000 7f5d6283d700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T10:47:52.304 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:51.988+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:52.304 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:51.988+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:52.304 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:51.988+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:52.305 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:51 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:51.988+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:52.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:51 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:51.989+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:52.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:51 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:51.989+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:52.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:51 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:51.989+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:52.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:51 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:51.989+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:52.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:51 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:51.989+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:52.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:52.304+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:52.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:52.304+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:52.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:52.304+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:52.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:52.304+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:52.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:52 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:52.304+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:25.856935+0000 front 2024-04-20T10:47:25.856974+0000 (oldest deadline 2024-04-20T10:47:51.757185+0000) 2024-04-20T10:47:52.803 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:52 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:52.640+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:52.803 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:52 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:52.640+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:52.804 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:52 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:52.640+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:52.804 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:52 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:52.640+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:52.804 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:52 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:52.640+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:53.276 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:53.012+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:53.276 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:53.012+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:53.277 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:53.012+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:53.277 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:53.012+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:53.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:53 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:53.030+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:53.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:53 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:53.030+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:53.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:53 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:53.030+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:53.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:53 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:53.030+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:53.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:53 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:53.030+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:53.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:53.276+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:53.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:53.276+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:53.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:53.276+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:53.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:53.276+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:53.651 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:53.276+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:25.856935+0000 front 2024-04-20T10:47:25.856974+0000 (oldest deadline 2024-04-20T10:47:51.757185+0000) 2024-04-20T10:47:54.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:53.663+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:54.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:53.663+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:54.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:53.663+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:54.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:53.663+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:54.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:53.663+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:54.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:53.663+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:47:54.240 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:53.977+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:54.240 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:53.977+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:54.240 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:53.977+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:54.240 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:53 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:53.977+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:54.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:53.999+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:54.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:53.999+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:54.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:53.999+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:54.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:53.999+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:54.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:53.999+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:54.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:53.999+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:29.008690+0000 front 2024-04-20T10:47:29.008790+0000 (oldest deadline 2024-04-20T10:47:53.708508+0000) 2024-04-20T10:47:54.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:54.240+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:54.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:54.240+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:54.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:54.240+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:54.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:54.240+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:54.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:54.240+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:25.856935+0000 front 2024-04-20T10:47:25.856974+0000 (oldest deadline 2024-04-20T10:47:51.757185+0000) 2024-04-20T10:47:55.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:54 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:54.691+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:55.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:54 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:54.691+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:55.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:54 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:54.691+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:55.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:54 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:54.691+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:55.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:54 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:54.691+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:55.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:54 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:54.691+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:47:55.289 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:54.956+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:55.290 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:54.956+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:55.290 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:54.956+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:55.290 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:54 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:54.956+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:55.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:54.996+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:55.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:54.996+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:55.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:54.996+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:55.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:54.996+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:55.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:54.996+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:55.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:54 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:54.996+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:29.008690+0000 front 2024-04-20T10:47:29.008790+0000 (oldest deadline 2024-04-20T10:47:53.708508+0000) 2024-04-20T10:47:55.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:55.290+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:55.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:55.290+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:55.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:55.290+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:55.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:55.290+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:55.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:55.290+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:25.856935+0000 front 2024-04-20T10:47:25.856974+0000 (oldest deadline 2024-04-20T10:47:51.757185+0000) 2024-04-20T10:47:56.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:55.719+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:56.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:55.719+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:56.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:55.719+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:56.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:55.719+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:56.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:55.719+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:56.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:55 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:55.719+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:47:56.335 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:55.944+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:56.335 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:55.944+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:56.336 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:55.944+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:56.336 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:55.944+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:56.336 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:55 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:55.944+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:47:56.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:56 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:56.033+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:56.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:56 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:56.033+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:56.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:56 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:56.033+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:56.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:56 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:56.033+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:56.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:56 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:56.033+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:56.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:56 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:56.033+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:29.008690+0000 front 2024-04-20T10:47:29.008790+0000 (oldest deadline 2024-04-20T10:47:53.708508+0000) 2024-04-20T10:47:56.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:56.335+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:55.551703+0000 front 2024-04-20T10:46:55.552044+0000 (oldest deadline 2024-04-20T10:47:21.952503+0000) 2024-04-20T10:47:56.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:56.335+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.953445+0000 front 2024-04-20T10:47:01.953212+0000 (oldest deadline 2024-04-20T10:47:27.853313+0000) 2024-04-20T10:47:56.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:56.335+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:12.554723+0000 front 2024-04-20T10:47:12.554836+0000 (oldest deadline 2024-04-20T10:47:39.555132+0000) 2024-04-20T10:47:56.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:56.335+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:19.555637+0000 front 2024-04-20T10:47:19.555778+0000 (oldest deadline 2024-04-20T10:47:43.055577+0000) 2024-04-20T10:47:56.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8[40665]: 2024-04-20T10:47:56.335+0000 7f5d5ae29700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:25.856935+0000 front 2024-04-20T10:47:25.856974+0000 (oldest deadline 2024-04-20T10:47:51.757185+0000) 2024-04-20T10:47:57.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:56 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:56.728+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:57.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:56 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:56.728+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:57.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:56 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:56.728+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:57.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:56 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:56.728+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:57.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:56 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:56.728+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:57.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:56 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:56.728+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:47:57.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:57 smithi162 podman[59174]: 2024-04-20 10:47:57.123381686 +0000 UTC m=+5.568525851 container died 9864be4258293c5a70bf3b8878f04c58fa8c3f2b18be47a018ebc2ba969fcd20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-20T10:47:57.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:56.899+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:57.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:56.899+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:57.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:56.899+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:57.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:56.899+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:57.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:56.899+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:47:57.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:57 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:57.043+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:57.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:57 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:57.043+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:57.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:57 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:57.043+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:57.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:57 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:57.043+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:57.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:57 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:57.043+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:57.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:57 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:57.043+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:29.008690+0000 front 2024-04-20T10:47:29.008790+0000 (oldest deadline 2024-04-20T10:47:53.708508+0000) 2024-04-20T10:47:57.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:57 smithi162 podman[59174]: 2024-04-20 10:47:57.471004407 +0000 UTC m=+5.916148568 container cleanup 9864be4258293c5a70bf3b8878f04c58fa8c3f2b18be47a018ebc2ba969fcd20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-20T10:47:57.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:57 smithi162 bash[59174]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8 2024-04-20T10:47:58.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:57.727+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:58.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:57.727+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:58.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:57.727+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:58.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:57.727+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:58.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:57.727+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:58.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:57 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:57.727+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:47:58.072 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:57 smithi162 podman[59191]: 2024-04-20 10:47:57.772323345 +0000 UTC m=+0.648633449 container remove 9864be4258293c5a70bf3b8878f04c58fa8c3f2b18be47a018ebc2ba969fcd20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-20T10:47:58.072 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:57.941+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:58.072 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:57.941+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:58.072 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:57.941+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:58.073 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:57.941+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:58.073 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:57 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:57.941+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:47:58.342 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:58 smithi162 podman[59282]: 2024-04-20 10:47:57.975845057 +0000 UTC m=+0.018210403 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:47:58.342 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:58 smithi162 podman[59282]: 2024-04-20 10:47:58.103048996 +0000 UTC m=+0.145414331 container create fd6ea39d1e9789b224c1c47b55ef6608e88dbf499b14538f9ac19b08dfe2942c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-20T10:47:58.342 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:58 smithi162 podman[59282]: 2024-04-20 10:47:58.342705881 +0000 UTC m=+0.385071216 container init fd6ea39d1e9789b224c1c47b55ef6608e88dbf499b14538f9ac19b08dfe2942c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-20T10:47:58.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:58.004+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:58.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:58.004+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:58.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:58.004+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:58.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:58.004+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:58.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:58.004+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:58.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:58.004+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:29.008690+0000 front 2024-04-20T10:47:29.008790+0000 (oldest deadline 2024-04-20T10:47:53.708508+0000) 2024-04-20T10:47:58.633 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:58 smithi162 podman[59282]: 2024-04-20 10:47:58.346038285 +0000 UTC m=+0.388403627 container start fd6ea39d1e9789b224c1c47b55ef6608e88dbf499b14538f9ac19b08dfe2942c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T10:47:58.633 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:58 smithi162 podman[59282]: 2024-04-20 10:47:58.425953132 +0000 UTC m=+0.468318477 container attach fd6ea39d1e9789b224c1c47b55ef6608e88dbf499b14538f9ac19b08dfe2942c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-20T10:47:58.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:58 smithi162 podman[59282]: 2024-04-20 10:47:58.634053258 +0000 UTC m=+0.676418601 container died fd6ea39d1e9789b224c1c47b55ef6608e88dbf499b14538f9ac19b08dfe2942c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-20T10:47:59.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:58 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:58.701+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:47:59.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:58 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:58.701+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:47:59.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:58 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:58.701+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:47:59.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:58 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:58.701+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:47:59.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:58 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:58.701+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:47:59.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:58 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:58.701+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:47:59.146 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.8.service' 2024-04-20T10:47:59.169 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:59 smithi162 podman[59308]: 2024-04-20 10:47:59.129564525 +0000 UTC m=+0.490016537 container remove fd6ea39d1e9789b224c1c47b55ef6608e88dbf499b14538f9ac19b08dfe2942c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-04-20T10:47:59.169 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:59 smithi162 systemd[1]: ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.8.service: Deactivated successfully. 2024-04-20T10:47:59.169 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:59 smithi162 systemd[1]: Stopped Ceph osd.8 for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:47:59.169 INFO:journalctl@ceph.osd.8.smithi162.stdout:Apr 20 10:47:59 smithi162 systemd[1]: ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.8.service: Consumed 1min 35.820s CPU time. 2024-04-20T10:47:59.170 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:58 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:58.965+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:47:59.170 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:58 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:58.965+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:47:59.170 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:58 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:58.965+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:47:59.170 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:58 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:58.965+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:47:59.170 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:58 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:58.965+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:47:59.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:47:59.186 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-20T10:47:59.186 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-20T10:47:59.186 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.9 2024-04-20T10:47:59.267 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:58.959+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:47:59.267 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:58.959+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:47:59.267 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:58.959+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:47:59.267 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:58.959+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:47:59.267 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:58.959+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:47:59.268 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:58 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:58.959+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:29.008690+0000 front 2024-04-20T10:47:29.008790+0000 (oldest deadline 2024-04-20T10:47:53.708508+0000) 2024-04-20T10:47:59.603 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:59 smithi042 systemd[1]: Stopping Ceph osd.9 for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:48:00.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:59 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:59.719+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:00.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:59 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:59.719+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:00.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:59 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:59.719+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:00.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:59 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:59.719+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:00.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:59 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:59.719+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:00.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:47:59 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:47:59.719+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:00.104 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:59 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:59.795+0000 7fb6e5c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:48:00.104 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:59 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:59.795+0000 7fb6e5c00700 -1 osd.9 103 *** Got signal Terminated *** 2024-04-20T10:48:00.104 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:47:59 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:47:59.795+0000 7fb6e5c00700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T10:48:00.104 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:00.001+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:48:00.105 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:00.001+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:48:00.105 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:00.001+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:48:00.105 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:00.001+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:48:00.105 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:00.001+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:48:00.105 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:00 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:00.001+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:29.008690+0000 front 2024-04-20T10:47:29.008790+0000 (oldest deadline 2024-04-20T10:47:53.708508+0000) 2024-04-20T10:48:00.323 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:59 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:59.915+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:00.323 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:59 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:59.915+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:00.324 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:59 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:59.915+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:00.324 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:59 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:59.915+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:00.324 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:47:59 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:47:59.915+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:01.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:00 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:00.725+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:01.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:00 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:00.725+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:01.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:00 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:00.725+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:01.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:00 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:00.725+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:01.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:00 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:00.725+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:01.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:00 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:00.725+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:01.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:00 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:00.866+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:01.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:00 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:00.866+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:01.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:00 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:00.866+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:01.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:00 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:00.866+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:01.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:00 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:00.867+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:01.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:01.017+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:48:01.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:01.017+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:48:01.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:01.017+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:48:01.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:01.017+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:48:01.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:01.017+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:48:01.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:01.017+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:29.008690+0000 front 2024-04-20T10:47:29.008790+0000 (oldest deadline 2024-04-20T10:47:53.708508+0000) 2024-04-20T10:48:01.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:01 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:01.017+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:39.010612+0000 front 2024-04-20T10:47:39.010757+0000 (oldest deadline 2024-04-20T10:48:00.710444+0000) 2024-04-20T10:48:02.007 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:01 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:01.696+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:02.007 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:01 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:01.696+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:02.008 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:01 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:01.696+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:02.008 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:01 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:01.696+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:02.008 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:01 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:01.696+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:02.008 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:01 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:01.696+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:02.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:01 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:01.859+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:02.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:01 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:01.859+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:02.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:01 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:01.859+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:02.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:01 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:01.859+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:02.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:01 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:01.859+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:02.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:02 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:02.057+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:48:02.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:02 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:02.057+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:48:02.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:02 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:02.057+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:48:02.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:02 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:02.057+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:48:02.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:02 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:02.057+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:48:02.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:02 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:02.057+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:29.008690+0000 front 2024-04-20T10:47:29.008790+0000 (oldest deadline 2024-04-20T10:47:53.708508+0000) 2024-04-20T10:48:02.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:02 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:02.057+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:39.010612+0000 front 2024-04-20T10:47:39.010757+0000 (oldest deadline 2024-04-20T10:48:00.710444+0000) 2024-04-20T10:48:03.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:02 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:02.710+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:03.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:02 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:02.710+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:03.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:02 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:02.710+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:03.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:02 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:02.710+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:03.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:02 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:02.710+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:03.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:02 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:02.710+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:03.149 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:02 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:02.844+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:03.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:02 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:02.844+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:03.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:02 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:02.844+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:03.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:02 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:02.844+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:03.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:02 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:02.844+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:03.604 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:03 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:03.104+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:48:03.604 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:03 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:03.104+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:48:03.604 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:03 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:03.104+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:48:03.604 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:03 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:03.104+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:48:03.604 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:03 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:03.104+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:48:03.605 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:03 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:03.104+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:29.008690+0000 front 2024-04-20T10:47:29.008790+0000 (oldest deadline 2024-04-20T10:47:53.708508+0000) 2024-04-20T10:48:03.605 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:03 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:03.104+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:39.010612+0000 front 2024-04-20T10:47:39.010757+0000 (oldest deadline 2024-04-20T10:48:00.710444+0000) 2024-04-20T10:48:04.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:03 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:03.663+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:04.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:03 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:03.663+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:04.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:03 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:03.663+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:04.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:03 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:03.663+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:04.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:03 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:03.663+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:04.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:03 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:03.663+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:04.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:03 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:03.856+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:04.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:03 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:03.856+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:04.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:03 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:03.856+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:04.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:03 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:03.856+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:04.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:03 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:03.856+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:04.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:03 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:03.856+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:04.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:04 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:04.076+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:41.593607+0000 front 2024-04-20T10:46:41.594238+0000 (oldest deadline 2024-04-20T10:47:07.493298+0000) 2024-04-20T10:48:04.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:04 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:04.076+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:54.398314+0000 front 2024-04-20T10:46:54.398232+0000 (oldest deadline 2024-04-20T10:47:17.898473+0000) 2024-04-20T10:48:04.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:04 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:04.076+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:01.301639+0000 front 2024-04-20T10:47:01.301729+0000 (oldest deadline 2024-04-20T10:47:26.001829+0000) 2024-04-20T10:48:04.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:04 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:04.076+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:11.305364+0000 front 2024-04-20T10:47:11.304658+0000 (oldest deadline 2024-04-20T10:47:37.204977+0000) 2024-04-20T10:48:04.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:04 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:04.076+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.206055+0000 front 2024-04-20T10:47:17.206168+0000 (oldest deadline 2024-04-20T10:47:43.106331+0000) 2024-04-20T10:48:04.354 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:04 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:04.076+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:29.008690+0000 front 2024-04-20T10:47:29.008790+0000 (oldest deadline 2024-04-20T10:47:53.708508+0000) 2024-04-20T10:48:04.355 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:04 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9[51943]: 2024-04-20T10:48:04.076+0000 7fb6dc400700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:39.010612+0000 front 2024-04-20T10:47:39.010757+0000 (oldest deadline 2024-04-20T10:48:00.710444+0000) 2024-04-20T10:48:05.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:04 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:04.643+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:05.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:04 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:04.643+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:05.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:04 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:04.643+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:05.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:04 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:04.643+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:05.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:04 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:04.643+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:05.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:04 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:04.643+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:05.055 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:04 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:04.643+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:38.391380+0000 front 2024-04-20T10:47:38.391342+0000 (oldest deadline 2024-04-20T10:48:04.291267+0000) 2024-04-20T10:48:05.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:04 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:04.862+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:05.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:04 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:04.862+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:05.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:04 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:04.862+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:05.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:04 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:04.862+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:05.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:04 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:04.862+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:05.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:04 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:04.862+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:05.753 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:05 smithi042 podman[92999]: 2024-04-20 10:48:05.472889254 +0000 UTC m=+5.802824399 container died 9528fd875ce11cf28119c42100db9e410b1a90e222a97437aebe2c3706a012c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-20T10:48:06.040 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:05 smithi042 podman[92999]: 2024-04-20 10:48:05.864028032 +0000 UTC m=+6.193963208 container cleanup 9528fd875ce11cf28119c42100db9e410b1a90e222a97437aebe2c3706a012c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-20T10:48:06.041 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:05 smithi042 bash[92999]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9 2024-04-20T10:48:06.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:05 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:05.650+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:06.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:05 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:05.650+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:06.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:05 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:05.650+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:06.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:05 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:05.650+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:06.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:05 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:05.650+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:06.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:05 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:05.650+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:06.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:05 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:05.650+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:38.391380+0000 front 2024-04-20T10:47:38.391342+0000 (oldest deadline 2024-04-20T10:48:04.291267+0000) 2024-04-20T10:48:06.149 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:05 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:05.881+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:06.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:05 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:05.881+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:06.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:05 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:05.881+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:06.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:05 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:05.881+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:06.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:05 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:05.882+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:06.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:05 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:05.882+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:06.353 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:06 smithi042 podman[93020]: 2024-04-20 10:48:06.040102646 +0000 UTC m=+0.557114132 container remove 9528fd875ce11cf28119c42100db9e410b1a90e222a97437aebe2c3706a012c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-20T10:48:07.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:06 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:06.665+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:07.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:06 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:06.665+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:07.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:06 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:06.665+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:07.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:06 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:06.665+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:07.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:06 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:06.665+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:07.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:06 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:06.665+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:07.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:06 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:06.665+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:38.391380+0000 front 2024-04-20T10:47:38.391342+0000 (oldest deadline 2024-04-20T10:48:04.291267+0000) 2024-04-20T10:48:07.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:06 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:06.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:07.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:06 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:06.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:07.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:06 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:06.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:07.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:06 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:06.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:07.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:06 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:06.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:07.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:06 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:06.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:07.161 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:06 smithi042 podman[93108]: 2024-04-20 10:48:06.793147133 +0000 UTC m=+0.037201104 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:48:07.162 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:06 smithi042 podman[93108]: 2024-04-20 10:48:06.891129402 +0000 UTC m=+0.135183408 container create f6cc2b33933231bdf7f617276fe0a87b8a15523e7111ca9f89da8a952bde1982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-20T10:48:07.162 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:07 smithi042 podman[93108]: 2024-04-20 10:48:07.161565114 +0000 UTC m=+0.405619163 container init f6cc2b33933231bdf7f617276fe0a87b8a15523e7111ca9f89da8a952bde1982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-20T10:48:07.431 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:07 smithi042 podman[93108]: 2024-04-20 10:48:07.172892052 +0000 UTC m=+0.416946031 container start f6cc2b33933231bdf7f617276fe0a87b8a15523e7111ca9f89da8a952bde1982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-20T10:48:07.431 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:07 smithi042 podman[93108]: 2024-04-20 10:48:07.251672302 +0000 UTC m=+0.495726321 container attach f6cc2b33933231bdf7f617276fe0a87b8a15523e7111ca9f89da8a952bde1982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-20T10:48:07.745 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 20 10:48:07 smithi042 podman[93108]: 2024-04-20 10:48:07.645565196 +0000 UTC m=+0.889619206 container died f6cc2b33933231bdf7f617276fe0a87b8a15523e7111ca9f89da8a952bde1982 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-20T10:48:08.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:07 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:07.688+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:08.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:07 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:07.688+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:08.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:07 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:07.688+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:08.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:07 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:07.688+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:08.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:07 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:07.688+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:08.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:07 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:07.688+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:08.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:07 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:07.688+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:38.391380+0000 front 2024-04-20T10:47:38.391342+0000 (oldest deadline 2024-04-20T10:48:04.291267+0000) 2024-04-20T10:48:08.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:07 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:07.832+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:08.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:07 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:07.833+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:08.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:07 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:07.833+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:08.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:07 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:07.833+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:08.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:07 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:07.833+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:08.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:07 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:07.833+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:08.333 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.9.service' 2024-04-20T10:48:08.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:48:08.433 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-20T10:48:08.433 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-20T10:48:08.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.10 2024-04-20T10:48:08.726 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:08 smithi046 systemd[1]: Stopping Ceph osd.10 for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:48:08.727 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:08 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:08.724+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:08.728 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:08 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:08.724+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:08.728 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:08 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:08.724+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:08.728 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:08 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:08.724+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:08.728 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:08 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:08.724+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:08.728 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:08 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:08.724+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:08.728 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:08 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:08.724+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:38.391380+0000 front 2024-04-20T10:47:38.391342+0000 (oldest deadline 2024-04-20T10:48:04.291267+0000) 2024-04-20T10:48:09.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:08 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:08.758+0000 7faf1d3d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:48:09.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:08 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:08.758+0000 7faf1d3d2700 -1 osd.10 103 *** Got signal Terminated *** 2024-04-20T10:48:09.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:08 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:08.758+0000 7faf1d3d2700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T10:48:09.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:08 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:08.876+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:09.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:08 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:08.876+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:09.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:08 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:08.876+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:09.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:08 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:08.876+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:09.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:08 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:08.876+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:09.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:08 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:08.876+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:10.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:09 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:09.769+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:10.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:09 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:09.769+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:10.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:09 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:09.769+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:10.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:09 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:09.769+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:10.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:09 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:09.769+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:10.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:09 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:09.769+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:10.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:09 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:09.769+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:38.391380+0000 front 2024-04-20T10:47:38.391342+0000 (oldest deadline 2024-04-20T10:48:04.291267+0000) 2024-04-20T10:48:10.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:09 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:09.874+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:10.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:09 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:09.874+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:10.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:09 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:09.874+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:10.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:09 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:09.874+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:10.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:09 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:09.874+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:10.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:09 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:09.874+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:10.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:10 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:10.835+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:10.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:10 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:10.835+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:10.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:10 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:10.835+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:10.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:10 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:10.835+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:10.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:10 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:10.835+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:10.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:10 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:10.835+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:11.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:10 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:10.783+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:11.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:10 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:10.783+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:11.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:10 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:10.783+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:11.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:10 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:10.783+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:11.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:10 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:10.783+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:11.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:10 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:10.783+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:11.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:10 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:10.783+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:38.391380+0000 front 2024-04-20T10:47:38.391342+0000 (oldest deadline 2024-04-20T10:48:04.291267+0000) 2024-04-20T10:48:12.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:11 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:11.847+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:12.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:11 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:11.847+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:12.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:11 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:11.847+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:12.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:11 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:11.847+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:12.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:11 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:11.847+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:12.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:11 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:11.847+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:12.303 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:11 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:11.810+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:12.304 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:11 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:11.810+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:12.304 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:11 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:11.810+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:12.304 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:11 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:11.810+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:12.304 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:11 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:11.810+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:12.304 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:11 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:11.810+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:12.304 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:11 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:11.810+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:38.391380+0000 front 2024-04-20T10:47:38.391342+0000 (oldest deadline 2024-04-20T10:48:04.291267+0000) 2024-04-20T10:48:13.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:12 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:12.771+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:13.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:12 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:12.771+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:13.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:12 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:12.771+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:13.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:12 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:12.771+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:13.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:12 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:12.771+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:13.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:12 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:12.771+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:13.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:12 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:12.771+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:38.391380+0000 front 2024-04-20T10:47:38.391342+0000 (oldest deadline 2024-04-20T10:48:04.291267+0000) 2024-04-20T10:48:13.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:12 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:12.802+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:13.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:12 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:12.802+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:13.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:12 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:12.802+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:13.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:12 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:12.802+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:13.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:12 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:12.802+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:13.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:12 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:12.802+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:13.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:12 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:12.802+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6820 osd.7 since back 2024-04-20T10:47:46.768215+0000 front 2024-04-20T10:47:46.768281+0000 (oldest deadline 2024-04-20T10:48:12.668222+0000) 2024-04-20T10:48:14.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:13 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:13.726+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.881011+0000 front 2024-04-20T10:46:44.880981+0000 (oldest deadline 2024-04-20T10:47:09.580868+0000) 2024-04-20T10:48:14.053 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:13 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:13.726+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:51.882080+0000 front 2024-04-20T10:46:51.882209+0000 (oldest deadline 2024-04-20T10:47:17.782185+0000) 2024-04-20T10:48:14.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:13 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:13.726+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:03.583976+0000 front 2024-04-20T10:47:03.584002+0000 (oldest deadline 2024-04-20T10:47:29.483918+0000) 2024-04-20T10:48:14.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:13 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:13.726+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-20T10:47:10.585090+0000 front 2024-04-20T10:47:10.584925+0000 (oldest deadline 2024-04-20T10:47:35.885022+0000) 2024-04-20T10:48:14.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:13 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:13.726+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:22.187443+0000 front 2024-04-20T10:47:22.187470+0000 (oldest deadline 2024-04-20T10:47:44.487580+0000) 2024-04-20T10:48:14.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:13 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:13.726+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:28.489563+0000 front 2024-04-20T10:47:28.489488+0000 (oldest deadline 2024-04-20T10:47:53.189550+0000) 2024-04-20T10:48:14.054 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:13 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10[46511]: 2024-04-20T10:48:13.726+0000 7faf159be700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:38.391380+0000 front 2024-04-20T10:47:38.391342+0000 (oldest deadline 2024-04-20T10:48:04.291267+0000) 2024-04-20T10:48:14.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:13 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:13.824+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:14.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:13 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:13.824+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:14.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:13 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:13.824+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:14.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:13 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:13.824+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:14.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:13 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:13.824+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:14.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:13 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:13.824+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:14.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:13 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:13.824+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6820 osd.7 since back 2024-04-20T10:47:46.768215+0000 front 2024-04-20T10:47:46.768281+0000 (oldest deadline 2024-04-20T10:48:12.668222+0000) 2024-04-20T10:48:14.518 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:14 smithi046 podman[58608]: 2024-04-20 10:48:14.216866372 +0000 UTC m=+5.578112039 container died 7ceaef752129368158568807b30199a0b291dfa150037e9f5d5bcd84db0f3442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-20T10:48:14.518 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:14 smithi046 podman[58608]: 2024-04-20 10:48:14.518268096 +0000 UTC m=+5.879513752 container cleanup 7ceaef752129368158568807b30199a0b291dfa150037e9f5d5bcd84db0f3442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:48:14.803 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:14 smithi046 bash[58608]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10 2024-04-20T10:48:14.803 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:14 smithi046 podman[58640]: 2024-04-20 10:48:14.760954549 +0000 UTC m=+0.543162997 container remove 7ceaef752129368158568807b30199a0b291dfa150037e9f5d5bcd84db0f3442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T10:48:15.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:14 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:14.840+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:15.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:14 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:14.840+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:15.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:14 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:14.840+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:15.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:14 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:14.840+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:15.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:14 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:14.840+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:15.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:14 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:14.840+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:15.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:14 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:14.840+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6820 osd.7 since back 2024-04-20T10:47:46.768215+0000 front 2024-04-20T10:47:46.768281+0000 (oldest deadline 2024-04-20T10:48:12.668222+0000) 2024-04-20T10:48:15.329 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:15 smithi046 podman[58728]: 2024-04-20 10:48:14.963288362 +0000 UTC m=+0.018470723 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:48:15.330 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:15 smithi046 podman[58728]: 2024-04-20 10:48:15.069951752 +0000 UTC m=+0.125134104 container create 914bffc785c4852404f1e5ec2f4e803fd28d6884dae9cf5d8daccf67d476fadd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10-deactivate, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:48:15.330 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:15 smithi046 podman[58728]: 2024-04-20 10:48:15.247187667 +0000 UTC m=+0.302370017 container init 914bffc785c4852404f1e5ec2f4e803fd28d6884dae9cf5d8daccf67d476fadd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-20T10:48:15.330 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:15 smithi046 podman[58728]: 2024-04-20 10:48:15.25042607 +0000 UTC m=+0.305608424 container start 914bffc785c4852404f1e5ec2f4e803fd28d6884dae9cf5d8daccf67d476fadd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-20T10:48:15.330 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:15 smithi046 podman[58728]: 2024-04-20 10:48:15.3304132 +0000 UTC m=+0.385595580 container attach 914bffc785c4852404f1e5ec2f4e803fd28d6884dae9cf5d8daccf67d476fadd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-20T10:48:15.803 INFO:journalctl@ceph.osd.10.smithi046.stdout:Apr 20 10:48:15 smithi046 podman[58728]: 2024-04-20 10:48:15.559144407 +0000 UTC m=+0.614326751 container died 914bffc785c4852404f1e5ec2f4e803fd28d6884dae9cf5d8daccf67d476fadd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-20T10:48:16.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:15 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:15.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:16.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:15 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:15.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:16.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:15 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:15.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:16.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:15 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:15.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:16.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:15 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:15.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:16.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:15 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:15.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:16.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:15 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:15.860+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6820 osd.7 since back 2024-04-20T10:47:46.768215+0000 front 2024-04-20T10:47:46.768281+0000 (oldest deadline 2024-04-20T10:48:12.668222+0000) 2024-04-20T10:48:16.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.10.service' 2024-04-20T10:48:16.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:48:16.301 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-20T10:48:16.302 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-20T10:48:16.302 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.11 2024-04-20T10:48:16.622 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:16 smithi162 systemd[1]: Stopping Ceph osd.11 for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:48:16.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:16 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:16.622+0000 7f9bc5abf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:48:16.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:16 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:16.622+0000 7f9bc5abf700 -1 osd.11 103 *** Got signal Terminated *** 2024-04-20T10:48:16.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:16 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:16.622+0000 7f9bc5abf700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T10:48:16.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:16 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:16.825+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:16.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:16 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:16.825+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:16.901 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:16 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:16.825+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:16.901 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:16 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:16.825+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:16.901 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:16 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:16.825+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:16.901 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:16 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:16.825+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:16.901 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:16 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:16.825+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6820 osd.7 since back 2024-04-20T10:47:46.768215+0000 front 2024-04-20T10:47:46.768281+0000 (oldest deadline 2024-04-20T10:48:12.668222+0000) 2024-04-20T10:48:18.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:17 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:17.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:18.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:17 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:17.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:18.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:17 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:17.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:18.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:17 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:17.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:18.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:17 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:17.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:18.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:17 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:17.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:18.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:17 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:17.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6820 osd.7 since back 2024-04-20T10:47:46.768215+0000 front 2024-04-20T10:47:46.768281+0000 (oldest deadline 2024-04-20T10:48:12.668222+0000) 2024-04-20T10:48:19.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:18 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:18.812+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:19.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:18 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:18.812+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:19.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:18 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:18.812+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:19.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:18 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:18.812+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:19.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:18 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:18.812+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:19.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:18 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:18.812+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:19.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:18 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:18.812+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6820 osd.7 since back 2024-04-20T10:47:46.768215+0000 front 2024-04-20T10:47:46.768281+0000 (oldest deadline 2024-04-20T10:48:12.668222+0000) 2024-04-20T10:48:20.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:19 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:19.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:20.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:19 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:19.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:20.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:19 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:19.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:20.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:19 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:19.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:20.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:19 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:19.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:20.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:19 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:19.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:20.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:19 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:19.843+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6820 osd.7 since back 2024-04-20T10:47:46.768215+0000 front 2024-04-20T10:47:46.768281+0000 (oldest deadline 2024-04-20T10:48:12.668222+0000) 2024-04-20T10:48:21.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:20 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:20.834+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-20T10:46:44.457884+0000 front 2024-04-20T10:46:44.457996+0000 (oldest deadline 2024-04-20T10:47:08.557704+0000) 2024-04-20T10:48:21.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:20 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:20.834+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6804 osd.1 since back 2024-04-20T10:46:52.559389+0000 front 2024-04-20T10:46:52.559231+0000 (oldest deadline 2024-04-20T10:47:17.259397+0000) 2024-04-20T10:48:21.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:20 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:20.834+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-04-20T10:47:04.261215+0000 front 2024-04-20T10:47:04.261295+0000 (oldest deadline 2024-04-20T10:47:29.561261+0000) 2024-04-20T10:48:21.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:20 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:20.834+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6812 osd.4 since back 2024-04-20T10:47:17.663799+0000 front 2024-04-20T10:47:17.663778+0000 (oldest deadline 2024-04-20T10:47:43.563634+0000) 2024-04-20T10:48:21.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:20 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:20.834+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-04-20T10:47:30.565032+0000 front 2024-04-20T10:47:30.565049+0000 (oldest deadline 2024-04-20T10:47:55.265084+0000) 2024-04-20T10:48:21.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:20 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:20.834+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-20T10:47:37.466933+0000 front 2024-04-20T10:47:37.466920+0000 (oldest deadline 2024-04-20T10:48:03.366610+0000) 2024-04-20T10:48:21.151 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:20 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11[47264]: 2024-04-20T10:48:20.834+0000 7f9bc20b3700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.46:6820 osd.7 since back 2024-04-20T10:47:46.768215+0000 front 2024-04-20T10:47:46.768281+0000 (oldest deadline 2024-04-20T10:48:12.668222+0000) 2024-04-20T10:48:22.239 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:21 smithi162 podman[59399]: 2024-04-20 10:48:21.951980998 +0000 UTC m=+5.458694744 container died d446a051b90ff02153d303e171490d7026d09e5f0528b67b0f8093ab52aaa8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-20T10:48:22.564 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:22 smithi162 podman[59399]: 2024-04-20 10:48:22.321774958 +0000 UTC m=+5.828488697 container cleanup d446a051b90ff02153d303e171490d7026d09e5f0528b67b0f8093ab52aaa8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T10:48:22.564 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:22 smithi162 bash[59399]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11 2024-04-20T10:48:22.890 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:22 smithi162 podman[59418]: 2024-04-20 10:48:22.564524645 +0000 UTC m=+0.607523985 container remove d446a051b90ff02153d303e171490d7026d09e5f0528b67b0f8093ab52aaa8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-20T10:48:23.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:22 smithi162 podman[59508]: 2024-04-20 10:48:22.89045019 +0000 UTC m=+0.111596046 container create e2384d53b5f71964056aeb1cf208298aa7df43e1fcc92eb6ed20cc0a8cf615d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-20T10:48:23.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:22 smithi162 podman[59508]: 2024-04-20 10:48:22.796985526 +0000 UTC m=+0.018131391 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T10:48:23.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:23 smithi162 podman[59508]: 2024-04-20 10:48:23.101009831 +0000 UTC m=+0.322155688 container init e2384d53b5f71964056aeb1cf208298aa7df43e1fcc92eb6ed20cc0a8cf615d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:48:23.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:23 smithi162 podman[59508]: 2024-04-20 10:48:23.104001826 +0000 UTC m=+0.325147690 container start e2384d53b5f71964056aeb1cf208298aa7df43e1fcc92eb6ed20cc0a8cf615d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-20T10:48:23.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:23 smithi162 podman[59508]: 2024-04-20 10:48:23.192945322 +0000 UTC m=+0.414091197 container attach e2384d53b5f71964056aeb1cf208298aa7df43e1fcc92eb6ed20cc0a8cf615d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T10:48:23.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Apr 20 10:48:23 smithi162 podman[59508]: 2024-04-20 10:48:23.401030522 +0000 UTC m=+0.622176372 container died e2384d53b5f71964056aeb1cf208298aa7df43e1fcc92eb6ed20cc0a8cf615d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-20T10:48:24.001 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@osd.11.service' 2024-04-20T10:48:24.038 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:48:24.039 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-20T10:48:24.039 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-20T10:48:24.039 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.a 2024-04-20T10:48:24.604 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 20 10:48:24 smithi042 systemd[1]: Stopping Ceph mds.a for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:48:25.104 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 20 10:48:24 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-a[58461]: 2024-04-20T10:48:24.640+0000 7f6a56400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:48:25.104 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 20 10:48:24 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-a[58461]: 2024-04-20T10:48:24.640+0000 7f6a56400700 -1 mds.a *** got signal Terminated *** 2024-04-20T10:48:27.690 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 20 10:48:27 smithi042 podman[93227]: 2024-04-20 10:48:27.388830638 +0000 UTC m=+2.880216700 container died 957181e577d744c24a1b87a54431045743718996c0696f8427aaa2f2a8b50162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-20T10:48:28.025 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 20 10:48:27 smithi042 podman[93227]: 2024-04-20 10:48:27.807533215 +0000 UTC m=+3.298919287 container cleanup 957181e577d744c24a1b87a54431045743718996c0696f8427aaa2f2a8b50162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-20T10:48:28.026 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 20 10:48:27 smithi042 bash[93227]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-a 2024-04-20T10:48:28.353 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 20 10:48:28 smithi042 podman[93241]: 2024-04-20 10:48:28.025296053 +0000 UTC m=+0.618039475 container remove 957181e577d744c24a1b87a54431045743718996c0696f8427aaa2f2a8b50162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T10:48:28.520 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.a.service' 2024-04-20T10:48:28.631 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:48:28.632 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-20T10:48:28.632 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-20T10:48:28.632 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.d 2024-04-20T10:48:28.854 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:48:28 smithi042 systemd[1]: Stopping Ceph mds.d for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:48:29.603 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:48:29 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-d[57465]: 2024-04-20T10:48:29.259+0000 7f62b9c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:48:29.604 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:48:29 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-d[57465]: 2024-04-20T10:48:29.259+0000 7f62b9c00700 -1 mds.d *** got signal Terminated *** 2024-04-20T10:48:32.261 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:48:31 smithi042 podman[93346]: 2024-04-20 10:48:31.964181714 +0000 UTC m=+2.850473181 container died 3f1f0f598855bc25fb42b1d47db5379bd5648d6df37827b319a3e2695e9af862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:48:32.603 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:48:32 smithi042 podman[93346]: 2024-04-20 10:48:32.436979711 +0000 UTC m=+3.323271171 container cleanup 3f1f0f598855bc25fb42b1d47db5379bd5648d6df37827b319a3e2695e9af862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-20T10:48:32.603 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:48:32 smithi042 bash[93346]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-d 2024-04-20T10:48:33.005 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:48:32 smithi042 podman[93360]: 2024-04-20 10:48:32.754828698 +0000 UTC m=+0.778090228 container remove 3f1f0f598855bc25fb42b1d47db5379bd5648d6df37827b319a3e2695e9af862 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-20T10:48:33.005 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 20 10:48:33 smithi042 systemd[1]: ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.d.service: Deactivated successfully. 2024-04-20T10:48:33.014 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.d.service' 2024-04-20T10:48:33.125 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:48:33.125 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-20T10:48:33.126 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-20T10:48:33.126 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.g 2024-04-20T10:48:33.603 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:48:33 smithi042 systemd[1]: Stopping Ceph mds.g for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:48:34.103 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:48:33 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-g[58075]: 2024-04-20T10:48:33.705+0000 7f26c1600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:48:34.104 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:48:33 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-g[58075]: 2024-04-20T10:48:33.705+0000 7f26c1600700 -1 mds.g *** got signal Terminated *** 2024-04-20T10:48:35.520 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:48:35 smithi042 podman[93466]: 2024-04-20 10:48:35.224308492 +0000 UTC m=+1.642519651 container died 45239af92595ee9df7e10eb5096c3acbbedbc3816bb69f3723c9c73a55be0ba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-20T10:48:35.817 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:48:35 smithi042 podman[93466]: 2024-04-20 10:48:35.64069447 +0000 UTC m=+2.058905623 container cleanup 45239af92595ee9df7e10eb5096c3acbbedbc3816bb69f3723c9c73a55be0ba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-g, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2) 2024-04-20T10:48:35.817 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:48:35 smithi042 bash[93466]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-g 2024-04-20T10:48:36.103 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 20 10:48:35 smithi042 podman[93480]: 2024-04-20 10:48:35.816727887 +0000 UTC m=+0.568358737 container remove 45239af92595ee9df7e10eb5096c3acbbedbc3816bb69f3723c9c73a55be0ba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-g, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-20T10:48:36.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.g.service' 2024-04-20T10:48:36.417 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:48:36.417 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-20T10:48:36.417 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-20T10:48:36.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.j 2024-04-20T10:48:36.853 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:48:36 smithi042 systemd[1]: Stopping Ceph mds.j for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:48:37.353 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:48:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-j[57081]: 2024-04-20T10:48:37.027+0000 7fabd8e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:48:37.354 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:48:37 smithi042 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-j[57081]: 2024-04-20T10:48:37.027+0000 7fabd8e00700 -1 mds.j *** got signal Terminated *** 2024-04-20T10:48:40.311 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:48:39 smithi042 podman[93583]: 2024-04-20 10:48:39.853560754 +0000 UTC m=+2.946890887 container died f99cf4e9e713dc9c99495999a07f419ea8788f31e9a995e94ea9309b627f8ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-20T10:48:40.603 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:48:40 smithi042 podman[93583]: 2024-04-20 10:48:40.421899872 +0000 UTC m=+3.515230036 container cleanup f99cf4e9e713dc9c99495999a07f419ea8788f31e9a995e94ea9309b627f8ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-20T10:48:40.604 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:48:40 smithi042 bash[93583]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-j 2024-04-20T10:48:41.104 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 20 10:48:40 smithi042 podman[93608]: 2024-04-20 10:48:40.731155623 +0000 UTC m=+0.866622498 container remove f99cf4e9e713dc9c99495999a07f419ea8788f31e9a995e94ea9309b627f8ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-j, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-20T10:48:41.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.j.service' 2024-04-20T10:48:41.344 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:48:41.344 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-20T10:48:41.344 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-20T10:48:41.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.b 2024-04-20T10:48:41.625 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:48:41 smithi046 systemd[1]: Stopping Ceph mds.b for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:48:41.909 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:48:41 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b[52186]: 2024-04-20T10:48:41.660+0000 7f73e8209700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:48:41.909 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:48:41 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b[52186]: 2024-04-20T10:48:41.660+0000 7f73e8209700 -1 mds.b *** got signal Terminated *** 2024-04-20T10:48:43.560 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:48:43 smithi046 podman[58834]: 2024-04-20 10:48:43.308622588 +0000 UTC m=+1.769055194 container died bc113dea996017e7fc033c439d1d7b5b87c67ca8c3364bbe1ddea7dfdb5ef9c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-20T10:48:43.843 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:48:43 smithi046 podman[58834]: 2024-04-20 10:48:43.651273475 +0000 UTC m=+2.111706080 container cleanup bc113dea996017e7fc033c439d1d7b5b87c67ca8c3364bbe1ddea7dfdb5ef9c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:48:43.843 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:48:43 smithi046 bash[58834]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b 2024-04-20T10:48:44.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.b.service' 2024-04-20T10:48:44.099 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:48:43 smithi046 podman[58860]: 2024-04-20 10:48:43.843933045 +0000 UTC m=+0.529537570 container remove bc113dea996017e7fc033c439d1d7b5b87c67ca8c3364bbe1ddea7dfdb5ef9c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-20T10:48:44.100 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:48:44 smithi046 systemd[1]: ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.b.service: Deactivated successfully. 2024-04-20T10:48:44.100 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:48:44 smithi046 systemd[1]: Stopped Ceph mds.b for 83e8f766-feff-11ee-bc92-c7b262605968. 2024-04-20T10:48:44.100 INFO:journalctl@ceph.mds.b.smithi046.stdout:Apr 20 10:48:44 smithi046 systemd[1]: ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.b.service: Consumed 11min 52.819s CPU time. 2024-04-20T10:48:44.113 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:48:44.113 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-20T10:48:44.114 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-20T10:48:44.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.e 2024-04-20T10:48:44.414 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:48:44 smithi046 systemd[1]: Stopping Ceph mds.e for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:48:44.803 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:48:44 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-e[51566]: 2024-04-20T10:48:44.413+0000 7f78448b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:48:44.803 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:48:44 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-e[51566]: 2024-04-20T10:48:44.413+0000 7f78448b0700 -1 mds.e *** got signal Terminated *** 2024-04-20T10:48:47.097 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:48:46 smithi046 podman[58970]: 2024-04-20 10:48:46.836876486 +0000 UTC m=+2.505613857 container died e15407ae12275cc1561cd340a3afd7c0d007e967e92b7938799c25b8bb4d2295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-20T10:48:47.553 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:48:47 smithi046 podman[58970]: 2024-04-20 10:48:47.171709453 +0000 UTC m=+2.840446815 container cleanup e15407ae12275cc1561cd340a3afd7c0d007e967e92b7938799c25b8bb4d2295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:48:47.553 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:48:47 smithi046 bash[58970]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-e 2024-04-20T10:48:47.553 INFO:journalctl@ceph.mds.e.smithi046.stdout:Apr 20 10:48:47 smithi046 podman[58984]: 2024-04-20 10:48:47.34781206 +0000 UTC m=+0.504597929 container remove e15407ae12275cc1561cd340a3afd7c0d007e967e92b7938799c25b8bb4d2295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:48:47.607 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.e.service' 2024-04-20T10:48:47.640 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:48:47.640 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-20T10:48:47.640 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-20T10:48:47.640 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.h 2024-04-20T10:48:47.983 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:48:47 smithi046 systemd[1]: Stopping Ceph mds.h for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:48:48.303 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:48:47 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-h[51869]: 2024-04-20T10:48:47.983+0000 7f330874c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:48:48.303 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:48:47 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-h[51869]: 2024-04-20T10:48:47.983+0000 7f330874c700 -1 mds.h *** got signal Terminated *** 2024-04-20T10:48:52.303 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:48:51 smithi046 podman[59093]: 2024-04-20 10:48:51.945161949 +0000 UTC m=+4.085975191 container died f397eec9989b20c70ebfd92d06e7317f71bb12ea04b17b1774c49c4151c5e268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-h, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T10:48:52.676 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:48:52 smithi046 podman[59093]: 2024-04-20 10:48:52.417863963 +0000 UTC m=+4.558677211 container cleanup f397eec9989b20c70ebfd92d06e7317f71bb12ea04b17b1774c49c4151c5e268 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-20T10:48:52.677 INFO:journalctl@ceph.mds.h.smithi046.stdout:Apr 20 10:48:52 smithi046 bash[59093]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-h 2024-04-20T10:48:52.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.h.service' 2024-04-20T10:48:52.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:48:52.880 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-20T10:48:52.880 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-20T10:48:52.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.k 2024-04-20T10:48:53.205 INFO:journalctl@ceph.mds.k.smithi046.stdout:Apr 20 10:48:52 smithi046 systemd[1]: Stopping Ceph mds.k for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:48:53.553 INFO:journalctl@ceph.mds.k.smithi046.stdout:Apr 20 10:48:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-k[51242]: 2024-04-20T10:48:53.205+0000 7fe34d928700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:48:53.553 INFO:journalctl@ceph.mds.k.smithi046.stdout:Apr 20 10:48:53 smithi046 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-k[51242]: 2024-04-20T10:48:53.205+0000 7fe34d928700 -1 mds.k *** got signal Terminated *** 2024-04-20T10:48:56.053 INFO:journalctl@ceph.mds.k.smithi046.stdout:Apr 20 10:48:55 smithi046 podman[59213]: 2024-04-20 10:48:55.748056512 +0000 UTC m=+2.649372880 container died e50c2e20b7c2bcb4d91d9d93db1dd8357b0ec9915fc2a59824386827da4962bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-20T10:48:56.384 INFO:journalctl@ceph.mds.k.smithi046.stdout:Apr 20 10:48:56 smithi046 podman[59213]: 2024-04-20 10:48:56.064032542 +0000 UTC m=+2.965348858 container cleanup e50c2e20b7c2bcb4d91d9d93db1dd8357b0ec9915fc2a59824386827da4962bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416) 2024-04-20T10:48:56.384 INFO:journalctl@ceph.mds.k.smithi046.stdout:Apr 20 10:48:56 smithi046 bash[59213]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-k 2024-04-20T10:48:56.384 INFO:journalctl@ceph.mds.k.smithi046.stdout:Apr 20 10:48:56 smithi046 podman[59227]: 2024-04-20 10:48:56.273202886 +0000 UTC m=+0.506875954 container remove e50c2e20b7c2bcb4d91d9d93db1dd8357b0ec9915fc2a59824386827da4962bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:48:56.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.k.service' 2024-04-20T10:48:56.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:48:56.441 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-20T10:48:56.441 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-20T10:48:56.442 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.c 2024-04-20T10:48:56.752 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:48:56 smithi162 systemd[1]: Stopping Ceph mds.c for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:48:57.014 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:48:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-c[52243]: 2024-04-20T10:48:56.751+0000 7f497adb8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:48:57.015 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:48:56 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-c[52243]: 2024-04-20T10:48:56.751+0000 7f497adb8700 -1 mds.c *** got signal Terminated *** 2024-04-20T10:49:01.027 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:49:00 smithi162 podman[59616]: 2024-04-20 10:49:00.758335813 +0000 UTC m=+4.127389318 container died 669991ee670dd2e0bde5bfd32a0ad2c5e66105ccc4f82e100fa823520fa760a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-20T10:49:01.378 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:49:01 smithi162 podman[59616]: 2024-04-20 10:49:01.127223651 +0000 UTC m=+4.496277162 container cleanup 669991ee670dd2e0bde5bfd32a0ad2c5e66105ccc4f82e100fa823520fa760a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-c, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-20T10:49:01.378 INFO:journalctl@ceph.mds.c.smithi162.stdout:Apr 20 10:49:01 smithi162 bash[59616]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-c 2024-04-20T10:49:01.551 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.c.service' 2024-04-20T10:49:01.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:49:01.585 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-20T10:49:01.586 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-20T10:49:01.586 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.f 2024-04-20T10:49:01.899 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:49:01 smithi162 systemd[1]: Stopping Ceph mds.f for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:49:02.399 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:49:02 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-f[51935]: 2024-04-20T10:49:02.064+0000 7ff42216c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:49:02.400 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:49:02 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-f[51935]: 2024-04-20T10:49:02.064+0000 7ff42216c700 -1 mds.f *** got signal Terminated *** 2024-04-20T10:49:06.150 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:49:05 smithi162 podman[59750]: 2024-04-20 10:49:05.794602163 +0000 UTC m=+3.983742154 container died 22ceba1b0f8b997926759a89ec09327cfaae81cf28ffc2dcf97dac34a23f973c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-20T10:49:06.523 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:49:06 smithi162 podman[59750]: 2024-04-20 10:49:06.330706493 +0000 UTC m=+4.519846485 container cleanup 22ceba1b0f8b997926759a89ec09327cfaae81cf28ffc2dcf97dac34a23f973c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T10:49:06.523 INFO:journalctl@ceph.mds.f.smithi162.stdout:Apr 20 10:49:06 smithi162 bash[59750]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-f 2024-04-20T10:49:06.747 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.f.service' 2024-04-20T10:49:06.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:49:06.781 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-20T10:49:06.781 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-20T10:49:06.781 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.i 2024-04-20T10:49:07.134 INFO:journalctl@ceph.mds.i.smithi162.stdout:Apr 20 10:49:06 smithi162 systemd[1]: Stopping Ceph mds.i for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:49:07.400 INFO:journalctl@ceph.mds.i.smithi162.stdout:Apr 20 10:49:07 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-i[52560]: 2024-04-20T10:49:07.133+0000 7f9020088700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:49:07.400 INFO:journalctl@ceph.mds.i.smithi162.stdout:Apr 20 10:49:07 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-i[52560]: 2024-04-20T10:49:07.133+0000 7f9020088700 -1 mds.i *** got signal Terminated *** 2024-04-20T10:49:11.400 INFO:journalctl@ceph.mds.i.smithi162.stdout:Apr 20 10:49:11 smithi162 podman[59869]: 2024-04-20 10:49:11.148969348 +0000 UTC m=+4.144268052 container died dd8bfdadbfd492ed9982ac520064526c802a0a53ce5d11a4d6792de398553717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T10:49:11.776 INFO:journalctl@ceph.mds.i.smithi162.stdout:Apr 20 10:49:11 smithi162 podman[59869]: 2024-04-20 10:49:11.525828765 +0000 UTC m=+4.521127523 container cleanup dd8bfdadbfd492ed9982ac520064526c802a0a53ce5d11a4d6792de398553717 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-20T10:49:11.777 INFO:journalctl@ceph.mds.i.smithi162.stdout:Apr 20 10:49:11 smithi162 bash[59869]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-i 2024-04-20T10:49:11.889 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.i.service' 2024-04-20T10:49:11.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:49:11.921 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-20T10:49:11.921 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-20T10:49:11.922 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.l 2024-04-20T10:49:12.149 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:49:12 smithi162 systemd[1]: Stopping Ceph mds.l for 83e8f766-feff-11ee-bc92-c7b262605968... 2024-04-20T10:49:12.650 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:49:12 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-l[51627]: 2024-04-20T10:49:12.245+0000 7fe67417a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T10:49:12.650 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:49:12 smithi162 ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-l[51627]: 2024-04-20T10:49:12.245+0000 7fe67417a700 -1 mds.l *** got signal Terminated *** 2024-04-20T10:49:16.562 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:49:16 smithi162 podman[59990]: 2024-04-20 10:49:16.253554124 +0000 UTC m=+4.112900705 container died d0514c17601a4f8bf2a936e6fb4cf6c1f59904d0012db13abdd0e69114d738ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-l, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-20T10:49:16.854 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:49:16 smithi162 podman[59990]: 2024-04-20 10:49:16.637109418 +0000 UTC m=+4.496455994 container cleanup d0514c17601a4f8bf2a936e6fb4cf6c1f59904d0012db13abdd0e69114d738ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-20T10:49:16.854 INFO:journalctl@ceph.mds.l.smithi162.stdout:Apr 20 10:49:16 smithi162 bash[59990]: ceph-83e8f766-feff-11ee-bc92-c7b262605968-mds-l 2024-04-20T10:49:16.982 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-83e8f766-feff-11ee-bc92-c7b262605968@mds.l.service' 2024-04-20T10:49:17.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T10:49:17.013 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-20T10:49:17.013 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 83e8f766-feff-11ee-bc92-c7b262605968 --force --keep-logs 2024-04-20T10:49:17.347 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:49:22.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 83e8f766-feff-11ee-bc92-c7b262605968 --force --keep-logs 2024-04-20T10:49:22.979 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:49:25.550 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 83e8f766-feff-11ee-bc92-c7b262605968 --force --keep-logs 2024-04-20T10:49:25.740 INFO:teuthology.orchestra.run.smithi162.stdout:Deleting cluster with fsid: 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:49:28.022 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T10:49:28.111 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T10:49:28.143 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T10:49:28.169 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-20T10:49:28.171 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445/remote/smithi042/crash 2024-04-20T10:49:28.172 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/crash -- . 2024-04-20T10:49:28.265 INFO:teuthology.orchestra.run.smithi042.stderr:tar: /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T10:49:28.265 INFO:teuthology.orchestra.run.smithi042.stderr:tar: Error is not recoverable: exiting now 2024-04-20T10:49:28.269 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445/remote/smithi046/crash 2024-04-20T10:49:28.270 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/crash -- . 2024-04-20T10:49:28.312 INFO:teuthology.orchestra.run.smithi046.stderr:tar: /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T10:49:28.313 INFO:teuthology.orchestra.run.smithi046.stderr:tar: Error is not recoverable: exiting now 2024-04-20T10:49:28.315 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445/remote/smithi162/crash 2024-04-20T10:49:28.316 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/crash -- . 2024-04-20T10:49:28.354 INFO:teuthology.orchestra.run.smithi162.stderr:tar: /var/lib/ceph/83e8f766-feff-11ee-bc92-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T10:49:28.354 INFO:teuthology.orchestra.run.smithi162.stderr:tar: Error is not recoverable: exiting now 2024-04-20T10:49:28.355 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-20T10:49:28.356 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-20T10:49:28.462 INFO:tasks.cephadm:Compressing logs... 2024-04-20T10:49:28.463 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T10:49:28.467 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T10:49:28.470 DEBUG:teuthology.orchestra.run.smithi162:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T10:49:28.493 INFO:teuthology.orchestra.run.smithi162.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-20T10:49:28.494 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.log 2024-04-20T10:49:28.494 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.2.log 2024-04-20T10:49:28.494 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.f.log 2024-04-20T10:49:28.495 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.audit.log 2024-04-20T10:49:28.495 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.5.log 2024-04-20T10:49:28.495 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.c.log 2024-04-20T10:49:28.496 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.audit.log: /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mon.c.log 2024-04-20T10:49:28.496 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mgr.z.log 2024-04-20T10:49:28.498 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.c.log: /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mon.c.log: 86.9% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.log.gz 2024-04-20T10:49:28.498 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.i.log 2024-04-20T10:49:28.498 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mgr.z.log: 90.4% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.audit.log.gz 2024-04-20T10:49:28.501 INFO:teuthology.orchestra.run.smithi162.stderr: 94.5% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.f.log.gz 2024-04-20T10:49:28.502 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.cephadm.log 2024-04-20T10:49:28.502 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.11.log 2024-04-20T10:49:28.502 INFO:teuthology.orchestra.run.smithi162.stderr: 94.5% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.c.log.gz 2024-04-20T10:49:28.502 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.cephadm.log: 83.2% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.cephadm.log.gz 2024-04-20T10:49:28.502 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.8.log 2024-04-20T10:49:28.507 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.11.log: 92.0% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mgr.z.log.gz 2024-04-20T10:49:28.508 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-volume.log 2024-04-20T10:49:28.510 INFO:teuthology.orchestra.run.smithi046.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-20T10:49:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.log 2024-04-20T10:49:28.511 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.audit.log 2024-04-20T10:49:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.7.log 2024-04-20T10:49:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.log: /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.b.log 2024-04-20T10:49:28.512 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.4.log 2024-04-20T10:49:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.1.log 2024-04-20T10:49:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mgr.y.log 2024-04-20T10:49:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.h.log 2024-04-20T10:49:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.10.log 2024-04-20T10:49:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.cephadm.log 2024-04-20T10:49:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.e.log 2024-04-20T10:49:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-volume.log 2024-04-20T10:49:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mon.b.log 2024-04-20T10:49:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.k.log 2024-04-20T10:49:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T10:49:28.525 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.log 2024-04-20T10:49:28.525 INFO:teuthology.orchestra.run.smithi042.stderr:: No such file or directory 2024-04-20T10:49:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.audit.log 2024-04-20T10:49:28.527 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.g.log 2024-04-20T10:49:28.528 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.j.log 2024-04-20T10:49:28.529 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.3.log 2024-04-20T10:49:28.530 INFO:teuthology.orchestra.run.smithi042.stderr: 86.9% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.log.gz 2024-04-20T10:49:28.531 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.0.log 2024-04-20T10:49:28.532 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.l.log 2024-04-20T10:49:28.533 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.3.log: 90.2% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.audit.log.gz 2024-04-20T10:49:28.533 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mgr.x.log 2024-04-20T10:49:28.534 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mon.a.log 2024-04-20T10:49:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mgr.x.log: 94.4% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.j.log.gz 2024-04-20T10:49:28.536 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.6.log 2024-04-20T10:49:28.536 INFO:teuthology.orchestra.run.smithi042.stderr: 94.5% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.g.log.gz 2024-04-20T10:49:28.537 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.cephadm.log 2024-04-20T10:49:28.538 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.a.log 2024-04-20T10:49:28.539 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-volume.log 2024-04-20T10:49:28.541 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.a.log: 83.8% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.cephadm.log.gz 2024-04-20T10:49:28.544 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.d.log 2024-04-20T10:49:28.545 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T10:49:28.548 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.9.log 2024-04-20T10:49:28.548 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.d.log: 94.4% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.a.log.gz 2024-04-20T10:49:28.548 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T10:49:28.552 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-volume.log.gz 2024-04-20T10:49:28.552 INFO:teuthology.orchestra.run.smithi162.stderr: 94.5% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.l.log.gz 2024-04-20T10:49:28.554 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T10:49:28.554 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 94.3% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.d.log.gz 2024-04-20T10:49:28.560 INFO:teuthology.orchestra.run.smithi042.stderr: 90.9% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-volume.log.gz 2024-04-20T10:49:28.564 INFO:teuthology.orchestra.run.smithi042.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T10:49:28.611 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 83.3% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.cephadm.log.gz 2024-04-20T10:49:28.627 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.log.gz 2024-04-20T10:49:28.655 INFO:teuthology.orchestra.run.smithi046.stderr: 94.5% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.h.log.gz 2024-04-20T10:49:28.714 INFO:teuthology.orchestra.run.smithi046.stderr: 92.0% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mgr.y.log.gz 2024-04-20T10:49:28.719 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-volume.log.gz 2024-04-20T10:49:28.725 INFO:teuthology.orchestra.run.smithi046.stderr: 90.3% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph.audit.log.gz 2024-04-20T10:49:28.739 INFO:teuthology.orchestra.run.smithi046.stderr: 94.4% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.k.log.gz 2024-04-20T10:49:28.756 INFO:teuthology.orchestra.run.smithi046.stderr: 94.5% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.e.log.gz 2024-04-20T10:49:28.788 INFO:teuthology.orchestra.run.smithi046.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T10:49:28.832 INFO:teuthology.orchestra.run.smithi042.stderr: 90.2% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mgr.x.log.gz 2024-04-20T10:49:29.116 INFO:teuthology.orchestra.run.smithi162.stderr: 91.7% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mon.c.log.gz 2024-04-20T10:49:29.934 INFO:teuthology.orchestra.run.smithi042.stderr: 90.3% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mon.a.log.gz 2024-04-20T10:49:33.517 INFO:teuthology.orchestra.run.smithi046.stderr: 91.8% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mon.b.log.gz 2024-04-20T10:49:40.699 INFO:teuthology.orchestra.run.smithi162.stderr: 95.3% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.i.log.gz 2024-04-20T10:49:43.985 INFO:teuthology.orchestra.run.smithi042.stderr: 93.1% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.9.log.gz 2024-04-20T10:49:45.489 INFO:teuthology.orchestra.run.smithi162.stderr: 92.8% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.8.log.gz 2024-04-20T10:49:47.411 INFO:teuthology.orchestra.run.smithi162.stderr: 92.7% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.5.log.gz 2024-04-20T10:49:48.207 INFO:teuthology.orchestra.run.smithi042.stderr: 93.0% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.6.log.gz 2024-04-20T10:49:48.611 INFO:teuthology.orchestra.run.smithi162.stderr: 92.8% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.2.log.gz 2024-04-20T10:49:49.885 INFO:teuthology.orchestra.run.smithi042.stderr: 93.3% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.0.log.gz 2024-04-20T10:49:50.321 INFO:teuthology.orchestra.run.smithi042.stderr: 93.1% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.3.log.gz 2024-04-20T10:49:50.324 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-04-20T10:49:50.325 INFO:teuthology.orchestra.run.smithi042.stderr:real 0m21.825s 2024-04-20T10:49:50.325 INFO:teuthology.orchestra.run.smithi042.stderr:user 1m14.089s 2024-04-20T10:49:50.325 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m5.717s 2024-04-20T10:49:51.217 INFO:teuthology.orchestra.run.smithi162.stderr: 93.3% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.11.log.gz 2024-04-20T10:49:51.219 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-04-20T10:49:51.219 INFO:teuthology.orchestra.run.smithi162.stderr:real 0m22.735s 2024-04-20T10:49:51.219 INFO:teuthology.orchestra.run.smithi162.stderr:user 1m28.610s 2024-04-20T10:49:51.219 INFO:teuthology.orchestra.run.smithi162.stderr:sys 0m2.836s 2024-04-20T10:50:36.340 INFO:teuthology.orchestra.run.smithi046.stderr: 92.2% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.4.log.gz 2024-04-20T10:51:24.619 INFO:teuthology.orchestra.run.smithi046.stderr: 92.8% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.10.log.gz 2024-04-20T10:52:04.730 INFO:teuthology.orchestra.run.smithi046.stderr: 93.0% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.1.log.gz 2024-04-20T10:52:17.143 INFO:teuthology.orchestra.run.smithi046.stderr: 93.2% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-osd.7.log.gz 2024-04-20T10:55:37.761 INFO:teuthology.orchestra.run.smithi046.stderr: 93.1% -- replaced with /var/log/ceph/83e8f766-feff-11ee-bc92-c7b262605968/ceph-mds.b.log.gz 2024-04-20T10:55:37.763 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-20T10:55:37.763 INFO:teuthology.orchestra.run.smithi046.stderr:real 6m9.278s 2024-04-20T10:55:37.763 INFO:teuthology.orchestra.run.smithi046.stderr:user 4m45.777s 2024-04-20T10:55:37.763 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m9.432s 2024-04-20T10:55:37.764 INFO:tasks.cephadm:Archiving logs... 2024-04-20T10:55:37.765 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445/remote/smithi042/log 2024-04-20T10:55:37.766 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T10:55:49.267 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445/remote/smithi046/log 2024-04-20T10:55:49.268 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T10:56:32.651 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445/remote/smithi162/log 2024-04-20T10:56:32.652 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T10:56:44.906 INFO:tasks.cephadm:Removing cluster... 2024-04-20T10:56:44.906 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 83e8f766-feff-11ee-bc92-c7b262605968 --force 2024-04-20T10:56:45.237 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:56:46.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 83e8f766-feff-11ee-bc92-c7b262605968 --force 2024-04-20T10:56:46.349 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:56:46.909 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 83e8f766-feff-11ee-bc92-c7b262605968 --force 2024-04-20T10:56:47.100 INFO:teuthology.orchestra.run.smithi162.stdout:Deleting cluster with fsid: 83e8f766-feff-11ee-bc92-c7b262605968 2024-04-20T10:56:47.720 INFO:tasks.cephadm:Removing cephadm ... 2024-04-20T10:56:47.720 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T10:56:47.769 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T10:56:47.789 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T10:56:47.805 INFO:tasks.cephadm:Teardown complete 2024-04-20T10:56:47.805 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-20T10:56:47.816 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-20T10:56:47.816 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T10:56:47.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T10:56:47.831 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T10:56:47.975 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-20T10:56:47.985 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-20T10:56:47.985 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:56:47.992 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:56:47.995 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:56:48.008 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:56:48.010 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:56:48.013 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:56:48.014 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-04-20T10:56:48.014 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 62 +570us[ +570us] +/- 82ms 2024-04-20T10:56:48.014 INFO:teuthology.orchestra.run.smithi162.stdout:^* hv02.front.sepia.ceph.com 3 6 377 62 +22us[ +35us] +/- 70ms 2024-04-20T10:56:48.014 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -504us[ -504us] +/- 83ms 2024-04-20T10:56:48.014 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:56:48.038 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:56:48.038 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-04-20T10:56:48.038 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 +536us[ +527us] +/- 82ms 2024-04-20T10:56:48.038 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 -20us[ -29us] +/- 71ms 2024-04-20T10:56:48.039 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 -536us[ -545us] +/- 83ms 2024-04-20T10:56:48.039 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:56:48.042 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:56:48.053 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:56:48.053 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-04-20T10:56:48.053 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 58 +551us[ +564us] +/- 82ms 2024-04-20T10:56:48.053 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 -11us[+1741ns] +/- 71ms 2024-04-20T10:56:48.054 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -519us[ -506us] +/- 83ms 2024-04-20T10:56:48.054 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:56:48.055 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-20T10:56:48.066 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-20T10:56:48.066 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-20T10:56:48.116 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T10:56:48.325 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T10:56:48.399 DEBUG:teuthology.orchestra.run.smithi162:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T10:56:48.451 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:56:48.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:56:48.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:56:48.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:56:48.584 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:56:48.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:56:48.610 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-20T10:56:48.622 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-20T10:56:48.671 INFO:teuthology.task.internal:Duration was 2677.454223 seconds 2024-04-20T10:56:48.672 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-20T10:56:48.722 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-20T10:56:48.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T10:56:48.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T10:56:48.733 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T10:56:48.769 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:56:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:56:48.877 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:56:49.203 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-20T10:56:49.203 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-04-20T10:56:49.203 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T10:56:49.267 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-04-20T10:56:49.268 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T10:56:49.304 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2024-04-20T10:56:49.304 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T10:56:49.324 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-20T10:56:49.325 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T10:56:49.333 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T10:56:49.346 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T10:56:49.422 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-20T10:56:49.423 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T10:56:49.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T10:56:49.432 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T10:56:49.899 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-20T10:56:49.911 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-20T10:56:49.912 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T10:56:50.002 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T10:56:50.028 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T10:56:50.053 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-20T10:56:50.067 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T10:56:50.075 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T10:56:50.077 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T10:56:50.112 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-04-20T10:56:50.116 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2024-04-20T10:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-04-20T10:56:50.216 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T10:56:50.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:56:50.266 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T10:56:50.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:56:50.281 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T10:56:50.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:56:50.295 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-20T10:56:50.307 INFO:teuthology.task.internal:Transferring archived files... 2024-04-20T10:56:50.308 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445/remote/smithi042 2024-04-20T10:56:50.309 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T10:56:50.412 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445/remote/smithi046 2024-04-20T10:56:50.413 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T10:56:50.446 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445/remote/smithi162 2024-04-20T10:56:50.447 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T10:56:50.484 INFO:teuthology.task.internal:Removing archive directory... 2024-04-20T10:56:50.485 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T10:56:50.494 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T10:56:50.497 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T10:56:50.544 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-20T10:56:50.557 INFO:teuthology.task.internal:Not uploading archives. 2024-04-20T10:56:50.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-20T10:56:50.581 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-20T10:56:50.582 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T10:56:50.590 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T10:56:50.592 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T10:56:50.605 INFO:teuthology.orchestra.run.smithi046.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 10:56 /home/ubuntu/cephtest 2024-04-20T10:56:50.606 INFO:teuthology.orchestra.run.smithi162.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 10:56 /home/ubuntu/cephtest 2024-04-20T10:56:50.643 INFO:teuthology.orchestra.run.smithi042.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 10:56 /home/ubuntu/cephtest 2024-04-20T10:56:50.644 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-20T10:56:50.660 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-20T10:56:50.718 INFO:teuthology.nuke:Checking targets against current locks 2024-04-20T10:56:50.746 DEBUG:teuthology.nuke:shortname: smithi042 2024-04-20T10:56:50.746 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T10:56:50.770 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 09:51:24.035993', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKCLxedFLQQooUioetMNOWMSW3V3DZBT3SZGtYGjkEavLNXRfqwvYiBnhj37FmDrLfhFL546HlP9/18VyZRadts='} 2024-04-20T10:56:50.775 DEBUG:teuthology.nuke:shortname: smithi046 2024-04-20T10:56:50.776 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T10:56:50.800 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 09:51:24.032904', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNvWiyW+vBj4WJa04epY2fu4M+1srgsmyOzK2JFbH4XVy32KcTmP8AxkpXBroArg2JrtL/lomIreJnIn9bWJW9I='} 2024-04-20T10:56:50.801 DEBUG:teuthology.nuke:shortname: smithi162 2024-04-20T10:56:50.801 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T10:56:50.829 INFO:teuthology.orchestra.console.smithi042:Power off 2024-04-20T10:56:50.829 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T10:56:50.843 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664445', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 09:51:24.034627', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEGMrSYtx1ReTq6hR4AO9tNxgdLp5UhF+fXUZTlcHIimdAt8vJ5tY0gsMEKMLNemlxgMlXB8HEOfgG+5vfck9E8='} 2024-04-20T10:56:50.855 DEBUG:teuthology.orchestra.console.smithi042:power off output: Chassis Power Control: Down/Off 2024-04-20T10:56:50.856 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T10:56:50.878 INFO:teuthology.orchestra.console.smithi046:Power off 2024-04-20T10:56:50.878 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T10:56:50.888 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2024-04-20T10:56:50.901 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off 2024-04-20T10:56:50.901 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T10:56:50.917 INFO:teuthology.orchestra.console.smithi162:Power off 2024-04-20T10:56:50.917 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T10:56:50.928 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2024-04-20T10:56:50.941 DEBUG:teuthology.orchestra.console.smithi162:power off output: Chassis Power Control: Down/Off 2024-04-20T10:56:50.941 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T10:56:50.963 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2024-04-20T10:56:54.889 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T10:56:54.930 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T10:56:54.964 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T10:56:55.000 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2024-04-20T10:56:55.041 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2024-04-20T10:56:55.075 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2024-04-20T10:56:59.002 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T10:56:59.042 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T10:56:59.076 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T10:56:59.113 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2024-04-20T10:56:59.152 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2024-04-20T10:56:59.186 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2024-04-20T10:56:59.254 INFO:teuthology.orchestra.console.smithi046:Power off completed 2024-04-20T10:56:59.287 INFO:teuthology.orchestra.console.smithi162:Power off completed 2024-04-20T10:57:03.114 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T10:57:03.226 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is off 2024-04-20T10:57:03.327 INFO:teuthology.orchestra.console.smithi042:Power off completed 2024-04-20T10:57:03.434 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/kernel_untar_build}} duration: 2677.4542231559753 failure_reason: 'Command failed (workunit test kernel_untar_build.sh) on smithi042 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=9e1e488ef85640df8d29cacccc3dcc74 status: fail success: false 2024-04-20T10:57:03.434 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T10:57:03.522 INFO:teuthology.run:FAIL